Methods, apparatuses, devices, and products for displaying components

Abstract

This disclosure relates to methods, apparatus, devices, and products for displaying components. The method includes displaying first media content and a first component on a first interface. The method also includes displaying a second component in response to an operation on the first component, the second component including the first interactive content, and the second component being generated based on the first media content.

Description

Technical Field

[0001] This article relates to the field of human-computer interaction, and more specifically to methods, apparatuses, devices and products for display components. Background Technology

[0002] On content distribution platforms, many content creators publish instructional videos containing content such as recipes, travel guides, and skill tutorials. Content creators can use visuals, voice, subtitles, or any combination thereof in audio or video to present information for users to reference or follow. For example, cooking tutorial videos may include ingredient lists, cooking step lists, cooking times, or seasoning parameters. Travel guide videos may include itinerary plans, destination lists, time schedules, transportation information, accommodation information, or budget information. Users can summarize structured information such as recipe preparation steps or itinerary plans from these audio or video clips. Summary of the Invention

[0003] In a first aspect of the examples in this paper, a method for displaying a component is provided. The method includes displaying first media content and a first component on a first interface. The method also includes displaying a second component in response to an operation on the first component, the second component including the first interactive content, and the second component being generated based on the first media content. In this way, media information originally presented in audio and video format can be displayed in a structured and operable manner, facilitating users' quick understanding and use of key information in the media content, reducing the operational cost for users to organize media content themselves, and improving the user experience.

[0004] In a second aspect of the examples herein, an apparatus for displaying components is provided. The apparatus includes a first component display module configured to display first media content and the first component on a first interface. The apparatus also includes a second component display module configured to display a second component in response to an operation on the first component. The second component includes first interactive content and is generated based on the first media content. In this way, media information originally presented in audio and video format can be displayed in a structured and operable manner, facilitating users' quick understanding and use of key information in the media content, reducing the operational cost for users to organize media content themselves, and improving the user experience.

[0005] In a third aspect of the examples herein, an electronic device is provided. The electronic device includes one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement a method for displaying a component. The method includes displaying first media content and a first component on a first interface. The method also includes displaying a second component in response to an operation on the first component, the second component including the first interactive content, and the second component being generated based on the first media content. In this way, media information originally presented in audio and video format can be displayed in a structured and operable manner, facilitating users' quick understanding and use of key information in the media content, reducing the operational cost for users to organize media content themselves, and improving the user experience.

[0006] In a fourth aspect of the examples herein, a computer program product is provided. This computer program product is tangibly stored on a non-transitory computer-readable medium and includes machine-executable instructions that, when executed, cause a machine to implement a method for displaying a component. The method includes displaying first media content and a first component on a first interface. The method also includes displaying a second component in response to an operation on the first component, the second component including the first interactive content, and the second component being generated based on the first media content. In this way, media information originally presented in audio-visual form can be displayed in a structured and operable manner, facilitating users' quick understanding and use of key information in the media content, reducing the operational cost for users to organize media content themselves, and improving the user experience.

[0007] The summary section is provided to present the chosen concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or principal features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0008] The above and other features, advantages, and aspects of the examples herein will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. In the accompanying drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram is shown illustrating the example environment in which several examples in this article can be implemented;

[0010] Figure 2 A flowchart illustrating methods for displaying components, based on some examples in this article, is shown;

[0011] Figure 3 A schematic diagram of an example system for generating interactive components, based on some examples in this paper, is shown;

[0012] Figure 4 A schematic diagram illustrating an example process for generating interactive components, based on some examples in this article;

[0013] Figures 5A to 5D The diagram illustrates examples of user interfaces for generating interactive components, based on some examples in this article;

[0014] Figures 6A to 6D A schematic diagram is shown illustrating an example of generating a second interactive component based on a first interactive component, according to some examples in this paper;

[0015] Figure 7 A block diagram of a device for displaying components, according to some examples in this document, is shown;

[0016] Figure 8 A block diagram of a device that can implement several of the examples in this paper is shown.

[0017] In all the accompanying figures, the same or similar reference numerals denote the same or similar elements. Detailed Implementation

[0018] It is understood that all user-related data involved in this technical solution should be obtained and used only after authorization from the user. This means that if it is necessary to use a user's personal information in this technical solution, the user's explicit consent and authorization are required before obtaining this data; otherwise, no related data collection and use will be carried out. It should also be understood that when implementing this technical solution, relevant laws and regulations should be strictly followed in the process of data collection, use, and storage, and necessary technical measures should be taken to protect user data security and ensure the secure use of data.

[0019] It is understandable that before using the technical solutions disclosed in the examples in this article, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this article in an appropriate manner in accordance with relevant laws and regulations, and their authorization should be obtained.

[0020] For example, upon receiving a user's proactive request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations described herein, based on the prompt message.

[0021] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0022] It is understood that the above notification and user authorization process is merely illustrative and does not limit the implementation method described in this article. Other methods that comply with relevant laws and regulations may also be applied to the implementation method described in this article.

[0023] The examples herein will now be described in more detail with reference to the accompanying drawings. While some examples of this document are shown in the drawings, it should be understood that this document can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these examples are provided to provide a more thorough and complete understanding of this document. It should be understood that the accompanying drawings and examples are for illustrative purposes only and are not intended to limit the scope of this document.

[0024] In the description of the examples in this document, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "an example" or "the example" should be understood as "at least one example". The terms "first", "second", etc., may refer to different or the same objects unless explicitly stated otherwise. Other explicit and implicit definitions may also be included below.

[0025] As mentioned above, many content creators on content distribution platforms publish instructional videos containing content such as recipes, travel guides, and skill tutorials. Content creators can use visuals, audio, subtitles, or any combination thereof in audio or video to present information for users to reference or follow. For example, cooking tutorial videos may include ingredient lists, cooking step lists, cooking times, or seasoning parameters. Travel guide videos may include itinerary plans, destination lists, time schedules, transportation information, accommodation information, or budget information. Users can summarize structured information such as recipe preparation steps or itinerary plans from these audio or video clips.

[0026] After watching videos or listening to audio, users may need to further structure the content. For example, cooking tutorial videos might be organized into actionable ingredient lists or cooking step lists, or travel guide videos might be organized into itinerary plans. In some related technologies, to obtain such structured information, users need to take screenshots of the video, summarize the video or audio content into text, and then use note-taking tools to organize this content. However, this process requires users to expend effort to understand and summarize the media content, making it quite difficult. Furthermore, this process is numerous and cumbersome, resulting in high operational costs for users. In other related technologies, users can input media content into a Large Language Model (LLM), which then summarizes the media content to generate ingredient lists or itinerary plans. However, cooking step lists or itinerary plans generated in this way only include images and text; users cannot interact with them, such as checking completed cooking steps or setting the status of expired itineraries. In addition, summaries generated in this way are stored on the user's device or note-taking app and cannot be easily published to the original app of the media content so that the user can interact or discuss with other users.

[0027] To address this, the examples in this paper provide a solution for displaying components. In this solution, a terminal device can display first media content and a first component on a first interface. In response to an operation on the first component, the terminal device can display a second component, wherein the second component includes the first interactive content and is generated based on the first media content. In this way, media information originally presented in audio and video format can be displayed in a structured and operable manner, facilitating users' quick understanding and use of key information in the media content, reducing the operational costs for users to organize media content themselves, and improving the user experience.

[0028] Figure 1 A schematic diagram of an example environment 100 in which several examples of this article can be implemented is shown. For example... Figure 1 As shown, environment 100 includes terminal device 102. Terminal device 102 can be any device with processing capabilities. For example, terminal device 102 can include smartphones, tablets, laptops, desktop computers, workstations, or smart wearable devices. In environment 100, terminal device 102 can be a client device, and terminal device 102 can run application 104. Users can interact with application 104 through terminal device 102. For example, application 104 can be a content publishing platform, content creation platform, content editing platform, social platform, video platform, music platform, etc.

[0029] In environment 100, terminal device 102 can display media content 108 and a first component 110 (also referred to herein as a prompt component) on a first interface 106 in application 104. For example, the first interface 106 can be a media content playback interface, which can be configured to play media content such as video or audio. Media content 108 may be related to content such as cooking tutorials, travel guides, makeup tutorials, event planning, or equipment repair. In some cases, terminal device 102 can display a first prompt message associated with the first component 110, which is related to media content 108. For example, the first prompt message may prompt the user to trigger the first component 110 to display content associated with media content 108. For example, when media content 108 is a cooking tutorial video about boiled fish, the prompt message "How to make boiled fish" can be displayed on the first component 110 to prompt the user to trigger the first component 110. In this way, the user can be informed about the media content or applicable scenarios corresponding to the first component 110, enabling the user to more accurately understand the function of the first component 110 and its relationship with the media content 108.

[0030] In environment 100, in response to an operation on the first component 110, terminal device 102 can display a second component 112 (also referred to herein as the first interactive component). The second component 112 is an interactive component and may include interactive content 114. In environment 100, the second component 112 is generated based on media content 108. Herein, interactive content 114 refers to content containing user-operable content elements and capable of displaying information, updating status, or generating corresponding results based on user actions. For example, in response to an operation on the first component 110, terminal device 102 can send a request to a server to obtain data for the second component 112, the request including an identifier of media content 108. The server can obtain media content 108 based on its identifier and generate data for the second component 112 based on media content 108. The server can then send the data for the second component 112 to terminal device 102. Terminal device 102 can receive data from the server of the second component 112 and render the second component 112 on the first interface 106 or a new interface based on the data of the second component 112.

[0031] For example, when media content 108 is a cooking tutorial video, interactive content 114 may include a list of cooking steps, which may include multiple interactive cooking steps. Users can check the completed cooking steps or view the ingredient quantities and detailed instructions for a specific step through interactive operations. As another example, when media content 108 is a travel guide video, interactive content 114 may include a travel planner, which may include multiple interactive travel parameters (e.g., number of travelers, budget, or dates). Users can modify these travel parameters to update the itinerary displayed in the second component 112. As yet another example, when media content 108 is a fitness experience sharing video, interactive content 114 may include a list of fitness exercises, which may include multiple training exercises and the corresponding number of repetitions for each exercise. Users can check the completed training exercises or modify the number of repetitions for a specific exercise through interactive operations.

[0032] In this way, media information that was originally presented in audio and video format can be displayed in a structured and operable manner, making it easier for users to quickly understand and use the key information in media content 108, reducing the operational cost for users to organize media content 108 themselves, and improving the user experience.

[0033] Figure 2 A flowchart of a method 200 for displaying components, according to some examples in this document, is shown. Method 200 can be executed by a terminal device. For example, method 200 can be performed by... Figure 1 The terminal device 102 performs the operation. The terminal device may include, for example, a smartphone, tablet computer, laptop computer, desktop computer, workstation, or smart wearable device.

[0034] like Figure 2 As shown in box 202, the terminal device can display first media content and a first component on a first interface. For example, in... Figure 1 In the illustrated environment 100, terminal device 102 can display media content 108 and first component 110 on a first interface 106 in application 104. For example, the first interface 106 can be a media content playback interface, which can be configured to play media content such as video or audio. Media content 108 may be related to content such as cooking tutorials, travel guides, makeup tutorials, event planning, or equipment repair. In some cases, a prompt message associated with media content 108 can be displayed on the first component 110, prompting the user to trigger the first component 110 to display content associated with media content 108. For example, when media content 108 is a cooking tutorial video about boiled fish, the prompt message "How to make boiled fish" can be displayed on the first component 110 to prompt the user to trigger the first component 110.

[0035] In box 204, in response to an operation on the first component, the terminal device can display a second component, the second component including the first interactive content, and the second component being generated based on the first media content. For example, in... Figure 1 In the illustrated environment 100, in response to an operation on the first component 110, the terminal device 102 can display a second component 112. The second component 112 is an interactive component and may include interactive content 114. In environment 100, the second component 112 is generated based on media content 108. Herein, interactive content 114 refers to content containing user-operable content elements and capable of displaying information, updating status, or generating corresponding results based on user actions. For example, in response to an operation on the first component 110, the terminal device 102 can send a request to a server to obtain data for the second component 112, the request including an identifier of the media content 108. The server can obtain the media content 108 based on its identifier and generate data for the second component 112 based on the media content 108. The server can then send the data for the second component 112 to the terminal device 102. Terminal device 102 can receive data from the server of the second component 112 and render the second component 112 on the first interface 106 or a new interface based on the data of the second component 112.

[0036] In this way, media information that was originally presented in audio and video format can be displayed in a structured and operable manner, making it easier for users to quickly understand and use the key information in the media content, reducing the operational cost for users to organize media content themselves, and improving the user experience.

[0037] Figure 3 A schematic diagram of an example system 300 for generating interactive components, based on some examples in this paper, is shown. Figure 3 As shown, system 300 includes an artificial intelligence (AI) generation engine 302, a preview and editing module 304, a work packaging module 306, a distribution module 308, a derivative creation module 310, and a traffic feedback module 312. The AI ​​generation engine 302 is configured to be based on media content (e.g., Figure 1 Media content 108 in the middle) to generate interactive components (e.g., Figure 1The second component 112 in the text. In some examples, the AI ​​generation engine 302 can acquire content information of the media content, such as speech-to-text, title information, tag information, creator information, image recognition results, comment information, or user input information. The AI ​​generation engine 302 can determine the theme, key steps, entity information, relationships between entities, time nodes, or parameter information of the media content based on its content information. Furthermore, the AI ​​generation engine 302 can determine the type of interactive component based on the content information of the media content and generate structured component data corresponding to that type. In some cases, the type of interactive component may include a checklist, planner, comparison table, or timeline. This structured component data may include multiple content nodes, attribute information corresponding to each content node, relationships between nodes, editable parameters, interaction configurations, or time information corresponding to the media content.

[0038] The preview editing module 304 is configured to preview, edit, and confirm interactive components generated by the AI ​​generation engine 302. For example, the preview editing module 304 can call the corresponding renderer based on structured component data to generate a preview interface for displaying the interactive component. Users can view and edit the content nodes, text information, parameter information, interaction methods, or component types in the component through the preview interface. The preview editing module 304 can also send user editing instructions, parameter adjustment information, or natural language modification instructions for the interactive component to the AI ​​generation engine 302, so that the AI ​​generation engine 302 updates the structured component data based on the user's instructions and returns the updated component content to the preview editing module 304. Thus, users can adjust and confirm the AI-generated component before it is published.

[0039] The work encapsulation module 306 is configured to encapsulate interactive components into publishable content works. For example, the work encapsulation module 306 can associate and store structured component data, component type information, component rendering configuration, information associated with media content, component creator information, version information, permission information, or link information used to navigate to the component's interactive interface, to generate data containing the interactive component (e.g., in the form of a comment). Information associated with media content may include, for example, the media content's identifier, the media content's creator identifier, the time position within the media content, or the correspondence between nodes in the component and segments of the media content.

[0040] The distribution module 308 is configured to distribute the interactive work generated by the work encapsulation module 306 to at least one content display scenario. In some examples, the distribution module 308 can publish the interactive work as comment content to the comment area of ​​media content, and display the work in the comment area as a link, card, or other rich media object. In response to the user's operation on the link, card, or rich media object, the terminal device can display interactive components corresponding to the interactive work, allowing the user to perform checkmarks, expansions, filters, parameter modifications, content viewing, or other interactive operations within the application.

[0041] The traffic feedback module 312 is configured to collect and process data from the generated interactive works during the distribution process. For example, the traffic feedback module 312 can acquire interaction data (e.g., comments, reposts, likes, or views) of the work and its subsequent derivative works, and determine the relationship between the interactive works and the media content. The traffic feedback module 312 can generate feedback information based on this interaction data, which may include, for example, the dissemination effect of the interactive works, the activity level of derivative creations, recommendation weight adjustment information, or creator incentive information. This feedback information can be used to adjust the recommendation strategies for media content, interactive works, or derivative works.

[0042] The derivative creation module 310 is configured to generate new derivative interactive works based on published interactive works. For example, in response to a second user triggering a derivative creation operation on a published first interactive work, the derivative creation module 310 can obtain the structured component data of the first interactive work, the semantic information of the media content associated with the first interactive work, and the derivative creation input provided by the second user. This derivative creation input can include natural language instructions or parameter values ​​corresponding to the editable parameters of the interactive component. The derivative creation module 310 can generate new structured component data based on the structured component data of the first interactive work, the semantic information of the media content, and the derivative creation input. For example, the derivative creation module 310 can generate new interactive components that correspond to the first interactive work but have different content, based on natural language instructions such as "generate a version for one person," "change to a version suitable for young children," or "add accessibility information." Furthermore, the derivative creation module 310 can also adapt the first interactive work based on user-input parameter values ​​(e.g., budget, number of people, date, or location).

[0043] In some examples, the derivative creation module 310 can also record the version inheritance relationship between the first interactive work and the derivative interactive work (e.g., the second interactive work). For example, when generating a derivative interactive work, the derivative creation module 310 can record the parent work identifier, parent version identifier, or derivative creation instruction for the derivative interactive work, so that a version inheritance chain is formed between multiple interactive works. The derivative interactive work generated by the derivative creation module 310 can be sent to the work packaging module 306 for packaging, and published and distributed through the distribution module 308. The traffic feedback module 312 can also send feedback information related to the derivative interactive work to the derivative creation module 310 so as to adjust the derivative creation function or content generation strategy.

[0044] Figure 4 A schematic diagram of an example process 400 for generating interactive components, based on some examples in this document, is shown. Example process 400 can be executed by a server. For example... Figure 4 As shown in box 402, the server can construct a structured semantic representation of the media content. For example, the server can obtain the identifier of the media content. Based on this identifier, the server can obtain at least one type of content information associated with the media content. This content information may include the speech-to-text of the media content, title, tags, creator information, image recognition results, subtitle information, comment information, or user input information. Based on this content information, the server can construct a structured semantic representation of the media content.

[0045] In some examples, this structured semantic representation may include topic categories, multiple content nodes, node attributes corresponding to each content node, relationships between content nodes, time positions corresponding to content nodes, or one or more editable parameters. Content nodes may correspond to, for example, operational steps, characters, locations, events, options, or precautions in media content. For example, in a cooking tutorial video, this structured semantic representation may include ingredient nodes, cooking step nodes, seasoning parameter nodes, and their corresponding time positions. For a travel guide video, this structured semantic representation may include destination nodes, transportation nodes, accommodation nodes, activity nodes, or budget nodes.

[0046] In box 404, the server can use a large language model to generate structured data for the first interactive component based on the structured semantic representation. For example, the server can determine the component type suitable for the media content based on the structured semantic representation. The server can then input the structured semantic representation into the large language model to generate structured data corresponding to that component type. In some cases, the component type may include at least one of the following: a checklist, a planner, a comparison table, or a timeline. A checklist is configured to display multiple items to be completed in a preset order, allowing users to view, check, or mark items as completed. A planner is configured to generate an execution plan based on user-input parameters (e.g., time, budget, or number of people). A comparison table is configured to display information about multiple objects across multiple dimensions in tabular form, allowing users to compare, filter, or sort them. A timeline is configured to display multiple event nodes in chronological order, allowing users to view details corresponding to each time node. In this way, appropriate interactive forms can be adopted for different media content and information organization needs, improving the applicability of the component generation mechanism to various media content such as food and travel guides.

[0047] In some examples, the structured data of the first interactive component can be represented using JavaScript Object Schema (JSON Schema) or other data formats. Structured data can include a list of nodes, the content displayed by each node, node attributes, relationships between nodes, editable fields, interaction configurations, or rendering configurations. Interaction configurations can, for example, indicate whether nodes support clicking, expanding, checking, filtering, sorting, parameter modification, or navigating to the corresponding media content segment.

[0048] In some cases, in response to an operation on the first component, the terminal device can display a second prompt indicating that the generation process of the second component includes summarizing media content and constructing the component's interactive interface. In this way, the user understands that the second component is obtained by summarizing media content and constructing an interactive interface, thereby improving the understandability of the component generation process and the transparency of the interaction process.

[0049] In box 406, the server can display the first interactive component based on its structured data. For example, the server can send structured data to the terminal device, causing the terminal device to invoke the renderer corresponding to the component type, thereby generating the interactive interface of the first interactive component. The first interactive component can be displayed along with media content in the media content playback interface, media content details interface, component editing interface, or other interfaces. Users can use the first interactive component to select steps, view node details, adjust parameters, expand sub-content, filter content, or perform other interactive operations.

[0050] In some examples, when the first interactive component includes multiple nodes, in response to an operation on the first node, the terminal device can display an interaction result associated with that first node. This allows users to further view and explore specific information nodes, thereby improving the hierarchical display of structured content and the efficiency of information retrieval. The interaction result may include node details, associated nodes, sub-components, supplementary explanations, modifiable parameters, or corresponding segments from media content.

[0051] In some examples, the terminal device can display a third component (also referred to herein as a time component) corresponding to a second node among multiple nodes. This third component is associated with a first time point on the timeline, which in turn is associated with the second node. In response to an operation on the third component, the terminal device can begin playing the first media content from the first time point of that first media content. For example, the terminal device can control the media content to begin playing from the time position corresponding to that node, based on the time information associated with that node. This establishes a locational relationship between interactive components and segments of media content, facilitating users to review or supplement their understanding of the original content corresponding to the nodes and improving the user experience.

[0052] In optional box 408, the server can publish a first comment on the media content, which is associated with a first interactive component. In some cases, the terminal device can display a fourth component (also referred to herein as a comment component) associated with a second component. In response to an operation on the fourth component, the terminal device can publish a first comment on the first media content, which includes first content associated with the second component. This allows interactive components to be natively distributed within the comment context of media content, enhancing the information capacity of the comment area and facilitating discovery and use by other users. For example, the server can encapsulate the component identifier, access link, thumbnail information, or component card information corresponding to the first interactive component into the first comment and publish it to the comment area of ​​the media content. The first comment can be displayed as a hyperlink, rich media card, or other comment format that triggers access to the first interactive component. In response to an operation on the first comment, the terminal device can display the first interactive component, allowing other users to access it within the application.

[0053] In optional box 410, the server can generate a second interactive component based on the first interactive component. In some cases, the terminal device can display a first comment in the comment area of ​​the first media content. In response to an operation on the first comment, the terminal device can display a second component and a fifth component (also referred to herein as a follow-up question component). In response to an operation on the fifth component, the terminal device can display a sixth component (also referred to herein as a second interactive component), which includes the second interactive content and is generated based on the second component. In this way, an interactive link from component access in the comment to the generation of derivative components can be realized, enabling the components to be continuously expanded in the social context of the first media content. For example, in response to a derivative creation operation on the first interactive component, the server can obtain the structured data of the first interactive component, the structured semantic representation of the media content, and the derivative creation information input by the user. The server can generate the second interactive component based on this structured data, the structured semantic representation, and the derivative creation information.

[0054] In some cases, in response to an operation on the fifth component, the terminal device can obtain a natural language instruction. In response to a confirmation operation on the natural language instruction, the terminal device can display a sixth component, which is generated based on the second component and the natural language instruction. In this way, users can express personalized needs in natural language, thereby obtaining interactive components that meet specific needs with lower operational costs. For example, derivative creative information can include natural language instructions. For instance, a user can input instructions such as "generate a version for one person," "change it to a version suitable for young children," or "add accessibility information." The server can input the natural language instruction, the structured data of the first interactive component, and the structured semantic representation of the media content into a large language model to generate structured data for the second interactive component.

[0055] In some cases, in response to an operation on the fifth component, the terminal device can obtain a first parameter value, which is associated with a first parameter of the second component. The terminal device can then display a sixth component, generated based on the second component and the first parameter value. This approach allows users to controllably modify interactive components by adjusting parameters, improving the efficiency, accuracy, and predictability of derivative creations. For example, derivative creation information can include parameter values ​​corresponding to editable fields in the first interactive component. For instance, users can modify parameters such as the number of people, budget, date, location, duration, or difficulty level. The server can adjust the corresponding content nodes, node attributes, node relationships, or interaction configurations in the first interactive component based on the user-input parameter values ​​to generate a second interactive component. When generating the second interactive component, the server can record the version inheritance relationship between the first and second interactive components. For example, the server can record the parent component identifier, parent version identifier, derivative creation user identifier, or derivative creation instruction for the second interactive component, so that the first and second interactive components form a version inheritance chain.

[0056] In some cases, the second component is generated by the first user, and the sixth component is generated by the second user. In response to the generation of the sixth component, the terminal device can display a first message on the second interface associated with the first user, indicating that the second component has been used by other users. This allows the first user to be promptly informed that the second component they generated has been used by other users, providing visual feedback for the component's dissemination and helping to enhance the sustainability of component creation.

[0057] In optional box 412, the server can publish a second comment on the media content, which is associated with a second interactive component. In some cases, the terminal device can display a fourth component in association with a sixth component. In response to an operation on the fourth component, the terminal device can publish a second comment on the first media content, which includes second content associated with the sixth component. In this way, derivative components can continue to spread in the comment area of ​​the first media content, facilitating the continuous expansion of component versions. For example, in response to a user's publishing operation on the second interactive component, the terminal device can publish a comment including an access link to the second interactive component to the comment area of ​​the media content. Thus, other users can access the second interactive component based on the second comment and further interact or create derivative works based on the second interactive component.

[0058] Figures 5A to 5D A schematic diagram of example 500 for generating a user interface with interactive components, based on some examples in this document, is shown. Figure 5AAs shown, Example 500 includes a media content playback interface 502, which is configured to play media content 504, such as video or audio. When the media content 504 is audio, information associated with the audio can be displayed on the media playback interface 502, such as the cover art, author information, or subtitles. The media content playback interface 502 can also display author information 506 for the media content 504, which may be, for example, the name or image of the author of the media content 504. In Example 500, a prompt component 508 can be displayed on the media content playback interface 502. The prompt component 508 can display prompt information associated with the media content 504, prompting the user to trigger the prompt component 508 to display the information associated with the media content 504. For example, when media content 504 is a cooking tutorial video about boiled fish posted by user A, the prompt component 508 can display the prompt message "How to make boiled fish" to prompt the user to obtain information related to the method of making boiled fish in this video by clicking the prompt component 508.

[0059] In some cases, when user A publishes media content 504, they can choose whether to allow AI to access the information in media content 504. When user A allows AI to access the information in media content 504, a prompt component 508 can be displayed on the media playback interface 502. When user A disallows AI to access the information in media content 504, the server cannot obtain the information in media content 504, and therefore cannot generate an interactive component, thus the prompt component 508 is not displayed on the media playback interface 502.

[0060] like Figure 5B As shown in Example 500, in response to an operation on Prompt component 508, the terminal device can display Dialogue component 510 on media content playback interface 502. Dialogue component 510 is configured to engage in dialogue with an AI object (e.g., a large language model), and this dialogue is associated with media content 504. In other words, in the dialogue corresponding to dialogue component 510, the context of the dialogue includes information from media content 504.

[0061] In Example 500, in response to an operation on Prompt Component 508, the terminal device can also acquire a prompt word 512 corresponding to Prompt Component 508. Prompt word 512 is associated with media content 504, and prompt word 512 can prompt the AI ​​object to generate content associated with media content 504. Prompt word 512 can be associated with prompt information on Prompt Component 508. For example, prompt word 512 could be "Compile the recipe for boiled fish from the video into a checklist." In some cases, in response to an operation on Prompt Component 508, the terminal device can display prompt word 512 in Dialogue Component 510 and send prompt word 512 to the AI ​​object without user interaction. In some cases, in response to an operation on Prompt Component 508, the terminal device can display prompt word 512 in Dialogue Component 510, and the user can modify prompt word 512. In response to a sending operation for the modified prompt word, the terminal device can send the modified prompt word to the AI ​​object.

[0062] In Example 500, in response to the sending of prompt 512, responses 514 and 516 can be displayed on dialog component 510. Response 514 may include a natural language response to prompt 512. Response 516 may include information associated with the interactive component being generated (e.g., the name, type, or functional description of the interactive component), information associated with media content 504 (e.g., the cover image of media content 504), and actions performed during the generation of the interactive component (e.g., summarizing video information).

[0063] like Figure 5C As shown, during the execution of the interactive component generation task, progress information 518 of the generation task can also be displayed on the dialogue component 510. In example 500, progress information 518 can indicate that the interactive component generation task includes the following steps: summarizing video information, constructing the interactive interface of the interactive component and injecting visual style, and refining the interface structure and deploying the interactive component. Furthermore, information 520 can also be displayed on the dialogue component 510, which can indicate the thought process of the large language model during the generation of the interactive component.

[0064] like Figure 5D As shown, when the task of generating the interactive component is completed, the generated interactive component 522 can be displayed on the dialogue component 510 (e.g., Figure 1The second component 112 in Example 500. In Example 500, the interactive component 522 includes a checklist (e.g., a checklist for cooking boiled fish) generated based on the media content 504. The checklist includes multiple steps (e.g., step 1, step 2, step 3, etc.) and at least one sub-step corresponding to each step (e.g., step 1-1, step 1-2, and step 2-1, etc.). In Example 500, the sub-steps are implemented as check items in the checklist. In response to an operation on a sub-step (e.g., checking or dechecking), the state of the sub-step (e.g., completed or incomplete) can be changed. Furthermore, in Example 500, multiple time components corresponding to the multiple steps can be displayed on the interactive component 522, wherein the time components are configured to start playing the media content 504 from the corresponding time. For example, time component 526 corresponds to step 1, and "02:24" is displayed on time component 526. In response to an operation on the time component 526, the terminal device can display media content 504 on the media content playback interface 502, and start playback from 02:24 on the media content 504. Furthermore, a full-screen component 524 can also be displayed on the interactive component 522. In response to an operation on the full-screen component 524, the content of the interactive component 522 can be displayed in full-screen mode on a new interface.

[0065] In Example 500, a question component 528 may also be displayed on the dialogue component 510. Users can use the question component 528 to input natural language commands for the interactive component 522. In response to the sending of the natural language command, the terminal device can display a modified interactive component on the dialogue component 510, which is generated based on the interactive component 522 and the natural language command. In Example 500, a comment component 530 may also be displayed on the dialogue component 510. In response to an operation on the comment component 530, the terminal device can post a comment on the media content 504, which is associated with the interactive component 522. After the comment is posted, it can be displayed in the comment area of ​​the media content 504. Other users can interact with the comment to view the interactive component 522.

[0066] Figures 6A to 6D A schematic diagram of example 600, illustrating how a user interface for generating a second interactive component is based on a first interactive component, is shown according to some examples in this paper. (See example 600.) Figure 6A As shown, Example 600 includes a media content playback interface 602. The media content playback interface 602 is configured to play media content 604. A comment area 606 for the media content 604 can be displayed on the media content playback interface 602. Comments 608 can be displayed in the comment area 606, where comment 608 is associated with a generated interactive component. For example, comment 608 could be in response to comments such as... Figure 5D The comment 608 is a comment posted in connection with the interactive component 522, generated by the operation of the comment component 530. The comment 608 may include the poster's identifier (e.g., a user's name or avatar), content associated with the interactive component (e.g., a prompt used to generate the interactive component), and a link 610 associated with the interactive component.

[0067] In response to an operation on link 610, the terminal device can display as follows: Figure 6B The component viewing interface 612 is shown. Interactive components 622 generated based on media content 604 (e.g., such as...) can be displayed on the component viewing interface 612. Figure 5D The interactive component 522 shown is a checklist for making boiled fish. Furthermore, a prompt message 616 can be displayed on the component viewing interface 612. The prompt message 616 instructs the user who created the interactive component 622 (e.g., user B) and the user who published the media content 604 (e.g., user A). This increases the exposure of users A and B, which is beneficial for increasing interaction between other users and users A and B.

[0068] In addition, prompts 618 for generating interactive components 622 can also be displayed on the component viewing interface 612 (e.g., such as...). Figure 5B (See prompt 512). In this way, the current user can clearly perceive the theme of the interactive component 622. Furthermore, a prompt 620 can be displayed on the component viewing interface 612, indicating that the interactive component 622 was automatically generated based on the media content 604. This encourages the user to further utilize the component generator to create derived interactive components. In addition, a follow-up question component 624 can be displayed on the component viewing interface 612.

[0069] In response to an operation on the follow-up component 624, the terminal device can display as follows: Figure 6C The text input component 626 is shown. Users can use the text input component 626 to input natural language instructions 628 (e.g., "Eating alone, I don't have ingredient A at home, so I'll use ingredient B instead") to create derivative works based on media content 604 and interactive component 622. Natural language instructions 628 can include modification instructions for interactive component 622 to prompt the large language model to generate derivative interactive components based on interactive component 622.

[0070] In response to an operation on the sending component 630, the terminal device can obtain a derived interactive component, which is generated based on the interactive component 622, media content 604, and natural language instructions 628. For example... Figure 6DAs shown, in response to an operation on sending component 630, the terminal device can display a dialog interface 632. On the dialog interface 632, a prompt 634 corresponding to the natural language instruction 628, a response 636 in natural language form to the prompt 634, and a derived interactive component 638 can be displayed. On the derived interactive component 638, a modified checklist can be displayed; for example, in step 3-1, the original material A is replaced with material B, and the steps associated with material A in the checklist can also be replaced with steps associated with material B. Furthermore, a question component 640 can be displayed on the dialog interface 632. The user can use the question component 640 to continue sending prompts to the AI ​​object to obtain more information in the context of media content 604 or to continue modifying the derived interactive component 638. In example 600, when the derived interactive component 638 is generated, the terminal device can display a notification message on the interface associated with the author of the interactive component 622 (e.g., user B), indicating that the interactive component 622 has been invoked for derivative creation. This approach can increase user B's sense of accomplishment and motivate user B to continue creating content.

[0071] Figure 7 A block diagram of a device 700 for displaying components, according to some examples herein, is shown. Figure 7 As shown, device 700 includes a first component display module 702, configured to display first media content and a first component on a first interface. Device 700 also includes a second component display module 704, configured to display a second component in response to an operation on the first component. The second component includes the first interactive content and is generated based on the first media content. In this way, media information originally presented in audio and video format can be displayed in a structured and operable manner, facilitating users' quick understanding and use of key information in the media content, reducing the operational cost for users to organize media content themselves, and improving the user experience.

[0072] In some cases, device 700 further includes a first prompt information display module, configured to display first prompt information in association with the first component, the first prompt information being associated with first media content. In this way, the user can be informed of the media content or applicable scenario corresponding to the first component, enabling the user to more accurately understand the function of the first component and its association with the media content.

[0073] In some cases, device 700 further includes a second prompt information display module, configured to display a second prompt information in response to an operation on the first component. The second prompt information indicates that the generation process of the second component includes summarizing media content and constructing the component's interactive interface. In this way, the user can understand that the second component is obtained by summarizing media content and constructing an interactive interface, thereby improving the understandability of the component generation process and the transparency of the interaction process.

[0074] In some cases, the first interactive content includes multiple nodes, and the device 700 further includes an interaction result display module configured to display the interaction result associated with the first node in response to an operation on the first node among the multiple nodes. In this way, users can be supported in further viewing and exploring around specific information nodes, thereby improving the hierarchical display capability of structured content and the efficiency of information retrieval. The interaction result may include node details, associated nodes, sub-components, supplementary explanations, modifiable parameters, or corresponding segments from media content.

[0075] In some cases, device 700 further includes: a first node usage module configured to display a third component corresponding to a second node among a plurality of nodes, the third component being associated with a first time point on a timeline, the first time point being associated with the second node; and a media content playback module configured to, in response to an operation on the third component, begin playing the first media content from the first time point of the first media content. In this way, a positioning relationship can be established between interactive components and segments of media content, facilitating users to trace back or supplement their understanding of the original content corresponding to the nodes, thus improving the user experience.

[0076] In some cases, the second component includes at least one of the following: a checklist, a planner, a comparison table, or a timeline. This approach allows for the adoption of appropriate interactive formats to meet the diverse needs of different media content and information organization, improving the applicability of the component generation mechanism to various media content types, such as food and travel guides.

[0077] In some cases, device 700 further includes: a fourth component display module configured to display the fourth component in association with the second component; and a first comment publishing module configured to publish a first comment on the first media content in response to an operation on the fourth component, the first comment including the first content associated with the second component. In this way, interactive components can be natively disseminated within the comment context of media content, enhancing the information capacity of the comment area and facilitating discovery and use of the component by other users.

[0078] In some cases, device 700 further includes: a first comment display module configured to display a first comment in the comment area of ​​the first media content; a fifth component display module configured to display a second component and a fifth component in response to an operation on the first comment; and a sixth component display module configured to display a sixth component in response to an operation on the fifth component, the sixth component including second interactive content, the sixth component being generated based on the second component. In this way, an interactive link from component access in the comment to the generation of derived components can be realized, enabling components to be continuously expanded within the social context of the first media content.

[0079] In some cases, the sixth component display module includes: an instruction acquisition module configured to acquire natural language instructions in response to an operation on the fifth component; and an instruction usage module configured to display the sixth component, which is generated based on the second component and the natural language instructions. In this way, users can express personalized needs in natural language, thereby obtaining interactive components that meet specific requirements with lower operational costs.

[0080] In some cases, the sixth component display module includes: a first parameter value acquisition module, configured to acquire a first parameter value in response to an operation on the fifth component, the first parameter value being associated with a first parameter of the second component; and a first parameter value usage module, configured to display the sixth component, the sixth component being generated based on the second component and the first parameter value. This approach allows users to controllably modify interactive components by adjusting parameters, improving the efficiency, accuracy, and predictability of derivative creations based on components.

[0081] In some cases, the second component is generated by the first user, and the sixth component is generated by the second user. The device 700 also includes a first message display module configured to display a first message on a second interface associated with the first user in response to the generation of the sixth component. The first message indicates that the second component has been used by other users. In this way, the first user can be promptly informed that the second component they generated has been used by other users, providing visual feedback for the dissemination of the component and helping to enhance the sustainability of component creation.

[0082] In some cases, device 700 further includes: a sixth component usage module configured to display the fourth component in association with the sixth component; and a second comment publishing module configured to publish a second comment on the first media content in response to an operation on the fourth component, the second comment including second content associated with the sixth component. In this way, derivative components can continue to spread in the comment area of ​​the first media content, facilitating the continuous expansion of component versions.

[0083] Figure 8A block diagram of a device 800 capable of implementing several examples of this document is shown. Device 800 may be, for example, as follows: Figure 1 The terminal device 102 shown is an example. Figure 8 As shown, device 800 includes a central processing unit (CPU) and / or a graphics processing unit (GPU) 801, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 802 or loaded from storage unit 808 into random access memory (RAM) 803. The RAM 803 can also store various programs and data required for the operation of device 800. The CPU / GPU 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804. Although not shown in... Figure 8 As shown, device 800 may also include a coprocessor.

[0084] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0085] The various methods or processes described above can be executed by CPU / GPU 801. For example, in some cases, the methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 808. In some cases, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by CPU / GPU 801, one or more steps or actions in the methods or processes described above can be performed.

[0086] In some cases, the methods and processes described above can be implemented as a computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing the various aspects of this document.

[0087] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0088] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network (LAN), a wide area network (WAN), and / or a wireless network, to an external computer or external storage device. The network may include copper cables, fiber optic cables, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0089] The computer program instructions used to perform the operations described herein can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​and conventional procedural programming languages. The computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider). In some cases, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can be customized using the status information from the computer-readable program instructions to implement various aspects of this document.

[0090] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0091] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0092] The flowcharts and block diagrams in the accompanying figures illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to several examples herein. In this regard, each box in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the boxes may occur in a different order than those marked in the figures. For example, two consecutive boxes may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each box in the block diagrams and / or flowcharts, and combinations of boxes in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0093] The examples described above are exemplary and not exhaustive, nor are they limited to the disclosed examples. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described examples. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to technology in the market, or to enable others skilled in the art to understand the examples disclosed herein.

Claims

1. A method for displaying a component, comprising: The first interface displays the first media content and the first component; as well as In response to an operation on the first component, a second component is displayed, the second component including first interactive content, and the second component is generated based on the first media content.

2. The method according to claim 1, further comprising: A first prompt message is displayed in association with the first component, and the first prompt message is associated with the first media content.

3. The method according to claim 1, further comprising: In response to an operation on the first component, a second prompt message is displayed, indicating that the generation process of the second component includes summarizing media content and constructing the component's interactive interface.

4. The method according to claim 1, wherein the first interactive content comprises multiple nodes, and the method further comprises: In response to an operation performed on the first of the plurality of nodes, the interaction result associated with the first node is displayed.

5. The method according to claim 4, further comprising: Displays a third component corresponding to the second node among the plurality of nodes, the third component being associated with a first time on the timeline, the first time being associated with the second node; as well as In response to an operation on the third component, playback of the first media content begins from the first time point of the first media content.

6. The method of claim 1, wherein the second component comprises at least one of: a checklist, a planner, a comparison table, or a timeline.

7. The method according to claim 1, further comprising: The fourth component is displayed in association with the second component; as well as In response to an operation on the fourth component, a first comment is published on the first media content, the first comment including first content associated with the second component.

8. The method according to claim 7, further comprising: The first comment is displayed in the comment area of ​​the first media content; In response to an action on the first comment, the second and fifth components are displayed; as well as In response to an operation on the fifth component, a sixth component is displayed, the sixth component including second interactive content, the sixth component being generated based on the second component.

9. The method of claim 8, wherein displaying the sixth component in response to an operation on the fifth component comprises: In response to an operation on the fifth component, natural language instructions are obtained; as well as The sixth component is displayed, which is generated based on the second component and the natural language instructions.

10. The method of claim 8, wherein displaying the sixth component in response to an operation on the fifth component comprises: In response to an operation on the fifth component, a first parameter value is obtained, the first parameter value being associated with a first parameter of the second component; as well as The sixth component is displayed, which is generated based on the second component and the first parameter value.

11. The method of claim 8, wherein the second component is generated by the first user, the sixth component is generated by the second user, and the method further comprises: In response to the generation of the sixth component, a first message is displayed on a second interface associated with the first user, indicating that the second component is being used by other users.

12. The method of claim 8, further comprising: The fourth component is displayed in association with the sixth component; as well as In response to an operation on the fourth component, a second comment is published on the first media content, the second comment including second content associated with the sixth component.

13. An apparatus for displaying a component, comprising: The first component display module is configured to display the first media content and the first component on the first interface; as well as The second component display module is configured to display a second component in response to an operation on the first component. The second component includes first interactive content and is generated based on the first media content.

14. An electronic device comprising: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, which, when executed by the processor, cause the electronic device to perform the method according to any one of claims 1 to 13.

15. A computer program product tangibly stored on a non-transitory computer-readable medium and comprising machine-executable instructions that, when executed, cause a machine to perform the method according to any one of claims 1 to 13.

Images