Interface interaction method and device, equipment and storage medium

Abstract

The embodiment of the invention relates to an interface interaction method and device, equipment and a storage medium. A method presented herein includes presenting an interactive window associated with published reference media content, the interactive window associated with a virtual object; receiving a user message via the interactive window; and providing response content generated by the virtual object based on the user message in the interaction window, wherein the response content is generated at least based on reference information of the reference media content. In this way, according to the embodiment of the invention, the interface interaction efficiency can be improved, and the efficiency of obtaining the information related to the reference media content by the user can be improved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and particularly to methods, apparatuses, devices, and computer-readable storage media for interface interaction. Background Art

[0002] With the development of computer technology, more and more platforms or applications provide users with intelligent dialogue services, such as intelligent dialogue systems, digital assistants, and so on. Users can ask some questions to these virtual objects such as intelligent dialogue systems and digital assistants, and thus the virtual objects can reply to these questions of the users to achieve dialogue interaction with the users. Summary of the Invention

[0003] In a first aspect of the present disclosure, there is provided a method for interface interaction. The method includes: presenting an interaction window associated with published reference media content, the interaction window being associated with a virtual object; receiving a user message via the interaction window; and providing, in the interaction window, response content generated by the virtual object based on the user message, the response content being generated at least based on reference information of the reference media content.

[0004] In a second aspect of the present disclosure, there is provided an apparatus for interface interaction. The apparatus includes: a first presentation module configured to present an interaction window associated with published reference media content, the interaction window being associated with a virtual object; a receiving module configured to receive a user message via the interaction window; and a providing module configured to provide, in the interaction window, response content generated by the virtual object based on the user message, the response content being generated at least based on reference information of the reference media content.

[0005] In a third aspect of the present disclosure, there is provided an electronic device. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. The instructions, when executed by the at least one processing unit, cause the device to execute the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, there is provided a computer-readable storage medium. A computer program is stored on the computer-readable storage medium and can be executed by a processor to implement the method of the first aspect.

[0007] It should be understood that the content described in this content part is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Brief Description of the Drawings

[0008] In conjunction with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent. In the drawings, the same or similar reference numerals denote the same or similar elements, where:

[0009] Figure 1 A schematic diagram showing an example environment in which embodiments according to the present disclosure can be implemented;

[0010] Figures 2A through 2K An example interface according to some embodiments of the present disclosure;

[0011] Figures 3A through 3F An example interface according to some other embodiments of the present disclosure;

[0012] Figure 4 A flowchart showing an example process of information processing according to some embodiments of the present disclosure;

[0013] Figure 5 A flowchart showing an example process of interface interaction according to some embodiments of the present disclosure;

[0014] Figure 6 A schematic structural block diagram showing an example device for interface interaction according to some embodiments of the present disclosure; and

[0015] Figure 7 A block diagram of an electronic device capable of implementing multiple embodiments of the present disclosure. Detailed Description of Specific Embodiments

[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Instead, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0017] It should be noted that the titles of any section / subsection provided herein are not restrictive. Various embodiments are described throughout this document, and any type of embodiment can be included under any section / subsection. In addition, the embodiments described in any section / subsection can be combined with any other embodiments described in the same section / subsection and / or different section / subsections in any manner.

[0018] In the description of the embodiments of the present disclosure, the term "including" and its similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "an embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". There may also be other explicit and implicit definitions hereinafter. The terms "first", "second", etc. may refer to different or the same objects. There may also be other explicit and implicit definitions hereinafter.

[0019] In the embodiments of the present disclosure, it may involve the user's data, the acquisition and / or use of data, etc. These aspects all comply with the corresponding laws, regulations and related provisions. In the embodiments of the present disclosure, the collection, acquisition, processing, processing, forwarding, use, etc. of all data are carried out on the premise that the user is aware of and confirms. Correspondingly, when implementing the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the data or information that may be involved should be informed to the user and the user's authorization should be obtained through appropriate means according to the relevant laws and regulations. The specific notification and / or authorization methods may vary according to the actual situation and application scenarios, and the scope of the present disclosure is not limited in this regard.

[0020] In the solutions of this specification and embodiments, if personal information processing is involved, it will be processed on the premise of having a legal basis (such as obtaining the consent of the personal information subject, or being necessary for performing a contract), and will only be processed within the specified or agreed scope. The user's refusal to process personal information other than the necessary information required for the basic functions will not affect the user's use of the basic functions.

[0021] The embodiments of the present disclosure propose a solution for interface interaction. According to this solution, an interaction window associated with the published reference media content can be presented, and the interaction window is associated with a virtual object. Further, user messages can be received via the interaction window. Additionally, response content generated by the virtual object based on the user messages can be provided in the interaction window, and the response content is at least generated based on the reference information of the reference media content.

[0022] Based on such a method, the embodiments of the present disclosure can support the virtual object to provide response content related to the reference media content, and the user can obtain information related to the reference media content without complex interaction or complex problem description. Therefore, the embodiments of the present disclosure can greatly improve the efficiency of interface interaction and the efficiency of the user to obtain information related to the reference media content.

[0023] Therefore, the embodiments of the present disclosure can improve the efficiency of interface interaction and can also improve the efficiency of the user to obtain information related to the reference media content.

[0024] The following further describes various example implementations of the solution in detail with reference to the accompanying drawings.

[0025] Example environment

[0026] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. As Figure 1 shown, the example environment 100 may include an electronic device 110.

[0027] In this example environment 100, the electronic device 110 may run an application 120 that supports interface interaction. The application 120 may be any suitable type of application for interface interaction, examples of which may include, but are not limited to: media applications, social applications, or other suitable applications. The user 140 may interact with the application 120 via the electronic device 110 and / or its attached devices.

[0028] In Figure 1 the environment 100, if the application ¹²⁰ is active, the electronic device ¹¹⁰ may present an interface ¹⁵⁰ for supporting interface interaction through the application ¹²⁰.

[0029] In some embodiments, the electronic device 110 communicates with the server 130 to implement the supply of services for the application 120. The electronic device 110 may be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, palmtop computers, portable game terminals, VR / AR devices, Personal Communication System (PCS) devices, personal navigation devices, Personal Digital Assistant (PDA), audio / video players, digital cameras / cameras, positioning devices, television receivers, radio broadcast receivers, e-book devices, game devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the electronic device 110 can also support any type of interface for the user (such as a "wearable" circuit, etc.).

[0030] It should be noted that there seems to be an error in the original text where "¹²⁰" and "¹¹⁰" and "¹⁵⁰" are used instead of normal numbers or correct notations. This has been retained in the translation as it is part of the original text.The server 130 can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. The server 130 can include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, and so on. The server 130 can provide background services for the application 120 that supports content presentation in the electronic device 110.

[0031] A communication connection can be established between the server 130 and the electronic device 110. The communication connection can be established by wired or wireless means. The communication connection can include, but is not limited to, a Bluetooth connection, a mobile network connection, a Universal Serial Bus (USB) connection, a Wireless Fidelity (WiFi) connection, etc., and the embodiments of the present disclosure are not limited in this regard. In the embodiments of the present disclosure, the server 130 and the electronic device 110 can implement signaling interaction through the communication connection therebetween.

[0032] It should be understood that the structures and functions of the various elements in the environment 100 are described only for exemplary purposes, without implying any limitation on the scope of the present disclosure.

[0033] Some exemplary embodiments of the present disclosure will be described below with continued reference to the drawings.

[0034] Example interactions of some interfaces

[0035] Some example interaction processes according to some embodiments of the present disclosure will be described below with reference to Figures 2A through 2K Some example interaction processes according to some embodiments of the present disclosure will be described below with reference to Figures 2A through 2K Exemplary interfaces 200A to 200K according to some embodiments of the present disclosure are shown, and the interfaces 200A to 200K can be provided, for example, by Figure 1 the electronic device 110 shown.

[0036] In some embodiments, such interfaces 200A to 200K can include a session interface between the user and the virtual object. Such a session interface can be, for example, a one-on-one chat session between the user 140 and the virtual object, or a group chat session with at least the user 140 and the virtual object as participants. In some scenarios, such a dialogue interface can also include a conversational search interface associated with the virtual object, which can search for some content based on the conversation between the user 140 and the virtual object, and then present the search results in the form of a reply message from the virtual object.

[0037] In some other scenarios, such a dialogue interface may, for example, also have other appropriate forms. For example, the dialogue interface can correspond to the interaction space of a virtual object, which can, for example, support the user 140 to have various modal dialogue interactions with the virtual object.

[0038] In some embodiments, the user 140 can view some media content through the media application of the electronic device 110. In some scenarios, the user 140 can input some description content associated with the media content to the electronic device 110 to further understand such media content, and the efficiency of such an interaction method is low.

[0039] To further improve the user interaction efficiency, the embodiments of the present disclosure support the user 140 to send media content (or identify content indicating the media content, etc.) to the virtual object through such a session interface, so as to further understand such media content through the response message provided by the virtual object. As an example, such media content can include video content, atlas content, and so on. As an example, the electronic device 110 can support the user 140 to quickly have a conversation with the virtual object by providing some entrances to obtain information associated with the media content.

[0040] In some embodiments, the electronic device 110 can present an interface 200A as Figure 2A shown. Such an interface 200A can present an interaction window 201 associated with the virtual object. Further, the conversation between the user 140 and the virtual object can be realized in such an interaction window 201.

[0041] Further, such an interface 2000A can provide an entrance 211 (for example, an aggregation entrance) that aggregates multiple interaction entrances. Furthermore, the user 140 can trigger the presentation of multiple interaction entrances by triggering the entrance 211. For example, according to Figure 2B the interface 200B shown, the electronic device 110 can present a set of interaction entrances based on the user's triggering of the entrance 211. Examples thereof include an entrance 221-1 (also called, an album entrance), an entrance 221-2 (also called, a shooting entrance), an entrance 221-3 (also called, a file entrance), an entrance 221-4 (also called, a work entrance), and so on. As an example, such an entrance 221-4 can be a media addition entrance. Thus, the user 140 can trigger the addition of some media content by interacting with the entrance 221-4.

[0042] To further improve the user interaction efficiency, the electronic device 110 can also provide such a media addition entrance in the session interface in the form of a quick entrance. As an example, according to Figure 2CThe interface 200C shown. Such a media addition entry may include an interactive entry 222. Thus, the user 140 can more quickly trigger the addition of media content in the session interface.

[0043] In some embodiments, the electronic device 110 may provide options for media content that can be added to the user 140 based on the user 140's historical interactions with the published media content. As an example, the electronic device 110 may receive a first operation from the user in the session interface to present an interactive window associated with the media content 231-3. Such a first operation may indicate the published reference media content. Thus, the virtual object may refer to such reference media content to the user 140 to reply to the user's message, thereby helping the user to more conveniently understand such reference media content.

[0044] In some embodiments, such a first operation may indicate the process of the user 140 selecting such reference media content. Specifically, if the user 140 selects the entry 221-4 (or selects the interactive entry 222), the electronic device 110 may present a set of candidate media content associated with the user 140. Such a set of candidate media content may be determined based on the user 140's historical interactions with the published media content. As an example, such historical interactions include but are not limited to favorites, sharing, and so on.

[0045] For example, according to Figure 2D the interface 200D shown, such a set of candidate media content may include but is not limited to media content 231-1, media content 231-2, media content 231-3, and so on. As an example, such a set of candidate media content may be presented in the window 232. Such a window 232 may be presented by the electronic device 110 based on the user's selection of the entry 221-4 or the entry 222.

[0046] As an example, the user 140 may select the media content 231-3. The selected media content 231-3 may also be referred to as the reference media content. Thus, the electronic device 110 may present an interactive window associated with such reference media content. Such an interactive window is associated with a virtual object. Thus, the user 140 may further understand the information of such reference media content based on the conversation with the virtual object.

[0047] In some embodiments, if the user selects the media content 231-3, the electronic device 110 may present an input component associated with the media content 231-3. Such an input component may receive the user's input content. As an example, according to Figure 2EIn the interface 200E shown, the electronic device 110 can present the input component 232 in the session interface and receive the first input content 233 through the input component 232. Such first input content 233 can indicate the access address of such media content 231-3. Further, the user 140 can trigger the control 234 to send a message to the virtual object. As an example, the user 140 can also input some text, voice, etc. through such input component 232 as supplementary content to the first input content 233. For example, the user 140 can input some questions through the input component 232. Further, the user 140 can send a message indicating the first input content 233 and this question to the virtual object.

[0048] In some embodiments, according to Figure 2F In the interface 200F shown, the user 140 can send a message 241 (also referred to as the user message) to the virtual object. Such message 241 can indicate such media content 231-3. Further, the electronic device 110 can provide the message 242 provided by the virtual object. Such message 242 can be the response content generated by the virtual object based on such user message, and such response content is at least generated based on the reference information of such media content 231-3. As an example, the specific implementation of the generation of such response content can refer to the information processing process described below and will not be elaborated here.

[0049] In some embodiments, the electronic device 110 can provide a quick control in the playback interface of the media content to support the user to more conveniently understand the media content during the process of watching the media content. As an example, such quick control can be presented at a preset position in such playback interface and can be presented in a preset style.

[0050] For example, according to Figure 2G In the interface 200G shown, such quick control can include the control 252 presented in the playback interface of the media content 251 (which can also be referred to as the reference media content). Such control 252 can be presented adjacent to the music control. Also for example, according to Figure 2H In the interface 200H shown, such quick control can include the control 253 presented in the playback interface of the media content 251. Such control 253 can be presented adjacent to the user identification. Still for example, according to Figure 2I In the interface 200I shown, such quick control can include the control 251 presented in the playback interface of the media content 251. Such control 254 can be presented in the search area associated with the media content 251.

[0051] In some embodiments, the electronic device 110 may, in response to receiving a second operation in such a playback interface, present an interaction window associated with such media content 251 in the playback interface. As an example, such a second operation may include triggering of such shortcut controls presented in the playback interface, etc. As an example, according to Figure 2J the interface 200J shown, such an interaction window may include window 261, and the electronic device 110 may provide information associated with the media content 251 (such as media content summary, character analysis, story overview, etc.) through such an interaction window.

[0052] In some embodiments, the electronic device 110 may present a set of interaction items associated with the media content 251 in such an interaction window, and such a set of interaction items may be determined based on the media content 251. For example, such a set of interaction items may include Figure 2J the interaction items 271-1, interaction item 271-2, interaction item 271-3, interaction item 271-4, etc. shown. Taking the interaction item 271-4 as an example, such an interaction item 271-4 may indicate a prompt content "What is this plant in the picture?" and a media element 272, etc. Such a media element 272 may be derived from the picture content presented by the media content 251 at the current moment. As an example, the electronic device 110 may update such a set of interaction items based on the playback progress of the media content 251.

[0053] Further, according to Figure 2K the interface 200K shown, if the user triggers and selects the interaction item 271-4 (also known as the target interaction item), the electronic device 110 may present a message 281 (also known as the second message) from a virtual object, and such a message 281 is generated based on such a target interaction item.

[0054] Based on such an interaction method, the embodiments of the present disclosure can improve the efficiency of users to obtain information associated with media content.

[0055] Example interactions of some other interfaces

[0056] The following will refer to Figures 3A through 3F to describe some example interaction processes according to other embodiments of the present disclosure. Figures 3A through 3F The example interfaces 300A to 300F according to other embodiments of the present disclosure are shown, and the interfaces 300A to 300F may be provided by, for example, Figure 1 the electronic device 110 shown.

[0057] In some embodiments, the electronic device 110 may present Figure 3AThe interface 300A shown. Such an interface 300A can be a playback interface for media content. For example, such an interface 300A can be a playback interface for media content 311 (also referred to as reference media content). User 140 can watch such media content 311 through such an interface 300A and can interact with such media content 311 (such as commenting, favoriting, sharing, etc.). As an example, such media content can be played in player 310, and such player 310 can also provide various types of controls to support user 140 in adjusting the played content, such as supporting bullet screen sending, bullet screen number adjustment, media content playback speed adjustment, etc.

[0058] In some embodiments, in response to receiving a second operation in such a playback interface, electronic device 110 can present an interaction window associated with media content 311 in the playback interface. As an example, such a second operation can indicate the triggering of a preset shortcut key or a combination of preset shortcut keys. To reduce the difficulty of user operation, electronic device 110 can present a prompt window 321 in such a playback interface, and such a prompt window 321 can present the text "Have a problem watching the video? Press Q to ask".

[0059] Further, taking the example that such a second operation can indicate the triggering of the preset shortcut key "Q", electronic device 110 can receive the triggering of the preset shortcut key "Q" in the playback interface to present an interaction window associated with media content 311. As an example, according to Figure 3B The interface 300B shown, electronic device 110 can present an interaction window 330 associated with media content 311. Such an interaction window 330 can be associated with a virtual object to support user 140 in interacting with the virtual object.

[0060] In some embodiments, if such an interaction window 330 is presented, electronic device 110 can present a message 331 (also referred to as the first message) from the virtual object in the interaction window 330. As an example, such a message 331 can include descriptive text about media content 311, and such descriptive text can describe the characters, scenes, stories, etc. in media content 311. In this way, user 140 can more quickly understand such media content 311.

[0061] Further, user 140 can have a conversation interaction with the virtual object through such an interaction window 330. As an example, according to Figure 3CFor the interface 300C shown, user 140 can send a message 332 in such an interaction window. Furthermore, the electronic device 110 can present a message 333 provided by a virtual object as a response content for the message 332. In this way, embodiments of the present disclosure can support users to further understand information associated with media content during the process of watching media content, thereby improving the information acquisition efficiency of users.

[0062] In some embodiments, the electronic device 110 supports user 140 to select some content in the reference media content and initiate a corresponding question. Thus, the electronic device 110 can provide more accurate response content for user 140. As an example, according to Figure 3D For the interface 300D shown, user 140 can select an image area 341 of the reference media content in the playback interface and drag such an image area 341 from the player 310 in the playback interface to the interaction window 330.

[0063] Furthermore, according to Figure 3E For the interface 300E shown, the electronic device 110 can determine a message 334 (also referred to as a user message) based on the image area 341. As an example, such a message 334 can also include supplementary content (also referred to as second input content) input by user 140 through the interaction window 330, such as the text "What kind of tree is this". Thus, embodiments of the present disclosure can improve the accuracy of user messages, thereby improving the accuracy of response content generated based on such user messages.

[0064] In some embodiments, such an interaction window 330 can also present a set of interaction items determined based on the reference media content. For example, according to Figure 3F For the interface 300F shown, the electronic device 110 can present a set of interaction items determined based on the media content 311, which for example includes interaction item 351-1 (for example, indicating "Help me search for information: What trees are suitable to plant in spring"), interaction item 351-2 (for example, indicating "Help me make a decision: Spring travel guide"), interaction item 351-3 (for example, indicating "Help me find a method: How to prepare travel food"), interaction item 351-4 (for example, indicating "Help me create: Create spring travel works"), and so on. As an example, such a set of interaction items can also be updated based on the playback progress of the media content 311 played in the player 310. Based on such a set of interaction items, embodiments of the present disclosure can help users interact with virtual objects more conveniently, thereby further improving the interface interaction efficiency.

[0065] Based on such an interaction method, embodiments of the present disclosure can improve the efficiency of users obtaining information from media content.

[0066] Example information processing procedure

[0067] Figure 4 The flowchart of process 400 of an example information processing according to some embodiments of the present disclosure is shown. Process 400 may be implemented at electronic device 110. The following refers to Figure 1 Describe process 400.

[0068] In some embodiments, electronic device 110 samples (410) a plurality of images from reference media content. As an example, electronic device 110 may obtain such a plurality of images by any suitable sampling method. For example, in the case where such reference media content includes video content, electronic device 110 may sample such a plurality of images by video frame extraction. As another example, in the case where such reference media content includes atlas content, electronic device 110 may sample such a plurality of images from such atlas content through a predetermined image selection strategy.

[0069] Furthermore, electronic device 110 may use a vision model to generate descriptive text for such a plurality of images. As an example, such a vision model may be any suitable model, which can, for example, understand the content of the input image and output the descriptive content for the input image.

[0070] To improve the accuracy of such descriptive text, electronic device 110 may sequentially use the vision model to process such a plurality of images and generate multiple descriptive texts corresponding to such a plurality of images respectively. As an example, electronic device 110 may select (420) the currently to-be-processed image from such a plurality of images. Further, electronic device 110 may preprocess (430) such a to-be-processed image to improve the accuracy of the subsequent vision model for image recognition. As an example, such preprocessing methods include but are not limited to adjusting the image size, image encoding, and the like.

[0071] After completing the preprocessing of the to-be-processed image, electronic device 110 may use the vision model to understand (440) the preprocessed image to generate descriptive text corresponding to the preprocessed image.

[0072] Further, electronic device 110 may determine (450) whether all images in the plurality of images have been understood. If all such a plurality of images have been understood, electronic device 110 may generate reference information corresponding to the reference media content based on the descriptive text of such a plurality of images. If there are images in such a plurality of images that have not been understood, step 420 may be continued until the understanding of the plurality of images is completed.

[0073] In some embodiments, after completing the understanding of each image, the electronic device 110 may temporarily store the corresponding description text (460). Furthermore, after completing the understanding of all images, the electronic device 110 may generate reference information based on the description texts of the multiple images and information associated with such reference media content.

[0074] To improve the accuracy of the generated reference information, electronic device 110 may refer to multiple pieces of information to generate such reference information. Specifically, such reference information may be determined based on at least one of the following: a transcript of the audio content corresponding to the reference media content; recognized text determined by detecting the image of the reference media content; or metadata information of the reference media content.

[0075] As an example, such transcribed text may indicate semantic information in the reference media content. For example, such transcribed text may be derived from character dialogues, narrations, interpretations, and the like in such reference media content. As an example, such recognized text may include, but is not limited to, the title, comments, scene descriptions, item introductions, and the like of the media content. Thus, the embodiments of the present disclosure may help subsequent models to more comprehensively understand the reference media content. As an example, metadata information of the reference media content includes, but is not limited to, the title, classification tags, and the like of the reference media content. Thus, the electronic device 110 may further improve the accuracy of subsequent models in understanding such reference media content.

[0076] Based on this approach, electronic device 110 can generate more accurate reference information. As an example, such reference information may include at least one of the following categories of information: first information indicating a summary of the reference media content; second information indicating a set of keywords associated with the reference media content; and third information indicating a set of tags associated with the reference media content.

[0077] Furthermore, the electronic device 110 may input at least such reference information into a model and use the model to summarize (470) such reference information, thereby generating the response content described above. As an example, such a model may include any appropriate generative model, which may, for example, generate corresponding summarized reference information based on multiple pieces of input information.

[0078] Based on such an information processing process, the embodiments of the present disclosure can improve the accuracy of the generated reference information and the accuracy of the response content generated based on such reference information.

[0079] Example procedure

[0080] Figure 5FIG. 500 is a flowchart showing an example interface interaction process according to some embodiments of the present disclosure. Process 500 may be implemented at electronic device 110. The following will refer to Figure 1 describe Process 500.

[0081] As Figure 5 shown, at block 510, the electronic device presents an interaction window associated with the published reference media content, and the interaction window is associated with a virtual object.

[0082] At block 520, the electronic device receives a user message via the interaction window.

[0083] At block 530, the electronic device provides response content generated by the virtual object based on the user message in the interaction window, and the response content is at least generated based on the reference information of the reference media content.

[0084] In some embodiments, presenting an interaction window associated with the published reference media content includes: presenting a session interface with the virtual object; and in response to receiving a first operation via the session interface, presenting an interaction window associated with the reference media content, where the first operation indicates the published reference media content.

[0085] In this way, the embodiments of the present disclosure can support users to understand information related to the reference media content through the session interface, thereby improving the efficiency of users obtaining information.

[0086] In some embodiments, receiving the first operation includes: presenting a media addition entry in the session interface; in response to the selection of the media addition entry, presenting a set of candidate media content associated with the current user, where the set of candidate media content is determined based on the historical interaction of the current user with the published media content; and receiving the selection of the reference media content from the set of candidate media content.

[0087] In this way, the embodiments of the present disclosure can support selecting media content from the published media content of historical interactions without complex interactions such as users downloading or taking screenshots of the media content. Thus, the embodiments of the present disclosure can improve the efficiency of users obtaining information about the published media content.

[0088] In some embodiments, receiving the first operation includes: receiving first input content via the session interface, where the first input content indicates the access address of the reference media content.

[0089] In this way, the embodiments of the present disclosure can reduce the resource efficiency caused by users downloading and uploading media content and can improve the input efficiency of users for the reference media content.

[0090] In some embodiments, presenting an interactive window associated with the published reference media content includes: presenting a playback interface of the reference media content; and in response to receiving a second operation in the playback interface, presenting an interactive window associated with the reference media content in the playback interface.

[0091] In this way, the embodiments of the present disclosure can support users to timely understand relevant information during the process of watching media content (reference media content), thereby helping users to more efficiently understand such media content.

[0092] In some embodiments, receiving the second operation includes: in the playback interface, receiving a trigger for a preset shortcut key or a preset combination of shortcut keys.

[0093] In this way, the embodiments of the present disclosure further improve the interaction efficiency between the user and the interface through such a preset shortcut key or a preset combination of shortcut keys.

[0094] In some embodiments, receiving a user message via the interactive window includes: in the playback interface, receiving a selection operation on an image area of the reference media content; and receiving a drag operation of dragging the image area from the player in the playback interface to the interactive window to determine the user message based on the image area.

[0095] In this way, the embodiments of the present disclosure can help users more conveniently query information related to a specified part in the media content during the process of watching the media content.

[0096] In some embodiments, the user message further includes second input content received via the interactive window.

[0097] In this way, the embodiments of the present disclosure can support users to supplement more question content, and thus can improve the accuracy of the response content provided by the virtual object.

[0098] In some embodiments, before receiving the user message via the interactive window, process 500 further includes: in the interactive window, presenting a first message from a virtual object, the first message including descriptive text about the reference media content.

[0099] In this way, the embodiments of the present disclosure can automatically describe such reference media content for users, thereby further improving the efficiency of users obtaining information about the reference media content.

[0100] In some embodiments, process 500 further includes: in the interactive window, presenting a set of interactive items determined based on the reference media content; and in response to a selection of a target interactive item in the set of interactive items, presenting a second message from the virtual object, the second message being generated based on the target interactive item.

[0101] In this way, embodiments of the present disclosure can help users understand more information related to media content, thereby further improving the efficiency of users in obtaining information related to media content.

[0102] In some embodiments, process 500 further includes: updating a set of interactive items based on the playback progress of the reference media content.

[0103] In this way, embodiments of the present disclosure can improve the timeliness and accuracy of such a set of interactive items.

[0104] In some embodiments, the reference media content includes video content or picture set content.

[0105] In this way, embodiments of the present disclosure can support providing users with information related to video content or picture set content. Users can obtain accurate and comprehensive information without the need for complex search or problem description methods, thereby further improving the efficiency of users in obtaining information.

[0106] In some embodiments, the reference information is determined based on the following process: sampling multiple images from the reference media content; using a visual model to generate descriptive texts for the multiple images; and generating reference information for the reference media content based on the descriptive texts.

[0107] In this way, embodiments of the present disclosure can improve the use of the visual model to more accurately understand the reference media content, thereby improving the quality of the reference information generated for the reference media content.

[0108] In some embodiments, the reference information is determined based on at least one of the following: the transcribed text of the audio content corresponding to the reference media content; the recognition text determined by detecting the picture of the reference media content; the metadata information of the reference media content.

[0109] In this way, embodiments of the present disclosure can refer to more comprehensive information about the reference media content in the process of generating the reference information, thereby further improving the quality of the reference information generated for the reference media content.

[0110] In some embodiments, the reference information includes at least one of the following categories of information: first information, where the first information indicates the summary content of the reference media content; second information, where the second information indicates a set of keywords associated with the reference media content; third information, where the third information indicates a set of tags associated with the reference media content.

[0111] In this way, embodiments of the present disclosure can improve the comprehensiveness and quality of the reference information, and further improve the quality of the response content generated based on such reference information.

[0112] Based on such a manner, embodiments of the present disclosure can support a virtual object to provide response content related to reference media content, and a user can obtain information related to the reference media content without complex interactions or complex problem descriptions. Therefore, embodiments of the present disclosure can significantly improve the efficiency of interface interaction and the efficiency of the user to obtain information related to the reference media content.

[0113] Therefore, embodiments of the present disclosure can improve the efficiency of interface interaction and can also improve the efficiency of the user to obtain information related to the reference media content.

[0114] Example devices and equipment

[0115] Embodiments of the present disclosure also provide corresponding apparatuses for implementing the above methods or processes. Figure 6 FIG. shows a schematic structural block diagram of an exemplary apparatus 600 for interface interaction according to certain embodiments of the present disclosure. The apparatus 600 can be implemented as or included in an electronic device 110. Each module / component in the apparatus 600 can be implemented by hardware, software, firmware, or any combination thereof.

[0116] As Figure 6 shown, the apparatus 600 includes a first presentation module 610 configured to present an interaction window associated with the published reference media content, the interaction window being associated with a virtual object; a receiving module 620 configured to receive a user message via the interaction window; and a providing module 630 configured to provide, in the interaction window, response content generated by the virtual object based on the user message, the response content being generated at least based on reference information of the reference media content.

[0117] In some embodiments, the first presentation module 610 is further configured to: present a session interface of the virtual object; and in response to receiving a first operation via the session interface, present an interaction window associated with the reference media content, the first operation indicating the published reference media content.

[0118] In some embodiments, the first presentation module 610 is further configured to: in the session interface, present a media addition entry; in response to a selection of the media addition entry, present a set of candidate media content associated with the current user, the set of candidate media content being determined based on the historical interactions of the current user with the published media content; and receive a selection of the reference media content from the set of candidate media content.

[0119] In some embodiments, the first presentation module 610 is further configured to: receive, via the session interface, a first input content indicating an access address of the reference media content.

[0120] In some embodiments, the first presentation module 610 is further configured to: present a playback interface of the reference media content; and in response to receiving a second operation in the playback interface, present an interaction window associated with the reference media content in the playback interface.

[0121] In some embodiments, the first presentation module 610 is further configured to: in the playback interface, receive a trigger for a preset shortcut key or a preset combination of shortcut keys.

[0122] In some embodiments, the receiving module 620 is further configured to: in the playback interface, receive a selection operation on an image region of the reference media content; and receive a drag operation of dragging the image region from the player in the playback interface to the interaction window to determine a user message based on the image region.

[0123] In some embodiments, the user message further includes second input content received via the interaction window.

[0124] In some embodiments, the apparatus 600 further includes a second presentation module configured to: in the interaction window, present a first message from a virtual object, the first message including descriptive text about the reference media content.

[0125] In some embodiments, the apparatus 600 further includes a third presentation module configured to: in the interaction window, present a set of interaction items determined based on the reference media content; and in response to a selection of a target interaction item in the set of interaction items, present a second message from the virtual object, the second message being generated based on the target interaction item.

[0126] In some embodiments, the apparatus 600 further includes an update module configured to: update a set of interaction items based on the playback progress of the reference media content.

[0127] In some embodiments, the reference media content includes video content or picture set content.

[0128] In some embodiments, the apparatus 600 further includes an information generation module configured to: sample a plurality of images from the reference media content; use a visual model to generate descriptive text for the plurality of images; and generate reference information for the reference media content based on the descriptive text.

[0129] In some embodiments, the apparatus 600 further includes an information determination module configured to: the transcription text of the audio content corresponding to the reference media content; the recognition text determined by detecting the picture of the reference media content; the metadata information of the reference media content.

[0130] In some embodiments, the reference information includes at least one of the following categories of information: first information, where the first information indicates the summary content of the reference media content; second information, where the second information indicates a set of keywords associated with the reference media content; third information, where the third information indicates a set of tags associated with the reference media content.

[0131] The modules included in apparatus 600 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to the machine-executable instructions, some or all of the modules in apparatus 600 can be at least partially implemented by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0132] Figure 7 A block diagram of an electronic device 700 in which one or more embodiments of the present disclosure can be implemented is shown. It should be understood that Figure 7 The illustrated electronic device 700 is merely exemplary and should not impose any limitation on the functionality and scope of the embodiments described herein. Figure 7 The illustrated electronic device 700 can be used to implement Figure 1 electronic device 110 or Figure 6 apparatus 600.

[0133] As Figure 7 shown, the electronic device 700 is in the form of a general-purpose electronic device. The components of the electronic device 700 can include, but are not limited to, one or more processors or processing units 710, a memory 720, a storage device 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. The processing unit 710 can be an actual or virtual processor and is capable of performing various processes according to the programs stored in the memory 720. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to enhance the parallel processing capabilities of the electronic device 700.

[0134] The electronic device 700 generally includes multiple computer storage media. Such media can be any available media accessible to the electronic device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 720 can be volatile memory (such as registers, caches, random access memory (RAM)), non-volatile memory (such as read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 730 can be removable or non-removable media and can include machine-readable media, such as a flash drive, a magnetic disk, or any other media that can be used to store information and / or data and can be accessed within the electronic device 700.

[0135] The electronic device 700 can further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in Figure 7 it, a disk drive for reading from and writing to a removable, non-volatile magnetic disk (such as a "floppy disk") and an optical disk drive for reading from and writing to a removable, non-volatile optical disk can be provided. In these cases, each drive can be connected to a bus (not shown) by one or more data media interfaces. The memory 720 can include a computer program product 725 having one or more program modules that are configured to perform the various methods or actions of the various embodiments of the present disclosure.

[0136] The communication unit 740 enables communication with other electronic devices via a communication medium. Additionally, the functions of the components of the electronic device 700 can be implemented by a single computing cluster or multiple computer machines that are capable of communicating via a communication connection. Thus, the electronic device 700 can operate in a networked environment using a logical connection with one or more other servers, network personal computers (PCs), or another network node.

[0137] The input device 750 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. The output device 760 can be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 700 can also communicate with one or more external devices (not shown) as needed via the communication unit 740, such as storage devices, display devices, etc., communicate with one or more devices that enable a user to interact with the electronic device 700, or communicate with any device that enables the electronic device 700 to communicate with one or more other electronic devices (such as a network card, a modem, etc.). Such communication can be performed via an input / output (I / O) interface (not shown).

[0138] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, and the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, the computer program product being tangibly stored on a non-transitory computer-readable medium and including computer-executable instructions, and the computer-executable instructions being executed by a processor to implement the method described above.

[0139] Aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and the combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0140] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is produced that implements the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause a computer, a programmable data processing device, and / or other devices to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured article that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0141] The computer-readable program instructions can be loaded onto a computer, other programmable data processing device, or other device, so that a series of operation steps are executed on the computer, other programmable data processing device, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing device, or other device implement the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0142] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various implementations of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0143] The various implementations of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art in the field of this technology without departing from the scope and spirit of the described implementations. The choice of terms used herein is intended to best explain the principles of the implementations, the practical application, or the improvement of the technology in the market, or to enable other ordinary skilled artisans in the field of this technology to understand the various implementation manners disclosed herein.

Claims

1. A method for interface interaction, comprising: Presenting an interactive window associated with published reference media content, the interactive window being associated with a virtual object; Receiving a user message via the interactive window; And Providing, in the interactive window, response content generated by the virtual object based on the user message, the response content being generated at least based on reference information of the reference media content.

2. The method according to claim 1, wherein presenting an interactive window associated with published reference media content comprises: Presenting a session interface of the virtual object; And In response to receiving a first operation via the session interface, presenting the interactive window associated with the reference media content, the first operation indicating the published reference media content.

3. The method according to claim 2, wherein receiving the first operation comprises: Presenting a media addition entry in the session interface; In response to a selection of the media addition entry, presenting a set of candidate media content associated with the current user, the set of candidate media content being determined based on the current user's historical interactions with published media content; And Receiving a selection of the reference media content from the set of candidate media content.

4. The method according to claim 2, wherein receiving the first operation comprises: Receiving, via the session interface, a first input content indicating an access address of the reference media content.

5. The method according to claim 1, wherein presenting an interactive window associated with published reference media content comprises: Presenting a playback interface of the reference media content; And In response to receiving a second operation in the playback interface, presenting the interactive window associated with the reference media content in the playback interface.

6. The method according to claim 5, wherein receiving the second operation comprises: Receiving, in the playback interface, a trigger of a preset shortcut key or a preset combination of shortcut keys.

7. The method according to claim 5, wherein receiving a user message via the interactive window comprises: Receiving, in the playback interface, a selection operation on an image area of the reference media content; And Receiving a drag operation of dragging the image area from a player in the playback interface to the interactive window to determine the user message based on the image area.

8. The method according to claim 7, wherein the user message further comprises a second input content received via the interactive window.

9. The method according to claim 1, wherein before receiving a user message via the interactive window, the method further comprises: Presenting, in the interactive window, a first message from the virtual object, the first message comprising descriptive text about the reference media content.

10. The method according to claim 1, further comprising: Presenting, in the interactive window, a set of interactive items determined based on the reference media content; And In response to the selection of a target interaction item from the set of interaction items, a second message from the virtual object is presented, and the second message is generated based on the target interaction item.

11. The method according to claim 10, further comprising: Updating the set of interaction items based on the playback progress of the reference media content.

12. The method according to claim 1, wherein the reference media content includes video content or atlas content.

13. The method according to claim 1, wherein the reference information is determined based on the following process: Sampling a plurality of images from the reference media content; Using a visual model to generate descriptive text for the plurality of images; and Generating the reference information of the reference media content based on the descriptive text.

14. The method according to claim 1, wherein the reference information is determined based on at least one of the following: The transcription text of the audio content corresponding to the reference media content; The recognition text determined by detecting the screen of the reference media content; The metadata information of the reference media content.

15. The method according to claim 13, wherein the reference information includes at least one of the following categories of information: First information, the first information indicating the summary content of the reference media content; Second information, the second information indicating a set of keywords associated with the reference media content; Third information, the third information indicating a set of tags associated with the reference media content.

16. An apparatus for interface interaction, comprising: A first presentation module configured to present an interaction window associated with the published reference media content, the interaction window being associated with a virtual object; A receiving module configured to receive a user message via the interaction window; And A providing module configured to provide response content generated by the virtual object based on the user message in the interaction window, the response content being generated based at least on the reference information of the reference media content.

17. An electronic device, comprising: At least one processing unit; And At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions when executed by the at least one processing unit cause the electronic device to execute the method according to any one of claims 1 to 15.

18. A computer-readable storage medium having a computer program stored thereon, the computer program when executed by a processor implements the method according to any one of claims 1 to 15.

Images