Method, apparatus, device and product for interface interaction

Abstract

Provided are an interface interaction method, device, equipment and product. The method proposed herein includes: triggering presentation of a first interface, interface content of the first interface being provided for a conversation with a first object; in response to receiving a first operation, presenting a first graphical element at the first interface, the first operation being related to the first object; and outputting first conversation content, the first conversation content being from the first object, the first conversation content being related to the first graphical element. In this way, not only can the efficiency of the user obtaining information be improved, but also the interaction process can be simplified, thereby reducing the consumption of computing resources.

Description

Technical Field

[0001] The examples in this article generally relate to the field of computer science, and in particular to methods, apparatuses, devices, and products for user interface interaction. Background Technology

[0002] With the development of computer technology, the field of artificial intelligence continues to make breakthroughs. Users can communicate continuously and fluently with models through conversational text, completing various tasks such as information inquiry, logical thinking, and content interpretation without using complex commands. How to enable users to interact with models more effectively has become an urgent problem to be solved. Summary of the Invention

[0003] In a first aspect, a method for interface interaction is provided. The method includes: triggering the presentation of a first interface, the interface content of which is provided for dialogue interaction with a first object; in response to receiving a first operation, presenting a first graphical element on the first interface, the first operation being associated with the first object; and outputting first dialogue content derived from the first object, the first dialogue content being associated with the first graphical element.

[0004] In a second aspect, an apparatus for interface interaction is provided. The apparatus includes: a triggering module configured to trigger the presentation of a first interface, the interface content of which is provided for dialogue interaction with a first object; a first presentation module configured to, in response to receiving a first operation, present a first graphical element on the first interface, the first operation being associated with the first object; and a first output module configured to output first dialogue content derived from the first object, the first dialogue content being associated with the first graphical element.

[0005] In a third aspect, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. When executed by the at least one processor, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect, a computer program product is provided, which is tangibly stored in a computer storage medium and includes computer-executable instructions that, when executed by a device, cause the device to perform the method of the first aspect.

[0007] In this way, the example in this paper can output the first dialogue content from the first object and associated with the first graphical element, thereby providing users with more accurate first dialogue content and improving the efficiency of users obtaining information. Furthermore, the example in this paper allows users to trigger the presentation of the first graphical element and the content output from the first object through a first operation, simplifying the interaction process and reducing the consumption of computing resources.

[0008] It should be understood that the content described in this section is not intended to limit the key or important features of the examples in this article, nor is it intended to restrict the scope of the solution. Other features will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the various examples herein will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. In the accompanying drawings, the same or similar reference numerals denote the same or similar elements, wherein: Figure 1 A schematic diagram of the example environment is shown; Figures 2A to 2I Example interfaces for some scenarios are shown; Figures 3A to 3H Example interfaces for some scenarios are shown; Figures 4A to 4D Example interfaces for some scenarios are shown; Figure 5 The flowcharts show example processes of interface interactions in some scenarios; Figure 6 Schematic block diagrams of example devices for interface interaction in some scenarios are shown; and Figure 7 A block diagram of an electronic device capable of implementing multiple illustrative scenarios is shown. Detailed Implementation

[0010] The examples in this document will now be described in more detail with reference to the accompanying drawings. While some examples are shown in the drawings, it should be understood that solutions can be implemented in various forms and should not be construed as limited to the examples presented herein. Rather, these examples are provided to provide a more thorough and complete understanding of the solutions. It should be understood that the drawings and examples in this document are for illustrative purposes only and are not intended to limit the scope of protection of the solutions.

[0011] It should be noted that the headings of any section / subsection provided herein are not restrictive. Various examples are described throughout this document, and examples of any type may be included under any section / subsection. Furthermore, examples described in any section / subsection may be combined in any way with any other examples described in the same section / subsection and / or different sections / subsections.

[0012] In the description of the examples in this document, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "an example" or "the example" should be understood as "at least one example". The term "some examples" should be understood as "at least some examples". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0013] The examples in this document may involve user data, data acquisition, and / or use. All of these aspects comply with relevant laws, regulations, and rules. In the examples, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, when implementing each example, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained through appropriate means, in accordance with relevant laws and regulations. The specific methods of notification and / or authorization can vary depending on the actual situation and application scenario; the scope of the solution is not limited in this regard.

[0014] In this manual and the sample solutions, any processing of personal information will be conducted only under legal grounds (such as obtaining the consent of the data subject or being necessary for the performance of a contract) and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0015] As mentioned above, with the development of computer technology, the field of artificial intelligence continues to make breakthroughs. Users can communicate continuously and fluently with models through conversational text, completing various tasks such as information inquiry, logical thinking, and content interpretation without using complex commands. How to enable users to interact with models more effectively has become an urgent problem to be solved.

[0016] A user interface interaction scheme is proposed. The scheme includes: triggering the presentation of a first interface, the content of which is provided for dialogue with a first object. In response to receiving a first operation, a first graphical element is presented on the first interface. The first operation is related to the first object. Further, first dialogue content can be output. The first dialogue content originates from the first object. The first dialogue content is related to the first graphical element.

[0017] In this way, the example in this paper can output the first dialogue content from the first object and associated with the first graphical element, thereby providing users with more accurate first dialogue content and improving the efficiency of users obtaining information. Furthermore, the example in this paper allows users to trigger the presentation of the first graphical element and the content output from the first object through a first operation, simplifying the interaction process and reducing the consumption of computing resources.

[0018] The following describes various examples of this scheme in further detail with reference to the accompanying drawings.

[0019] Example Environment Figure 1 A schematic diagram of example environment 100 is shown. (e.g.) Figure 1 As shown, example environment 100 may include electronic device 110.

[0020] In this example environment 100, electronic device 110 can run an application 120 that supports user interface interaction. Application 120 can be any suitable type of application for user interface interaction, including but not limited to: content sharing applications, content viewing applications, or other suitable applications. User 140 can interact with application 120 via electronic device 110 and / or its attached devices.

[0021] exist Figure 1 In environment 100, if application 120 is active, electronic device 110 can use application 120 to present interface 150 for supporting interface interaction.

[0022] In some cases, electronic device 110 communicates with server 130 to provide services to application 120. Electronic device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some cases, electronic device 110 can also support any type of user-facing interface (such as "wearable" circuitry).

[0023] Server 130 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Server 130 may include, for example, computing systems / servers such as mainframes, edge computing nodes, computing devices in a cloud environment, etc. Server 130 can provide backend services for applications 120 that support user interface interaction in electronic devices 110.

[0024] A communication connection can be established between server 130 and electronic device 110. This communication connection can be established via wired or wireless means. The communication connection can include, but is not limited to, Bluetooth, mobile network, Universal Serial Bus (USB), and Wireless Fidelity (WiFi) connections. In some cases, server 130 and electronic device 110 can exchange signaling information through their communication connection.

[0025] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of the scheme.

[0026] The following description of the example will continue with reference to the accompanying drawings.

[0027] Example Interaction Figures 2A to 2I Example interfaces 200A to 200I are shown, illustrating interface interactions under various scenarios. Interfaces 200A to 200I can, for example, be provided by... Figure 1 The electronic device 110 shown is provided.

[0028] In some situations, electronic device 110 triggers the presentation of a first interface. The content of the first interface is provided for a dialogue with a first object. As an example, electronic device 110 may trigger the presentation of the first interface when preset conditions are met. For example, electronic device 110 may provide appropriate controls. When the control is clicked, electronic device 110 may present the first interface. As another example, electronic device 110 may receive a screen sharing request. Upon receiving a screen sharing request, electronic device 110 may enable a screen sharing event associated with the first object. After the screen sharing event is initiated, electronic device 110 may trigger the presentation of the first interface by the first object. Such a first interface can be any appropriate interface, such as a content viewing interface, a content editing interface, a content management interface, a product retrieval interface, etc.

[0029] After the first interface is presented, the electronic device 110 can provide the interface content of the first interface to the first object. For example, the electronic device 110 can continuously record the display screen of the first interface to generate a continuous video stream, thereby capturing dynamic changes in the interface and user interaction processes. The electronic device 110 can then provide this continuous video stream to the first object. Alternatively, the electronic device 110 can also acquire static images of the first interface at preset time intervals (e.g., 100 milliseconds) to construct a set of structured image sequences. The electronic device 110 can then provide this constructed set of structured image sequences to the first object.

[0030] After providing the interface content of the first interface to the first object, the electronic device 110 can receive the user's interaction request to the first object (e.g., sending a message to the first object). Further, the electronic device 110 can provide the interaction request to the first object. The first object can use the interface content of the first interface as context information and generate a corresponding response message based on the interaction request to conduct a dialogue interaction with the user. Dialogue interaction can represent the information exchange process between the user and the first object through natural language or interface operations. As an example, dialogue interaction can involve the user inputting a request in text or voice form, and the first object understanding and generating a corresponding response. The first object can proactively ask follow-up questions, clarify, or prompt the user to provide supplementary information during the interaction. Dialogue interaction can include, for example, voice calls, video calls, text chat, and other interaction forms.

[0031] In this paper, the first object refers to a system (e.g., an intelligent system, intelligent component, or virtual object) capable of autonomous control based on a machine learning model. The first object, for example, is a virtual object or physical entity capable of making decisions and autonomously executing actions based on a machine learning model to achieve preset goals or complete preset tasks. The first object can be an automated program that understands the user's intent and can utilize models or invoke tools to complete various types of tasks. In some cases, examples of the first object may include, but are not limited to: agents, bots, chatbots, digital avatars, intelligent customer service, digital assistants, etc. Alternatively, the first object can also be an intelligent role implemented based on a machine learning model. The "first object" can process requests from the user based on generative models (e.g., language models, multimodal models) to perform specified types of tasks. In some cases, the first object may also refer to a virtual account, which may have a corresponding avatar or nickname.

[0032] In some cases, electronic device 110 can receive a first message input, which is related to a first topic. For example, the first topic may be associated with a specific service request. Further, electronic device 110 can present a first interface. The first interface includes interface content associated with the first topic. As an example, electronic device 110 can receive the first message in any suitable manner. For example, electronic device 110 can provide appropriate controls in an interactive interface and receive the first message through those controls. As another example, electronic device 110 can acquire the user's voice input and use that voice input as the first message. Such a first message is associated with a first topic. The first topic can represent a problem, task, or goal indicated by the user that needs to be solved through a first object. Examples of first topics may include, but are not limited to, information query requests, task execution requests, content Q&A requests, and operation guidance requests. The first message can be used to indicate the first topic; for example, the first message can indicate that the first object needs to answer questions about any appropriate content (such as the screen parameters of a work published by the user). Such answers may, for example, indicate optimization suggestions for the screen parameters of the work (such as adjusting the resolution of the work, removing noise from the screen).

[0033] Taking the intelligent customer service as an example, the electronic device 110 can receive a first message related to a first topic. This first message can provide optimization suggestions for the image parameters of the user's published work. The electronic device 110 can then provide this first message to the first object. The first object can then provide the user with optimization suggestions regarding image parameters based on the first message.

[0034] Upon receiving a first message, the electronic device 110 can present a first interface determined based on a first topic. The content of the interface in the first interface is associated with the first topic. Taking the first topic as providing optimization suggestions for the screen parameters of work A as an example, after receiving the first message, the first object can process the first message using an appropriate method (such as semantic understanding) to determine that the first topic indicated by the first message is a screen parameter optimization requirement. After determining that the first topic is a screen parameter optimization requirement, the first object can determine the first interface corresponding to work A. Further, the electronic device 110 can present the first interface corresponding to work A and display work A, which requires optimization suggestions, in the first interface. Taking the first topic as an order inquiry as an example, after determining that the first topic is associated with an order, the first object can determine the first interface associated with the order. The electronic device 110 can then present the first interface associated with the order; such a first interface could be, for example, an order management interface. That is, when it is determined that the first topic is associated with an order, the electronic device 110 can present an order management interface.

[0035] In this way, the example in this paper can provide the user with a first interface containing interface content related to the first topic after receiving the first message related to the first topic, thereby reducing the interaction chain that the user triggers to present the corresponding interface content and improving the efficiency of human-computer interaction.

[0036] In some cases, electronic device 110 may display a second interface. The second interface is a dialogue interface with the first object. Electronic device 110 can receive the first message on the second interface and display the first interface. For example, as shown... Figure 2A As shown, electronic device 110 can present a second interface. The second interface can be, for example, interface 200A. Interface 200A can be, for example, a dialogue interface with a first object. Electronic device 110 can support dialogue interaction between the user and the first object via interface 200A. Electronic device 110 can present an input component in interface 200A. Such an input component can be, for example, component 205. Electronic device 110 can receive a first message via component 205. For example, electronic device 110 can receive the user's first input content via component 205, and use this first input content as a first message. After receiving the first message, as... Figure 2B As shown, the electronic device 110 can present a first interface. Such a first interface can be, for example, interface 200B. Interface 200B can be determined, for example, based on a first topic indicated by a first message.

[0037] For example, when the first theme is the optimization requirement of the screen parameters of work A, the first object can determine that the first interface to be presented is interface 200B associated with work A. Interface 200B can be used to manage a collection of works, or to view a specific work. Interface 200B can include a user's personal interface, which can include multiple works associated with the user. These multiple works can include works published by the user, or works shared by the user, etc. In some scenarios, interface 200B can also be the viewing interface for work A, where the work can be determined by the first object based on a first message. For example, if the first message indicates that optimization suggestions are provided for the screen parameters of work A, then the first object can determine that the first interface to be presented is the viewing interface for work A based on such a first message. After the first interface is presented, the electronic device 110 can maintain the dialogue interaction (e.g., a call) between the user and the first object, and provide the interface content of the first interface to the first object, so that the first object can provide appropriate responses to the user during the dialogue interaction based on the interface content.

[0038] In some cases, electronic device 110 may present a first entry point in response to receiving a first message. Further, electronic device 110 may present a first interface in response to triggering the first entry point. For example, such as... Figure 2A As shown, after receiving the first message, the electronic device 110 can present a first entry point in interface 200A. Such a first entry point could be, for example, entry point 210. Entry point 210 can be associated with the first interface; for example, entry point 210 can be used to trigger the presentation of the first interface. Furthermore, when entry point 210 is triggered (e.g., by clicking or double-clicking), the electronic device 110 can present the first interface. Such a first interface could be, for example, interface 200B.

[0039] In some cases, in response to receiving a first operation, electronic device 110 presents a first graphical element on a first interface. The first operation is associated with a first object. For example, such as... Figure 2B As shown, the electronic device 110 can display a preview image 215 of the artwork on the interface 200B. When the preview image 215 is clicked, as... Figure 2C As shown, electronic device 110 can present a work associated with preview image 215 on a first interface (e.g., interface 200C). Such a work may include, for example, media content 220. During the presentation of media content 220, electronic device 110 can maintain a dialogue interaction (e.g., a call) between the user and the first object. For ease of description, a call will be used as an example of dialogue interaction below. During the call between the user and the first object, electronic device 110 can receive a first operation. The first operation can be an appropriate type of interactive operation, such as clicking, long-pressing, swiping, or dragging an appropriate control on the interface, or clicking, swiping, or selecting interface content (e.g., the screen of media content 220). After receiving the first operation, electronic device 110 can determine a first graphic element based on the first operation. The first graphic element here can be an appropriate marker that can be used to represent the specific interface content indicated by the user. The first graphic element can correspond to different forms of expression, such as lines, geometric shapes (e.g., circles, rectangles), arrow symbols, etc. Further, electronic device 110 can present the first graphic element on the first interface. Such a first graphic element may be, for example, as shown in the image. Figure 2F Element 225 is shown. During the process of receiving the first operation and presenting the first graphical element, the electronic device 110 can maintain communication between the user and the first object. Alternatively, the first operation may be received during a dialogue interaction with the first object.

[0040] Taking a swipe operation on the interface content as an example, after receiving the swipe operation, the electronic device 110 can determine the movement trajectory corresponding to the swipe operation. Further, the electronic device 110 can determine the line corresponding to the movement trajectory as a first graphic element. The first graphic element can be matched with the movement trajectory of the swipe operation. After the first graphic element is generated, the electronic device 110 can display the first graphic element on the first interface.

[0041] In some scenarios, taking a click operation on interface content as an example, after receiving the click operation, the first object can determine the interface area corresponding to the click operation. Further, the first object can match an object (e.g., noise in the interface content) corresponding to the interface area from the interface content of the first interface. After determining the object, the first object can determine a first graphic element (e.g., a circle or square icon) from a set of graphic elements, and can adjust the size of the first graphic element so that it can indicate the object (e.g., noise) in the screen content. Further, the electronic device 110 can display the first graphic element on the first interface to indicate a specific object in the interface content.

[0042] In some cases, the electronic device 110 may provide a canvas corresponding to the first interface. Furthermore, the electronic device 110 may, in response to receiving a first operation on the canvas, present a first graphical element.

[0043] As an example, such as Figure 2C As shown, after a first interface (e.g., interface 200C) is presented, the electronic device 110 can provide a canvas corresponding to the first interface. Such a canvas can be, for example, canvas 230. The canvas can be a transparent or semi-transparent interactive area provided in the first interface. The canvas can be used to receive user clicks, swipes, selections, and other interactive operations. The interactive operations received via the canvas can be provided to a first object so that the first object can provide the user with its generated content based on such interactive operations. Further, the electronic device 110 can receive the first operation via canvas 230. After receiving the first operation, the first object can determine a first graphic element based on the first operation. After the first graphic element is determined, as... Figure 2F As shown, the electronic device 110 can display a first graphic element (e.g., element 225) in the interface 200F.

[0044] In this way, the example in this article can provide a canvas corresponding to the first interface, and obtain the first operation through the canvas, which can effectively improve the efficiency of obtaining the first operation.

[0045] In some cases, the electronic device 110 may present an interactive component on a first interface. The interactive component is associated with a first object. Further, the electronic device 110 may provide a canvas in response to a second operation associated with the interactive component. As an example, such as... Figure 2C As shown, during a dialogue interaction (e.g., a phone call) between a user and a first object, electronic device 110 can present interactive components in interface 200C. These interactive components can be, for example, visual components provided by electronic device 110 to enable interaction between the user and the first object, such as component 235. Electronic device 110 can support the dialogue interaction (e.g., a phone call) between the user and the first object via component 235. For example, electronic device 110 can receive a second input from the user for the first object via component 235. Such second input could be, for example, "What screen parameters do I need to optimize?" After receiving such second input, electronic device 110 can provide it to the first object. The first object can determine the semantic information of the second input using an appropriate method (e.g., semantic understanding). After determining the semantic information, the first object can generate a response based on the semantic information for the second input. Such a response could correspond to different modalities, such as a text modal or an audio modal. The response could be, for example, "Your work has noise and needs to be removed." After the response is generated, electronic device 110 can present the response in component 235. In some scenarios, after receiving the user's second input content, the electronic device 110 can also present the text content corresponding to the second input content in component 235.

[0046] Furthermore, the electronic device 110 can present the control 236 in the component 235. The electronic device 110 can receive a second operation via the control 236. Such a second operation can be, for example, a click, double-click, or other operation on the control 236. After the control 236 is clicked, the electronic device 110 can provide a canvas 230 to receive the first operation.

[0047] In this way, the example in this paper allows users to manually trigger the provision of a canvas to perform operations such as selection and drawing, effectively avoiding various user errors and thus improving the human-computer interaction experience. Furthermore, by reducing user errors, the consumption of computing resources can be effectively reduced.

[0048] In some cases, electronic device 110 may stop presenting interactive components in response to receiving a second operation. For example, such as... Figure 2C As shown, the second operation here can be, for example, a click operation on control 236. After control 236 is clicked, as... Figure 2DAs shown, electronic device 110 can stop displaying component 235. In other cases, electronic device 110 can also display a prompt message in response to receiving a second operation. The prompt message is related to the canvas. As an example, after control 236 is clicked, electronic device 110 can display a prompt message in interface 200D. The prompt message may, for example, indicate the operation that the user can perform through the canvas. Such a prompt message may be, for example, message 237. Message 237 may include graphic elements and text content. Such text content may be, for example, "Draw a circle on the screen to ask the first object a question."

[0049] In this way, the example in this paper can stop presenting interactive components or present prompts after receiving the first operation, which can help users more easily indicate at least part of the interface content through the canvas, thereby improving the efficiency of human-computer interaction.

[0050] In some cases, the canvas can also be provided proactively by the first object. Specifically, the electronic device 110 can provide the canvas in response to a dialogue interaction with the first object satisfying a triggering condition. As an example, such as Figure 2C As shown, during a dialogue interaction (e.g., a phone call) between a user and a first object, electronic device 110 can receive third input content from the user. Upon receiving the third input content, electronic device 110 can provide it to the first object. The first object can perform semantic understanding on the third input content to determine its semantic information. When the semantic information meets the triggering conditions, electronic device 110 can provide a canvas. For example, the third input content could be "How should the noise in work A be removed?" After receiving such third input content, the first object can perform semantic understanding on it to determine the semantic information corresponding to the third input content. When the semantic information indicates that the user needs to interact to indicate which part of work A's image needs optimization, the first object can determine that the semantic information meets the triggering conditions. Furthermore, electronic device 110 can provide a canvas 230 to allow the user to select the parts that need optimization. In some scenarios, such as... Figure 2E As shown, when the first object actively provides the canvas 230, the electronic device 110 can also present information 238 in the interface 200E. Information 238 can indicate that the canvas 230 has been provided, and the user can select the interface content through the canvas 230 to trigger the first object to generate a corresponding response based on the selected content.

[0051] In this way, the example in this article supports the first object to actively trigger the provision of the canvas, thereby reducing the interaction chain of triggering the provision of the canvas, which can effectively improve the efficiency of human-computer interaction.

[0052] In some cases, the first interface is used to play first media content. The electronic device 110 can pause the playback of the first media content on the first interface in response to providing a canvas. As an example, such as... Figure 2C As shown, electronic device 110 can present a first interface. The first interface can be, for example, interface 200C. Electronic device 110 can play first media content in interface 200C. The first media content can be, for example, media content 220. When canvas 230 is provided, electronic device 110 can pause the playback of the first media content (e.g., media content 220) to allow the user to more accurately select at least one object (e.g., noise in the image) within the media content 220 via the canvas.

[0053] In some scenarios, before providing the canvas 230, the first object can determine the time point at which the media content 220 needs to be paused based on contextual information. This contextual information may include, for example, dialogue messages between the user and the first object, and the media content 220 itself. For instance, a dialogue message might include, "Where in work A is there noise, and how can it be removed?" Based on such dialogue messages and the visual content of the media content 220, the first object can determine the specific video frames that need optimization for the media content 220. Once the video frames are determined and the canvas 230 needs to be provided, the first object can trigger an adjustment to the playback progress of the media content 220 to pause the media content 220 at the time point corresponding to the specific video frame.

[0054] In this way, the example in this paper can pause the playback of the first media content on the first interface when the canvas is provided, thereby enabling users to select at least a portion of the first media content more accurately and conveniently through the canvas, which can effectively improve the efficiency of human-computer interaction.

[0055] In some cases, electronic device 110 can output first dialogue content. The first dialogue content originates from a first object. The first dialogue content is associated with a first graphical element. For example, such as... Figure 2E As shown, after the first graphical element is presented, the first object generates first dialogue content based on the position, shape, corresponding area, or semantic information of the first graphical element in the interface. The first dialogue content can correspond to different modalities, such as text, voice, image, video, etc. The first dialogue content can explain, supplement, or evaluate the content indicated by the first graphical element. After the first dialogue content is generated, the electronic device 110 can output the first dialogue content. For example, when the first dialogue content is audio content, the electronic device 110 can play the audio content. When the first dialogue content is text content, such as... Figure 2G As shown, the electronic device 110 can present the first dialogue content in the interactive components (such as component 235) in the interface 200G.

[0056] In some scenarios, after the initial dialogue content is presented, the first object also supports follow-up questions from the user. Electronic device 110 can receive a fourth input from the user via component 235. After receiving the fourth input, electronic device 110 can provide it to the first object. The first object can then generate a corresponding response based on the fourth input to answer the user's follow-up questions.

[0057] In some cases, the first dialogue content is related to the first part of the interface content. The first part is indicated by a first graphic element. As an example, when the first object determines the first graphic element based on the user's first operation, it can also determine the first part indicated by the first graphic element based on the area where the first graphic element is located in the interface. For example, the first operation may be to select noise in the interface content. The electronic device 110 can determine the first graphic element based on such a first operation. After the first graphic element is presented, the first object can perform object recognition on the interface content based on the area where the first graphic element is located to determine that the first graphic element is the indicated object (e.g., noise in the image). After determining the object indicated by the first graphic element, the first object can take that object as the first part indicated by the first graphic element. When the first part is determined to be noise present in the image, the first object can generate the first dialogue content based on the noise present in the image and the first message. For example, the first message is "How can I remove this noise in my work A?" The first object can then generate the first dialogue content based on such a first message and the noise indicated by the first graphic element. Such initial dialogue content could be a reply to the first message, such as "You can click the edit button in the upper right corner to enter the editing interface and remove noise from the image using the adjustment tools in the editing interface."

[0058] In this way, the example in this paper can use a first object to generate first dialogue content based on the first part indicated by the first graphical element, thereby effectively improving the generation efficiency of the first dialogue content.

[0059] In some scenarios, electronic device 110 can send first image content to a first object to generate first dialogue content. The first image content includes at least a portion of the interface content and a first graphical element. As an example, after the first graphical element is rendered, electronic device 110 can trigger a screenshot event. The screenshot event can instruct a screenshot of the first interface. After the screenshot event is executed, electronic device 110 can acquire the first image content. Such first image content can include the interface content (e.g., the complete image of media content 220) and the first graphical element. After the first image content is acquired, electronic device 110 can send the first image content to the first object to trigger the first object to generate first dialogue content based on the first image content.

[0060] In some scenarios, a screenshot event can also instruct a screenshot to be taken of the area corresponding to the first graphic element. For example, electronic device 110 can take a screenshot of the area where the first graphic element (e.g., element 225) is located to obtain first image content. Such first image content may include element 225 and portions of interface content (e.g., the area where noise is located in media content 220). Furthermore, electronic device 110 can provide such first image content to a first object to trigger the first object to generate first dialogue content based on such first image content.

[0061] In this way, the example in this paper can send a first image content, including at least a portion of the interface content and a first graphical element, to a first object to trigger the first object to generate first dialogue content based on the first image content. This can effectively reduce the consumption of computing resources used to generate the first dialogue content and improve the generation efficiency of the first dialogue content.

[0062] In some cases, electronic device 110 can display a second graphic element on a first interface. The second graphic element is triggered to be displayed by the first object. For example, such as... Figure 2H As shown, electronic device 110 can present a first interface. The first interface can be, for example, interface 200H. Electronic device 110 can present a second graphic element in interface 200H. The second graphic element can represent a marker actively triggered by the first object and presented in the first interface during a dialogue interaction. It can be used to indicate at least a portion of the interface content to the user as a prompt. Such a second graphic element can be, for example, element 240. Element 240 can be triggered and displayed by the first object. As an example, when the first object determines that a graphic element is needed to indicate at least a portion of the interface content, it can trigger electronic device 110 to present the second graphic element in a specific area of ​​interface 200H. The second graphic element can indicate at least a portion of the interface content. For example, when it is determined that noise exists in the image of media content 220, the first object can determine that a graphic element is needed to indicate the noise in the image. The first object can then send parameters associated with the second graphic element to electronic device 110. Such parameters can, for example, indicate the position, style (e.g., arrow style), etc., of the second graphic element to be presented in the interface. After receiving these parameters, the electronic device 110 can display element 240 in the interface 200F to indicate noise in the image through element 240.

[0063] In this way, the examples in this paper can help users better perceive and understand the content output by the first object through the second graphical element, thereby effectively improving the efficiency of users in obtaining information.

[0064] In some cases, a second graphical element can indicate a second part of the interface content. Electronic device 110 can output second dialog content. The second dialog content originates from the first object. The second dialog content is related to the second graphical element or the second part. As an example, such as... Figure 2H As shown, electronic device 110 can output second dialogue content. The second dialogue content can correspond to an appropriate modality, such as audio, text, image, video, etc. For example, when the second dialogue content is audio, electronic device 110 can play the second dialogue content. When the second dialogue content is text, electronic device 110 can present the second dialogue content in an interactive component (e.g., component 235). The second dialogue content can be generated by a first object. The first object can generate the second dialogue content, for example, based on the second part of the interface content indicated by the second graphic element. For example, the second part indicated by the second graphic element is noise in the image. The first object can generate the second dialogue content based on the image corresponding to the noise. Such second dialogue content could be, for example, "There is obvious noise here; editing software can be used to remove the noise, or the overall tone can be adjusted to reduce the noise." In some scenarios, the first object can also generate the second dialogue content based on a second image element. For example, the second graphic element is an arrow element indicating noise in the image. The first object can generate the second dialogue content based on the style of the second graphic element. Such second dialogue content could be, for example, "As indicated by the arrow, there is noise here."

[0065] In this way, the examples in this paper can provide users with a second dialogue content related to the second graphic element or the second part, thereby providing a more accurate answer to the user's first topic and improving the efficiency of the user's information acquisition.

[0066] In some cases, the electronic device 110 not only supports the user in selecting at least a portion of the media content presented on the first interface, but also supports the user in selecting appropriate interface components on the first interface. As an example, such as... Figure 2IAs shown, electronic device 110 can present interface 200I. Electronic device 110 can receive a first operation via interface 200I. The first operation may be a selection operation on an appropriate interface component in interface 200I. After receiving the selection operation, electronic device 110 can also present a first graphic element (e.g., element 245). Further, the first object can generate response content based on the interface content indicated by element 245 and the user's interaction request (e.g., a query about the interface content). For example, the interaction request is "What is the use of this button?" The first object can generate a first response to the interaction request based on the first control indicated by element 245. The first response may be, for example, an explanation of the function of the first control. As another example, the interaction request is "There used to be a forward button here, but now it's gone." The first object can generate a second response to the interaction request based on the image indicated by element 245. The second response may be, for example, an explanation of the movement of the forward button and the current location of the forward button.

[0067] Figures 3A to 3H Example interfaces 300A to 300H are shown, illustrating interface interactions under various scenarios. Interfaces 300A to 300H can, for example, be... Figure 1 The electronic device 110 shown is provided.

[0068] In some cases, electronic device 110 may, in response to receiving a third operation, send second media content to the first object. Further, electronic device 110 may output third content. The third content originates from the first object. The third content is related to the second media content.

[0069] As an example, such as Figure 3AAs shown, electronic device 110 can present a second interface. The second interface can be interface 300A. Interface 300A can, for example, be a dialogue interface between a user and a first object. Electronic device 110 can receive a first message via interface 300A. This first message can, for example, indicate an operation guidance requirement. The first message can, for example, indicate "Page B is always not loading." After receiving the first message, electronic device 110 can present entry point 305 in interface 300A. When entry point 305 is clicked, electronic device 110 can present interface 300B. Interface 300B is used to present page B. Electronic device 110 can present interactive components associated with the first object in interface 300B. Such interactive components can, for example, be component 310. Electronic device 110 can receive a third operation via component 310. The third operation can be a selection or upload operation of second media content. The second media content can, for example, include video, images, audio, etc. After receiving the third operation, electronic device 110 can send the second media content to the first object. After the first object receives the second media content, it can generate third content based on the second media content and the first message. For example, the second media content includes the error code displayed on page B. The first object can then determine the reason why page B failed to load based on the error code, and generate third content based on that reason. After the third content is generated, electronic device 110 can output the third content. For example, electronic device 110 can play audio content indicating the reason why page B failed to load. Alternatively, electronic device 110 can present text content indicating the reason why page B failed to load in component 310.

[0070] In some cases, electronic device 110 may receive image content corresponding to the first interface in response to receiving a third operation. Furthermore, electronic device 110 may send the image content to the first object. For example, such as... Figure 3B As shown, electronic device 110 can display control 315 in component 310. When control 315 is clicked, as... Figure 3C As shown, the electronic device 110 can present a window 320 in the interface 300C. The window 320 includes a control 325. The control 325 can also be presented as a screenshot control. When the control 325 is clicked, the electronic device 110 can trigger a screenshot event to obtain image content corresponding to the first interface. Such image content can be a screenshot of the first interface. The image content corresponding to the first interface can include the completed interface content of the first interface. After obtaining the image content corresponding to the first interface, the electronic device 110 can send the image content to a first object to use the image content as context information. The first object can generate third content based on such image content.

[0071] In some scenarios, the first object can also autonomously trigger the acquisition of image content corresponding to the first interface. As an example, electronic device 110 can receive the user's fifth input content via component 310. After the fifth input content is received, the first object can perform semantic understanding on the fifth input content to determine the semantic information corresponding to it. When the semantic information corresponding to the fifth input content satisfies a first condition, the first object can actively trigger a screenshot event to acquire the image content corresponding to the first interface. For example, the fifth input content is "Look at the error code displayed on this page." The first object can determine the semantic information corresponding to the fifth input content; such semantic information may indicate the need to acquire the appropriate content displayed on the page (e.g., an error code). When the semantic information indicates the need to acquire the appropriate content displayed on the page, the first object can determine that the first condition is met. Furthermore, the first object can actively trigger a screenshot event. Before the screenshot event is triggered, such as... Figure 3D As shown, the electronic device 110 can present a content component. Such a content component could be, for example, component 325. The electronic device 110 can present an appropriate prompt in component 325, indicating that a screenshot event will be triggered. Furthermore, the electronic device 110 can also present multiple controls (e.g., controls 325-1 and 325-2) in component 325. These multiple controls can be used to determine whether authorization for screenshotting is granted. The electronic device 110 can receive a third operation via control 325-2. Such a third operation could be, for example, a click operation on control 325-2. After control 325-2 is clicked, the electronic device 110 can determine that the user has authorized screenshotting, and thus can receive the image content corresponding to the first interface.

[0072] In other scenarios, after the image content corresponding to the first interface is received, the electronic device 110 can not only output the third content, but also display the third graphic element corresponding to the third content. For example, such as... Figure 3E As shown, electronic device 110 can present third content in component 310. This third content can, for example, explain why page B cannot load and how to resolve this issue. This solution can be referred to as an operation guide, which instructs the user to perform appropriate actions to resolve the page B loading problem. Furthermore, electronic device 110 can also present a third graphic element in interface 300E. Such a third graphic element can be, for example, element 330. Element 330 can correspond to the solution for page B not loading in the third content. For example, the solution could instruct the user to click the refresh control in interface 300E to resolve the page loading problem. Therefore, element 330 can be used to instruct the user to click the refresh control in interface 300E.

[0073] Alternatively, the electronic device 110 can also demonstrate solutions to the problem of page B failing to load in steps. For example, the first object can first determine the first step in resolving the problem of page B failing to load, such a first step could be instructing the user to click the refresh control on page B. The electronic device 110 can display the graphic elements and text descriptions corresponding to the first step in the interface 300E. Furthermore, the first object can also determine the second step in resolving the problem of page B failing to load. Such a second step could be instructing the user to click the settings control and adjust the network configuration in the settings interface. After the second step is determined, the electronic device 110 can display the graphic elements and text descriptions corresponding to the second step in the interface 300E.

[0074] In this way, the example in this paper can send image content corresponding to the first interface to the first object, so as to use the first object to generate third content, which can effectively improve the generation quality of the third content.

[0075] In some cases, such as Figure 3C As shown, electronic device 110 can display control 335 in window 320. Control 335 can be referred to as a media content upload control. Electronic device 110 can receive a third operation via control 335. The third operation can be a click operation on control 335. After control 335 is clicked, as... Figure 3F As shown, electronic device 110 can present interface 300F. Interface 300F can be referred to as a media content upload interface, for example. Electronic device 110 can present multiple media contents in interface 300F, which may include media contents 335 and 336, for example. After at least one media content is selected, electronic device 110 can send at least one media content to a first object to trigger the first object to generate third content based on at least one media content. Such at least one media content may include second media content (e.g., media content 335).

[0076] In some cases, such as Figure 3GAs shown, the electronic device 110 also supports video calls between a user and a first object. As an example, the electronic device 110 can present an interface 300G. The electronic device 110 can present a component 310 in the interface 300G, which includes a control 340. The control 340 can be used to trigger a video call between the user and the first object. When the control 340 is clicked, the electronic device 110 can activate its associated image acquisition device (e.g., a camera). The electronic device 110 can acquire a video stream via the image acquisition device. After the video stream is acquired, the electronic device 110 can send the video stream to the first object to trigger the first object to interact with the user based on the video stream (e.g., generate corresponding content based on the video stream). Furthermore, during the video call between the user and the first object, the electronic device 110 can also present the screen corresponding to the video stream in the interface 300G. During the video call between the user and the first object, the electronic device 110 also supports the user to select or take screenshots of the screen in the video stream, providing the corresponding image content to the first object. The interactive process of selecting and taking screenshots of the video stream is similar to the process described above, and will not be elaborated on further here.

[0077] In some cases, such as Figure 3H As shown, electronic device 110 can present interface 300H. Interface 300H can be a dialogue interface between a user and a first object. Electronic device 110 can present the interactive content between the user and the first object during the dialogue interaction in interface 300H. Such interactive content may include, for example, content sent by the user to the first object, such as screenshots, text messages, voice messages, audio input received during the dialogue interaction, etc. For voice messages and audio input, electronic device 110 can transcribe the voice messages and audio input to determine the text corresponding to the voice messages and audio input. Further, electronic device 110 can present voice messages and audio input in text form in interface 300H. For example, electronic device 110 can present message 345-1 in interface 300H. Message 345-1 can be determined by electronic device 110 transcribing the audio input. In some scenarios, electronic device 110 can also present component 345-2 in interface 300H. Component 345-2 can indicate the screenshot received by the first object. In other scenarios, the electronic device 110 may also present message 345-3 on the interface 300H, such as a reply from the first object.

[0078] Figures 4A to 4D Example interfaces 400A to 400D are shown, illustrating interface interactions under various scenarios. Interfaces 400A to 400D can, for example, be... Figure 1The electronic device 110 shown is provided.

[0079] In some cases, electronic device 110 may present at least one service object. Electronic device 110 may receive a third operation. The third operation instructs the selection of a first service object from the at least one service object. Further, electronic device 110 may send the first service object to the first service object.

[0080] As an example, such as Figure 4A As shown, the electronic device 110 can present an interface 400A. Interface 400A includes interactive components associated with the first object. Such interactive components could be, for example, component 405. Component 405 includes a control 410. When control 410 is clicked, as... Figure 4B As shown, the electronic device 110 can present a window 415 in the interface 400B. The window 415 includes a control 420. The control 420 can be, for example, referred to as a business object sending control. Here, the business object can be an asset or entity with specific business meaning, which can be selected and sent by the user for interaction with a first object. Business objects can include, but are not limited to, works, orders, etc. After the control 420 is clicked, as... Figure 4C As shown, electronic device 110 can present interface 400C. Electronic device 110 can present at least one business object in interface 400C. This at least one business object can include objects 425-1 to 425-3. Taking a work as an example, electronic device 110 can present at least one work in interface 400C. This at least one work can be associated with the current user (e.g., a work published or forwarded by the current user) or with other users (e.g., a work published by another user). Electronic device 110 can receive a third operation via interface 400C. Such a third operation can be, for example, the selection of a first business object among the at least one business object. Such a first business object can be, for example, object 425-1. When object 425-1 is selected, electronic device 110 can send object 425-1 to the first object. After receiving object 425-1, the first object can generate third content based on the content of object 425-1. For example, the first object can analyze the color tone of object 425-1 and provide third content instructing for color tone adjustment. After the third content is generated, as... Figure 4D As shown, the electronic device 110 can display third content in component 405 of interface 400D. The third content could be, for example, "I have received your work, and the color tone of the work can be brightened a bit."

[0081] In this way, the example in this paper allows users to select at least one business object to send to a first object, triggering the first object to generate third content based on the selected at least one business object. This supports the first object to generate content based on multiple forms of content, which can provide users with richer information and improve the efficiency of information acquisition.

[0082] In some scenarios, the first interface displays a second business object. At least one first type of the business object is related to a second type of the second business object. For example, such as... Figure 4B As shown, electronic device 110 can present a second business object in a first interface (e.g., interface 400B). The second business object can be, for example, a work published by an appropriate user. Such a work can include, for example, videos, images, audio, etc. When control 420 is clicked, electronic device 110 can determine the second type of the second business object (e.g., work type, order type, etc.). After determining the second type, as... Figure 4C As shown, electronic device 110 can display at least one business object corresponding to a first type in interface 400C. This first type can, for example, be matched with a second type. For instance, when the second business object is a work, electronic device 110 can display at least one work in interface 400C. As another example, when the second business object is an order, electronic device 110 can display at least one order in interface 400C.

[0083] In this way, the example in this paper can output the first dialogue content from the first object and associated with the first graphical element, thereby providing users with more accurate first dialogue content and improving the efficiency of users obtaining information. Furthermore, the example in this paper allows users to trigger the presentation of the first graphical element and the content output from the first object through a first operation, simplifying the interaction process and reducing the consumption of computing resources.

[0084] Example process Figure 5 A flowchart illustrating an example process 500 for interface interaction under certain conditions is shown. Process 500 can be implemented at electronic device 110. See below for reference. Figure 1 To describe process 500.

[0085] like Figure 5 As shown, in box 510, electronic device 110 can trigger the presentation of a first interface, the content of which is provided for dialogue with a first object.

[0086] In box 520, electronic device 110 can respond to receiving a first operation by presenting a first graphical element on a first interface, the first operation being associated with a first object.

[0087] In box 530, electronic device 110 can output first dialogue content, which comes from the first object and is related to the first graphic element.

[0088] In some cases, triggering the presentation of the first interface includes: receiving a first message, the first message being related to a first topic; and presenting the first interface, wherein the first interface includes interface content associated with the first topic.

[0089] In this way, the example in this paper can provide the user with a first interface containing interface content related to the first topic after receiving the first message related to the first topic, thereby reducing the interaction chain that the user triggers to present the corresponding interface content and improving the efficiency of human-computer interaction.

[0090] In some cases, presenting the first interface includes: presenting a second interface, which is a dialogue interface with the first object; and receiving the first message on the second interface and presenting the first interface.

[0091] In some cases, presenting the first interface includes: presenting the first entry point in response to receiving the first message; and presenting the first interface in response to triggering the first entry point.

[0092] In some cases, presenting a first graphical element on a first interface includes: providing a canvas corresponding to the first interface; and presenting the first graphical element in response to receiving a first operation on the canvas.

[0093] In this way, the example in this article can provide a canvas corresponding to the first interface, and obtain the first operation through the canvas, which can effectively improve the efficiency of obtaining the first operation.

[0094] In some cases, providing a canvas corresponding to a first interface includes: presenting an interactive component in the first interface, the interactive component being associated with a first object; and providing a canvas in response to receiving a second operation associated with the interactive component.

[0095] In this way, the example in this paper allows users to manually trigger the provision of a canvas to perform operations such as selection and drawing, effectively avoiding various user errors and thus improving the human-computer interaction experience. Furthermore, by reducing user errors, the consumption of computing resources can be effectively reduced.

[0096] In some cases, process 500 may also include: stopping the rendering of interactive components in response to receiving a second operation; or displaying a prompt message in response to receiving a second operation, the prompt message being related to the canvas.

[0097] In this way, the example in this paper can stop presenting interactive components or present prompts after receiving the first operation, which can help users more easily indicate at least part of the interface content through the canvas, thereby improving the efficiency of human-computer interaction.

[0098] In some cases, providing a canvas corresponding to the first interface includes: providing a canvas in response to a dialogue interaction with the first object satisfying a triggering condition.

[0099] In this way, the example in this article supports the first object to actively trigger the provision of the canvas, thereby reducing the interaction chain of triggering the provision of the canvas, which can effectively improve the efficiency of human-computer interaction.

[0100] In some cases, the first interface is used to play the first media content, and process 500 also includes: in response to providing a canvas, pausing the playback of the first media content on the first interface.

[0101] In this way, the example in this paper can pause the playback of the first media content on the first interface when the canvas is provided, thereby enabling users to select at least a portion of the first media content more accurately and conveniently through the canvas, which can effectively improve the efficiency of human-computer interaction.

[0102] In some cases, the first dialogue content is related to the first part of the interface content, which is indicated by the first graphic element.

[0103] In this way, the example in this paper can use a first object to generate first dialogue content based on the first part indicated by the first graphical element, thereby effectively improving the generation efficiency of the first dialogue content.

[0104] In some cases, process 500 further includes: sending first image content to a first object to generate first dialogue content, the first image content including at least a portion of interface content and a first graphic element.

[0105] In this way, the example in this paper can send a first image content, including at least a portion of the interface content and a first graphical element, to a first object to trigger the first object to generate first dialogue content based on the first image content. This can effectively reduce the consumption of computing resources used to generate the first dialogue content and improve the generation efficiency of the first dialogue content.

[0106] In some cases, process 500 may also include: presenting a second graphic element on the first interface, the second graphic element being triggered for display by the first object.

[0107] In this way, the examples in this paper can help users better perceive and understand the content output by the first object through the second graphical element, thereby effectively improving the efficiency of users in obtaining information.

[0108] In some cases, the second graphical element indicates a second part of the interface content, and process 500 further includes: outputting second dialogue content, the second dialogue content being derived from the first object, and the second dialogue content being related to the second graphical element or the second part.

[0109] In this way, the examples in this paper can provide users with a second dialogue content related to the second graphic element or the second part, thereby providing a more accurate answer to the user's first topic and improving the efficiency of the user's information acquisition.

[0110] In some cases, process 500 further includes: during a dialogue interaction, in response to receiving a third operation, sending second media content to a first object; and outputting third content, which originates from the first object and is related to the second media content.

[0111] In this way, the example in this paper can send image content corresponding to the first interface to the first object, so as to use the first object to generate third content, which can effectively improve the generation quality of the third content.

[0112] In some cases, sending second media content to a first object in response to receiving a third operation includes: receiving image content corresponding to a first interface in response to receiving a third operation; and sending the image content to the first object.

[0113] In some cases, sending second media content to a first object in response to receiving a third operation includes: presenting at least one business object; receiving a third operation that instructs the selection of a first business object from at least one business object; and sending the first business object to the first object.

[0114] In this way, the example in this paper allows users to select at least one business object to send to a first object, triggering the first object to generate third content based on the selected at least one business object. This supports the first object to generate content based on multiple forms of content, which can provide users with richer information and improve the efficiency of information acquisition.

[0115] In some cases, the first interface displays a second business object, and at least one of the first types of the business object is related to the second type of the second business object.

[0116] Example devices and equipment A corresponding apparatus for implementing the above methods or processes is also provided. Figure 6 A schematic structural block diagram of an example device 600 for interface interaction is shown, according to some scenarios. Device 600 can be implemented as or included in electronic device 110. The various modules / components in device 600 can be implemented by hardware, software, firmware, or any combination thereof.

[0117] like Figure 6 As shown, the device 600 includes: a trigger module 610 configured to trigger the presentation of a first interface, the interface content of which is provided for dialogue interaction with a first object; a first presentation module 620 configured to present a first graphical element on the first interface in response to receiving a first operation, the first operation being related to the first object; and a first output module 630 configured to output first dialogue content, the first dialogue content originating from the first object and being related to the first graphical element.

[0118] In some cases, the trigger module 610 is also configured to: receive a first message, the first message being related to a first topic; and present a first interface, wherein the first interface includes interface content associated with the first topic.

[0119] In some cases, the trigger module 610 is also configured to: present a second interface, which is a dialogue interface with the first object; and receive the first message on the second interface and present the first interface.

[0120] In some cases, the trigger module 610 is also configured to: present a first entry point in response to receiving a first message; and present a first interface in response to triggering the first entry point.

[0121] In some cases, the first presentation module 620 is also configured to: provide a canvas corresponding to the first interface; and, in response to receiving a first operation in the canvas, present a first graphical element.

[0122] In some cases, the first presentation module 620 is also configured to: present an interactive component in a first interface, the interactive component being associated with a first object; and provide a canvas in response to receiving a second operation associated with the interactive component.

[0123] In some cases, device 600 also includes a stop module configured to: stop presenting interactive components in response to receiving a second operation; or present a prompt message related to the canvas in response to receiving a second operation.

[0124] In some cases, the first presentation module 620 is also configured to provide a canvas in response to a dialogue interaction with the first object satisfying a triggering condition.

[0125] In some cases, the first interface is used to play first media content, and the device 600 also includes a pause module configured to pause the playback of the first media content on the first interface in response to providing a canvas.

[0126] In some cases, the first dialogue content is related to the first part of the interface content, which is indicated by the first graphic element.

[0127] In some cases, the device 600 further includes a first sending module configured to send first image content to a first object to generate first dialogue content, the first image content including at least a portion of interface content and a first graphic element.

[0128] In some cases, device 600 also includes a second presentation module configured to: present a second graphic element on a first interface, the second graphic element being triggered for display by a first object.

[0129] In some cases, the second graphical element indicates a second part of the interface content, and the device 600 also includes a second output module configured to output second dialogue content, which is derived from the first object and is related to the second graphical element or the second part.

[0130] In some cases, the device 600 also includes a second sending module configured to: during a dialogue interaction, in response to receiving a third operation, send second media content to a first object; and output third content, the third content originating from the first object, and the third content being related to the second media content.

[0131] In some cases, the second sending module is also configured to: receive image content corresponding to the first interface in response to receiving a third operation; and send the image content to the first object.

[0132] In some cases, the second sending module is also configured to: present at least one service object; receive a third operation, the third operation indicating the selection of a first service object from at least one service object; and send the first service object to the first object.

[0133] In some cases, the first interface displays a second business object, and at least one of the first types of the business object is related to the second type of the second business object.

[0134] The modules included in device 600 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some cases, one or more modules can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units in device 600 can be implemented at least partially by one or more hardware logic components. By way of example, and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and so on.

[0135] Figure 7 A block diagram of an electronic device 700 in which one or more examples may be implemented is shown. It should be understood that... Figure 7 The electronic device 700 shown is merely exemplary and should not be construed as limiting the functionality and scope of the examples described herein. Figure 7 The illustrated electronic device 700 can be used to implement the electronic device 110 discussed above.

[0136] like Figure 7 As shown, electronic device 700 is in the form of a general-purpose electronic device. Components of electronic device 700 may include, but are not limited to, one or more processing units or processors 710, memory 720, storage devices 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. Processor 710 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 720. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 700.

[0137] Electronic device 700 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 720 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 730 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 700.

[0138] Electronic device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 7 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 720 may include computer program product 725 having one or more program modules configured to perform various methods or actions of various examples.

[0139] The communication unit 740 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 700 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 700 can operate in a networked environment using logical connections to one or more other servers, networked personal computers, or another network node.

[0140] Input device 750 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 760 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 700 can also communicate with one or more external devices (not shown) via communication unit 740 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 700, or with any device that enables electronic device 700 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0141] A computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. A computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0142] The flowcharts and / or block diagrams of the methods, apparatus, devices, and computer program products referred to herein describe various aspects. It should be understood that each block of the flowcharts and / or block diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0143] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0144] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0145] The flowcharts and block diagrams in the accompanying figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products under various scenarios. In this respect, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0146] Various examples have been described above. The foregoing descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for interface interaction, comprising: The first interface is triggered and its content is provided for dialogue with the first object. In response to receiving a first operation, a first graphical element is presented on the first interface, wherein the first operation is related to the first object; as well as Output the first dialogue content, which comes from the first object and is related to the first graphic element.

2. The method according to claim 1, wherein triggering the presentation of the first interface includes: Receive the first input message, which is related to the first topic; as well as The first interface is presented, wherein the first interface includes the interface content associated with the first topic.

3. The method according to claim 2, wherein presenting the first interface comprises: A second interface is presented, which is a dialogue interface with the first object; as well as The first message is received on the second interface, and the first interface is displayed.

4. The method of claim 3, wherein presenting the first interface comprises: Upon receiving the first message, the first entry point is presented; as well as In response to the triggering of the first entry point, the first interface is displayed.

5. The method according to claim 1, wherein presenting the first graphical element on the first interface includes: Provide a canvas corresponding to the first interface; as well as In response to receiving the first operation in the canvas, the first graphic element is rendered.

6. The method of claim 5, wherein providing the canvas corresponding to the first interface comprises: In the first interface, interactive components are presented, and the interactive components are associated with the first object; as well as In response to receiving a second operation associated with the interactive component, the canvas is provided.

7. The method of claim 6, further comprising: In response to receiving the second operation, the presentation of the interactive component is stopped; or In response to receiving the second operation, a prompt message is presented, which is related to the canvas.

8. The method according to claim 5, wherein providing the canvas corresponding to the first interface comprises: The canvas is provided in response to the dialogue interaction with the first object satisfying the triggering condition.

9. The method according to claim 5, wherein the first interface is used to play first media content, the method further comprising: In response to providing the canvas, pause playback of the first media content on the first interface.

10. The method of claim 1, wherein the first dialogue content is related to a first portion of the interface content, the first portion being indicated by the first graphical element.

11. The method according to claim 1, further comprising: Send first image content to the first object to generate the first dialogue content, the first image content including at least a portion of the interface content and the first graphic element.

12. The method according to claim 1, further comprising: On the first interface, a second graphic element is presented, which is triggered by the first object.

13. The method of claim 12, wherein the second graphical element indicates a second portion of the interface content, the method further comprising: Output a second dialogue content, which is derived from the first object and is related to the second graphic element or the second part.

14. The method of claim 1, further comprising: In response to receiving a third operation, send the second media content to the first object; as well as Output a third dialogue content, which is derived from the first object and is related to the second media content.

15. The method of claim 14, wherein sending the second media content to the first object in response to receiving the third operation comprises: In response to receiving the third operation, the image content corresponding to the first interface is received; as well as Send the image content to the first object.

16. The method of claim 14, wherein sending the second media content to the first object in response to receiving the third operation comprises: Present at least one business object; Receive the third operation, the third operation indicating the selection of a first business object among the at least one business objects; as well as Send the first business object to the first object.

17. The method of claim 16, wherein the first interface displays a second business object, and the first type of the at least one business object is related to the second type of the second business object.

18. A device for interface interaction, comprising: The trigger module is configured to trigger the presentation of the first interface, the content of which is provided for dialogue with the first object. The first presentation module is configured to present a first graphical element on a first interface in response to receiving a first operation, wherein the first operation is related to a first object; as well as The first output module is configured to output the first dialogue content, which comes from the first object and is related to the first graphical element.

19. An electronic device comprising: At least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 17 when executed by the at least one processor.

20. A computer program product tangibly stored in a computer storage medium and comprising computer-executable instructions that, when executed by a device, cause the device to perform the method according to any one of claims 1 to 17.

Images