Interaction method, device and equipment of graphical interface and storage medium

Abstract

A graphical user interface interaction method, apparatus, device, and storage medium are provided. The proposed method includes: displaying a workflow on a first interface, the workflow including a first node, a second node, and a preview node, wherein the first node and the second node are input nodes of the preview node; and, in response to the workflow being executed, displaying a first output of the first node in a first area of ​​the preview node and a second output of the second node in a second area of ​​the preview node, wherein the first output and the second output are of different types. This approach effectively improves the efficiency of information retrieval.

Description

Technical Field

[0001] The examples in this paper generally relate to the field of computers, and in particular to interactive methods, devices, apparatuses, and computer-readable storage media with graphical interfaces. Background Technology

[0002] With the development of computer technology, people can edit and publish various types of applications through various platforms. Workflow editing is an important task in the application development process. For example, users can automate processing flows by editing workflows. How to improve the efficiency of users obtaining information within workflows is a key concern. Summary of the Invention

[0003] In a first aspect, a graphical interface interaction method is provided. The method includes: displaying a workflow on a first interface, the workflow including a first node, a second node, and a preview node, the first node and the second node being input nodes of the preview node; and in response to the workflow being executed, displaying a first output of the first node in a first area of ​​the preview node, and displaying a second output of the second node in a second area of ​​the preview node, the first output and the second output being of different types.

[0004] In a second aspect, an interactive device for a graphical user interface is provided. The device includes: a first display module configured to display a workflow on a first interface, the workflow including a first node, a second node, and a preview node, the first node and the second node being input nodes of the preview node; and a second display module configured to, in response to the execution of the workflow, display a first output of the first node in a first area of ​​the preview node, and display a second output of the second node in a second area of ​​the preview node, the first output and the second output being of different types.

[0005] In a third aspect, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. When executed by the at least one processor, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions that can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect, a computer program product is provided, which is tangibly stored in a computer storage medium and includes computer-executable instructions that, when executed by a device, cause the device to perform the method of the first aspect.

[0008] This method can effectively improve the efficiency of information acquisition.

[0009] It should be understood that the content described in this section is not intended to limit the key or important features of the examples in this article, nor is it intended to restrict the scope of the solution. Other features will become readily apparent from the following description. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the various examples herein will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. In the accompanying drawings, the same or similar reference numerals denote the same or similar elements, wherein: Figure 1 A schematic diagram of the example environment is shown; Figures 2A to 2K Example interfaces for some scenarios are shown; Figure 3 The flowcharts show some examples of interactive processes in a graphical interface. Figure 4 Schematic block diagrams of interactive example devices for graphical user interfaces are shown in some scenarios; and Figure 5 A block diagram of an electronic device capable of implementing multiple illustrative scenarios is shown. Detailed Implementation

[0011] The examples in this document will now be described in more detail with reference to the accompanying drawings. While some examples are shown in the drawings, it should be understood that solutions can be implemented in various forms and should not be construed as limited to the examples presented herein. Rather, these examples are provided to provide a more thorough and complete understanding of the solutions. It should be understood that the drawings and examples in this document are for illustrative purposes only and are not intended to limit the scope of protection of the solutions.

[0012] It should be noted that the headings of any section / subsection provided herein are not restrictive. Various examples are described throughout this document, and examples of any type may be included under any section / subsection. Furthermore, examples described in any section / subsection may be combined in any way with any other examples described in the same section / subsection and / or different sections / subsections.

[0013] In the description of the examples in this document, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "an example" or "the example" should be understood as "at least one example". The term "some examples" should be understood as "at least some examples". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0014] The examples in this document may involve user data, data acquisition, and / or use. All of these aspects comply with relevant laws, regulations, and rules. In the examples, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, when implementing each example, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained through appropriate means, in accordance with relevant laws and regulations. The specific methods of notification and / or authorization can vary depending on the actual situation and application scenario; the scope of the solution is not limited in this regard.

[0015] In this manual and the sample solutions, any processing of personal information will be conducted only under legal grounds (such as obtaining the consent of the data subject or being necessary for the performance of a contract) and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0016] As mentioned above, traditionally, users can create workflows by configuring different processing nodes within the workflow. However, once a traditional workflow is executed, it is difficult for users to present multiple types of input through a single preview node, thus affecting the efficiency of information retrieval within the workflow.

[0017] A graphical user interface (GUI) interaction scheme is proposed. The scheme includes: displaying a workflow on a first interface, the workflow comprising a first node, a second node, and a preview node, wherein the first node and the second node are input nodes of the preview node; and, in response to the workflow being executed, displaying the first output of the first node in a first area of ​​the preview node and the second output of the second node in a second area of ​​the preview node, wherein the first output and the second output are of different types.

[0018] In this way, preview nodes associated with workflow outputs can be added to the first interface, allowing multiple types of input to be presented in different areas of the preview node, thereby improving the efficiency of users obtaining information in the workflow.

[0019] The following describes various examples of this scheme in further detail with reference to the accompanying drawings.

[0020] Example Environment Figure 1 A schematic diagram of example environment 100 is shown. (e.g.) Figure 1 As shown, example environment 100 may include electronic device 110.

[0021] In this example environment 100, electronic device 110 may run an interactive application 120 that supports a graphical user interface. Application 120 may be any suitable type of interactive application for a graphical interface, including but not limited to: workflow applications, development applications, browsers, or other suitable applications. User 140 may interact with application 120 via electronic device 110 and / or its attached devices.

[0022] exist Figure 1 In environment 100, if application 120 is active, electronic device 110 can use application 120 to present interactive interface 150 for supporting graphical interface.

[0023] In some cases, electronic device 110 communicates with server 130 to provide services to application 120. Electronic device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some cases, electronic device 110 can also support any type of user-facing interface (such as "wearable" circuitry).

[0024] Server 130 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Server 130 may include, for example, computing systems / servers such as mainframes, edge computing nodes, computing devices in a cloud environment, etc. Server 130 can provide backend services for interactive applications 120 that support graphical interfaces in electronic devices 110.

[0025] A communication connection can be established between server 130 and electronic device 110. This communication connection can be established via wired or wireless means. The communication connection can include, but is not limited to, Bluetooth, mobile network, Universal Serial Bus (USB), and Wireless Fidelity (WiFi) connections. In some cases, server 130 and electronic device 110 can exchange signaling information through their communication connection.

[0026] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of the scheme.

[0027] The following description of the example will continue with reference to the accompanying drawings.

[0028] Example Interaction Figures 2A to 2K Examples of first interfaces 200A to 200K according to some scenarios are shown. First interfaces 200A to 200K can, for example, be... Figure 1 The electronic device 110 shown is provided. It is understood that the first interfaces 200A to 200K are merely illustrative examples and are not intended to limit their specific style.

[0029] like Figure 2A As shown, the electronic device 110 can present a first interface 200A for displaying a workflow. In some cases, such a first interface 200A may be an interface that includes existing nodes, or it may be a blank interface that does not include nodes. In some cases, such a first interface 200A may also be referred to as a canvas interface.

[0030] In some cases, electronic device 110 can add nodes to such a first interface 200A based on user operations to create workflows. Such users may include, but are not limited to, ordinary users, developer users, etc.

[0031] In some contexts, workflows can be used to define a series of steps or tasks within an automated business process. These steps or tasks can be implemented based on nodes with different and / or similar functionalities, such as, but not limited to, text generation, image generation, and video generation. It is understood that this article does not aim to limit the specific functions of the nodes.

[0032] In some scenarios, two related nodes in a workflow can be connected by any appropriate method, such as a straight line, polyline, or curve, while two unrelated nodes may not be connected. In other scenarios, two related nodes in a workflow can be in a sequential calling relationship. For example, during workflow execution, the first node in the workflow can be called first, followed by subsequent nodes, such as nodes directly connected to the first node.

[0033] In some cases, the electronic device 110 may present an add control 201 on the first interface 200A. This add control 201 can be used to add nodes on the first interface, for example, to preview nodes. As an example, in response to a user's selection of the add control 201, the electronic device 110 may present an interactive panel 202 on the first interface 200A. Furthermore, the electronic device 110 may add nodes on the first interface 200A based on the interactive panel 202.

[0034] For example, in the interactive panel 202, the electronic device 110 can present multiple interface elements corresponding to various node types, such as interface element 203-1, interface element 203-2, and interface element 203-3, etc. (referred to individually or collectively as interface element 203). As an example, interface element 203-1 can indicate the addition of a preview node, interface element 203-2 can indicate the addition of a text generation node, interface element 203-3 can indicate the addition of a video generation node, and so on. It is understood that, in addition to the specific nodes mentioned above, the addition of other appropriate types of nodes can also be supported. For example, the electronic device 110 can support users to treat other created workflows as single processing nodes.

[0035] In some cases, the electronic device 110 may also display a search control 204 in the interactive panel 202. As an example, the electronic device 110 may receive keywords (such as a first node) via the search control 204. Furthermore, in response to receiving a keyword, the electronic device 110 may display interface elements matching that keyword in the interactive panel 202. Additionally or alternatively, such interface elements may be, for example, elements with a matching degree to the keyword greater than a threshold.

[0036] In some cases, electronic device 110 can receive user selections of interface elements and add corresponding nodes to the first interface 200A. As an example, electronic device 110 can receive user selections of interface element 203-1, presenting the following... Figure 2B The first interface 200B shown here allows the electronic device 110 to display added preview nodes 210 (also known as canvas nodes). In some cases, such preview nodes 210 can be used to receive and present various types of input.

[0037] In some cases, electronic device 110 may present input port 205 in preview node 210, such input port 205 being used to receive input from preview node 210. In some cases, electronic device 110 may present multiple input ports in response to receiving a selection for input port 205, such as input port 206-1, input port 206-2, input port 206-3, and input port 206-4.

[0038] In some cases, such multiple input ports can be associated with multiple input types, which may include, but are not limited to, image types, video types, audio types, and text types. It is understood that this article does not aim to limit the specific content of these multiple input types.

[0039] As an example, each of these multiple input ports can be used to receive input of a corresponding type. For instance, input port 206-1 can be used to receive image input, input port 206-2 can be used to receive video input, input port 206-3 can be used to receive audio input, input port 206-4 can be used to receive text input, and so on. It is understood that this is not intended to limit the number of output ports for the preview node 210.

[0040] In some cases, the workflow may also include a first node and a second node, such as indicating the input node of preview node 210. For example, the first output of the first node may be provided to preview node 210, and the second output of the second node may be provided to preview node 210. In some cases, such a first node and a second node correspond to different output types. In other cases, the first node and the second node may have multiple output types. For ease of description, the following description uses the example of the first output and the second output each corresponding to one type of output.

[0041] As an example, the output type of such a first node can be any one of image, video, audio, or text types, and the output type of such a second node can be a different type than the output type of the first node. For example, the output type of the first node can be image, and the output type of the second node can be video, audio, or text.

[0042] refer to Figure 2C The electronic device 110 can display a first node 211 and a second node 212 in a first interface 200C. Such a first node 211 and a second node 212 can, for example, indicate a node directly connected to the preview node 210. Such a first output of the first node 211 can be provided to the preview node 210, and a second output of the second node 212 can be provided to the preview node 210.

[0043] It is understandable that although this article refers to the first node and the second node, it is also understandable that this article may include the third node, the fourth node, etc. This article does not aim to limit the function or number of input nodes of the preview node 210, but aims to illustrate that the preview node 210 includes at least two types of input.

[0044] In some cases, the first node 211 and the second node 212 may also include corresponding aggregated output ports. In some cases, in response to connecting the aggregated output ports corresponding to the first node 211 and the second node 212 to the output port of the preview node 210 respectively, the electronic device 110 may display a first connection element 270 connecting the first node 211 and the preview node 210, and a second connection element 271 connecting the second node 212 and the preview node 210 in the first interface 200C. In some cases, such connection elements may be presented in the form of, for example, straight lines, polylines, or curves.

[0045] In some cases, such a first connection element 270 can be connected to the first input port 206-1 of the preview node 210, and such a second connection element 271 can be connected to the second input port 206-2 of the preview node 210, with the first and second input ports corresponding to different types of inputs.

[0046] In some cases, such a first connection element 270 can indicate that the first output of the first node 211 is provided as the input of the preview node 210. Such a second connection element 271 can indicate that the second output of the second node 212 is provided as the input of the preview node 210.

[0047] For example, in response to connecting the aggregated output port of the first node 211 to the input port 206-1 of the preview node 210, the electronic device 110 can instruct the image output by the first node 211 to be used as the input of the preview node 210. Similarly, in response to connecting the aggregated output port of the second node 212 to the input port 206-2 of the preview node 210, the electronic device 110 can instruct the video output by the second node 212 to be used as the input of the preview node 210, and so on. Further details are omitted here.

[0048] It is understood that the above description takes the aggregated output port of the first node 211 and the second node 212 as an example. The first node 211 and the second node 212 may also include at least one output port. The electronic device 110 may also present the corresponding connection element based on the connection between at least one output port and the aggregated input port or at least one input port of the preview node 210.

[0049] The above description uses the example of the first node 211 and the second node 212 each corresponding to one different type of output. In other cases, a single input node may also include multiple types of outputs; see reference [link / reference]. Figure 2C When the second node 212 includes multiple output types (e.g., video type, audio type), connection elements 271 and 272 can be presented to indicate that the video type and audio type outputs are provided to the preview node 210.

[0050] For easier viewing, Figures 2D to 2K Instead of showing the connection between the aggregated output port of node 211 and the aggregated input port 205 of preview node 210, the connection between multiple ports will no longer be shown.

[0051] In some cases, electronic device 110 can also delete the connection between the corresponding output port and input port based on the user's deletion operation on the connection element.

[0052] In some cases, in response to the execution of a workflow, electronic device 110 may display the first output of the first node 211 in the first area of ​​preview node 210 and the second output of the second node 212 in the second area of ​​preview node 210, such that the first output and the second output are of different types.

[0053] As an example, in response to receiving an interactive operation associated with preview node 210, electronic device 110 can present various types of input in preview node 210, such as a first output of first node 211 and a second output of second node 212. (See reference) Figure 2CIn some cases, electronic device 110 can be associated with preview node 210 to provide preset controls 214. Furthermore, in response to receiving a user's click on the preset controls 214, electronic device 110 can trigger workflow execution. In some cases, in response to workflow execution, electronic device 110 can present different types of areas in preview node 210 based on the different output types of the first node 211 and the second node 212, and present the corresponding type of output result in each area. In other words, the number of areas in preview node 210 can be determined based on the number of output types.

[0054] For example, assuming the output type of the first node 211 is an image, the image area can be presented in the preview node 210, and the output result of the image type can be presented. Assuming the output type of the second node 212 is a video, the video area can be presented in the preview node 210, and the output result of the video type can be presented.

[0055] refer to Figure 2D In response to the execution of the workflow, electronic device 110 can present, as follows: Figure 2D The first interface 200D is shown. In the first interface 200D, the electronic device 110 can display the first output of the first node 211 (e.g., image a, image b, and image c) in the first area 222 of the preview node 210, and the second output of the second node 212 (e.g., video a, video b, and video c) in the second area 223. Additionally or alternatively, the electronic device 110 can also display other outputs of other input nodes (e.g., audio a, audio b, and audio c) in the third area 224 of the preview node 210.

[0056] As described above, the first node 211 and the second node 212 can have different types of output. In other cases, the first node 211 and the second node 212 themselves can also have different types of output. For example, the first node 211 can generate different types of output (e.g., image type and video type), and the first node 212 can also generate different types of output (e.g., image type and text type).

[0057] In some cases, the input node of preview node 210 may also include a third node. Electronic device 110 may display a third output of the same type as the first output of the first node 211 in a first area, such a third output may come from the second node 212 or the third node.

[0058] As an example, a region in preview node 210 can correspond to a type of input. If the first region displays the first type of input, the same type of output input from other nodes (such as the second node 212 or the third node) can also be displayed in the first region.

[0059] Assume that the first node 211, the second node 212, and the third node all include image-type outputs, and output image a, image b, and image c respectively. (Reference) Figure 2D The electronic device 110 can display image a from the first node 211, image b from the second node 212, and image c from the third node in the first region 222.

[0060] In some cases, the presentation order of these multiple regions in preview node 210 can be determined based on the output type. For example, the presentation order of the regions can be determined based on the reception time of the corresponding output type; the earlier the reception time, the earlier the corresponding region is presented.

[0061] refer to Figure 2C Assuming the first output type received by preview node 210 is an image, electronic device 110 can display the corresponding image area 222 in preview node 210. Assuming the second output type received by preview node 210 is a video, electronic device 110 can display the corresponding video area 223 in preview node 210. Assuming the third output type received by preview node 210 is an audio, electronic device 110 can display the corresponding audio area 224 in preview node 210, and so on.

[0062] In order to improve the efficiency of users' information acquisition, in some cases, electronic device 110 can filter the various types of input presented in preview node 210 based on the filtering operation associated with the user and preview node 210, so as to present input that meets the user's needs.

[0063] In some cases, electronic device 110 may be associated with preview node 210 to present multiple options, which may indicate various types of input to preview node 210. Furthermore, in response to receiving a user's selection of a first option (corresponding to a second type) among the multiple options, electronic device 110 may stop presenting input that does not match the second type at preview node 210.

[0064] As an example, assuming that preview node 210 includes input of image type, video type, audio type and text type, electronic device 110 can be associated with preview node 210 to provide corresponding multiple options, such as option 1 corresponding to image type, option 2 corresponding to video type, option 3 corresponding to audio type and option 4 corresponding to text type. Furthermore, in response to receiving the user's selection of option 1 among the multiple options, electronic device 110 can present image type input in preview node 210 and stop presenting other types of input.

[0065] To reduce user operating costs, in some cases, the electronic device 110 can be associated with the preview node 210 to present a second control. Furthermore, based on the user's triggering operation on the second control, the electronic device 110 can download various types of input presented in the preview node 210, such as the first output of the first node 211 and the second output of the second node 212. Such triggering operations can, for example, indicate action instructions, such as click operations, swipe operations, long press operations, etc.

[0066] refer to Figure 2D The electronic device 110 can be associated with the preview node 210 to present the second control 213. Furthermore, based on the user's trigger operation on the second control 213, the electronic device 110 can download multiple types of input presented in the preview node 210 at once, such as image a to image c, video a to video c, and audio a to audio c.

[0067] In some cases, electronic device 110 may display content items of a first type in a first area of ​​preview node 210. For example, electronic device 110 may present image-type content items, such as image a, image b, and image c, in area 222 of preview node 210. Electronic device 110 may present video-type content items, such as video a, video b, and video c, in area 223 of preview node 210. Electronic device 110 may present audio-type content items, such as audio a, audio b, and audio c, in area 224 of preview node 210. It is understood that this document is not intended to limit the specific number of content items.

[0068] In some cases, the various areas of preview node 210 can be associated with multiple view modes. Such view modes can, for example, indicate the arrangement of content items within the area, and different view modes can correspond to different arrangements. As an example, such view modes may include, but are not limited to, a first view mode and a second view mode.

[0069] In some cases, the electronic device 110, in response to input from the preview node 210, may display multiple content items of a first type in a first area, in a first view mode. In some cases, such a first view mode may, for example, indicate a tiled view mode, indicating that multiple content items are presented simultaneously in the first area.

[0070] refer to Figure 2DThe electronic device 110, in response to input from the preview node 210 including multiple content items of the image type, can simultaneously display image a, image b, and image c in area 222. The electronic device 110, in response to input from the preview node 210 including multiple content items of the video type, can simultaneously display video a, video b, and video c in area 222. The electronic device 110, in response to input from the preview node 210 including multiple content items of the audio type, can simultaneously display audio a, audio b, and audio c in area 222.

[0071] In some cases, in response to receiving a first operation associated with preview node 210, electronic device 110 may display multiple content items in a first area in a second view mode. Such a second view mode may, for example, indicate an aggregated view mode, indicating that one of the multiple content items is displayed at a time in the first area.

[0072] In some cases, such a first operation can be used to adjust the display size of the preview node 210. In some cases, the electronic device 110 displays multiple content items in a second view mode in response to the display size of the preview node 210 being adjusted to meet the triggering conditions.

[0073] As an example, the electronic device 110 may adjust the arrangement of multiple content items in the first area in response to the size of the preview node 210 being adjusted to a preset threshold, and switch the first area from a first view mode to a second view mode, so as to display one of the multiple content items in the first area at a time.

[0074] For example, in response to receiving a zoom operation from a user on the preview node 210, the electronic device 110 can dynamically adjust the arrangement of multiple content items in the first area of ​​the preview node 210.

[0075] refer to Figure 2D In response to receiving a zoom-in operation (e.g., drag-to-zoom) from a user on the preview node 210, the electronic device 110 can display as follows: Figure 2E The first interface 200E is shown. In the first interface 200E, the electronic device 110 can display a magnified first area. For example, areas 222 to 224 have a larger display area compared to before the magnification operation. In some cases, the electronic device 110 can adaptively adjust the content items displayed in the first area to match the display area of ​​the corresponding area.

[0076] As another example, electronic device 110 may be associated with displaying a first control in a first area. In response to receiving a trigger operation on the first control, electronic device 110 may display multiple content items in the first area in a second view mode.

[0077] refer to Figure 2D Taking region 222 in preview node 210 as an example, the view mode in region 222 is the first view mode. In some cases, electronic device 110 can be associated with region 222 to provide a first control 226. Furthermore, in response to receiving a user's selection of the first control 226, electronic device 110 can present, as shown below... Figure 2F The first interface 200F is shown. In the first interface 200F, the electronic device 110 can present a second view mode in area 222 to display one of a plurality of content items (e.g., image a) at a time in area 222.

[0078] Continue to refer to Figure 2D Taking region 223 in preview node 210 as an example, the view mode in region 223 is the first view mode. In some cases, electronic device 110 can be associated with region 223 to provide a first control 227. In response to receiving a user's selection of the first control 227, electronic device 110 can present, as shown below... Figure 2G The first interface 200G is shown. In the first interface 200G, the electronic device 110 can present a second view mode in area 223 to display one of a plurality of content items (e.g., video a) at a time in area 223.

[0079] In other scenarios, the electronic device 110 can also switch the second view mode to the first view mode based on a trigger operation on the first control. For user convenience, as an example, the electronic device 110 can be associated with the first control to display corresponding identification information. For example, in response to the current view mode being the first view mode, the electronic device 110 can be associated with the first control to display corresponding identification information, such as a tiled view. In response to the current view mode being the second view mode, the electronic device 110 can be associated with the first control to display corresponding identification information, such as an aggregated view.

[0080] In some cases, the first or second output received by preview node 210 may include multiple visual contents; in other cases, such multiple visual contents may include inputs of the same type. Alternatively, such multiple visual contents may also include inputs of different types. Such visual contents may include, for example, images, videos, tables, text, etc. It is understood that this is not intended to limit the specific content of the visual contents.

[0081] In some cases, electronic device 110 may present the first visual content among multiple visual content items in the fourth area of ​​preview node 210. To enable an immersive preview of the visual content for the user, in some cases, electronic device 110 may present a viewing component and display the first visual content in the viewing component in response to receiving a trigger operation on the first visual content. In some cases, such a fourth area may be the same as or different from the first and second areas. For ease of description, the following description continues to use areas 222 and 223 as examples.

[0082] As an example, in response to receiving a click operation (e.g., a single click or double click) on the first visual content, the electronic device 110 can present the first visual content in a viewing component. Alternatively, the electronic device 110 can provide a preset control associated with the first visual content, and in response to receiving a selection of the preset control, the electronic device 110 can present the first visual content in a viewing component.

[0083] In some cases, the electronic device 110 can present a corresponding viewing component based on the type of the first visual content. For example, if the type of the first visual content is an image, the electronic device 110 can present a corresponding image viewing component. If the type of the first visual content is video, the electronic device 110 can present a corresponding video viewing component.

[0084] refer to Figure 2F The electronic device 110 can present image a from a set of images in region 222. As an example, such image a can be presented, for instance, in a second view mode. Furthermore, in response to receiving a trigger operation for image a, the electronic device 110 can present, as... Figure 2H The first interface 200H is shown. In the first interface 200H, the electronic device 110 can display the image viewing component 230, and the image a can be displayed in the image viewing component 230.

[0085] refer to Figure 2G Electronic device 110 can display video a from a set of videos in region 223. As an example, such video a can be displayed, for instance, in a second view mode. Furthermore, in response to receiving a trigger operation for video a, electronic device 110 can display, as... Figure 2I The first interface 200I is shown. In the first interface 200I, the electronic device 110 can display the video viewing component 240, and the video viewing component 240 can display video a.

[0086] To provide quick editing and control of visual content and improve interaction efficiency, in some cases, the electronic device 110 can be associated with a viewing component to provide a first set of interactive controls corresponding to the first visual content. Furthermore, the electronic device 110 can, based on this first set of interactive controls, enable the user to perform editing and control operations on the first visual content. In some cases, the electronic device 110 can provide different interactive controls based on the type of different viewing components.

[0087] refer to Figure 2H The electronic device 110 can be associated with the image viewing component 230 to present a set of interactive controls 231 corresponding to image a. This set of interactive controls 231 includes, but is not limited to: switching controls, zoom controls, scaling controls, rotation controls, and download controls. It is understood that the above examples are merely illustrative and are not intended to limit the specific functions of a set of interactive controls.

[0088] refer to Figure 2I The electronic device 110 can be associated with the video viewing component 240 to present a set of interactive controls 241 corresponding to the video a. Such a set of interactive controls 241 includes, but is not limited to: switching controls, playback controls, progress bar controls, volume controls, download controls, and full-screen controls.

[0089] It is understood that the above examples are merely illustrative and are not intended to limit the specific functions of the first set of interactive controls.

[0090] In some cases, in response to an interactive operation associated with a viewing component, the electronic device 110 may present a second visual content among multiple visual contents in the viewing component. Such a second visual content may, for example, be visual content indicating whether it precedes or follows the first visual content. Such a second visual content may have the same type as the first visual content, such as an image or video, etc.

[0091] As one example, in response to receiving a selection for a toggle control, electronic device 110 can display a second visual content among multiple visual content items in a viewing component. As another example, in response to receiving a user's swipe operation on a preset area of ​​the viewing component, electronic device 110 can display a second visual content among multiple visual content items in the viewing component.

[0092] To improve user interaction efficiency, in some cases, in response to receiving an interactive operation on the first visual content, the electronic device 110 may display a second set of interactive controls in the area corresponding to the first visual content. Furthermore, the electronic device 110 may switch the first visual content to the second visual content based on a user's triggering operation on the second set of interactive controls. In some cases, such interactive operations may, for example, instruct a mouse pointer hovering over the first visual content or other operations.

[0093] refer to Figure 2J In response to receiving an interactive operation for image a, electronic device 110 may present a set of controls, such as controls 250 and 251, in the area 222 where image a is located, for presenting image c or image b from a set of images in the area 222.

[0094] refer to Figure 2K In response to receiving an interactive operation for video a, electronic device 110 may present a set of controls, such as controls 260 and 261, in the area 223 where video a is located, for presenting video c or video b from a set of videos in the area 223.

[0095] It is understood that the above is merely an illustrative example, and the electronic device 110 may present different interactive controls based on the different input types in the area.

[0096] return Figure 2D In some cases, electronic device 110 can also publish the workflow based on a trigger on publishing control 225. Such a workflow can be published or shared to other users, for example. In some cases, such a workflow can be published independently, or it can trigger the creation and / or publication of applications based on the workflow, such as robot programs or intelligent agents.

[0097] In this way, preview nodes associated with workflow execution results can be added to the first interface, allowing multiple types of input to be presented in multiple areas of this preview node, thereby improving the efficiency of users obtaining information in the workflow.

[0098] Example process Figure 3 A flowchart illustrating an example interactive process 300 using a graphical interface under certain conditions is shown. Process 300 can be implemented at electronic device 110. See below for reference. Figure 1 To describe process 300.

[0099] like Figure 3 As shown in box 310, the electronic device 110 displays a workflow on the first interface. The workflow includes a first node, a second node, and a preview node. The first node and the second node are input nodes of the preview node.

[0100] In box 320, in response to the execution of the workflow, electronic device 110 displays the first output of the first node in the first area of ​​the preview node and the second output of the second node in the second area of ​​the preview node. The first output and the second output are of different types.

[0101] In some cases, process 300 further includes: in a first area, displaying a third output, the third output being from a second node or a third node, the first output and the third output being of the same type, and the third node being an input node to the preview node.

[0102] In this way, the first area can display inputs of the same type as those displayed in the first area, which are entered by other input nodes of the preview node. This makes it easier for users to view multiple inputs of the same type, thereby improving the efficiency of information acquisition for users.

[0103] In some cases, the first area is used to display content items of a first type, and process 300 further includes: in response to input from the preview node including multiple content items of the first type, displaying multiple content items in the first area in a first view mode; receiving a first operation associated with the preview node; and in response to the first operation, displaying multiple content items in the first area in a second view mode.

[0104] In this way, multiple arrangements of content items can be presented in the first area, thus providing users with a richer visual experience.

[0105] In some cases, the first view mode or the second view mode includes: a tiled view mode, which indicates that multiple content items are presented simultaneously in a first area; or an aggregated view mode, which indicates that one of the multiple content items is displayed at a time in a first area.

[0106] In some cases, receiving a first operation includes: displaying a first control associated with a first region; and receiving a trigger operation on the first control.

[0107] In some cases, the first operation is used to adjust the display size of the preview node, and in response to the first operation, displaying multiple content items in a second view mode in a first area includes: in response to the display size of the preview node being adjusted to meet the triggering condition, displaying multiple content items in a second view mode.

[0108] In this way, the view mode of the first area and the display size of the preview node can be switched based on the adjustment of the display size of the preview node, thereby reducing the user's operating cost and improving the user's interaction efficiency.

[0109] In some cases, process 300 further includes: presenting multiple options indicating multiple types of input to the preview node; receiving a selection of a first option among the multiple options, the first option corresponding to a second type; and stopping the presentation of input that does not match the second type at the preview node.

[0110] In this way, it is possible to filter the target type of input from multiple types of input, thereby improving the efficiency of information acquisition.

[0111] In some cases, process 300 further includes: displaying a second control; and downloading a first output and a second output in response to a triggering operation on the second control.

[0112] In this way, a download control can be provided associated with the preview node to download the input content in the preview node, thereby improving the user's interaction efficiency.

[0113] In some cases, process 300 further includes: displaying a first connection element and a second connection element in a first interface, the first connection element connecting a first node and a preview node, and the second connection element connecting a second node and a preview node, wherein the first connection element is connected to a first input port of the preview node, and the second connection element is connected to a second input port of the preview node, and the first input port and the second input port correspond to different types of input.

[0114] In this way, the connection elements can be displayed on the first interface, which can more intuitively show the input nodes of the preview nodes to the user, thereby improving the efficiency of information transmission.

[0115] Example devices and equipment A corresponding apparatus for implementing the above methods or processes is also provided. Figure 4 A schematic structural block diagram of an interactive example device 400 for a graphical interface is shown, according to some scenarios. Device 400 can be implemented as or included in electronic device 110. The various modules / components in device 400 can be implemented by hardware, software, firmware, or any combination thereof.

[0116] like Figure 4 As shown, the device 400 includes a first display module 410 configured to display a workflow on a first interface, the workflow including a first node, a second node and a preview node, the first node and the second node being input nodes of the preview node; and a second display module 420 configured to, in response to the execution of the workflow, display a first output of the first node in a first area of ​​the preview node and a second output of the second node in a second area of ​​the preview node, the first output and the second output being of different types.

[0117] In some cases, the device 400 also includes a third display module configured to display a third output in a first area, the third output being from a second node or a third node, the first output and the third output being of the same type, and the third node being an input node for the preview node.

[0118] In some cases, the first area is used to display content items of a first type, and the device 400 also includes a fourth display module configured to, in response to input from the preview node including multiple content items of the first type, display multiple content items in the first area in a first view mode; receive a first operation associated with the preview node; and, in response to the first operation, display multiple content items in the first area in a second view mode.

[0119] In some cases, the first view mode or the second view mode includes: a tiled view mode, which indicates that multiple content items are presented simultaneously in a first area; or an aggregated view mode, which indicates that one of the multiple content items is displayed at a time in a first area.

[0120] In some cases, the fourth display module is also configured to display a first control associated with a first area; and to receive trigger operations on the first control.

[0121] In some cases, the first operation is used to adjust the display size of the preview node, and the fourth display module is also configured to display multiple content items in a second view mode in response to the preview node's display size being adjusted to meet the triggering conditions.

[0122] In some cases, device 400 also includes a processing module configured to present multiple options indicating various types of input to the preview node; receive a selection of a first option among the multiple options, the first option corresponding to a second type; and stop presenting input that does not match the second type at the preview node.

[0123] In some cases, device 400 also includes a fifth display module configured to display a second control; and to download a first output and a second output in response to a triggering operation on the second control.

[0124] In some cases, the device 400 also includes a sixth display module configured to display a first connection element and a second connection element in a first interface, wherein the first connection element connects a first node and a preview node, and the second connection element connects a second node and a preview node, wherein the first connection element is connected to a first input port of the preview node, and the second connection element is connected to a second input port of the preview node, and the first input port and the second input port correspond to different types of input.

[0125] The modules included in device 400 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some cases, one or more modules can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units in device 400 can be implemented at least partially by one or more hardware logic components. By way of example, and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and so on.

[0126] Figure 5 A block diagram of an electronic device 500 in which one or more examples may be implemented is shown. It should be understood that... Figure 5 The electronic device 500 shown is merely exemplary and should not be construed as limiting the functionality and scope of the examples described herein. Figure 5 The electronic device 500 shown can be used to implement the electronic device 110 discussed above.

[0127] like Figure 5 As shown, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, one or more processing units or processors 510, memory 520, storage devices 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processor 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0128] Electronic device 500 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 500.

[0129] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 5 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various examples.

[0130] The communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 500 can operate in a networked environment using logical connections to one or more other servers, networked personal computers, or another network node.

[0131] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via an input / output (I / O) interface (not shown).

[0132] A computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. A computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0133] The flowcharts and / or block diagrams of the methods, apparatus, devices, and computer program products referred to herein describe various aspects. It should be understood that each block of the flowcharts and / or block diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0134] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0135] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0136] The flowcharts and block diagrams in the accompanying figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products under various scenarios. In this respect, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0137] Various examples have been described above. The foregoing descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A graphical user interface interaction method, comprising: The workflow is displayed on the first interface. The workflow includes a first node, a second node, and a preview node. The first node and the second node are the input nodes of the preview node. as well as In response to the execution of the workflow, a first output of the first node is displayed in a first area of ​​the preview node, and a second output of the second node is displayed in a second area of ​​the preview node, wherein the first output and the second output are of different types.

2. The method according to claim 1, further comprising: In the first area, a third output is displayed. The third output comes from the second node or the third node. The first output and the third output are of the same type. The third node is the input node of the preview node.

3. The method according to claim 1, wherein the first area is used to display content items of the first type, the method further comprising: In response to input from the preview node, which includes multiple content items of the first type, the multiple content items are displayed in the first area in a first view mode. Receive a first operation, which is associated with the preview node; as well as In response to the first operation, the plurality of content items are displayed in the first area in a second view mode.

4. The method according to claim 3, wherein the first view mode or the second view mode comprises: Tiled view mode, indicating that the multiple content items are presented simultaneously in the first area; or Aggregated view mode indicates that one of the multiple content items is displayed at a time in the first area.

5. The method of claim 3, wherein receiving the first operation comprises: Display a first control, which is associated with the first area; as well as Receive the trigger operation on the first control.

6. The method of claim 3, wherein the first operation is used to adjust the display size of the preview node, and the step of displaying the plurality of content items in the first area in a second view mode in response to the first operation includes: In response to the preview node's display size being adjusted to meet the triggering condition, the plurality of content items are displayed in the second view mode.

7. The method according to claim 1, further comprising: Multiple options are presented, indicating various types of input for the preview node; Receive a selection of a first option from the plurality of options, the first option corresponding to a second type; as well as Stop displaying input that does not match the second type in the preview node.

8. The method according to claim 1, further comprising: Display the second control; as well as In response to a trigger operation on the second control, the first output and the second output are downloaded.

9. The method according to claim 1, further comprising: In the first interface, a first connection element and a second connection element are displayed. The first connection element connects the first node and the preview node, and the second connection element connects the second node and the preview node. The first connection element is connected to the first input port of the preview node, and the second connection element is connected to the second input port of the preview node. The first input port and the second input port correspond to different types of input.

10. An interactive device for a graphical user interface, comprising: The first display module is configured to display a workflow on a first interface. The workflow includes a first node, a second node, and a preview node. The first node and the second node are input nodes of the preview node. as well as A second display module is configured to, in response to the execution of the workflow, display a first output of the first node in a first area of ​​the preview node and a second output of the second node in a second area of ​​the preview node, wherein the first output and the second output are of different types.

11. An electronic device, comprising: At least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 9 when executed by the at least one processor.

12. A computer-readable storage medium having stored thereon computer-executable instructions that can be executed by a processor to implement the method according to any one of claims 1 to 9.

13. A computer program product tangibly stored in a computer storage medium and comprising computer-executable instructions that, when executed by a device, cause the device to perform the method according to any one of claims 1 to 9.

Images