Interaction method and apparatus, electronic device, and storage medium

By displaying composite node options on the media data processing flow configuration page, designers can directly call the encapsulated first media data processing flow as a node, solving the problem of high difficulty for designers to build node-based data processing flows and achieving more efficient media data processing flow construction.

WO2026233204A1PCT designated stage Publication Date: 2026-11-12BEIJING ZITIAO NETWORK TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2026/093901
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-08
Filing Date
2026-04-29
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

In existing technologies, the technical threshold for designers to build node-based data processing streams is relatively high, making it difficult to complete independently, thus affecting the efficiency and effectiveness of building media data processing streams.

Method used

By displaying the composite node option in its encapsulated state, designers are allowed to directly call the pre-encapsulated first media data processing stream as a node to be built in the media data processing stream configuration page, reducing the difficulty of building composite node-based media data processing streams.

Benefits of technology

It improves the efficiency and effectiveness of building node-based media data processing streams, lowers the technical threshold for designers, and makes the construction of media data processing streams more efficient and convenient.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2026093901_12112026_PF_FP_ABST
    Figure CN2026093901_12112026_PF_FP_ABST
Patent Text Reader

Abstract

An interaction method and apparatus, an electronic device, and a storage medium. The method comprises: in response to a first preset operation on a media data processing flow configuration page, displaying a node list, wherein the node list comprises composite node options, the composite node options are used for representing composite nodes, the composite nodes are first media data processing flows in an encapsulated state, and each first media data processing flow has a media data processing function corresponding to the first media data processing flow, and the media data processing flow configuration page comprises a canvas area; and in response to a selection operation on a target composite node option in the node list, displaying, in the canvas area, a target composite node corresponding to the target composite node option, so that the target composite node serves as a node in a second media data processing flow to be constructed.
Need to check novelty before this filing date? Find Prior Art

Description

Interaction methods, devices, electronic devices, and storage media

[0001] Cross-references to related applications

[0002] This application is based on and claims priority to Chinese application No. 202510593050.3, filed on May 8, 2025, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure relates to the field of media data processing technology, and more particularly to an interactive method, apparatus, electronic device, and storage medium. Background Technology

[0004] Today, media data processing tools that employ node-based data processing streams are profoundly reshaping the creative design field, empowering users to precisely customize and reliably reproduce media data designs through their sophisticated process control mechanisms. Summary of the Invention

[0005] In a first aspect, this disclosure provides an interaction method, comprising: in response to a first preset operation on a media data processing stream configuration page, displaying a node list, the node list including composite node options, the composite node options being used to characterize composite nodes; the composite node being a first media data processing stream in an encapsulated state; the first media data processing stream having a corresponding media data processing function; the media data processing stream configuration page including a canvas area; in response to a selection operation of a target composite node option in the node list, displaying a target composite node corresponding to the target composite node option in the canvas area, so that the target composite node is used as a node in a second media data processing stream to be constructed.

[0006] Secondly, this disclosure also provides an interactive device, comprising: a first display module configured to display a node list in response to a first preset operation on a media data processing stream configuration page, the node list including composite node options, the composite node options being used to characterize composite nodes; the composite node being a first media data processing stream in an encapsulated state; the first media data processing stream having a corresponding media data processing function; the media data processing stream configuration page including a canvas area; and a second display module configured to display a target composite node corresponding to the target composite node option in the canvas area in response to a selection operation of a target composite node option in the node list, so that the target composite node serves as a node in a second media data processing stream to be constructed.

[0007] Thirdly, this disclosure also provides an electronic device comprising: one or more processors; a storage device configured to store one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the interaction method as described above.

[0008] Fourthly, this disclosure also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the interaction method described above.

[0009] Fifthly, this disclosure also provides a computer program comprising: instructions that, when executed by a processor, cause the processor to perform the interaction method as described above. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0011] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 is a flowchart of an interaction method provided in an embodiment of this disclosure;

[0013] Figures 2-10 are schematic diagrams of several electronic device display interfaces provided in the embodiments of this disclosure;

[0014] Figure 11 is a schematic diagram of the structure of an interactive device according to an embodiment of this disclosure;

[0015] Figure 12 is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0016] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0017] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0018] In real-world work, building a complete and functional node-based data processing workflow from scratch demands a high level of professional skill from designers, making it difficult for some to handle this task independently. Therefore, effectively lowering the technical barrier for designers in building node-based data processing workflows is a pressing issue that needs to be addressed.

[0019] To address, or at least partially address, the aforementioned technical problems, this disclosure provides an interactive method, apparatus, electronic device, and storage medium. The technical solution provided by embodiments of this disclosure displays a node list in response to a first preset operation on a media data processing stream configuration page. The node list includes composite node options, which characterize composite nodes. Each composite node is a first media data processing stream in an encapsulated state. The first media data processing stream has corresponding media data processing functions. The media data processing stream configuration page includes a canvas area. In response to a selection operation on a target composite node option in the node list, a target composite node corresponding to the target composite node option is displayed in the canvas area, so that the target composite node serves as a node in the second media data processing stream to be constructed. Essentially, it pre-encapsulates a usable first media data processing stream with specific functions as a composite node. When a designer needs to create a second media data processing stream, they can directly call the composite node formed by the first media data processing stream, without having to build a first media data processing stream with the same functions from scratch. This reduces the difficulty of constructing node-based media data processing streams and improves the efficiency and effectiveness of building node-based media data processing streams.

[0020] Figure 1 is a flowchart of an interaction method provided in an embodiment of this disclosure. This embodiment is applicable to situations where interaction occurs on a client side. The method can be executed by an interaction device, which can be implemented in software and / or hardware. The device can be configured in an electronic device, such as a terminal, including but not limited to smartphones, PDAs, tablets, wearable devices with displays, desktop computers, laptops, all-in-one computers, smart home devices, etc. Alternatively, this embodiment can be applied to situations where interaction occurs on a server side. The method can be executed by an interaction device, which can be implemented in software and / or hardware. The device can be configured in an electronic device, such as a server.

[0021] As shown in Figure 1, the method may specifically include steps S110-S120.

[0022] S110. In response to a first preset operation on the media data processing stream configuration page, a node list is displayed. The node list includes a composite node option, which is used to characterize a composite node. The composite node is a first media data processing stream in an encapsulated state. The first media data processing stream has a corresponding media data processing function. The media data processing stream configuration page includes a canvas area.

[0023] A data processing flow is a workflow that achieves the orderly execution of specific tasks and operations through node connections and process design. A node is an operational unit in a data processing flow, used to execute a specific operational step. Data processing flows are used to process media data. Media data may include at least one of the following: text, images, audio, and video.

[0024] A data processing stream is like a production line, connecting a series of interconnected nodes in an orderly manner to collaboratively complete the entire process of media data processing. When raw material is provided as input to a pre-configured data processing stream, the stream follows its design, passing through each processing node sequentially to obtain the final processed media data. The raw material and the processed media data can be of the same type or different types. For example, for a data processing stream used for text-to-image conversion, the raw material is text, and the processed media data is an image. For a data processing stream used for image-to-image conversion, the raw material consists of both images and text, and the processed media data is an image.

[0025] For example, an image data processing stream may include the following nodes: a loading model node, a text encoder, a loading image node, an image encoding node, a sampler, an image decoding node, and a preview image node. The loading model node loads a pre-trained drawing model, which includes a denoising algorithm, a text encoding model, and an image encoding / decoding component. The denoising algorithm is responsible for noise prediction and image generation. The image encoding / decoding component performs a conversion between latent space and pixel space. The text encoder encodes text prompts into semantic vectors. These vectors are then conditionally input into the sampler to guide the image generation process. The loading image node receives a user-provided prompt image. The image encoding node converts the prompt image from pixel space to a latent space representation. The latent space representation of the prompt image is then conditionally input into the sampler to guide the image generation process. The sampler, based on the denoising algorithm, combines the semantic vectors of the text prompts and the latent space representation of the prompt image to generate a latent space representation of the target image. The image decoding node converts the latent space image output by the sampler into a pixel space image, obtaining the final visualization image. The preview image node displays the generated pixel space image.

[0026] A media data processing stream configuration page can be, for example, a page used to configure media data processing streams. Configuring a media data processing stream can be done, for example, by selecting appropriate nodes, configuring the parameters included in the nodes, and configuring the input and output relationships between different nodes, thus constructing the media data processing stream.

[0027] A node can be, for example, a step in a media data processing stream or a complete algorithm. Nodes are typically associated with a set of parameters that configure or define the node's behavior. For example, for a text encoding node, the associated parameters include the text content to be encoded. For an image resizing node, the associated parameters include the desired dimensions of the resized image. For a K-sampler node, the associated parameters include the model's stride, random seed, etc. Input-output relationships between different nodes can, for example, include using the output data of one node as the input data of another.

[0028] In some scenarios, nodes can include simple nodes and composite nodes. A simple node can be, for example, the smallest computational unit used to construct a media data processing stream, typically a single-function module whose corresponding function cannot be further decomposed. For example, the loading model node, text encoder, loading image node, image encoder node, sampler, image decoder node, and preview image node in the image data processing stream mentioned in the previous example are all simple nodes.

[0029] A composite node can be, for example, the encapsulation result of a media data processing stream (referred to as the first media data processing stream in this disclosure) formed by logically arranging multiple simple nodes. Therefore, a composite node is the first media data processing stream in an encapsulated state. Compared to simple nodes, the functions of a composite node are formed by the combined action of the simple nodes, and its corresponding functions can be further decomposed, the decomposition result being all the functions of the simple nodes constituting the composite node. For example, a media data processing stream (i.e., the first media data processing stream) capable of automatically detecting and repairing target objects is formed by logically arranging simple nodes such as a loading model node, a text encoder, a loading image node, a detection loader (used to detect and locate target objects in the image input by the loading image node), an image segmenter (used to call an image style model to segment the image input by the loading image node to obtain a mask of the target object), and a thinning node (used to repair the target object). Encapsulating this first media data processing stream results in a composite node. Here, "encapsulation" can, for example, mean combining all the simple nodes constituting the media data processing stream and using them as a whole. In actual use, the specific simple nodes used to constitute the first media data processing stream, how these simple nodes are arranged, and the specific parameter values ​​used by each node can be fully exposed to the user of the composite node, partially exposed, or no information can be exposed to the user of the composite node at all. A composite node corresponds to the first media data processing stream that forms it.

[0030] Simple nodes can be viewed as basic units that constitute a media data processing stream, like "basic building blocks"; composite nodes are standardized components built upon these basic units, similar to standardized modules formed by piecing together "basic building blocks". When constructing a second media data processing stream, either simple or composite nodes can be used.

[0031] It should be noted that the first media data processing stream is used to construct composite nodes. During its creation, it can use only simple nodes, only composite nodes, or both. If a composite node is used when creating the first media data processing stream, then the composite node used to create this processing stream is not the same node as the composite node formed after the first media data processing stream is encapsulated.

[0032] Composite node options can be, for example, options used to represent composite nodes. A node list can be, for example, a list that centrally displays nodes. In practice, a node list can include only composite node options; or it can include both composite node options and simple node options. Simple node options can be, for example, options used to represent simple nodes.

[0033] The canvas area could be, for example, the area in the media data processing stream configuration page used to display the currently configured data processing stream in real time.

[0034] The first preset operation on the media data processing stream configuration page can be, for example, an operation performed on the media data processing stream configuration page. This disclosure does not limit what specific operation the first preset operation refers to. For example, the first preset operation on the media data processing stream configuration page can be a click operation in the canvas area of ​​the media data processing stream configuration page, or clicking a control on the media data processing stream configuration page that can trigger the display of a node list, or other operations.

[0035] S120. In response to the selection operation of the target composite node option in the node list, the target composite node corresponding to the target composite node option is displayed in the canvas area so that the target composite node is used as a node in the second media data processing stream to be constructed.

[0036] The selection operation of a target composite node option in the node list can, for example, reflect the user's desire to construct a second media data processing stream using the target composite node. The target composite node option is a composite node option in the node list that is being selected. The selection operation of a target composite node option in the node list can include, for example, clicking, swiping, hovering, or dragging the target composite node option.

[0037] The second media data processing stream is the media data processing stream that needs to be constructed using the interaction method provided in this disclosure, and it is a media data processing stream that uses composite nodes.

[0038] For example, referring to Figure 2, the media data processing flow configuration page includes a canvas area and a "Components" option. If a user double-clicks on a blank area of ​​the canvas area or clicks the "Components" option, it is considered as detecting the first preset operation. Referring to Figure 3, a node list is displayed. This node list includes the "Specific Feature Preservation" option, the "Target Object Repair" option, and the "Target Object Detection" option. Specific Feature Preservation, Target Object Repair, and Target Object Detection are all composite nodes, and these options are all composite node options. The specific nodes used and / or the arrangement order between nodes in the first media data processing flow that constitutes different composite nodes are different. If the user drags the "Target Object Repair" option to the canvas area (i.e., a selection operation of the target composite node option in the node list is detected), the "Target Object Repair" option is used as the target composite node option. Referring to Figure 4, the target object repair node is displayed in the canvas area. Subsequently, the target object repair node can be used as a node in the second media data processing flow. That is, the entire first media data processing flow corresponding to the target object repair node is used as a node in the second media data processing flow.

[0039] The above technical solution, in response to a first preset operation on the media data processing stream configuration page, displays a node list. This list includes composite node options, which characterize composite nodes. Each composite node represents a first media data processing stream in an encapsulated state, possessing corresponding media data processing functions. The media data processing stream configuration page includes a canvas area. In response to selecting a target composite node option from the node list, the target composite node corresponding to that option is displayed in the canvas area, making it a node in the second media data processing stream to be constructed. Essentially, it pre-encapsulates usable first media data processing streams with specific functions as composite nodes. When a designer needs to create a second media data processing stream, they can directly call the composite node formed by the first media data processing stream, without having to build a first media data processing stream with the same functions from scratch. This reduces the difficulty of building node-based media data processing streams and improves their efficiency and effectiveness.

[0040] Based on the above technical solutions, in some embodiments, S120 may include: in response to the selection operation of the target composite node option in the node list, displaying a preview window, the preview window including detailed information of the target composite node and node application options; in response to the selection operation of the node application options, displaying the target composite node corresponding to the target composite node option in the canvas area.

[0041] A preview window could be, for example, a window that previews the target composite node. In practice, the detailed information of the target composite node may include at least one of the following: the author of the target composite node (i.e., the creator of the target composite node), the publication time, the version number, the function, and the structure of the corresponding first media data processing stream (such as the included nodes and the connection relationships between the nodes).

[0042] Node application options can be, for example, options displayed in the preview window. When a node application option is selected, it means that the user wants to use the composite node to build a second media data processing stream. The selection operation for a node application option can be, for example, clicking, swiping, or dragging the node application option.

[0043] For example, if a user hovers the mouse over the "Target Object Repair" option in Figure 3, with "Target Object Repair" as the target composite node, a preview window will be displayed (see Figure 5). This preview window includes detailed information about the "Target Object Repair" composite node and node application options. In Figure 5, the detailed information of the composite node specifically includes a thumbnail of the first media data processing stream corresponding to the composite node, the author, publication time, and version. The "Add to Canvas" option in Figure 5 is the node application option. If the user clicks the node application option (see Figure 4), the "Target Object Repair" node will be displayed in the canvas area.

[0044] In some cases, the area in the node list used to display information about individual nodes is limited; for example, only the names of each node may be shown. In actual use, users may not be able to clearly understand whether a node can perform the functions they need based solely on its name. By setting a preview window to appear in response to the selection of a target composite node option in the node list, the preview window includes detailed information about the composite node and node application options. The purpose is to use the preview window to further demonstrate the relevant information of the target composite node, helping users to further clarify whether it can achieve the functions they need.

[0045] In some embodiments, the preview window may include a first area, and the preview window may have a first state and a second state. When the preview window is in the first state, a thumbnail of the first media data processing stream is displayed in the first area; when the preview window is in the second state, the thumbnail of the first media data processing stream is not displayed in the first area. For example, in Figure 5, the preview window is in the first state, and a thumbnail of the first media data processing stream corresponding to the composite node "Target Object Repair" is displayed in the first area. In Figure 6, the preview window is in the second state, and the thumbnail of the first media data processing stream corresponding to the composite node "Target Object Repair" is not displayed in the first area. In practice, the creator, manager, or user of the composite node can configure whether the preview window is displayed in the first state or the second state when needed. Alternatively, it can be switched between the first and second states according to instructions.

[0046] In some embodiments, the display may be configured to display a thumbnail of the first media data processing stream in the first area in response to an operation that adjusts the display state of the preview window to a first state; and to stop displaying the thumbnail of the first media data processing stream in the first area in response to an operation that adjusts the display state of the preview window to a second state.

[0047] Adjusting the preview window's display state to the first state could be an operation such as selecting a specific control or option within the preview window, or a sliding operation within the preview window in a specific direction. Similarly, adjusting the preview window's display state to the second state could also be an operation such as selecting a specific control or option within the preview window, or a sliding operation within the preview window in a specific direction. However, it is necessary to ensure that the operation of adjusting the preview window's display state to the first state is not the same operation as the operation of adjusting the preview window's display state to the second state. For example, referring to Figure 5, an "Expand" option is displayed in the lower right corner of the first area. If the user clicks the "Expand" option, it is determined that an operation of adjusting the preview window's display state to the second state has been detected, and referring to Figure 6, the thumbnail of the first media data processing stream is stopped from being displayed in the first area. In Figure 6, a "Do Not Expand" option is displayed in the lower right corner of the first area. If the user clicks the "Do Not Expand" option, it is determined that an operation of adjusting the preview window's display state to the first state has been detected, and referring to Figure 5, the thumbnail of the first media data processing stream is displayed in the first area.

[0048] Since different users may have different viewing needs for the first media data processing stream in practice, by setting the operation to display a thumbnail of the first media data processing stream in the first area in response to the operation of adjusting the display state of the preview window to the first state, and to stop displaying the thumbnail of the first media data processing stream in the first area in response to the operation of adjusting the display state of the preview window to the second state, the essence is to allow users to adjust the display state of the preview window according to their own needs.

[0049] In some embodiments, when a thumbnail of a first media data processing stream is displayed in a first region, a region selection tool and a magnifying window are displayed; in response to a selection operation of a target region in the first region using the region selection tool, a partial magnified view of the first media data processing stream within the target region is displayed in the magnifying window.

[0050] Region selection tools may include, for example, region selection boxes or smearing tools (such as smearing brushes).

[0051] The target area could be, for example, a sub-region within the first area that the user wishes to zoom in on. Zooming in on the target area means that the image within that area will be presented in a clearer state, revealing more detail compared to before zooming. Selecting the target area within the first area using the region selection tool could be, for example, using the region selection tool to indicate which sub-region within the first area to use as the target area. Exemplarily, selecting the target area within the first area using the region selection tool could include moving the region selection box or using the smudge tool to smudge within the first area.

[0052] A magnified window could be, for example, a display window used to present the magnified results of an image of a target area.

[0053] For example, in Figure 7, the area selection tool is a region selection box, the area enclosed by the region selection box is the target area, and the magnified window displays a partial magnified view of the first media data processing stream within the target area.

[0054] In practice, due to the limited size of the first region, users cannot clearly see all the details of the first media data processing stream. By setting up a function that responds to the selection of a target region within the first region using the region selection tool, and displays a magnified view of a portion of the first media data processing stream within the target region in a zoom-in window, users can better understand the details of the first media data processing stream.

[0055] Based on the above technical solution, in some embodiments, after S120, the method further includes: configuring the values ​​of the parameters of the first media data processing stream corresponding to the composite node and / or configuring the connection relationship between the composite node and other nodes in the second media data processing stream.

[0056] Since the first media data processing stream includes multiple nodes, the purpose of configuring the parameter values ​​of the first media data processing stream corresponding to the composite node is to enable the first media data processing stream to meet the user's personalized usage needs, thereby making the second media data processing stream built based on the composite node more in line with the user's usage needs.

[0057] It should be noted that when actually constructing a composite node, the creator of the composite node can choose to expose only some parameters of the first media data processing stream to the composite node's users, or expose all parameters of the first media data processing stream to the composite node's users. Once a parameter is exposed to a user, the user can configure the value of that parameter according to their usage needs. In some embodiments, the creator of the composite node can specify which parameters(s) of the first media data processing stream will be exposed to the composite node's users.

[0058] Based on the above technical solutions, in some embodiments, the composite node is created through the following steps: when the canvas area of ​​the media data processing stream configuration page includes a first media data processing stream, an encapsulation option is displayed; in response to the selection operation of the encapsulation option, the first media data processing stream is encapsulated to obtain a composite node; and the composite node is displayed in the canvas area.

[0059] The encapsulation option can be, for example, an option corresponding to the first media data processing stream, having the function of encapsulating the corresponding first media data processing stream. The selection operation for the encapsulation option can be, for example, a click operation, a swipe operation, or a drag operation.

[0060] For example, referring to Figure 8, the media data processing stream configuration page includes a first media data processing stream and a "Create Component" option. This "Create Component" option is the encapsulation option. If the user clicks the encapsulation option, the first media data processing stream is encapsulated to obtain a composite node corresponding to the media data processing stream. Referring to Figure 9, this composite node is displayed in the canvas area. Essentially, this setting provides a method for creating composite nodes to meet the user's need to create composite nodes.

[0061] Based on the above technical solutions, in some embodiments, after displaying the composite node in the canvas area, the process may further include: displaying disassembly options corresponding to the composite node; in response to the selection operation of the disassembly options, performing decapsulation processing on the first media data processing stream corresponding to the composite node; and displaying the decapsulated first media data processing stream in the canvas area.

[0062] The decomposition option could be, for example, an option corresponding to a composite node, used to decapsulate the first media data processing stream corresponding to the composite node. The selection operation for the decomposition option could be, for example, a click operation, a swipe operation, or a drag operation.

[0063] For example, referring to Figure 9, the canvas area includes a "Component Decoupling" option. This "Component Decoupling" option is the decoupling option corresponding to the composite node. If the user clicks the "Component Decoupling" option, referring to Figure 8, the first media data processing flow is displayed.

[0064] In some scenarios, encapsulating the first media data processing stream may prevent or hinder its modification (e.g., it may be prone to accidental errors). Modifying the first media data processing stream can include, but is not limited to, adding, deleting, or replacing nodes, adjusting the connections between nodes, and adjusting node parameter values. By setting a response to the selection of the decapsulation option, the first media data processing stream corresponding to the composite node is decapsulated; displaying the decapsulated first media data processing stream in the canvas area allows the composite node to be restored to the first media data processing stream when the creator needs to modify it, thus enabling modification of the first media data processing stream.

[0065] In some embodiments, after displaying the composite node in the canvas area, the method further includes: displaying settings options corresponding to the composite node; displaying a composite node settings page in response to a selection operation of the settings options; the composite node settings page is used to set the attributes of the composite node; the attributes of the composite node include at least one of the following: usage permissions of the composite node; whether the first media data processing stream corresponding to the composite node is allowed to be displayed when in preview state; and whether the parameters associated with the first media data processing stream corresponding to the composite node are allowed to be configured during use.

[0066] Setting options could be options corresponding to composite nodes, used to configure the properties of composite nodes. The selection of setting options could be done through clicking, swiping, or dragging.

[0067] A composite node settings page could be a page for setting the properties of composite nodes.

[0068] The usage permissions for composite nodes can refer to, for example, the scope of users authorized to use the composite node. For instance, when the usage permissions for a composite node are set to members of the same creative team as the node creator, the composite node will be included in the node lists of every member within that creative team. Conversely, if the usage permissions for a composite node are limited to the creator, then the composite node will not be included in the node lists of any other member of the creator's creative team. Configuring usage permissions for composite nodes allows for more effective management and control.

[0069] The preview state could be, for example, the state where the composite node details are displayed through the preview window mentioned earlier. It could also specify whether the first media data processing stream corresponding to the composite node is allowed to be displayed, for example, whether the first media data processing stream corresponding to the composite node is allowed to be displayed in the first area of ​​the preview window. That is, whether the preview window is allowed to be displayed in the first state and whether it is allowed to be displayed in the second state. It should be noted that this is the composite node creator's configuration of which states(s) the preview window can be displayed during the use of the composite node. If the composite node creator configures that the preview window can be displayed in the first state but not in the second state, then the preview window can only be displayed in the first state during the use of the composite node. If the composite node creator configures that the preview window can be displayed in both the first and second states, then the preview window's display state can be switched during the use of the composite node.

[0070] The configuration option allows users to configure parameters associated with the first media data processing stream corresponding to the composite node. For example, it could specify which parameters in the first media data processing stream are allowed to be accessed by the composite node user. Once a parameter is accessible, the user can configure its value to suit their needs, ensuring the first media data processing stream operates according to their requirements. In practice, the first media data processing stream of a composite node is used as a standardized component; therefore, in some scenarios, users don't need to understand the internal workings of each component. By allowing configuration of parameters associated with the first media data processing stream during use, the essence is to expose users to parameters or functions related to personalized usage needs, thereby reducing the difficulty of using the composite node.

[0071] For example, referring to Figure 9, the canvas area includes a "Component Settings" option. This "Component Settings" option corresponds to the settings for the composite node. If the user clicks this "Component Settings" option, referring to Figure 10, the composite node settings page is displayed. On this settings page, "Owner File" is used to set the usage permissions for the composite node. "Support Expanding" configures the attribute of "whether the first media data processing stream corresponding to the composite node is allowed to be displayed when in preview mode". "Input Connection Point" and "Output Connection Point" configure the attribute of "whether the parameters associated with the first media data processing stream corresponding to the composite node are allowed to be configured during use".

[0072] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0073] Figure 11 is a schematic diagram of an interactive device according to an embodiment of this disclosure. The interactive device provided in this embodiment can be configured in a client or in a server. Referring to Figure 11, the interactive device specifically includes:

[0074] The first display module 310 is configured to display a node list in response to a first preset operation on the media data processing stream configuration page. The node list includes a composite node option, which is used to characterize a composite node. The composite node is a first media data processing stream in an encapsulated state. The first media data processing stream has a corresponding media data processing function. The media data processing stream configuration page includes a canvas area.

[0075] The second display module 320 is configured to display the target composite node corresponding to the target composite node option in the canvas area in response to the selection operation of the target composite node option in the node list, so that the target composite node is used as a node in the second media data processing stream to be constructed.

[0076] In some embodiments, the second display module 320 is configured as follows:

[0077] In response to the selection of a target composite node option in the node list, a preview window is displayed, which includes detailed information about the target composite node and node application options;

[0078] In response to the selection operation of the node application option, the target composite node corresponding to the target composite node option is displayed in the canvas area.

[0079] In some embodiments, the preview window includes a first area; the preview window's display state includes a first state and a second state; the second display module 320 is further configured to:

[0080] In response to the operation of adjusting the display state of the preview window to the first state, a thumbnail of the first media data processing stream is displayed in the first area;

[0081] In response to the operation of adjusting the display state of the preview window to the second state, the display of the thumbnail of the first media data processing stream in the first area is stopped.

[0082] In some embodiments, the second display module 320 is further configured to:

[0083] When a thumbnail of the first media data processing stream is displayed in the first area, a display area selection tool and a zoom window are provided.

[0084] In response to a selection operation of a target region in the first region using the region selection tool, a partial magnified view of the first media data processing stream within the target region is displayed in the magnified window.

[0085] In some embodiments, the device further includes a configuration module configured to:

[0086] In response to the selection operation of the target composite node option in the node list, after displaying the target composite node corresponding to the target composite node option in the canvas area, the values ​​of the parameters of the first media data processing stream corresponding to the composite node and / or the connection relationship between the composite node and other nodes in the second media data processing stream are configured.

[0087] In some embodiments, the apparatus further includes a creation module configured to:

[0088] If the canvas area of ​​the media data processing stream configuration page includes the first media data processing stream, the encapsulation option is displayed;

[0089] In response to the selection of the encapsulation option, the first media data processing stream is encapsulated to obtain a composite node;

[0090] The composite node is displayed in the canvas area.

[0091] In some embodiments, the creation module is configured as follows:

[0092] After displaying the composite node in the canvas area, the disassembly options corresponding to the composite node are displayed;

[0093] In response to the selection of the disassembly option, the first media data processing stream corresponding to the composite node is decapsulated.

[0094] The first media data processing stream after decapsulation is displayed in the canvas area.

[0095] In some embodiments, the creation module is configured as follows:

[0096] After displaying the composite node in the canvas area, the corresponding settings options for the composite node are displayed;

[0097] In response to the selection operation of the composite node setting options, the composite node setting page is displayed. The composite node setting page is used to set the attributes of the composite node. The attributes of the composite node include at least one of the following: the usage permissions of the composite node, whether the first media data processing stream corresponding to the composite node is allowed to be displayed when it is in preview state, and whether the parameters associated with the first media data processing stream corresponding to the composite node are allowed to be configured during use.

[0098] The interactive device provided in this disclosure embodiment can execute the steps performed by the client or server in the interactive method provided in this disclosure method embodiment, and has the execution steps and beneficial effects, which will not be described in detail here.

[0099] Referring specifically to Figure 12, which illustrates a structural schematic suitable for implementing the electronic device 1000 in the embodiments of this disclosure, the electronic device 1000 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. The electronic device shown in Figure 12 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this disclosure.

[0100] As shown in FIG12, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003 to implement the interaction method as described in the embodiments of this disclosure. The RAM 1003 also stores various programs and information required for the operation of the electronic device 1000. The processing device 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0101] Typically, the following devices can be connected to the I / O interface 1005: input devices 1006 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1007 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1008 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic device 1000 to communicate wirelessly or wiredly with other devices to exchange information. Although Figure 12 shows an electronic device 1000 with various devices, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0102] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the interactive methods as described above. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1009, or installed from storage device 1008, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of embodiments of this disclosure.

[0103] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include information signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated information signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0104] In some implementations, clients and servers may communicate using any known or future network protocol such as HTTP (Hypertext Transfer Protocol) and may interconnect with digital information communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any known or future networks.

[0105] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0106] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to:

[0107] In response to a first preset operation on the media data processing stream configuration page, a node list is displayed, the node list including a composite node option, the composite node option being used to characterize a composite node; the composite node is a first media data processing stream in an encapsulated state; the first media data processing stream has a corresponding media data processing function; the media data processing stream configuration page includes a canvas area.

[0108] In response to the selection of a target composite node option in the node list, the target composite node corresponding to the target composite node option is displayed in the canvas area so that the target composite node is used as a node in the second media data processing stream to be constructed.

[0109] In some embodiments, when one or more of the above-described programs are executed by the electronic device, the electronic device may also perform other steps described in the above embodiments.

[0110] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0111] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0112] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0113] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0114] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0115] According to one or more embodiments of this disclosure, this disclosure provides an electronic device, including:

[0116] One or more processors;

[0117] Memory, used to store one or more programs;

[0118] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the interaction methods provided in this disclosure.

[0119] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements an interaction method as described in any of the present disclosure.

[0120] This disclosure also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the interaction method described above.

[0121] This disclosure also provides a computer program, including: instructions that, when executed by a processor, cause the processor to perform the interaction method described above.

[0122] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0123] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An interaction method, comprising: In response to a first preset operation on the media data processing stream configuration page, a node list is displayed. The node list includes a composite node option, which is used to characterize a composite node. The composite node is a first media data processing stream in an encapsulated state. The first media data processing stream has media data processing functions corresponding to the first media data processing stream. The media data processing stream configuration page includes a canvas area. In response to the selection of a target composite node option in the node list, the target composite node corresponding to the target composite node option is displayed in the canvas area so that the target composite node is used as a node in the second media data processing stream to be constructed.

2. The interaction method according to claim 1, wherein, The step of displaying the target composite node corresponding to the target composite node option in the canvas area in response to the selection operation of the target composite node option in the node list includes: In response to the selection of a target composite node option in the node list, a preview window is displayed, which includes detailed information about the target composite node and node application options; In response to the selection operation of the node application option, the target composite node corresponding to the target composite node option is displayed in the canvas area.

3. The interaction method according to claim 2, wherein, The preview window includes a first area, and the preview window's display states include a first state and a second state, and also includes: In response to the operation of adjusting the display state of the preview window to the first state, a thumbnail of the first media data processing stream is displayed in the first area; In response to the operation of adjusting the display state of the preview window to the second state, the display of the thumbnail of the first media data processing stream in the first area is stopped.

4. The interaction method according to claim 3 further includes: When a thumbnail of the first media data processing stream is displayed in the first area, a region selection tool and a zoom-in window are displayed; In response to a selection operation of a target region in the first region using the region selection tool, a partial magnified view of the first media data processing stream within the target region is displayed in the magnified window.

5. The interaction method according to any one of claims 1 to 4, wherein, The step of responding to a selection operation of a target composite node option in the node list, after displaying the target composite node corresponding to the target composite node option in the canvas area, further includes: Configure the values ​​of the parameters of the first media data processing stream corresponding to the composite node and / or the connection relationship between the composite node and other nodes in the second media data processing stream.

6. The interaction method according to any one of claims 1 to 5, wherein, The composite node is created through the following steps: If the canvas area of ​​the media data processing stream configuration page includes the first media data processing stream, the encapsulation option is displayed; In response to the selection of the encapsulation option, the first media data processing stream is encapsulated to obtain a composite node; The composite node is displayed in the canvas area.

7. The interaction method according to claim 6, wherein, After displaying the composite node in the canvas area, the method further includes: Display the disassembly options corresponding to the composite node; In response to the selection of the disassembly option, the first media data processing stream corresponding to the composite node is decapsulated. The first media data processing stream after decapsulation is displayed in the canvas area.

8. The interaction method according to claim 6 or 7, wherein, After displaying the composite node in the canvas area, the method further includes: Display the setting options corresponding to the composite node; In response to the selection operation of the composite node setting options, the composite node setting page is displayed; the composite node setting page is used to set the attributes of the composite node, and the attributes of the composite node include at least one of the following: the usage permissions of the composite node, whether the first media data processing stream corresponding to the composite node is allowed to be displayed when it is in preview state, and whether the parameters associated with the first media data processing stream corresponding to the composite node are allowed to be configured during use.

9. The interaction method according to any one of claims 1 to 8, wherein, The first media data processing flow is a workflow that achieves the orderly execution of specific tasks and operations through node connections and process design.

10. The interaction method according to any one of claims 1 to 9, wherein, The selection operation includes clicking, swiping, hovering, or dragging.

11. An interactive device, comprising: The first display module is configured to respond to a first preset operation on the media data processing stream configuration page and display a node list. The node list includes a composite node option, which is used to characterize a composite node. The composite node is a first media data processing stream in an encapsulated state. The first media data processing stream has media data processing functions corresponding to the first media data processing stream. The media data processing stream configuration page includes a canvas area. The second display module is configured to display the target composite node corresponding to the target composite node option in the canvas area in response to the selection operation of the target composite node option in the node list, so that the target composite node is used as a node in the second media data processing stream to be constructed.

12. An electronic device, comprising: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the interaction method as described in any one of claims 1-10.

13. A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the interactive method as described in any one of claims 1-10.

14. A computer program comprising: Instructions, which, when executed by a processor, cause the processor to perform the interaction method as described in any one of claims 1 to 10.