Interaction method and device, electronic equipment and storage medium

Abstract

The invention relates to an interaction method and device, electronic equipment and a storage medium, and the method comprises the steps: responding to a first preset operation on a media data processing flow configuration page, and displaying a node list, the node list comprises a composite node option, and the composite node option is used for representing a composite node; the composite node is a first media data processing stream in a packaging state; the first media data processing stream has a media data processing function corresponding to the first media data processing stream; the media data processing stream configuration page comprises a canvas area; and in response to a selection operation on a target composite node option in the node list, displaying a target composite node corresponding to the target composite node option in the canvas area, so that the target composite node is used as a node in a second media data processing stream to be constructed. The method can achieve the purposes of reducing the difficulty of constructing the node type media data processing stream and improving the construction efficiency and effect of the node type media data processing stream.

Description

Technical Field

[0001] The present disclosure relates to the technical field of media data processing, and in particular to an interaction method, device, electronic device, and storage medium. Background Art

[0002] Today, media data processing tools that use node-based data processing flows are profoundly reshaping the field of creative design, enabling users to accurately customize and reliably reproduce media data designs with their sophisticated process control mechanisms.

[0003] However, in real-world applications, building a complete, operational, node-based data processing flow from scratch requires a high level of professional expertise, making it difficult for some designers to accomplish this task on their own. Therefore, effectively lowering the technical barriers for designers to build node-based data processing flows is a pressing issue. Summary of the Invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides an interaction method, device, electronic device and storage medium.

[0005] In a first aspect, the present disclosure provides an interaction method, comprising:

[0006] In response to a first preset operation on a media data processing flow configuration page, a node list is displayed, the node list including a composite node option, the composite node option being used to represent a composite node; the composite node is a first media data processing flow in an encapsulated state; the first media data processing flow has a corresponding media data processing function; the media data processing flow configuration page includes a canvas area;

[0007] In response to a selection operation on a target composite node option in the node list, a target composite node corresponding to the target composite node option is displayed in the canvas area, so that the target composite node serves as a node in a second media data processing flow to be constructed.

[0008] In a second aspect, the present disclosure further provides an interactive device, comprising:

[0009] A first display module is configured to display a node list in response to a first preset operation on a media data processing flow configuration page, wherein the node list includes a composite node option, the composite node option being used to represent a composite node; the composite node is a first media data processing flow in an encapsulated state; the first media data processing flow has a corresponding media data processing function; and the media data processing flow configuration page includes a canvas area;

[0010] The second display module is used to display a target composite node corresponding to the target composite node option in the node list in response to a selection operation of the target composite node option, so that the target composite node serves as a node in the second media data processing flow to be constructed.

[0011] In a third aspect, the present disclosure further provides an electronic device, comprising:

[0012] one or more processors;

[0013] a storage device for storing one or more programs;

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the interaction method described above.

[0015] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned interaction method when executed by a processor.

[0016] The technical solution provided by the embodiments of the present disclosure has the following advantages over the prior art:

[0017] The technical solution provided by the embodiment of the present disclosure is to display a node list in response to a first preset operation on the media data processing flow configuration page, wherein the node list includes a composite node option, and the composite node option is used to represent a composite node; the composite node is a first media data processing flow in an encapsulated state; the first media data processing flow has a media data processing function corresponding to it; the media data processing flow configuration page includes a canvas area; in response to a selection operation of a target composite node option in the node list, a target composite node corresponding to the target composite node option is displayed in the canvas area, so that the target composite node serves as a node in the second media data processing flow to be constructed. Its essence is to pre-encapsulate the first media data processing flow with a specific function that can be used as a composite node. When the designer needs to create a second media data processing flow, he can directly call the composite node formed by the first media data processing flow without having to build the first media data processing flow with the same function from scratch, thereby reducing the difficulty of building a node-based media data processing flow and improving the efficiency and effect of building a node-based media data processing flow. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0019] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A flowchart of an interactive method provided in an embodiment of the present disclosure;

[0021] Figure 2-Figure 10 Schematic diagrams of display interfaces of several electronic devices provided in embodiments of the present disclosure;

[0022] Figure 11 is a structural diagram of an interactive device in an embodiment of the present disclosure;

[0023] Figure 12 Schematic diagram of the structure of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0025] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0026] Figure 1 This is a flowchart of an interaction method provided in an embodiment of the present disclosure. This embodiment is applicable to situations where interaction occurs on a client. The method can be executed by an interaction device, which can be implemented using software and / or hardware. The device can be configured in an electronic device, such as a terminal, including but not limited to smartphones, PDAs, tablet computers, wearable devices with displays, desktop computers, laptop computers, all-in-one computers, smart home devices, etc. Alternatively, this embodiment can be applicable to situations where interaction occurs on a server. The method can be executed by an interaction device, which can be implemented using software and / or hardware. The device can be configured in an electronic device, such as a server.

[0027] like Figure 1 As shown, the method may specifically include:

[0028] S110. In response to a first preset operation on a media data processing flow configuration page, a node list is displayed, the node list includes a composite node option, the composite node option is used to represent a composite node; the composite node is a first media data processing flow in an encapsulated state; the first media data processing flow has a media data processing function corresponding thereto; the media data processing flow configuration page includes a canvas area.

[0029] A data processing flow is a workflow that achieves the orderly execution of specific tasks and operations through node connections and process design. A node is an operational unit in a data processing flow, used to perform a specific operation step. A data processing flow is used to process media data. Media data can include, for example, at least one of the following: text, images, audio, and video.

[0030] A data processing flow is like a production line, connecting a series of interconnected nodes in an orderly manner to collaboratively complete the entire media data processing process. When raw material is input to a pre-configured data processing flow, it follows the process design and sequentially passes through the processing stages of each node to produce the final processed media data. The raw material and processed media data can be of the same or different types. For example, in a data processing flow for text-to-image generation, the raw material is text, and the processed media data is an image. In a data processing flow for image-to-image generation, the raw material is images and text, and the processed media data is an image.

[0031] For example, an image data processing flow may include the following nodes: a load model node, a text encoder, a load image node, an image encoding node, a sampler, an image decoding node, and a preview image node. The load model node is used to load a pre-trained drawing model, which includes a denoising algorithm, a text encoding model, and an image encoding and decoding component. The denoising algorithm is responsible for noise prediction and image generation. The image encoding and decoding component is used to convert between latent space and pixel space. The text encoder is used to encode text prompt words into semantic vectors. These vectors are then input into the sampler as conditions to guide the image generation process. The load image node is used to receive a user-provided guidance image. The image encoding node is used to convert the guidance image from pixel space to a latent space representation. The latent space representation of the guidance image is then input into the sampler as a condition to guide the image generation process. The sampler generates a latent space representation of the target image based on the denoising algorithm, the semantic vectors of the text prompt words, and the latent space representation of the guidance image. The image decoding node is used to convert the latent space image output by the sampler into a pixel space image to obtain the final visual image. The preview image node is used to display the generated pixel space image.

[0032] The media data processing flow configuration page may be, for example, a page for configuring the media data processing flow. Configuring the media data processing flow may be, for example, constructing the media data processing flow by selecting appropriate nodes, configuring the parameters included in the nodes, and configuring the input and output relationships between different nodes.

[0033] A node can be, for example, a step in a media data processing flow or a complete algorithm. A node is usually associated with a set of parameters that are used to configure or limit the behavior of the node. For example, for a text encoding node, the parameters associated with it include the text content to be encoded. For an image resizing node, the parameters associated with it include the desired size of the image after resizing. For a K sampler node, the parameters associated with it include the step size of the model, the random seed, etc. The input-output relationship between different nodes can, for example, include using the output data of one node as the input data of another node.

[0034] In some scenarios, nodes can include simple nodes and composite nodes. A simple node, for example, can be the smallest computational unit used to construct a media data processing flow. It is typically a single functional module whose corresponding functionality cannot be further decomposed. For example, the load model node, text encoder, load image node, image encoding node, sampler, image decoding node, and preview image node in the image data processing flow mentioned in the previous example are all simple nodes.

[0035] A composite node, for example, can be the encapsulation result of a media data processing flow (referred to as the first media data processing flow in this disclosure) formed by logical arrangement of multiple simple nodes. Therefore, a composite node is the first media data processing flow in an encapsulated state. Compared to a simple node, the functions of a composite node are formed by the joint action of simple nodes, and the corresponding functions can be further decomposed, and the decomposition result is all the functions of the simple nodes that constitute the composite node. For example, by logically arranging simple nodes such as a loading model node, a text encoder, a loading image node, a detection loader (for detecting and locating the target object in the image input by the loading image node), an image segmenter (for calling an image style model to segment the image input by the loading image node to obtain a mask of the target object), and a refinement node (for repairing the target object), a media data processing flow (i.e., the first media data processing flow) that can automatically detect and repair the target object is formed, and the first media data processing flow is encapsulated, and the encapsulated result is a composite node. Here, "encapsulation" can, for example, mean combining all the simple nodes that constitute the media data processing flow and using them as a whole. In actual use, the specific simple nodes used to form the first media data processing flow, how the simple nodes are arranged, and the specific parameter values used by the parameters involved in each node can be fully disclosed to the user of the composite node, partially disclosed to the user of the composite node, or not disclosed at all. The composite node corresponds to the first media data processing flow that forms itself.

[0036] Simple nodes can be thought of as the fundamental building blocks of a media data processing flow, like "basic building blocks." Composite nodes are standardized components constructed from these fundamental building blocks, similar to standardized modules formed by combining these building blocks. When constructing the second media data processing flow, both simple and composite nodes can be used.

[0037] It should be noted that the first media data processing flow is used to construct a composite node. During its creation, only simple nodes, only composite nodes, or a combination of simple and composite nodes may be used. If a composite node is used when creating the first media data processing flow, the composite node used to create that processing flow is different from the composite node formed after encapsulating the first media data processing flow.

[0038] A composite node option can be, for example, an option for representing a composite node. A node list can be, for example, a list that displays nodes in a centralized manner. In practice, a node list can include only composite node options, or it can include both composite node options and simple node options. A simple node option can be, for example, an option for representing a simple node.

[0039] The canvas area may be, for example, an area in a media data processing flow configuration page that is used to display the currently configured data processing flow in real time.

[0040] The first preset operation on the media data processing flow configuration page can be, for example, an operation performed on the media data processing flow configuration page. The present disclosure does not limit the specific operation referred to by the first preset operation. For example, the first preset operation on the media data processing flow configuration page can be a click operation in the canvas area of the media data processing flow configuration page, or a click on a control on the media data processing flow configuration page that triggers the display of a node list, or other operations.

[0041] S120 . In response to a selection operation on a target composite node option in the node list, display a target composite node corresponding to the target composite node option in the canvas area, so that the target composite node serves as a node in the second media data processing flow to be constructed.

[0042] The selection operation on the target composite node option in the node list may, for example, reflect a user's desire to use the target composite node to construct the second media data processing flow. The target composite node option is a composite node option in the node list and is the selected composite node option. The selection operation on the target composite node option in the node list may, for example, be a click operation, a slide operation, a hover operation, or a drag operation on the target composite node option in the node list.

[0043] The second media data processing flow is the media data processing flow that needs to be constructed using the interactive method provided by the present disclosure, and is a media data processing flow that uses a composite node.

[0044] For example, see Figure 2 The media data processing flow configuration page includes a canvas area and a "component" option. If a user double-clicks a blank area in the canvas area or clicks a "component" option, it is considered as the first preset operation. Figure 3 , display the node list. The node list includes the "exclusive feature preservation" option, the "target object repair" option, and the "target object detection" option. Exclusive feature preservation, target object repair, and target object detection are all composite nodes, and these options are all composite node options. The specific nodes and / or the arrangement order between nodes used in the first media data processing flow that constitute different composite nodes are different. If the user drags the "target object repair" option to the canvas area (that is, the selection operation of the target composite node option in the node list is detected), the "target object repair" option will be used as the target composite node option, see Figure 4, displaying the target object repair node in the canvas area. Subsequently, the target object repair node can be used as a node in the second media data processing flow. That is, the entire first media data processing flow corresponding to the target object repair node can be used as a node in the second media data processing flow.

[0045] The above technical solution is configured to display a node list in response to a first preset operation on a media data processing flow configuration page. The node list includes a composite node option, which is used to represent a composite node. The composite node is a first media data processing flow in an encapsulated state. The first media data processing flow has a corresponding media data processing function. The media data processing flow configuration page includes a canvas area. In response to selecting a target composite node option in the node list, a target composite node corresponding to the target composite node option is displayed in the canvas area, so that the target composite node serves as a node in a second media data processing flow to be constructed. Essentially, the solution pre-encapsulates a usable first media data processing flow with a specific function into a composite node. When a designer needs to create a second media data processing flow, they can directly call the composite node formed by the first media data processing flow without having to build a first media data processing flow with the same function from scratch. This reduces the difficulty of constructing a node-based media data processing flow and improves the efficiency and effectiveness of building a node-based media data processing flow.

[0046] Based on the above technical solution, optionally, S120 may include: in response to a selection operation on a target composite node option in a node list, displaying a preview window, the preview window including detailed information of the target composite node and a node application option; in response to a selection operation on a node application option, displaying a target composite node corresponding to the target composite node option in the canvas area.

[0047] The preview window can be, for example, a window for previewing the target composite node. In practice, the detailed information of the target composite node may include at least one of the following: the author of the target composite node (i.e., the creator of the target composite node), release time, version number, function, and the structure of the corresponding first media data processing flow (e.g., the nodes included and the connections between the nodes).

[0048] The node application option can be, for example, an option displayed in a preview window. When the node application option is selected, it means that the user wishes to use the composite node to construct a second media data processing flow. The node application option selection operation can be, for example, a click, slide, or drag operation on the node application option.

[0049] For example, if the user hovers the mouse over Figure 3 In the "Target Object Repair" option, use the target object repair as the target compound node, see Figure 5, display the preview window. The preview window includes the detailed information of the "Target Object Repair" compound node and the node application options. Figure 5 In the example, the detailed information of the composite node specifically includes a thumbnail, author, release time, and version of the first media data processing flow corresponding to the composite node. Figure 5 The "Add to Canvas" option in the Node Application option is the node application option. If the user clicks the node application option, see Figure 4 , the target object repair node is displayed in the canvas area.

[0050] In some cases, the area used to display information related to a single node in the node list is limited. For example, only the names of each node are displayed in the node list. In actual use, the user may not be able to clearly understand whether the node can realize the function the user needs by just the node name. By setting a preview window in response to the selection operation of the target composite node option in the node list, the preview window includes detailed information of the composite node and node application options. Its purpose is to use the preview window to further display the relevant information of the target composite node, helping the user to further clarify whether it can realize the function the user needs.

[0051] Furthermore, the preview window may include a first area; the display state of the preview window may include a first state and a second state; when the display state of the preview window is the first state, a thumbnail of the first media data processing flow is displayed in the first area; when the display state of the preview window is the second state, the thumbnail of the first media data processing flow is not displayed in the first area. For example, Figure 5 The display state of the preview window is the first state, and the thumbnail of the first media data processing flow corresponding to the composite node "target object repair" is displayed in the first area. Figure 6 The preview window is displayed in the second state, and the thumbnail of the first media data processing flow corresponding to the "Target Object Repair" composite node is not displayed in the first area. In practice, the creator, manager, or user of the composite node can configure whether the preview window is displayed in the first state or the second state when needed. Alternatively, the preview window can be switched between the first and second states based on instructions.

[0052] Furthermore, it can be set that in response to the operation of adjusting the display state of the preview window to the first state, the thumbnail of the first media data processing flow is displayed in the first area; in response to the operation of adjusting the display state of the preview window to the second state, the thumbnail of the first media data processing flow is stopped from being displayed in the first area.

[0053] The operation of adjusting the display state of the preview window to the first state may be, for example, a selection operation on a specific control or option in the preview window, or a sliding operation in a specific direction within the preview window. Similarly, the operation of adjusting the display state of the preview window to the second state may be a selection operation on a specific control or option in the preview window, or a sliding operation in a specific direction within the preview window. However, it is necessary to ensure that the operation of adjusting the display state of the preview window to the first state and the operation of adjusting the display state of the preview window to the second state are not the same operation. For example, see Figure 5 , the "Expand" option is displayed in the lower right corner of the first area. If the user clicks the "Expand" option, it is determined that the operation of adjusting the display state of the preview window to the second state is detected. Figure 6 , stop displaying the thumbnail of the first media data processing stream in the first area. Figure 6 In the first area, the "Do Not Expand" option is displayed in the lower right corner. If the user clicks the "Do Not Expand" option, it is determined that the operation of adjusting the display state of the preview window to the first state is detected. Figure 5 , a thumbnail of the first media data processing flow is displayed in the first area.

[0054] In practice, different users may have different viewing requirements for the first media data processing stream. Therefore, in response to an operation of adjusting the display state of the preview window to a first state, a thumbnail of the first media data processing stream is displayed in the first area; and in response to an operation of adjusting the display state of the preview window to a second state, the thumbnail of the first media data processing stream is stopped from being displayed in the first area. In essence, this allows users to adjust the display state of the preview window according to their own needs.

[0055] Furthermore, when a thumbnail of the first media data processing flow is displayed in the first area, an area selection tool and a magnification window are displayed; in response to a selection operation of a target area in the first area using the area selection tool, a local magnified image of the first media data processing flow in the target area is displayed in the magnification window.

[0056] The area selection tool may include, for example, an area selection box or a smear tool (such as a smear pen).

[0057] The target area may be, for example, a sub-area within the first area that the user wishes to zoom in on. Zooming in on the target area means that the image within the area will be presented in a clearer state, showing more details than before zooming in. The operation of selecting the target area in the first area using the area selection tool may, for example, be an operation of indicating which sub-area in the first area is to be used as the target area using the area selection tool. Exemplarily, the operation of selecting the target area in the first area using the area selection tool may include an operation of moving the area selection box or an operation of smearing within the first area using the smear tool.

[0058] The magnification window may be, for example, a display window for presenting the magnification result of the target area image.

[0059] For example, in Figure 7 The area selection tool is an area selection box. The area enclosed by the area selection box is the target area. The zoom window displays a partial zoomed-in view of the first media data processing flow in the target area.

[0060] In practice, due to the limited size of the first area, the user may not be able to clearly see all the details of the first media data processing flow. By providing a configuration in which, in response to a selection operation of a target area in the first area using an area selection tool, a zoomed-in view of a portion of the first media data processing flow within the target area is displayed in a zoomed-in window, the user can understand the details of the first media data processing flow.

[0061] Based on the above technical solution, optionally, after S120, the method further includes: configuring parameter values of the first media data processing flow corresponding to the composite node and / or a connection relationship between the composite node and other nodes in the second media data processing flow.

[0062] Since the first media data processing flow includes multiple nodes, the purpose of configuring the parameter values of the first media data processing flow corresponding to the composite node is to enable the first media data processing flow to meet the user's personalized usage requirements, thereby making the second media data processing flow constructed based on the composite node more in line with the user's usage requirements.

[0063] It should be noted that when constructing a composite node, the creator of the composite node can make only some parameters in the first media data processing stream available to the composite node user, or make all parameters in the first media data processing stream available to the composite node user. After a parameter is available to the user, the user can configure the value of that parameter based on their needs. Furthermore, the creator of the composite node can specify which parameters in the first media data processing stream are available to the composite node user.

[0064] Based on the above technical solution, optionally, a composite node is created through the following steps: when the canvas area of the media data processing stream configuration page includes the first media data processing stream, a packaging option is displayed; in response to a selection operation of the packaging option, the first media data processing stream is packaged to obtain a composite node; and the composite node is displayed in the canvas area.

[0065] The encapsulation option may be, for example, an option corresponding to the first media data processing flow and having a function of encapsulating the corresponding first media data processing flow. The selection operation of the encapsulation option may be, for example, a click operation, a slide operation, or a drag operation on the encapsulation option.

[0066] For example, see Figure 8 The media data processing flow configuration page includes the first media data processing flow and the "Create Component" option. The "Create Component" option is the encapsulation option. If the user clicks the encapsulation option, the first media data processing flow is encapsulated to obtain a composite node corresponding to the media data processing flow. Figure 9 , and display the composite node in the canvas area. The essence of this setting is to provide a method for creating composite nodes to meet the needs of users who need to create composite nodes.

[0067] Based on the above technical solution, optionally, after displaying the composite node in the canvas area, it may also include: displaying a disassembly option corresponding to the composite node; in response to a selection operation of the disassembly option, unpacking the first media data processing stream corresponding to the composite node; and displaying the unpacked first media data processing stream in the canvas area.

[0068] The disassembly option may be, for example, an option corresponding to the composite node and used to decapsulate the first media data processing flow corresponding to the composite node. The selection operation on the disassembly option may be, for example, a click operation, a slide operation, or a drag operation on the disassembly option.

[0069] For example, see Figure 9 , the canvas area includes the "Component Decoupling" option. The "Component Decoupling" option is the disassembly option corresponding to the composite node. If the user clicks the "Component Decoupling" option, see Figure 8 , showing the first media data processing flow.

[0070] In some scenarios, after encapsulating the first media data processing stream, it may be impossible to modify or inconvenient to modify the first media data processing stream (for example, it is prone to misoperation, etc.). Here, the modification of the first media data processing stream may include, but is not limited to, adding, deleting, or replacing nodes, adjusting the connection relationship between nodes, and adjusting the parameter values of nodes. By setting a response to the selection operation of the disassembly option, the first media data processing stream corresponding to the composite node is unpacked; the unpacked first media data processing stream is displayed in the canvas area, so that when the creator needs to modify the first media data processing stream, the composite node can be restored to the first media data processing stream, thereby realizing the modification of the first media data processing stream.

[0071] Furthermore, after displaying the composite node in the canvas area, it also includes: displaying setting options corresponding to the composite node; displaying a composite node setting page in response to a selection operation on the setting option; the composite node setting page is used to set the properties of the composite node; the properties of the composite node include at least one of the following: usage permissions of the composite node; whether the first media data processing flow corresponding to the composite node is allowed to be displayed when in preview state; and whether parameters associated with the first media data processing flow corresponding to the composite node are allowed to be configured during use.

[0072] The setting option may be, for example, an option corresponding to the composite node and used to set the properties of the composite node. The selection operation on the setting option may be, for example, a click operation, a slide operation, or a drag operation on the setting option.

[0073] The composite node setting page may be, for example, a page for setting properties of a composite node.

[0074] The usage permissions for a composite node can, for example, refer to the range of users who have permission to use the composite node. For example, if the usage permissions for a composite node are set to only those members of the same creative team as the creator of the composite node, the composite node will be included in the node lists of all members of that creative team. However, if the usage permissions for a composite node are limited to the creator of the composite node, the composite node will not be included in the node lists of other members of the creative team to which the creator belongs. By configuring the usage permissions for composite nodes, composite nodes can be more effectively managed and controlled.

[0075] The preview state can be, for example, the state of displaying the detailed information of the composite node through the preview window mentioned above. Whether the first media data processing flow corresponding to the composite node is allowed to be displayed, for example, whether the first media data processing flow corresponding to the composite node is allowed to be displayed in the first area of the preview window. That is, whether the preview window is allowed to be displayed in the first state and whether the preview window is allowed to be displayed in the second state. It should be noted that here the composite node creator configures which state or states the preview window can be displayed in during the use of the composite node. If the composite node creator configures that the preview window can be displayed in the first state and cannot be displayed in the second state during the use of the composite node, then during the use of the composite node, the preview window can only be displayed in the first state. If the composite node creator configures that the preview window can be displayed in the first state and the second state during the use of the composite node, then during the use of the composite node, the display state of the preview window can be switched during the use of the composite node.

[0076] Whether the parameters associated with the first media data processing flow corresponding to the composite node are allowed to be configured during use, for example, it can refer to which parameters in the first media data processing flow are allowed to be opened to the composite node user. After a certain parameter is opened to the composite node user, the composite node user can configure the value of the parameter during use, so that the first media data processing flow corresponding to the composite node runs according to the use requirements of the composite node user. In practice, the first media data processing flow corresponding to the composite node is applied as a standardized component, so in some scenarios, the composite node user does not need to understand how the various links inside the first media data processing flow operate. By setting whether the parameters associated with the first media data processing flow corresponding to the composite node are allowed to be configured during use, its essence is to open parameters or functions related to personalized use needs to the user, thereby reducing the difficulty of using the composite node.

[0077] For example, see Figure 9 , the canvas area includes the "Component Settings" option. The "Component Settings" option is the setting option corresponding to the compound node. If the user clicks the "Component Settings" option, see Figure 10 , displaying the composite node settings page. On this settings page, "File Ownership" configures the usage permissions for the composite node. "Support Expansion" configures whether the first media data processing flow corresponding to the composite node is displayed when in preview mode. "Input Connection Point" and "Output Connection Point" configure whether parameters associated with the first media data processing flow corresponding to the composite node are configurable during use.

[0078] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0079] Figure 11 The diagram is a schematic diagram of the structure of an interactive device in an embodiment of the present disclosure. The interactive device provided in the embodiment of the present disclosure can be configured in the client or in the server. Figure 11 , the interactive device specifically includes:

[0080] A first display module 310 is configured to display a node list in response to a first preset operation on a media data processing flow configuration page, wherein the node list includes a composite node option, wherein the composite node option is used to represent a composite node; the composite node is a first media data processing flow in an encapsulated state; the first media data processing flow has a corresponding media data processing function; and the media data processing flow configuration page includes a canvas area.

[0081] The second display module 320 is used to display the target composite node corresponding to the target composite node option in the canvas area in response to the selection operation of the target composite node option in the node list, so that the target composite node serves as a node in the second media data processing flow to be constructed.

[0082] Furthermore, the second display module 320 is used to:

[0083] In response to a selection operation on a target composite node option in the node list, displaying a preview window, the preview window including detailed information of the target composite node and node application options;

[0084] In response to a selection operation on the node application option, a target composite node corresponding to the target composite node option is displayed in the canvas area.

[0085] Furthermore, the preview window includes a first area; the display state of the preview window includes a first state and a second state; the second display module 320 is further used to:

[0086] In response to an operation of adjusting the display state of the preview window to a first state, displaying a thumbnail of the first media data processing flow in the first area;

[0087] In response to the operation of adjusting the display state of the preview window to the second state, the thumbnail of the first media data processing flow is stopped from being displayed in the first area.

[0088] Furthermore, the second display module 320 is further configured to:

[0089] When the thumbnail of the first media data processing flow is displayed in the first area, a region selection tool and a magnification window are displayed;

[0090] In response to a selection operation of a target area in the first area using the area selection tool, a partially enlarged view of the first media data processing flow in the target area is displayed in the enlarged window.

[0091] Furthermore, the device also includes a configuration module, which is used to:

[0092] In response to the selection operation of the target composite node option in the node list, after the target composite node corresponding to the target composite node option is displayed in the canvas area, the values of the parameters of the first media data processing flow corresponding to the composite node and / or the connection relationship between the composite node and other nodes in the second media data processing flow are configured.

[0093] Furthermore, the device also includes a creation module, which is used to:

[0094] In a case where the canvas area of the media data processing flow configuration page includes a first media data processing flow, displaying a packaging option;

[0095] In response to a selection operation of the encapsulation option, encapsulating the first media data processing flow to obtain a composite node;

[0096] The composite node is displayed in the canvas area.

[0097] Furthermore, modules are created to:

[0098] After displaying the composite node in the canvas area, displaying a disassembly option corresponding to the composite node;

[0099] In response to a selection operation of the disassembly option, decapsulating the first media data processing flow corresponding to the composite node;

[0100] The first media data processing flow after decapsulation is displayed in the canvas area.

[0101] Furthermore, modules are created to:

[0102] After displaying the composite node in the canvas area, displaying setting options corresponding to the composite node;

[0103] In response to a selection operation on the composite node setting option, the composite node setting page is displayed; the composite node setting page is used to set the properties of the composite node; the properties of the composite node include at least one of the following: usage permissions of the composite node; whether the first media data processing flow corresponding to the composite node is allowed to be displayed when in preview state; and whether parameters associated with the first media data processing flow corresponding to the composite node are allowed to be configured during use.

[0104] The interactive device provided in the embodiment of the present disclosure can execute the steps executed by the client or server in the interactive method provided in the embodiment of the method of the present disclosure, and has the execution steps and beneficial effects, which will not be repeated here.

[0105] The following specific reference Figure 12 , which shows a schematic structural diagram of an electronic device 1000 suitable for implementing the embodiments of the present disclosure. The electronic device 1000 in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), wearable electronic devices, and fixed terminals such as digital TVs, desktop computers, smart home devices, and the like. Figure 12 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0106] like Figure 12 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes to implement the interactive method of the embodiment described in the present disclosure according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage device 1008 to the random access memory (RAM) 1003. Various programs and information required for the operation of the electronic device 1000 are also stored in the RAM 1003. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0107] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device 1000 to communicate with other devices wirelessly or by wire to exchange information. Although Figure 12 The electronic device 1000 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0108] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart, thereby implementing the interactive method as described above. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1009, or installed from the storage device 1008, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0109] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include an information signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated information signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0110] In some embodiments, the client and server can communicate using any known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital information communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any known or future developed network.

[0111] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0112] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0113] In response to a first preset operation on a media data processing flow configuration page, a node list is displayed, the node list including a composite node option, the composite node option being used to represent a composite node; the composite node is a first media data processing flow in an encapsulated state; the first media data processing flow has a corresponding media data processing function; the media data processing flow configuration page includes a canvas area;

[0114] In response to a selection operation on a target composite node option in the node list, a target composite node corresponding to the target composite node option is displayed in the canvas area, so that the target composite node serves as a node in a second media data processing flow to be constructed.

[0115] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0116] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0118] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0119] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0120] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0121] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

[0122] one or more processors;

[0123] a memory for storing one or more programs;

[0124] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the interaction methods provided in the present disclosure.

[0125] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which implements any of the interaction methods provided by the present disclosure when the program is executed by a processor.

[0126] An embodiment of the present disclosure further provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the interaction method described above is implemented.

[0127] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0128] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. An interactive method, characterized in that: include: In response to a first preset operation on the media data processing flow configuration page, displaying a node list, the node list including a composite node option, the composite node option being used to represent a composite node; The composite node is a first media data processing flow in an encapsulated state; the first media data processing flow has a corresponding media data processing function; the media data processing flow configuration page includes a canvas area; In response to a selection operation on a target composite node option in the node list, a target composite node corresponding to the target composite node option is displayed in the canvas area, so that the target composite node serves as a node in a second media data processing flow to be constructed.

2. The method according to claim 1, characterized in that In response to a selection operation on a target composite node option in the node list, displaying a target composite node corresponding to the target composite node option in the canvas area includes: In response to a selection operation on a target composite node option in the node list, displaying a preview window, the preview window including detailed information of the target composite node and node application options; In response to a selection operation on the node application option, a target composite node corresponding to the target composite node option is displayed in the canvas area.

3. The method according to claim 2, characterized in that The preview window includes a first area; the display state of the preview window includes a first state and a second state; and further includes: In response to an operation of adjusting the display state of the preview window to a first state, displaying a thumbnail of the first media data processing flow in the first area; In response to the operation of adjusting the display state of the preview window to the second state, the thumbnail of the first media data processing flow is stopped from being displayed in the first area.

4. The method according to claim 3, characterized in that Also includes: When the thumbnail of the first media data processing flow is displayed in the first area, a region selection tool and a magnification window are displayed; In response to a selection operation of a target area in the first area using the area selection tool, a partially enlarged view of the first media data processing flow in the target area is displayed in the enlarged window.

5. The method according to claim 1, wherein In response to a selection operation on a target composite node option in the node list, after displaying a target composite node corresponding to the target composite node option in the canvas area, the method further includes: The values of the parameters of the first media data processing flow corresponding to the composite node and / or the connection relationship between the composite node and other nodes in the second media data processing flow are configured.

6. The method according to claim 1, characterized in that The composite node is created by the following steps: In a case where the canvas area of the media data processing flow configuration page includes a first media data processing flow, displaying a packaging option; In response to a selection operation of the encapsulation option, encapsulating the first media data processing flow to obtain a composite node; The composite node is displayed in the canvas area.

7. The method according to claim 6, characterized in that After displaying the composite node in the canvas area, the method further includes: Displaying disassembly options corresponding to the composite node; In response to a selection operation of the disassembly option, decapsulating the first media data processing flow corresponding to the composite node; The first media data processing flow after decapsulation is displayed in the canvas area.

8. The method according to claim 6, characterized in that After displaying the composite node in the canvas area, the method further includes: Displaying setting options corresponding to the composite node; In response to a selection operation on the composite node setting option, the composite node setting page is displayed; the composite node setting page is used to set the properties of the composite node; the properties of the composite node include at least one of the following: usage permissions of the composite node; whether the first media data processing flow corresponding to the composite node is allowed to be displayed when in preview state; and whether parameters associated with the first media data processing flow corresponding to the composite node are allowed to be configured during use.

9. An interactive device, characterized in that: include: A first display module is configured to display a node list in response to a first preset operation on the media data processing flow configuration page, wherein the node list includes a composite node option, and the composite node option is used to represent a composite node; The composite node is a first media data processing flow in an encapsulated state; the first media data processing flow has a corresponding media data processing function; the media data processing flow configuration page includes a canvas area; The second display module is used to display a target composite node corresponding to the target composite node option in the node list in response to a selection operation of the target composite node option, so that the target composite node serves as a node in the second media data processing flow to be constructed.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images