Workflow deployment method and device, readable medium, equipment and program product

Abstract

The embodiment of the invention discloses a workflow deployment method and device, a readable medium, equipment and a program product. The method comprises the steps that a workflow design interface comprising a workflow editing component and a workflow deployment component is displayed; in response to an editing operation on the workflow editing component, obtaining a selected reference node to construct a to-be-deployed workflow; in response to a trigger operation on the workflow deployment component, obtaining execution performance data of the reference node in the process of executing the data processing task, and optimizing the to-be-deployed workflow based on the execution performance data corresponding to the reference node to obtain a target workflow; obtaining a dependent data packet required by operation of a target node in the target workflow, and generating an operation data packet corresponding to the target workflow based on the dependent data packet corresponding to the target node; and creating a container instance corresponding to the target workflow based on the operation data packet to deploy the target workflow. By adopting the embodiment of the invention, the deployment efficiency of the workflow can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to methods, apparatus, readable media, devices and program products for deploying workflows. Background Technology

[0002] Workflow is a high-level abstraction of the computational process in the form of a data flow graph. It encapsulates volatile artificial intelligence algorithms into stable nodes, managing system complexity through graph construction rather than coding. It is an engineering solution for handling multimodal, long-chain AI generation tasks. However, currently, workflow deployment is mainly done manually, which is extremely inefficient. Therefore, improving the deployment efficiency of workflows is a pressing technical problem that needs to be solved. Summary of the Invention

[0003] This application provides a workflow deployment method, apparatus, readable medium, device, and program product that can improve workflow deployment efficiency.

[0004] On the one hand, embodiments of this application provide a workflow deployment method, including: Displays a workflow design interface; wherein the workflow design interface includes a workflow editing component and a workflow deployment component; In response to an editing operation on the workflow editing component, a reference node selected from at least one preset node is obtained, and a workflow to be deployed is constructed based on the reference node; In response to a trigger operation on the workflow deployment component, the execution performance data of the reference node in the workflow to be deployed during the execution of data processing tasks is obtained, and the workflow to be deployed is optimized based on the execution performance data corresponding to the reference node to obtain the target workflow; Obtain the dependency data package required for the target node to run in the target workflow, and generate the running data package corresponding to the target workflow based on the dependency data package corresponding to the target node; Based on the runtime data package, a container instance corresponding to the target workflow is created to deploy the target workflow.

[0005] On one hand, embodiments of this application provide a workflow deployment apparatus, which includes a display unit, an editing response unit, a trigger response unit, an acquisition unit, and a deployment unit, wherein: The display unit is used to display the workflow design interface; wherein, the workflow design interface includes a workflow editing component and a workflow deployment component; The editing response unit is configured to respond to the editing operation of the workflow editing component, obtain a reference node selected from at least one preset node, and construct a workflow to be deployed based on the reference node; The trigger response unit is used to respond to the trigger operation of the workflow deployment component, obtain the execution performance data of the reference node in the workflow to be deployed during the execution of data processing tasks, and optimize the workflow to be deployed based on the execution performance data corresponding to the reference node to obtain the target workflow; The acquisition unit is used to acquire the dependency data packets required for the target node to run in the target workflow, and generate the running data packets corresponding to the target workflow based on the dependency data packets corresponding to the target node; The deployment unit is used to create a container instance corresponding to the target workflow based on the running data package, so as to deploy the target workflow.

[0006] In one embodiment of this application, the number of reference nodes includes at least one. Based on the aforementioned scheme, when the trigger response unit optimizes the workflow to be deployed based on the execution performance data corresponding to the reference nodes to obtain the target workflow, it can be used to: analyze and obtain the performance bottleneck nodes in the workflow to be deployed based on the execution performance data corresponding to at least one reference node; update the performance bottleneck nodes in the workflow to be deployed to obtain the updated workflow; and perform node pruning on the updated workflow to obtain the target workflow.

[0007] In one embodiment of this application, based on the aforementioned scheme, when the trigger response unit performs node pruning on the updated workflow to obtain the target workflow, it can specifically be used to: construct a node adjacency list based on the data input and data output relationships between every two nodes in the updated workflow; select a target endpoint node from the endpoint nodes contained in the updated workflow; obtain the dependent nodes of the target endpoint node in the updated workflow based on the node adjacency list; and prune the nodes in the updated workflow other than the target endpoint node and dependent nodes to obtain the target workflow.

[0008] In one embodiment of this application, the execution performance data corresponding to each reference node includes: the execution time of each reference node performing a data processing task, and the resource usage interval sequence data obtained based on the amount of computing resources occupied by each reference node during the execution of the data processing task. Based on the aforementioned scheme, when the trigger response unit analyzes and obtains the performance bottleneck node in the workflow to be deployed based on the execution performance data corresponding to at least one reference node, it can specifically be used to: construct the execution time sequence data corresponding to the workflow to be deployed based on the execution time corresponding to at least one reference node and the node execution order corresponding to at least one reference node; concatenate the resource usage interval sequence data corresponding to at least one reference node according to the node execution order corresponding to at least one reference node to obtain the resource usage interval sequence data corresponding to the workflow to be deployed; generate a node performance analysis interface based on the execution time sequence data corresponding to the workflow to be deployed and the resource usage interval sequence data corresponding to the workflow to be deployed; and display the node performance analysis interface to obtain the performance bottleneck node based on the node performance analysis interface.

[0009] In one embodiment of this application, based on the aforementioned scheme, when the trigger response unit obtains the execution performance data of the reference node in the workflow to be deployed during the execution of data processing tasks, it can be further configured to: obtain the amount of computing resources occupied by the reference node at each execution time point during the execution of data processing tasks by the reference node; construct a resource occupation interval sequence data corresponding to the reference node based on each execution time point and the resource amount corresponding to each execution time point; obtain the start execution time point when the reference node begins to execute the data processing task and the end execution time point when the reference node stops executing the data processing task from at least two execution time points corresponding to the reference node; obtain the execution duration of the reference node based on the start execution time point and the end execution time point corresponding to the reference node; and obtain the execution performance data corresponding to the reference node based on the resource occupation interval sequence data and the execution duration corresponding to the reference node.

[0010] In one embodiment of this application, based on the aforementioned scheme, after the trigger response unit optimizes the workflow to be deployed based on the execution performance data corresponding to the reference node to obtain the target workflow, it can specifically be used to: obtain the execution code segment corresponding to the target node in the target workflow; delete and update the interface display code segment contained in the execution code segment to obtain the target execution code segment. Then, when the acquisition unit generates the runtime data package corresponding to the target workflow based on the dependency data package corresponding to the target node, it can specifically be used to generate the runtime data package corresponding to the target workflow based on the dependency data package corresponding to the target node and the target execution code segment.

[0011] In one embodiment of this application, based on the foregoing scheme, the device further includes a data mapping unit. Before displaying the workflow design interface, the data mapping unit is used to acquire the code segments called during the registration process of each preset node; to parse the storage path of the acquired code segments to obtain the source data packets of the acquired code segments; and to establish a mapping relationship between each preset node and the corresponding parsed source data packets to obtain at least one mapping relationship. Therefore, when the acquisition unit acquires the dependency data packets required for the target node to run in the target workflow, it can specifically be used to acquire the source data packets corresponding to the target node based on at least one mapping relationship, and use the acquired source data packets as the dependency data packets required for the target node to run.

[0012] In one embodiment of this application, based on the aforementioned scheme, when the deployment unit creates a container instance corresponding to the target workflow based on the running data package, it can specifically be used to: obtain a preset image deployment script; wherein, the preset image deployment script contains the image identifier of the image required for each preset node to run; obtain a target image that matches the image identifier corresponding to the target node based on the preset image deployment script; and construct a container instance corresponding to the target workflow according to the target image and the running data package.

[0013] In one embodiment of this application, the apparatus further includes a testing unit. Based on the aforementioned scheme, after creating a container instance corresponding to the target workflow based on the running data packet to deploy the target workflow, the testing unit can be used to obtain the target node type to which the target node running in the container instance belongs; obtain the target parameter mapping rule corresponding to the target node type according to the pre-set correspondence between node type and parameter mapping rule; assign values ​​to the input parameters in the target node based on the target parameter mapping rule to obtain the workflow to be tested; call the workflow to be tested to execute the test task to obtain the execution test result for the target workflow; if the execution test result indicates that the target workflow passes the test, then the target workflow is deployed to the production environment.

[0014] On one hand, embodiments of this application provide an electronic device, which includes an input interface and an output interface, and further includes: A processor, adapted to implement one or more instructions; and, A computer storage medium storing one or more instructions adapted for loading and execution by the processor of the above-described workflow deployment method.

[0015] On one hand, embodiments of this application provide a computer-readable medium storing computer program instructions, which, when executed by a processor, are used to perform the deployment method of the above-described workflow.

[0016] On one hand, embodiments of this application provide a computer program product or computer program, the computer program product or computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium; a processor of an electronic device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, and when the computer instructions are executed by the processor, they are used to execute the deployment method of the above-mentioned workflow.

[0017] In the technical solution provided by the embodiments of this application, by obtaining the execution performance data of the nodes in the workflow to be deployed during the execution of data processing tasks, accurate optimization of the workflow to be deployed can be achieved based on the execution performance data. Simultaneously, by pulling the dependency data packages corresponding to the target nodes in the optimized target workflow, the embodiments of this application can generate the runtime data package corresponding to the target workflow. Thus, the runtime data package contains all the data resources required for the nodes in the target workflow to run, thereby automatically creating a container instance corresponding to the target workflow based on the runtime data package, achieving rapid deployment of the target workflow. It is evident that the embodiments of this application can achieve automated and rapid deployment of the entire lifecycle of a workflow from development to application, which is beneficial for significantly improving the deployment efficiency of workflows. Furthermore, in the embodiments of this application, the main human intervention is in the development and design of the workflow. Developers only need to trigger the workflow deployment component to achieve one-click rapid deployment of the workflow, thereby significantly improving the deployment efficiency of workflows while reducing labor costs. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the architecture of a workflow deployment system provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a workflow deployment method provided in an embodiment of this application; Figure 3 This is a schematic diagram of a workflow design interface provided in an embodiment of this application; Figure 4 This is a schematic diagram illustrating the online development phase during an automated deployment process provided in an embodiment of this application; Figure 5 This is a comparative schematic diagram of an executable code segment provided in an embodiment of this application; Figure 6 This is a schematic diagram illustrating the process of saving and publishing during an automated deployment, as provided in an embodiment of this application. Figure 7 This is a schematic diagram illustrating the service deployment phase during an automated deployment process provided in an embodiment of this application. Figure 8 This is a flowchart illustrating another workflow deployment method provided in an embodiment of this application; Figure 9 This is a schematic diagram of another workflow design interface provided in an embodiment of this application; Figure 10 This is a schematic diagram of a node performance analysis interface provided in an embodiment of this application; Figure 11 This is a schematic diagram of the structure of a workflow deployment device provided in an embodiment of this application; Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0021] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0022] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0023] The flowcharts shown in the accompanying diagrams are merely illustrative and do not necessarily include all content and operations, nor do they necessarily have to be executed in the described order. For example, some operations may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0025] It should also be noted that "multiple" as mentioned in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0026] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0027] Workflow: An AI-generated task process consisting of multiple nodes is a high-level abstraction of the computation process in the form of a data flow graph. It encapsulates volatile AI algorithms into stable nodes and manages system complexity by constructing graphs rather than coding them. It is an engineering solution for dealing with multimodal, long-chain AI-generated tasks.

[0028] AI-generated content (AIGC) refers to new digital content (such as text, images, audio, video, code, etc.) that automatically synthesizes according to specific semantic or stylistic requirements by using generative artificial intelligence models to learn the joint probability distribution of large-scale datasets.

[0029] ComfyUI (Customizable Modular Framework User Interface for Stable Diffusion): A node-based AI image / video generation workflow design tool.

[0030] Currently, many steps in the workflow deployment process require manual intervention. For example, developers need to optimize the workflow based on their experience (such as manually identifying and deleting redundant nodes), and they need to manually identify the dependency list required for each node to run and package and deploy it. It is clear that the current workflow deployment process is extremely cumbersome and requires a large amount of manual intervention, resulting in a workflow often taking several weeks from design to deployment, which is highly inefficient.

[0031] Based on this, this application provides a workflow deployment scheme. In this scheme, a workflow design interface is displayed, where developers can edit and select nodes to build a workflow to be deployed. When the workflow deployment component in the workflow design interface is triggered, the backend can automatically obtain the execution performance data of the nodes in the workflow to be deployed during the execution of data processing tasks. Based on the execution performance data of the nodes, the workflow to be deployed is optimized to obtain the target workflow. Furthermore, based on the dependency data packages of the target nodes in the target workflow, a runtime data package corresponding to the target workflow is generated, and a container instance corresponding to the target workflow is created based on the runtime data package to deploy the target workflow.

[0032] It is easy to see that this solution can accurately optimize the workflow to be deployed by obtaining the execution performance data of the nodes in the workflow during the execution of data processing tasks. At the same time, this solution can generate the runtime data package corresponding to the target workflow by pulling the dependency data package corresponding to the target node in the optimized target workflow. Based on the runtime data package, the container instance corresponding to the target workflow can be automatically created, thereby realizing the rapid deployment of the target workflow.

[0033] As can be seen, this solution enables automated and rapid deployment of workflows from development to application. Therefore, the main human intervention in this solution is in the development and design of the workflow. Developers only need to trigger the workflow deployment component to achieve one-click rapid deployment of the workflow, thereby significantly improving the deployment efficiency of the workflow.

[0034] Based on the above workflow deployment method, this application provides a workflow deployment system, which can be found in [reference needed]. Figure 1 , Figure 1 The workflow deployment system shown may include terminal devices 101 and servers 102. The number of terminal devices 101 may include at least one, and the number of servers 102 may include at least one. A communication connection is established between any terminal device and any server. Figure 1As shown, terminal device 101 may include any one or more of the following: sensors, smartphones, tablets, laptops, desktop computers, smart vehicles, and smart wearable devices. Terminal device 101 may run applications for developing workflows, as well as various other applications (APPs), such as shopping clients, payment clients, game clients, virtual interaction clients, multimedia playback clients, social clients, information streaming clients, and so on.

[0035] Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal device 101 and server 102 can communicate directly or indirectly via wired or wireless communication; this application does not impose any restrictions on this.

[0036] In one embodiment, the deployment method of the above workflow can be solely determined by... Figure 1 The terminal device 101 in the workflow deployment system executes the following process: Terminal device 101 can display a workflow design interface, which includes a workflow editing component and a workflow deployment component. Then, in response to an editing operation on the workflow editing component, terminal device 101 obtains a reference node selected from at least one preset node to construct the workflow to be deployed based on the reference node. Additionally, in response to a trigger operation on the workflow deployment component, terminal device 101 obtains the execution performance data of the reference node in the workflow to be deployed during the execution of data processing tasks, and optimizes the workflow to be deployed based on the execution performance data corresponding to the reference node to obtain the target workflow. Then, terminal device 101 obtains the dependency data package required for the target node to run in the target workflow, and generates a runtime data package corresponding to the target workflow based on the dependency data package corresponding to the target node. Finally, terminal device 101 creates a container instance corresponding to the target workflow based on the runtime data package to deploy the target workflow.

[0037] Alternatively, the deployment method of the above workflow can also be solely determined by... Figure 1 The workflow shown is executed by server 102 in the deployment system. The specific execution process can be found in the specific execution process of terminal device 101 mentioned above, and will not be repeated here.

[0038] In another embodiment, the above-described workflow deployment method can be provided by... Figure 1The deployment system of the workflow shown in the diagram is jointly executed by terminal device 101 and server 102. The specific execution process is as follows: Terminal device 101 can display a workflow design interface, which includes a workflow editing component and a workflow deployment component. Then, in response to the editing operation of the workflow editing component, terminal device 101 can obtain a reference node selected from at least one preset node to build a workflow to be deployed based on the reference node. In response to the triggering operation of the workflow deployment component, terminal device 101 can send the workflow to be deployed to server 102. After receiving the workflow to be deployed, server 102 can obtain the execution performance data of the reference node in the workflow to be deployed during the execution of data processing tasks. Based on the execution performance data corresponding to the reference node, server 102 can optimize the workflow to be deployed to obtain the target workflow. Server 102 can also obtain the dependency data package required for the target node to run in the target workflow and generate the running data package corresponding to the target workflow based on the dependency data package corresponding to the target node. Finally, server 102 can create a container instance corresponding to the target workflow based on the running data package to deploy the target workflow.

[0039] It should be noted that the embodiments of this application can be applied to various business scenarios that require the use of artificial intelligence to generate content and therefore need to develop and deploy corresponding workflows, including but not limited to smart finance, smart transportation, smart e-commerce, smart logistics, live streaming, games and other business scenarios, and are not limited thereto.

[0040] Specifically, when creative design is required, workflows for AI-powered painting, image editing, and style transfer can be deployed; when generating multimodal content, workflows for image generation, video generation, and audio generation can be deployed; when promoting holiday activities, workflows for AI-powered costume changes and virtual avatar generation can be deployed; in game applications, workflows for generating virtual characters and rendering scenes can be deployed; and in video production, workflows for generating AI-powered dances, motion transfer, and special effects can be deployed.

[0041] For example, in a smart e-commerce scenario, the above solution can be used to deploy a workflow for generating e-commerce product images using artificial intelligence, so that sellers can quickly list new e-commerce products; similarly, in a smart transportation scenario, the above solution can be used to deploy a workflow for generating predictions of traffic congestion in future time periods using artificial intelligence, so that drivers can make reasonable travel arrangements.

[0042] Furthermore, in specific embodiments of this application, if workflow, performance data, or other data or information involve objects, when the embodiments of this application are applied to specific products or technologies, permission or consent from the objects is required, and the collection, use, and processing of related data or information must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0043] Based on the above workflow deployment scheme and system, this application provides a workflow deployment method. See also... Figure 2 This is a flowchart illustrating a workflow deployment method provided in an embodiment of this application. Figure 2 The deployment method of the workflow shown can be... Figure 1 The workflow shown is executed by a server or terminal device in the deployment system, using this method. Figure 2 The workflow shown is illustrated using the terminal device execution in the deployment system as an example.

[0044] in, Figure 2 The deployment method of the workflow shown may include steps S201 to S205: S201 displays the workflow design interface; the workflow design interface includes a workflow editing component and a workflow deployment component.

[0045] In this embodiment of the application, the workflow editing component may include a node editing component and a connection relationship editing component; wherein, the node editing component may provide at least one preset node for developers to select.

[0046] The connection editing component is used to establish connections between nodes; these connections can specifically include input and output relationships between nodes, and logical links between nodes can be established to define data flow or control flow. The workflow deployment component is used to trigger the automated deployment of workflows.

[0047] For example, a workflow editing component may include an interactive canvas area, at least one draggable node module (i.e., a node editing component), and a connection tool (i.e., a connection relationship editing component). Developers can drag and drop node modules onto the canvas area and use the connection tool to establish logical links between these nodes to define data flow or control flow.

[0048] S202, in response to an editing operation on the workflow editing component, obtain a reference node selected from at least one preset node, and construct the workflow to be deployed based on the reference node.

[0049] In the embodiments of this application, each preset node refers to a pre-encapsulated software module unit with specific functions that can be directly invoked. As a basic functional block for building a workflow, a node represents an independent operation step or logical unit in the data processing flow.

[0050] Specifically, each preset node may include one of the following: a clearly defined input interface, an output interface, and configurable internal parameters. For example, when designing an image processing workflow, the preset nodes selected by the developer through the workflow editing component may include: an "image loading node" (whose input interface is a file path and whose output interface is an image data matrix), a "Gaussian blur filter node" (whose input interface receives image data, whose internal parameters include configurable kernel size and standard deviation, and whose output interface is the processed image), and an "image saving node" (whose input interface receives image data, and whose internal parameters include the save path and format).

[0051] Optionally, in this embodiment, the preset node library can adopt an extensible architecture, supporting developers to create and import new nodes as preset nodes according to specific domain needs (such as audio processing, text analysis, or 3D rendering), thereby expanding the functionality of the workflow and broadening the scope of workflow design applicability.

[0052] Furthermore, editing operations on workflow editing components can include selection operations on at least one preset node. Specifically, the selection operation on at least one preset node can vary depending on the form of the workflow editing component (which may specifically be a node editing component).

[0053] For example, if the node editing component included in the workflow editing component is a drop-down selector, then the selection operation for at least one preset node may include: a click operation on the drop-down selector (to display at least one preset node), and a click operation on any one of the at least one preset nodes (to select any preset node as a reference node). If the node editing component in the workflow editing component includes at least one draggable node module (one node module corresponds to one preset node), then the selection operation for at least one preset node may include a drag-and-drop operation on at least one draggable node module.

[0054] In some embodiments, editing operations on workflow editing components may also include editing operations on connection relationship editing components to establish connections between nodes. Specifically, editing operations on connection relationship editing components may vary depending on the form of the connection relationship editing components, and will not be elaborated here.

[0055] Therefore, in some embodiments, the specific process of constructing a workflow to be deployed based on reference nodes may include: obtaining the connection relationship between reference nodes; and constructing the workflow to be deployed based on the reference nodes and the connection relationship between reference nodes.

[0056] For example, please see the appendix. Figure 3 The diagram shows a schematic of a workflow design interface. Figure 3 The workflow design interface 301 includes a canvas area 302 for editing workflows and a workflow deployment component 303.

[0057] Developers can select one or more node modules from at least one node module (i.e., preset nodes) and drag them to canvas area 302. For example... Figure 3 As shown, canvas area 302 contains node modules 304 and 305 that can be dragged and dropped by the developer. Node module 304 is used to load the pre-trained model, and its ID is 4, indicating that node module 304 is the 4th node in the current workflow. Node module 304 has three output ports: the main model output port (the main model can be a diffusion model), the text encoder output port, and the variational autoencoder output port.

[0058] Node module 305 is used to encode the prompt words using a text encoder. Its ID is 6, indicating that node module 304 is the 6th node in the current design workflow. Node module 305 has a text encoder input port and a conditional vector output port. Additionally, node module 305 includes a prompt word input box 306 for inputting the prompt words for the image to be generated.

[0059] like Figure 3 As shown, the developer established a connection relationship 307 between the text encoder output port in node module 304 and the text encoder input port in node module 305 through the connection relationship editor. In this way, the text encoder output by node module 304 can be input to node module 305, so that node module 305 can call the text encoder to encode the input prompt words and output the encoded text vector as a condition vector.

[0060] Finally, after designing the workflow to be deployed, developers only need to click the workflow deployment component 303 in the workflow design interface 301 to achieve one-click deployment of the workflow.

[0061] It is not difficult to see that Figure 3The nodes in the example can be displayed in the workflow design interface as graphical elements (such as icons or frames), which developers can instantiate onto the canvas area by dragging and dropping. In this way, developers only need to configure the nodes and their parameters (such as the prompts in node module 305 mentioned above) and connect their input / output ports to other nodes via connection lines to quickly build complex data processing workflows. This pre-defined node mechanism encapsulates complex algorithms or operations into standardized, reusable components, greatly reducing the technical threshold and error probability of workflow construction. Simultaneously, by displaying the workflow deployment component 303 in the workflow design interface, the designed workflow can be deployed promptly and quickly, shortening the time from development and design to deployment and release, thereby significantly increasing the workflow iteration frequency and making it more adaptable to business needs.

[0062] S203, in response to the trigger operation of the workflow deployment component, obtain the execution performance data of the reference node in the workflow to be deployed during the execution of data processing tasks, and optimize the workflow to be deployed based on the execution performance data of the reference node to obtain the target workflow.

[0063] In this embodiment, the triggering operation includes at least one interactive operation on the workflow deployment component, used to initiate the deployment process of the workflow to be deployed. Specifically, the triggering operation may include a click operation, a long-press click operation, a double-click operation, etc., and is not limited thereto.

[0064] In addition, the number of reference nodes may include at least one; the execution performance data may include at least one of the following: the execution time of each reference node in executing the data processing task, the amount of computing resources occupied by each reference node in the process of executing the data processing task, and the resource occupation interval sequence data corresponding to each reference node.

[0065] The resource quantity corresponding to each reference node includes the amount of computing resources occupied by each reference node at each execution time point; each execution time point refers to each time point in the process of executing the data processing task.

[0066] A time point refers to a specific and unique position on a continuous timeline. Specifically, a time point represents a non-continuous instantaneous state, and its granularity depends on the time unit used. For example, in a system with "minutes" as the smallest time unit, a time point can be represented as "10:59" or "10:58"; in a system with "seconds" as the smallest time unit, a time point can be further refined to "10:59:41" or "10:59:42"; in higher precision systems, the resolution of time points can reach milliseconds, microseconds, or even nanoseconds, which is not limited in this embodiment.

[0067] In some embodiments, the workflow to be deployed can be invoked to execute a pre-defined data processing task in order to obtain the execution performance data of the reference node in the workflow to be deployed during the execution of the data processing task.

[0068] Optionally, simulation data related to the execution of data processing tasks can be obtained, and independent execution tests can be performed on each reference node in the workflow to be deployed based on the simulation data, thereby obtaining the execution performance data of each reference node during the execution of data processing tasks. Optionally, the execution performance data of reference nodes during the execution of data processing tasks can also be obtained through other methods, which are not limited here.

[0069] In some embodiments, the specific process of optimizing the workflow to be deployed based on the execution performance data corresponding to the reference node can be as follows: (1) Based on the execution performance data corresponding to at least one reference node, the performance bottleneck node in the workflow to be deployed is analyzed.

[0070] Among them, the performance bottleneck node refers to the node that consumes significantly more time or computing resources than other nodes in the workflow during execution, and that limits the overall execution efficiency and throughput of the workflow.

[0071] Optionally, a performance bottleneck node may have at least one of the following characteristics: the corresponding execution time is greater than the preset execution time, the ratio between the corresponding execution time and the total execution time of the workflow is greater than the preset ratio, the amount of computing resources occupied at any execution time point reaches the preset amount of resources, and the total amount of resources occupied during the execution of data processing tasks reaches the resource threshold, etc.

[0072] The preset execution duration, preset ratio, preset resource quantity, and resource quantity threshold can be set manually or by the server or terminal device in the system, and are not limited here. For example, if the server detects that the resource quantity occupied by a node at any execution time reaches a certain value, which will hinder the concurrent execution of other nodes, then that value can be used as the preset resource quantity.

[0073] (2) Update the performance bottleneck nodes in the workflow to be deployed to obtain the updated workflow.

[0074] Optionally, the update processing of performance bottleneck nodes in the deployment workflow may include at least one of the following: node replacement, node parameter tuning, node reconstruction, etc.

[0075] Node replacement refers to replacing a performance bottleneck node with a functionally similar but more efficient one. For example, a node encapsulating a Standard Variational Autoencoder Decoder (VAE decoder) can be replaced with a node encapsulating a Dedicated Simplified Autoencoder for Stable Diffusion (TAESD decoder). Node parameter tuning refers to adjusting the parameters in the performance bottleneck node. For example, the number of sampling steps in the performance bottleneck node can be reduced, or the sampler in the performance bottleneck node can be updated to a faster sampler. Node refactoring refers to changing nodes that are executed serially in the workflow to be executed in parallel.

[0076] (3) Perform node pruning on the updated workflow to obtain the target workflow.

[0077] Optionally, the specific process of performing node pruning on the updated workflow may include: constructing a node adjacency list based on the data input and data output relationships between every two nodes in the updated workflow; then, selecting a target endpoint node from the endpoint nodes contained in the updated workflow; subsequently, obtaining the dependent nodes of the target endpoint node in the updated workflow based on the node adjacency list; and finally, pruning the nodes in the updated workflow other than the target endpoint node and dependent nodes to obtain the target workflow.

[0078] Optionally, the specific construction process of the node adjacency table may include: traversing each node in the updated workflow, identifying the output ports connected to each node's input ports (specifically, the node's inputs field), and obtaining the node identifier (i.e., source_id) and output index (i.e., output_idx, indicating which output port of the source node is used) of the corresponding output port. Then, based on each node's node identifier, the index of the input port, the obtained node identifier, and the output index, a mapping relationship representing the data output from the source node's output port to each node's input port can be constructed, thus obtaining the node adjacency table.

[0079] Specifically, this embodiment traverses each node in the updated workflow, accurately identifies the specific output port connected to the input of each node, and obtains the node identifier and output index of the source node corresponding to that output port. This allows for the construction of a port-level node mapping relationship, thereby generating a high-precision node adjacency table. Furthermore, compared to traditional solutions that only record coarse-grained connections between nodes, this embodiment, through a high-precision port-level node adjacency table, enables more accurate identification of redundant nodes for the target endpoint during subsequent pruning. This effectively avoids incorrect or missed pruning due to ambiguous connection relationships, significantly improving the accuracy and reliability of the subsequent pruning process. Ultimately, this optimizes the workflow's execution efficiency and resource utilization, providing crucial technical support for the automated optimization and deployment of complex workflows.

[0080] Optionally, a node can be determined as an end point node by checking if it has an end point marker attribute configured. If configured, the node is considered an end point node; otherwise, it is not. Specifically, the end point marker attribute can be the node's OUTPUT_NODE field, used to mark the final output node in the workflow.

[0081] Optionally, some or all of the endpoint nodes contained in the updated workflow can be selected as the target endpoint node, which is not limited in this application.

[0082] Optionally, the target endpoint node can be selected manually, or it can be selected by the server or terminal device in the above system. This application does not limit this.

[0083] Taking the selection of a target endpoint node by a user as an example, the specific process of selecting a target endpoint node from the endpoint nodes contained in the updated workflow may include: displaying the updated workflow and highlighting the endpoint nodes in the updated workflow; in response to a click operation on an endpoint node in the updated workflow, obtaining the clicked endpoint node and using the clicked endpoint node as the target endpoint node. The highlighting may include at least one of the following: highlighting, displaying with a different color than other nodes, bolding the node identifier of the endpoint node, etc., and is not limited here.

[0084] Optionally, the specific process of obtaining dependent nodes based on the node adjacency list may include: starting from the target endpoint node, performing a depth-first traversal of the updated workflow; for the current node encountered during traversal, the node identifier of the current node can be written into the node set first; then, if the source node corresponding to the current node in the node adjacency list has not been traversed, the source node is used as the next node to be traversed until the traversal ends.

[0085] Thus, the nodes indicated by the node identifiers in the final node set are the dependent nodes of the target endpoint node in the updated workflow. Specifically, this embodiment starts from the target endpoint node and performs a depth-first traversal of the updated workflow based on a precise node adjacency list and port-level node connections. During the traversal, reverse dependency detection can be strictly performed according to the actual data flow, thereby avoiding the dependency omissions or misjudgments caused by the general connection relationships in traditional dependency analysis. This allows for the automated and high-precision identification of all upstream nodes that the target endpoint node depends on in the updated workflow, significantly improving the accuracy of node pruning.

[0086] In general, during the design and debugging iterations of a workflow, a large number of redundant nodes may accumulate that are not actually referenced by the execution path of the target endpoint node. These nodes will still be loaded and initialized in production deployment, consuming computing resources and potentially causing unnecessary computational scheduling. According to actual data analysis, such invalid computations generally cause 20-40% of resource and time waste in production environments.

[0087] This embodiment first accurately locates performance bottleneck nodes based on execution performance data and updates them accordingly. Then, it systematically performs node pruning on the updated workflow, automatically identifying and removing all redundant nodes without any manual intervention. This pruning algorithm is based on rigorous graph reachability analysis, tracing back from the target endpoint node and retaining only the node paths that can reach the endpoint, achieving 100% removal accuracy. This results in a simpler workflow without any manual intervention. Thus, the target workflow obtained through this embodiment significantly reduces the computational resources required for its operation compared to the original workflow to be deployed. The overall execution time of the workflow is also significantly shortened because there is no need to initialize and calculate redundant nodes, thereby improving overall throughput. It is clear that this embodiment greatly improves the overall efficiency and reliability of workflow optimization, thus contributing to improved overall deployment efficiency and reliability.

[0088] In some possible implementations, after obtaining the target workflow, the editable parameters in the target workflow can be set to empty (thus retaining only the workflow's connection structure); then, the target workflow with empty parameters is JSON serialized (the process of converting a data structure or object state into a JSON (JavaScript Object Notation) string for storage or transmission), and the hash value of the serialized data is calculated; finally, the calculated hash value can be stored as the version identifier of the target workflow.

[0089] Specifically, in traditional workflow management, version identification often relies on manual naming or hashing of complete files (including parameters and structure). This leads to workflows with identical underlying connection structures but different parameter configurations being misclassified as different versions, resulting in significant storage redundancy and version confusion in team collaboration. This embodiment removes volatile configuration information by setting all editable parameters in the target workflow to null, extracting the connection structure representing the workflow's essential topology and logic. This pure structural data is then serialized into JSON, generating a standardized, unambiguous string representation. Finally, a hash value is calculated for this string, and this hash value is stored and associated as a unique identifier for the workflow structure version. This ensures that as long as the workflow's node type and connection topology are identical, regardless of parameter settings, the same version identifier hash value will be generated. This embodiment achieves true structure deduplication storage, significantly saving storage space. In collaborative development, developers can quickly and accurately determine whether two workflows have the same underlying structure using the version identifier; in deployment and production environments, the version identifier can be used to accurately match and roll back to a specific workflow structure version.

[0090] In some possible implementations, during the node pruning process, at least one of the following information can also be recorded: the number of original nodes in the updated workflow, the number of dependent nodes of the target endpoint node obtained from the analysis, the number of nodes pruned and removed from the updated workflow (also known as the number of removed nodes), the number of nodes reduced in the target workflow relative to the updated workflow, and the difference in data volume between the dependency data package required for the updated workflow to run and the dependency data package required for the target workflow to run.

[0091] Then, a workflow pruning report can be generated based on the recorded information, so that developers can evaluate the target workflow based on the node pruning status reported in the node pruning report.

[0092] In practical applications, the deployment workflow can be optimized within an online development environment. Please see the appendix. Figure 4 This illustrates a process diagram of the online development phase during automated deployment. For example... Figure 4 As shown, the online development phase can include a customizable node-based user interface (i.e., ComfyUI), integrated development tools, workflow optimization modules, and detection modules.

[0093] The integrated development tools may include at least one of the following: Jupyter Laboratory (JupyterLab for short, a web-based integrated development environment that provides an interactive interface that allows you to create and run documents containing live code, visual charts, mathematical formulas and narrative text), Visual Studio Code (VSCode for short, a free, open-source, cross-platform source code editor), and File Browser (a core graphical user interface component that allows browsing, managing, uploading, downloading, renaming and deleting files and directories in the running file system).

[0094] Specifically, developers can design and edit workflows using ComfyUI; the integrated development environment (IDE) provides technical support for customizable node-based user interfaces, and developers can also use the IDE to create nodes and their corresponding dependency packages. After developers obtain the workflow to be deployed through ComfyUI, they can call the workflow optimization module in the online development environment to optimize the workflow to obtain the target workflow. Simultaneously, the detection module in the online development environment can perform log checks on ComfyUI and the workflow to be deployed, as well as performance checks on reference nodes in the workflow.

[0095] The workflow optimization module includes functions such as performance detection, performance optimization, and automatic pruning. Performance detection involves acquiring execution performance data of reference nodes in the workflow to be deployed during data processing tasks. Performance optimization involves analyzing the execution performance data of at least one reference node to identify performance bottleneck nodes in the workflow and updating these bottleneck nodes to obtain an updated workflow. Automatic pruning involves pruning nodes in the updated workflow to obtain the target workflow. Finally, as... Figure 4 As shown, the target workflow can be stored in the code repository for persistent storage.

[0096] Specifically, a depth-first search reverse traversal algorithm based on directed acyclic graphs can be used to automatically identify and remove redundant nodes in the workflow to be deployed, thereby achieving automatic pruning. The algorithm process is as follows: (1) Construct a node adjacency table: Traverse the inputs field (i.e., input port field) of each reference node in the workflow to be deployed, identify the source_id (i.e., the node identifier of the source node) and output_idx (i.e., the output port index) corresponding to the input port, thereby constructing a mapping relationship between each reference node (target node) and the corresponding source node (source nodes), and constructing a node adjacency table based on the mapping relationship.

[0097] (2) Automatic identification of target endpoint node: By checking the OUTPUT_NODE attribute of each reference node in the workflow to be deployed, the endpoint node in the workflow to be deployed is automatically identified without manual annotation, with an accuracy of 100%; then, the target endpoint node can be selected from the identified endpoint nodes.

[0098] (3) Reverse dependency detection: Starting from the target endpoint node, perform a depth-first recursive traversal to collect all dependent nodes of the target endpoint node in reverse order, and use a set to automatically remove duplicates; the time complexity of the entire algorithm process is O(V+E), which can ensure the efficiency of the algorithm.

[0099] (4) Generate version identifier: Set the editable parameters in the target workflow to empty, and serialize the target workflow after setting the parameters to empty into JSON; then, calculate the hash value of the serialized file; finally, store the calculated hash value as the version identifier of the target workflow and the target workflow accordingly.

[0100] (5) Generate workflow pruning report: Record information such as the number of original nodes, the number of dependent nodes, the number of removed nodes, the node reduction rate, and the dependency reduction status, and generate workflow pruning report based on the recorded information.

[0101] It is easy to see that developers can use the online development environment provided in this example to design workflows, develop custom plugins, debug in real time, and perform initial optimizations through the built-in performance optimization and automatic pruning functions, laying the foundation for subsequent intelligent optimization and deployment.

[0102] S204, obtain the dependency data package required for the target node to run in the target workflow, and generate the running data package corresponding to the target workflow based on the dependency data package corresponding to the target node.

[0103] In this embodiment, a target node refers to a node included in the target workflow. The dependency data package (also known as a plugin package) required for the target node to run includes the executable code segment, dependency declarations, resource files, and other data required for the target node's operation. The dependency declarations include a list of external files required for operation; the resource files include non-code files required for operation, such as model files, configuration files, sample files, UI resources, tool scripts, etc., which are not limited here. A dependency data package may include at least one piece of data required for the operation of a node.

[0104] In some embodiments, the specific process of generating the runtime data package corresponding to the target workflow may include: obtaining the dependency data package corresponding to each target node in the target workflow; and packaging the obtained dependency data packages to obtain the runtime data package corresponding to the target workflow.

[0105] Optionally, during the packaging process, a completeness check can be performed on each dependency package, such as checking whether the model files in the dependency package are corrupted or whether the critical path is valid.

[0106] Optionally, after packaging the runtime data package, the runtime data package can be uploaded to a preset remote repository so that the preset remote repository can perform subsequent management and deployment of the runtime data package of the target workflow.

[0107] Furthermore, the version identifier and runtime data package mentioned in step S203 can also be uploaded to a preset remote repository together, so that the preset remote repository can perform version management of the runtime data package of the target workflow based on the version identifier, thereby ensuring that each version update of the workflow has complete traceability. Any object can accurately obtain the exact same runtime data package by specifying the version identifier, and accurately reproduce the workflow and its runtime environment based on the runtime data package, which greatly improves the reliability of research, collaboration and deployment.

[0108] In some embodiments, before executing step S201, the server may first obtain the code segment called during the registration process of each preset node; then, the obtained code segment is parsed for storage path resolution to obtain the source data packet of the obtained code segment; subsequently, a mapping relationship between each preset node and the parsed source data packet can be established to obtain at least one mapping relationship. In this way, the source data packet corresponding to the target node can be obtained based on at least one mapping relationship, and the obtained source data packet can be used as the dependency data packet required for the target node to run.

[0109] The specific process of parsing the storage path of the acquired code segment may include: obtaining the call stack when registering each preset node, then extracting the file storage path pointing to the specific source data packet from the call stack; and finally, obtaining the source data packet from the file storage path.

[0110] Specifically, data packets are typically organized in a directory structure, containing source code files (i.e., the executable code segments corresponding to nodes). When a data packet is loaded, the module files within it are executed, and registration calls are initiated from these module files. The call stack records the sequence of function calls, including module information for each call point (such as filename and path). Therefore, by parsing the call stack, the file storage path of the module files in the data packet can be found. This file storage path is the storage path of the executable code segment corresponding to the current preset node, thus determining that the data packet stored at that file storage path is the source data packet corresponding to the current preset node.

[0111] This embodiment captures and parses the call stack of a preset node in real time during the node registration process, obtaining the recorded module file paths. This allows for precise identification of the actual source data packet of the code segment executed by the current preset node, and establishes a reliable mapping relationship between the node and the data packet. Therefore, this embodiment fully utilizes the real function call chain and module information recorded in the call stack at runtime, achieving accurate tracing of the source of the node's executed code. This effectively solves the problem of accurately determining which data packet a node depends on at runtime in environments with dynamic loading, multiple versions coexisting, or complex dependencies. Furthermore, by establishing and maintaining the mapping relationship between the preset node and its source data packet, this embodiment enables the accurate and rapid acquisition of the dependent data packets corresponding to the target node, thereby shortening the overall deployment time of the workflow and further improving its deployment efficiency.

[0112] In some embodiments, after optimizing the workflow to be deployed based on the execution performance data corresponding to the reference node to obtain the target workflow, the executable code segment corresponding to the target node in the target workflow can also be obtained, and the interface display code segment contained in the executable code segment can be deleted and updated to obtain the target executable code segment.

[0113] The interface display code segment describes at least one of the following: the target node's display position, display size, layer display order, node name, and display appearance (such as color, icon, etc.) in the display interface. The execution code segment corresponding to the target node is stored in the dependency data package corresponding to the target node.

[0114] In addition, deleting and updating the UI display code segment contained in the executable code segment can specifically include: deleting the UI display code segment in the executable code segment and refactoring the remaining code segments in the executable code segment to obtain the target executable code segment that can ensure normal operation.

[0115] For example, please see the appendix. Figure 5 This illustrates a comparative diagram of an executable code segment. For example... Figure 5 As shown, the execution code segment 501 corresponding to the target node identified as 3 in the target workflow contains interface display code segments such as "pos": [1845,130] (i.e., the pixel coordinates of target node 3 on the canvas area are (1845, 130)) and "size": {"0": 315, "1": 262} (i.e., the node width of target node 3 on the canvas area is 315 pixels and the node height is 262 pixels), which describe the display information of target node 3 on the canvas area. It should be noted that the ellipsis in the execution code segment 501 indicates that there are other target nodes whose corresponding code segments are not displayed.

[0116] Deleting and updating the UI display code segment contained in execution code segment 501 will yield the following result: Figure 5 The target execution code segment 502 is shown. Here, "class_type": "KSampler" indicates that the node type is a K-sampler, used to perform diffusion model sampling to generate images; "inputs" defines the input parameters of target node 3; "model": ["75",0] indicates that the "model" input comes from the 0th output port of target node 75; "positive": ["76", 0] indicates that the "positive prompt word" input comes from the 0th output of node 76; "negative": ["77", 0] indicates that the "negative prompt word" input comes from the 0th output of node 77; "latent_image": ["194", 0] indicates that the "latent image" input comes from the 0th output of node 194.

[0117] It is easy to see that the target execution code segment 502 obtained after deleting and updating the interface display code segment contained in the execution code segment 501 does not contain any code segment describing the display information of the target node 3 on the display interface. Compared with the execution code segment 501, the target execution code segment 502 is more concise.

[0118] The specific process of generating the runtime data package corresponding to the target workflow based on the dependency data package corresponding to the target node can include: generating the runtime data package corresponding to the target workflow based on the dependency data package corresponding to the target node and the target execution code segment.

[0119] Specifically, the executable code segment corresponding to the target node in the dependency data package corresponding to the target node can be updated to the target executable code segment to obtain the target dependency data package corresponding to the target node; finally, the target dependency data package corresponding to the target node is packaged to obtain the runtime data package corresponding to the target workflow.

[0120] Specifically, traditional workflows embed UI / UX code within the execution code of nodes to support visual editing. However, when a workflow needs to be deployed in a production environment, since the production environment only needs to obtain the workflow's output and does not require editing or development of the workflow itself, the UI / UX code is redundant. It only increases the size of the workflow's corresponding data package, reduces parsing and loading speed, and may introduce unnecessary dependencies.

[0121] This embodiment obtains the target executable code segment by deleting and updating the UI display code segment within the executable code segment. This removes all UI-related parts from the target node's executable code segment, generating a clean target executable code segment. Subsequently, a streamlined workflow (i.e., a workflow without UI display) suitable for deployment in a real production environment can be built based on this target executable code segment. Thus, the regular workflow version, including the UI display, continues to provide developers with visual editing capabilities, while the streamlined workflow without UI display can be quickly deployed and launched in a real production environment due to its smaller file size. Therefore, this embodiment provides an automatic dual-version workflow generation mechanism. On the one hand, it eliminates the tediousness and error risk of developers manually maintaining two sets of code, achieving a balance between flexibility in the development phase and efficiency in the production phase, greatly improving the overall efficiency of workflow from development to deployment. On the other hand, it further improves workflow deployment efficiency by reducing the workflow's file size.

[0122] In practical implementation, it undertakes Figure 4 For example, after obtaining the target workflow through the online development environment, you can proceed to the save and publish phase. See the appendix. Figure 6 This diagram illustrates the process of saving and publishing during an automated deployment.

[0123] like Figure 6 As shown, during the save and publish phase, the target workflow can first be parsed. The workflow parsing module mainly includes a dual-version workflow generation unit and a dependency data package acquisition unit.

[0124] Specifically, the dual-version workflow generation unit can automatically delete and update the interface display code segment in the executable code segment corresponding to the target workflow (i.e., the normal version of the target workflow), thereby obtaining the target executable code segment (i.e., the simplified version of the target workflow). In this way, the normal version of the target workflow can still support editing of the target workflow, while the simplified version of the target workflow removes the interface display-related data and is a pure execution format.

[0125] The dependency data packet acquisition unit can non-intrusively detect the data packets loaded for registering each preset node through hook functions during the registration process of each preset node, and establish a mapping relationship between each preset node and the data packet, thereby forming a complete mapping table, so that the dependency data packets (including the executable code segments in the dependency data packets) corresponding to the node can be obtained according to the mapping relationship in the mapping table.

[0126] Furthermore, after detecting the target workflow, the dependency data packet acquisition unit can perform static analysis based on the abstract syntax tree of the target workflow according to the previously constructed completion mapping table to obtain the dependency data packets corresponding to each target node in the target workflow. Based on the dependency data packets corresponding to each target node, the runtime data packet of the target workflow can be obtained. Simultaneously, the dependency data packet acquisition unit can also acquire the metadata corresponding to the runtime data packet; the metadata may include at least one of the following: the version identifier of the target workflow, the generation timestamp of the target workflow, and the total number of nodes in the target workflow.

[0127] In this way, the dependency packaging module in the save and publish phase can generate a dependency manifest (specifically, a manifest in JavaScript Object Notation (JSON) format) based on the dependency packages contained in the runtime package and the metadata corresponding to the runtime package. Furthermore, the dependency packaging module can pull the runtime package and its metadata based on the dependency manifest and automatically package and publish it to a remote location (the remote location can be a server or a terminal device). This allows for the subsequent creation of a container instance corresponding to the target workflow on the remote location, enabling the deployment of the target workflow to the production environment. Since the metadata carried in the runtime package includes a version identifier, the version management module can subsequently trace and reproduce the target workflow based on the version identifier.

[0128] In actual testing, the above-mentioned save and publish phases resulted in an accuracy rate of over 95% in identifying the dependency data packets corresponding to nodes in the workflow, far exceeding the 80% accuracy rate achievable by manually configuring dependency data packets. At the same time, the workflow parsing process in the save and publish phases does not require manual intervention and is fully automated, reducing the workflow parsing process from the original 1 hour required manually to within 10 seconds, which is conducive to further improving the deployment efficiency of the target workflow.

[0129] S205: Create a container instance corresponding to the target workflow based on the runtime data package to deploy the target workflow.

[0130] In this embodiment, the container instance corresponding to the target workflow can be created on a server, in the cloud, or on a terminal device; this application does not limit this. Furthermore, if it is detected that the container instance corresponding to the target workflow has been created, it can be determined that the target workflow deployment is complete.

[0131] Optionally, the specific process of creating a container instance corresponding to the target workflow based on the runtime data package may include: obtaining a preset image deployment script, wherein the preset image deployment script contains the image identifier of the image required for each preset node to run; obtaining a target image that matches the image identifier corresponding to the target node based on the preset image deployment script; and constructing a container instance corresponding to the target workflow based on the target image and the runtime data package.

[0132] The image contains the environment, files, code, etc. required for each preset node to run. The image required for each preset node to run can be pre-built and cached, allowing the preset image deployment script to be obtained based on the image identifier of the built image.

[0133] In some embodiments, the specific process of constructing a container instance corresponding to the target workflow based on the target image and the runtime data package may include: first, performing static analysis based on the runtime data package to obtain the runtime resource requirements of the target workflow; then, calling a preset intelligent allocation algorithm to determine the target runtime resources allocated to the target workflow based on the runtime resource requirements; finally, obtaining a target deployment service template from a preset deployment service template based on the target runtime resources and the target image, and creating a container instance corresponding to the target workflow based on the target deployment service template and the runtime data package.

[0134] Specifically, the operating resources can be physical operating resources such as Graphics Processing Unit (GPU) and Central Processing Unit (CPU).

[0135] The deployment service template contains the complete image and resource configuration information for different runtime resources; similarly, the target deployment service template contains the complete target image and resource configuration information for the target runtime resources. Therefore, the target image and target runtime resources can be directly pulled through the target deployment service template, allowing the construction of container instances corresponding to the target workflow based on the target image, target runtime resources, and runtime data packages.

[0136] Specifically, this embodiment performs static analysis on the runtime data package (including workflow logic and parameters) to accurately assess its runtime resource requirements (such as GPU model and memory size, CPU cores and memory), enabling pre-planning of resource allocation and avoiding resource contention or waste. Secondly, by pre-configuring different images for different nodes, this embodiment allows for the rapid determination of the target image required for workflow execution regardless of workflow changes. Subsequently, by matching the target runtime resources and the target image, the most suitable target deployment service template can be matched from pre-set deployment service templates, thereby quickly instantiating a container instance. It is clear that this embodiment effectively avoids the need to rebuild the container image for each deployment and the manual configuration of deployment-related parameters, automating container instance construction and significantly improving the deployment efficiency of the target workflow.

[0137] In some embodiments, after creating a container instance corresponding to the target workflow based on the running data package to deploy the target workflow, the target node type of the target node running in the container instance can be further obtained; then, according to the pre-defined correspondence between node type and parameter mapping rules, the target parameter mapping rule corresponding to the target node type can be obtained; then, based on the target parameter mapping rule, the input parameters in the target node can be assigned values ​​to obtain the workflow to be tested; by calling the workflow to be tested to execute the test task, the execution test result for the target workflow can be obtained; if the execution test result indicates that the target workflow passes the test, the target workflow is deployed to the production environment.

[0138] The production environment, also known as the actual production environment, refers to the final deployment environment that provides stable and reliable services to real external users; its core characteristics are stability, high availability, security, and scalability.

[0139] Node types can include image loading (LoadImage), text encoders (such as CLIP Text Encode), K-samplers (KSampler), or model loading and preprocessing (ControlNetLoade), etc. Corresponding parameter mapping rules can be defined for each node type; each parameter mapping rule can include the parameter name, type, format, required parameter, default value, etc., which are not limited here. Optionally, each parameter mapping rule can also include parameter constraints for the parameters to be filled, such as image format, integer range, string length, etc.

[0140] Optionally, if the test results indicate that the target workflow has failed the test, a message indicating the failure of the target workflow test can be generated and sent to the developers.

[0141] In practical implementation, it undertakes Figure 6 For example, after publishing the runtime data package and metadata corresponding to the target workflow to the remote endpoint during the save and publish phase, the deployment service phase of the automated deployment process can begin. Taking a remote server as an example, please refer to the appendix. Figure 7 This diagram illustrates a process for the service deployment phase in an automated deployment process.

[0142] like Figure 7 As shown, after the remote server detects the running data packets and metadata corresponding to the target workflow, it can manage the version of the running data packets corresponding to the target workflow through the metadata; at the same time, the terminal device can enter the deployment service phase and start allocating resources to the target workflow.

[0143] The specific process of resource allocation may include: obtaining the runtime resource requirements of the target workflow, and obtaining runtime resources such as GPU, CPU, and memory that match the runtime resource requirements of the target workflow from runtime resources such as GPU card specification library.

[0144] The GPU card specification library can maintain memory and computing power configuration tables for various GPUs, such as the Zixiao Card, NVIDIA L20 GPU Accelerator (a data center-grade GPU), and NVIDIA H20 Tensor Core GPU Accelerator (a GPU designed for large-scale artificial intelligence training and inference).

[0145] Afterwards, the terminal device can match a deployment service template to the target workflow. Each pre-defined deployment service template contains a pre-built and cached complete container image. This container image contains the basic environment built upon the algorithm libraries / code libraries that the workflow nodes depend on. Specifically, the container image can include the container images required for the preset nodes to run. Optionally, different deployment service templates can be configured according to different runtime resource requirements, such as a deployment service template specifically for the Zixiao Card. This way, only the corresponding deployment service template needs to be updated synchronously when the preset node and its corresponding dependency data package change, without needing to temporarily rebuild the entire image when deploying the workflow. Optionally, the deployment service templates can be updated periodically or irregularly, and version management and rollback of the deployment service templates are also supported.

[0146] Specifically, the terminal device can automatically match the target deployment service template for the target workflow based on the previously analyzed runtime resource requirements and the target nodes contained in the target workflow.

[0147] Afterwards, the terminal device can directly create a container instance corresponding to the target workflow based on the target deployment service template. Specifically, the terminal device can pull the runtime data package corresponding to the target workflow from the remote server and directly create a container instance based on the runtime data package and the selected target deployment service template (without rebuilding the image).

[0148] After the container instance corresponding to the target workflow is created on the terminal device, then as follows: Figure 7 As shown, online testing can be initiated. Specifically, the same approach can be used to deploy a container instance corresponding to the target workflow in the cloud-based online testing system, with the target workflow running within the container instance. The cloud-based online testing system can then scan and identify each target node in the target workflow, determine the target node type of each target node, and obtain the target parameter mapping rules corresponding to the target node type.

[0149] Then, based on the target parameter mapping rules, each target node can be assigned a value according to the unified task interface parameter format (such as the unified task interface parameter format of Venus Software Development Kit (Venus SDK, a software development kit)). This generates a standard JSON structured workflow to be tested, ensuring that the workflow to be tested remains compatible with the Venus SDK task management interface.

[0150] For the standard JSON structure of the workflow to be tested obtained in the previous step, a custom parameter template file can be automatically generated. This file not only lists all parameter keys but also provides detailed annotations on parameter types, whether they are required, Chinese descriptions, and example values. Simultaneously, a directly executable test code file can be generated for the standard JSON structure of the workflow to be tested. This test code file integrates the initialization logic of the Venus SDK and pre-sets example parameters read from the template. Developers can use the test code file and the custom parameter template file to automate testing of the workflow to be tested.

[0151] Optionally, in the online testing system's testing interface, when a tester selects a target workflow, the system can automatically pre-populate a structured form with the editable parameters of each target node in the target workflow, as well as the parameter values ​​to be assigned to each editable parameter. Testers can then directly modify any parameter value on this form. After clicking the "Run Test" button in the testing interface, the system assigns values ​​to the editable parameters of each target node in the target workflow based on the parameter values ​​in the form, resulting in a new workflow to be tested. Optionally, the testing interface can also dynamically display the complete testing process of the new workflow, including the data request process, data response process, execution status, and the final execution test, thereby achieving end-to-end, visual verification of the entire workflow testing process.

[0152] In summary, the online development, save and publish, and deployment service phases of the automated deployment process mentioned in the examples above enable online development, instant deployment, and real-time testing of workflows. This improves development and deployment efficiency, facilitating rapid workflow iteration and allowing workflows better suited to business needs to be quickly deployed to the production environment. Furthermore, the automated resource allocation and deployment service templates in the deployment service phase allow developers to accurately and quickly instantiate workflow containers without needing expertise in resource or container configuration. In actual testing, the automated deployment process in the examples significantly reduced the workflow deployment cycle from weeks to 30 minutes, resulting in a substantial improvement in deployment efficiency. Moreover, the automated deployment process in the examples can flexibly adapt to various business scenarios; in practical applications, it has supported the development and deployment of workflows across at least 40 business types, reaching millions of calls per day.

[0153] In this embodiment, by acquiring the execution performance data of nodes in the workflow to be deployed during the execution of data processing tasks, accurate optimization of the workflow can be achieved based on the execution performance data. Simultaneously, by pulling the dependency data packages corresponding to the target nodes in the optimized target workflow, this embodiment can generate the runtime data package corresponding to the target workflow. Based on the runtime data package, a container instance corresponding to the target workflow can be automatically created, thereby achieving rapid deployment of the target workflow. Therefore, this embodiment enables automated and rapid deployment of workflows from development to application. Thus, in this embodiment, the main human intervention is in the development and design of the workflow; developers only need to trigger the workflow deployment component to achieve one-click rapid deployment of the workflow, thereby significantly improving the deployment efficiency of the workflow.

[0154] Based on the above workflow deployment scheme and system, this application provides another workflow deployment method. See also... Figure 8 This is a flowchart illustrating another workflow deployment method provided in an embodiment of this application. Figure 8 The deployment method of the workflow shown can be... Figure 1 The workflow shown is executed by a server or terminal device in the deployment system, using this method. Figure 1 The workflow shown is illustrated using the terminal device execution in the deployment system as an example.

[0155] in, Figure 8 The deployment method of the workflow shown may include steps S801 to S811: S801 displays the workflow design interface; the workflow design interface includes a workflow editing component and a workflow deployment component.

[0156] As can be seen from the foregoing embodiments, the automatic deployment process of the workflow includes stages such as online development, saving and publishing, and deployment services; among which, the performance optimization of the workflow is mainly carried out in the online development environment, including workflow updates and node pruning based on performance bottleneck nodes.

[0157] Therefore, in this embodiment of the application, the workflow deployment component may include a performance detection enabling component, a node pruning confirmation component, and a save and publish component.

[0158] The performance detection enablement component is used to enable the execution of the process based on the execution performance data corresponding to at least one reference node, analyze the performance bottleneck nodes in the workflow, and update the performance bottleneck nodes in the workflow to be deployed to obtain the updated workflow.

[0159] The node pruning confirmation component is used to confirm that node pruning has been performed on the workflow to obtain the target workflow.

[0160] The save and publish component triggers the steps of obtaining the dependency data packages required for the target node to run in the target workflow, and generating the corresponding runtime data package for the target workflow based on the dependency data packages corresponding to the target node. Specifically, the save and publish component corresponds to the save and publish phase mentioned in the above embodiments; through the save and publish phase, the workflow can be automatically advanced from the online development environment to the deployment service phase, realizing automatic connection from the online development environment to the deployment service phase, thereby avoiding the time and manpower costs increased by manual connection, which is conducive to improving the deployment efficiency of the workflow.

[0161] For example, please see the appendix. Figure 9 This illustrates a different workflow design interface. For example... Figure 9As shown, the developer edited a relatively complete workflow to be deployed in canvas area 902 of the workflow design interface 901. The workflow to be deployed includes a node for loading the pre-trained model (node ​​identifier 4), two nodes for encoding the prompt words with a text encoder (node ​​identifiers 6 and 7), a node for inputting a blank latent representation image (node ​​identifier 5), a node for model sampling (node ​​identifier 3), a node for image generation (node ​​identifier 8), a node for saving the generated image (node ​​identifier 9), and a node for previewing the generated image (node ​​identifier 10).

[0162] The workflow design interface 901 also includes a "Save" button, a "Save Simplified Version" button, a "Save and Publish" button (corresponding to the Save and Publish component), a "Load" button, a "Refresh" button, a "Condition Space Switch" button, a "Enable Performance Detection" edit box (corresponding to the Performance Detection Enable component), a "Enable Workflow Pruning" edit box (corresponding to the Node Pruning Confirmation component), and a "Last Output Node" edit box.

[0163] The "Save" button saves the workflow in the canvas area; the "Save Simplified Version" button generates the executable code segment after removing the corresponding UI display code segment from the canvas area; the "Load" button imports external resources such as model files and workflow images into the current interface; and the "Refresh" button updates nodes. The "Condition Space Switch" button switches between two input modes: "Natural Language Description" and "Internal Feature Representation Sequence." The former allows input of human-readable prompts, while the latter allows direct input of feature representations for model processing.

[0164] The "Save and Publish" button triggers the save and publish phase. If the "Enable Performance Monitoring" edit box is checked (e.g., ...), ... Figure 9 As shown), this allows for performance optimization of the workflow in the canvas area during the automatic deployment process (specifically, after the "Save and Publish" button is clicked and before entering the save and publish stage). This optimization mainly includes the aforementioned steps of detecting performance bottleneck nodes and updating the workflow based on performance bottleneck nodes.

[0165] If the "Enable Workflow Pruning" edit box is checked ( Figure 9 If the option is not selected, then node pruning can be performed on the workflow in the canvas area during automatic deployment (specifically, after the "Save and Publish" button is clicked but before entering the save and publish phase). Additionally, developers can input the target endpoint node of the workflow in the selected canvas area through the "Last Output Node" edit box. Figure 9The selected target endpoint node is node 9, so that node pruning can be performed subsequently based on the target endpoint node.

[0166] As can be seen in this example, developers can flexibly choose between workflow performance optimization and node pruning; they can check only the "Enable performance detection" edit box, only the "Enable workflow pruning" edit box, or both the "Enable performance detection" and "Enable workflow pruning" edit boxes.

[0167] S802, in response to an editing operation on a workflow editing component, obtain at least one reference node selected from at least one preset node, and construct a workflow to be deployed based on at least one reference node.

[0168] In the embodiments of this application, the specific implementation of step S802 can be found in the specific implementation of step S202, and will not be repeated here.

[0169] S803, in response to the trigger operation of the workflow deployment component, obtains the execution performance data of each reference node in the workflow to be deployed during the execution of data processing tasks; wherein, the execution performance data corresponding to each reference node includes: the execution time of each reference node in executing data processing tasks, and the resource usage interval sequence data corresponding to each reference node.

[0170] In this embodiment, the resource occupancy interval sequence data corresponding to each reference node can specifically be obtained based on the amount of computing resources occupied by each reference node during the execution of data processing tasks. The resource occupancy interval sequence data can include at least one resource occupancy interval, each resource occupancy interval corresponding to multiple consecutive execution time points, forming a time period. Each resource occupancy interval can characterize the amount of computing resources stably occupied by a node within the corresponding time period.

[0171] Specifically, the process of obtaining the resource occupancy interval sequence data corresponding to each reference node may include: during the data processing task performed by the reference node, obtaining the amount of computing resources occupied by the reference node at each execution time point; then, based on each execution time point and the amount of resources corresponding to each execution time point, constructing the resource occupancy interval sequence data corresponding to the reference node.

[0172] Optionally, the amount of computing resources occupied by the reference node at each execution time point can constitute time-series data of computing resource usage. Then, by performing a stationarity test on the time-series data of resource usage, at least one resource usage interval can be divided to obtain the resource usage interval sequence data.

[0173] The resource quantity corresponding to each resource occupancy interval can be less than a preset stability threshold; optionally, the resource quantity corresponding to each resource occupancy interval is the average of the resource quantities corresponding to all execution time points within each resource occupancy interval. Optionally, resource occupancy intervals can also be divided in other ways, which are not limited in this application.

[0174] Optionally, the resource occupancy interval sequence data for each reference node can be different for different computing resources.

[0175] In some embodiments, the process of obtaining the execution duration corresponding to each reference node may include: obtaining the start execution time point when the reference node begins to execute the data processing task and the end execution time point when the reference node stops executing the data processing task from at least two execution time points corresponding to the reference node; and then, obtaining the execution duration of the reference node based on the start execution time point and the end execution time point corresponding to the reference node.

[0176] Specifically, the time difference between the start and end execution times can be calculated, and this time difference can be used as the execution duration.

[0177] In practical applications, taking ComfyUI, an application that provides a workflow design interface, as an example, we can create scheduling hook functions for the scheduler in the ComfyUI execution engine, as well as execution hook functions for the executor in the ComfyUI execution engine.

[0178] Specifically, scheduling hook functions can be used to detect the scheduling order of each node in the workflow, as well as the scheduling overhead data such as data packets and computing resources scheduled by the scheduler to start each node. Based on the scheduling overhead data, the amount of computing resources occupied by each node at each execution time point can be obtained, thus obtaining the resource usage interval sequence for each node. Furthermore, by executing hook functions, the execution duration of each node in the workflow can be detected.

[0179] This dual-perspective detection mechanism, which detects both the scheduler and the executor separately, allows for the separate detection of computing resources and execution time. This helps to accurately distinguish between these two types of overhead, thus leading to more precise performance testing data.

[0180] Optionally, during the detection of the scheduler and executor, a high-precision timer (such as time.perf_counter) can be used to measure the detected data with microsecond-level precision, so that the time precision error is less than 1%. At the same time, cross-process time synchronization can be supported to clarify the order of data occurrence, thereby obtaining accurate sequence data (such as execution duration sequence data, resource usage interval sequence data).

[0181] S804, based on the execution duration corresponding to at least one reference node and the node execution order corresponding to at least one reference node, construct the execution duration sequence data corresponding to the workflow to be deployed.

[0182] In this embodiment of the application, the execution duration sequence data includes the execution duration arranged according to the node execution order of the corresponding reference node.

[0183] For example, suppose the execution time of node 1 is 20 seconds, the execution time of node 2 is 10 seconds, and the execution time of node 3 is 5 seconds; the execution order of the nodes is node 2, node 1, and node 3; then the execution time sequence data can be [10, 20, 5].

[0184] S805: According to the execution order of the nodes corresponding to at least one reference node, the resource occupancy interval sequence data corresponding to at least one reference node are concatenated to obtain the resource occupancy interval sequence data corresponding to the workflow to be deployed.

[0185] In this embodiment of the application, the resource occupancy interval sequence data corresponding to the workflow to be deployed includes: resource occupancy interval sequence data corresponding to at least one reference node; at the same time, these resource occupancy interval sequence data are concatenated according to the execution order of the nodes corresponding to at least one reference node; thus, the resource occupancy interval sequence data corresponding to the workflow to be deployed is essentially arranged according to the time sequence of the resource occupancy intervals.

[0186] Optionally, step S804 mentions that the resource occupancy interval sequence data corresponding to each reference node can be different for different computing resources; then, the target resource occupancy interval sequence data corresponding to each reference node for the same target computing resource can be obtained; then, according to the execution order of the nodes corresponding to at least one reference node, the target resource occupancy interval sequence data corresponding to at least one reference node are concatenated to obtain the resource occupancy interval sequence data corresponding to the target computing resource for the workflow to be deployed.

[0187] S806 generates a node performance analysis interface based on the execution duration sequence data and resource usage interval sequence data corresponding to the workflow to be deployed.

[0188] In this embodiment, an execution duration display area can be generated based on the execution duration sequence data corresponding to the workflow to be deployed; wherein, the execution duration display area contains a duration display block corresponding to each execution duration in the execution duration sequence data corresponding to the workflow to be deployed, and the length of the duration display block is positively correlated with the execution duration.

[0189] Furthermore, a resource occupancy display area can be generated based on the resource occupancy interval sequence data corresponding to the workflow to be deployed; wherein, the resource occupancy display area contains resource display blocks corresponding to each resource occupancy interval in the resource occupancy interval sequence data corresponding to the workflow to be deployed, and the area length of the resource display block is positively correlated with the length of the time interval corresponding to the resource occupancy interval.

[0190] Finally, a node performance analysis interface can be generated, which includes an area displaying execution time and an area displaying resource usage.

[0191] Optionally, a node scheduling display area representing the scheduling order of each reference node in the workflow to be deployed can be generated based on the duration display block corresponding to each execution duration in the execution duration sequence data corresponding to the workflow to be deployed, and the node name of each reference node in the workflow to be deployed.

[0192] This allows the generation of a node performance analysis interface that includes areas for displaying execution time, resource usage, and node scheduling.

[0193] For example, please see the appendix. Figure 10 The diagram illustrates a node performance analysis interface. Figure 10 As shown, the node performance analysis interface 1001 includes an execution time display area 1002, a node scheduling display area 1003, and a resource display area 1004.

[0194] The workflow to be deployed is workflow 1401258. The execution duration display area 1002 contains a duration display block corresponding to each node in workflow 1401258. Optionally, each duration display block can also display the node name and node identifier of the corresponding node; for example... Figure 10 As shown, the duration display block 1005 displays the node name as "Loading Pre-trained Model" and the node identifier as "4". Meanwhile, as... Figure 10 As shown, the node performance analysis interface 1001 can also include a timeline. By using the timeline and the block length of each duration display block, the execution time of the node corresponding to each duration display block can be clearly defined. For example, from... Figure 10 As can be seen from the time display, the execution time corresponding to block 1005 is approximately 295 seconds.

[0195] And, such as Figure 10As shown, the vertical axis in the node scheduling display area 1003 represents each node in workflow 1401258; for example, workflow 1401258_4 represents the node identified as 4 in workflow 1401258. The horizontal axis in the node scheduling display area 1003 displays the duration display block corresponding to each node. Through the node scheduling display area 1003, it can be clearly seen that workflow 1401258 is scheduled and executed in the order of node 4 → node 8 → node 9 during execution.

[0196] Finally, as Figure 10 As shown, the resource occupancy display area 1004 contains workflow 1401258 containing sequence data of resource occupancy intervals corresponding to four types of computing resources: CPU, ARM (a processor architecture), GPU, and VRAM (representing memory or video memory). The time interval length of the resource occupancy interval corresponding to the resource display block can be determined by the time axis and the block length of the resource display area.

[0197] Optional, such as Figure 10 As shown, each resource display block can also display the amount of resources occupied by the corresponding resource occupancy range. For example, by using the block length of resource display block 1006, the number 1.2 contained in resource display block 1006, and the timeline, it can be determined that node 4 continuously occupied 1.2 cores of CPU during the time period from 0 to 175 seconds.

[0198] S807 displays the node performance analysis interface to identify performance bottleneck nodes based on the node performance analysis interface.

[0199] In this embodiment of the application, in response to the triggering operation of the node included in the workflow to be deployed in the node performance analysis interface, the triggered node can be obtained and the triggered node can be regarded as the performance bottleneck node.

[0200] In some embodiments, the node performance analysis interface may also include an automatic analysis component. In response to the triggering operation of the automatic analysis component, a performance analysis tool may be invoked to analyze and obtain the performance bottleneck node in the workflow to be deployed based on the execution duration sequence data and resource usage interval sequence data corresponding to the workflow to be deployed.

[0201] In specific implementations, performance analysis tools may include Perfetto System Tracing Framework (Perfetto for short, a system testing platform), Windows Performance Analyzer (a deep performance analysis tool), Speedscope (an interactive platform that imports various analyzers), etc., and are not limited here.

[0202] Optionally, the performance bottleneck nodes obtained from the performance analysis tool can be further output. If the developer's confirmation operation on the performance bottleneck node is detected, step S808 is triggered.

[0203] S808 updates the performance bottleneck nodes in the workflow to be deployed, resulting in an updated workflow.

[0204] In this application embodiment, the specific implementation of step S808 can be found in the specific implementation of the update process in step S203, and will not be repeated here.

[0205] S809 performs node pruning on the updated workflow to obtain the target workflow.

[0206] In this application embodiment, the specific implementation of step S809 can be found in the specific implementation of node pruning in step S203, and will not be repeated here.

[0207] S810: Obtain the dependency data package required for the target node to run in the target workflow, and generate the running data package corresponding to the target workflow based on the dependency data package corresponding to the target node.

[0208] In the embodiments of this application, the specific implementation of step S810 can be found in the specific implementation of step S204, and will not be repeated here.

[0209] S811 creates a container instance corresponding to the target workflow based on the runtime data package to deploy the target workflow.

[0210] In the embodiments of this application, the specific implementation of step S811 can be found in the specific implementation of step S205, and will not be repeated here.

[0211] Specifically, currently, developers rely on their own development experience to subjectively identify performance bottlenecks in workflows, which is inaccurate. However, in this embodiment, a node performance analysis interface visualizes execution performance data such as execution time sequence data and resource usage interval sequence data. This provides quantitative data support for identifying performance bottlenecks, reducing errors from subjective human judgment. This allows developers to fully understand the execution time and computing resources consumed by each reference node in the workflow to be deployed. Using this information, they can accurately select performance bottlenecks from the nodes included in the workflow. Accurate identification of performance bottlenecks facilitates precise optimization of the workflow, improving the reliability of subsequent deployments and avoiding redundant development and deployment, thus increasing workflow deployment efficiency.

[0212] Furthermore, as mentioned above, the embodiments of this application significantly shorten the optimization time of the workflow and improve the optimization accuracy of the workflow by automatically parsing the dependency data packets corresponding to the nodes in the workflow, using a dual-view detection mechanism for execution performance, and node pruning, thereby enabling the deployment of the workflow with zero configuration, high efficiency, and high quality, and achieving efficient management of the entire lifecycle of the AIGC workflow.

[0213] Based on the description of the workflow deployment method above, this application also discloses a workflow deployment apparatus. This workflow deployment apparatus can be a computer program (including program code) running on the computer device mentioned above. This workflow deployment apparatus can execute the workflow deployment method described above; please refer to [link to relevant documentation]. Figure 11 The deployment device for this workflow may include at least: a display unit 1101, an editing response unit 1102, a trigger response unit 1103, an acquisition unit 1104, and a deployment unit 1105, wherein: Display unit 1101 is used to display workflow design interface; wherein, workflow design interface includes workflow editing component and workflow deployment component; The editing response unit 1102 is used to respond to the editing operation of the workflow editing component, obtain a reference node selected from at least one preset node, and construct the workflow to be deployed based on the reference node; The trigger response unit 1103 is used to respond to the trigger operation of the workflow deployment component, obtain the execution performance data of the reference node in the workflow to be deployed during the execution of data processing tasks, and optimize the workflow to be deployed based on the execution performance data of the reference node to obtain the target workflow. The acquisition unit 1104 is used to acquire the dependency data package required for the target node to run in the target workflow, and generate the running data package corresponding to the target workflow based on the dependency data package corresponding to the target node; Deployment unit 1105 is used to create a container instance corresponding to the target workflow based on the running data package in order to deploy the target workflow.

[0214] In an optional implementation, the number of reference nodes includes at least one. When the trigger response unit 1103 optimizes the workflow to be deployed based on the execution performance data corresponding to the reference nodes to obtain the target workflow, it can be used to: analyze and obtain the performance bottleneck nodes in the workflow to be deployed based on the execution performance data corresponding to at least one reference node; update the performance bottleneck nodes in the workflow to be deployed to obtain the updated workflow; and perform node pruning on the updated workflow to obtain the target workflow.

[0215] In an optional implementation, when the trigger response unit 1103 performs node pruning on the updated workflow to obtain the target workflow, it can specifically be used to: construct a node adjacency list based on the data input and data output relationships between every two nodes in the updated workflow; select a target endpoint node from the endpoint nodes contained in the updated workflow; obtain the dependent nodes of the target endpoint node in the updated workflow based on the node adjacency list; and prune the nodes in the updated workflow other than the target endpoint node and dependent nodes to obtain the target workflow.

[0216] In an optional implementation, the execution performance data corresponding to each reference node includes: the execution time of each reference node performing the data processing task, and the resource usage interval sequence data obtained based on the amount of computing resources occupied by each reference node during the execution of the data processing task; when the trigger response unit 1103 analyzes and obtains the performance bottleneck node in the workflow to be deployed based on the execution performance data corresponding to at least one reference node, it can specifically be used to: construct the execution time sequence data corresponding to the workflow to be deployed based on the execution time corresponding to at least one reference node and the node execution order corresponding to at least one reference node; concatenate the resource usage interval sequence data corresponding to at least one reference node according to the node execution order corresponding to at least one reference node to obtain the resource usage interval sequence data corresponding to the workflow to be deployed; generate a node performance analysis interface based on the execution time sequence data corresponding to the workflow to be deployed and the resource usage interval sequence data corresponding to the workflow to be deployed; and display the node performance analysis interface to obtain the performance bottleneck node based on the node performance analysis interface.

[0217] In an optional implementation, when the trigger response unit 1103 acquires the execution performance data of the reference node in the workflow to be deployed during the execution of the data processing task, it may further be used to: acquire the amount of computing resources occupied by the reference node at each execution time point during the execution of the data processing task; construct a resource occupation interval sequence data corresponding to the reference node based on each execution time point and the resource amount corresponding to each execution time point; acquire the start execution time point when the reference node starts executing the data processing task and the end execution time point when the reference node stops executing the data processing task from at least two execution time points corresponding to the reference node; obtain the execution duration of the reference node based on the start execution time point and the end execution time point corresponding to the reference node; and obtain the execution performance data corresponding to the reference node based on the resource occupation interval sequence data and the execution duration corresponding to the reference node.

[0218] In an optional implementation, after the trigger response unit 1103 optimizes the workflow to be deployed based on the execution performance data corresponding to the reference node to obtain the target workflow, it can specifically be used to: obtain the execution code segment corresponding to the target node in the target workflow; delete and update the interface display code segment contained in the execution code segment to obtain the target execution code segment. Then, when the acquisition unit 1104 generates the runtime data package corresponding to the target workflow based on the dependency data package corresponding to the target node, it can specifically be used to generate the runtime data package corresponding to the target workflow based on the dependency data package corresponding to the target node and the target execution code segment.

[0219] In an optional implementation, the device further includes a data mapping unit. Before displaying the workflow design interface, the data mapping unit is used to acquire the code segments called during the registration process of each preset node; to parse the storage path of the acquired code segments to obtain the source data packets of the acquired code segments; and to establish a mapping relationship between each preset node and the corresponding parsed source data packets to obtain at least one mapping relationship. Therefore, when the acquisition unit 1104 acquires the dependency data packets required for the execution of the target node in the target workflow, it can specifically be used to acquire the source data packets corresponding to the target node based on at least one mapping relationship, and use the acquired source data packets as the dependency data packets required for the execution of the target node.

[0220] In an optional implementation, when the deployment unit 1105 creates a container instance corresponding to the target workflow based on the runtime data package, it may specifically be used to: obtain a preset image deployment script; wherein the preset image deployment script contains the image identifier of the image required for each preset node to run; obtain the target image that matches the image identifier corresponding to the target node based on the preset image deployment script; and construct the container instance corresponding to the target workflow according to the target image and the runtime data package.

[0221] In an optional implementation, the apparatus further includes a testing unit. After creating a container instance corresponding to the target workflow based on the running data package to deploy the target workflow, the testing unit can be used to obtain the target node type to which the target node running in the container instance belongs; obtain the target parameter mapping rule corresponding to the target node type according to the pre-set correspondence between node type and parameter mapping rule; assign values ​​to the input parameters in the target node based on the target parameter mapping rule to obtain the workflow to be tested; call the workflow to be tested to execute the test task to obtain the execution test result for the target workflow; if the execution test result indicates that the target workflow passes the test, then the target workflow is deployed to the production environment.

[0222] According to one embodiment of this application, the various steps involved in the aforementioned method can be performed by... Figure 11The workflow shown is executed by the individual units in the deployment device.

[0223] According to another embodiment of this application, Figure 11 The units in the workflow deployment apparatus shown are based on logical functional divisions. These units can be individually or entirely merged into one or more other units, or some of these units can be further divided into functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. In other embodiments of this application, the workflow-based deployment apparatus may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0224] According to another embodiment of this application, a computer program (including program code) capable of performing the steps involved in the aforementioned method can be run on a general-purpose computing device, such as a computer device, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), to construct a system as described above. Figure 11 The diagram illustrates a workflow deployment apparatus and a workflow deployment method for implementing embodiments of this application. A computer program may be recorded on, for example, a computer storage medium, loaded onto the aforementioned computer device via the computer storage medium, and run therein.

[0225] In this embodiment, by acquiring the execution performance data of nodes in the workflow to be deployed during the execution of data processing tasks, accurate optimization of the workflow can be achieved based on the execution performance data. Simultaneously, by pulling the dependency data packages corresponding to the target nodes in the optimized target workflow, this embodiment can generate the runtime data package corresponding to the target workflow. Based on the runtime data package, a container instance corresponding to the target workflow can be automatically created, thereby achieving rapid deployment of the target workflow. It is evident that this embodiment enables automated and rapid deployment of workflows from development to application. Therefore, in this embodiment, the main human intervention is in the development and design of the workflow. Developers only need to trigger the workflow deployment component to achieve one-click rapid deployment of the workflow, thereby significantly improving the deployment efficiency of the workflow.

[0226] Based on the above method and apparatus embodiments, this application also provides an electronic device. See also Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 12The electronic device shown may include at least a processor 1201, an input interface 1202, an output interface 1203, and a computer storage medium 1204. The processor 1201, input interface 1202, output interface 1203, and computer storage medium 1204 may be connected via a bus or other means.

[0227] Computer storage medium 1204 can be stored in the memory of electronic device. Computer storage medium 1204 is used to store computer programs, which include program instructions. Processor 1201 is used to execute the program instructions stored in computer storage medium 1204. Processor 1201 (or CPU (Central Processing Unit)) is the computing and control core of electronic device. It is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to realize the deployment method or corresponding function of the above-mentioned workflow.

[0228] This application embodiment also provides a computer storage medium (Memory), which is a memory device in an electronic device used to store programs and data. It is understood that the computer storage medium here can include the built-in storage medium in a terminal, or it can include an extended storage medium supported by the terminal. The computer storage medium provides storage space, which stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by the processor 1201. These instructions can be one or more computer programs (including program code). It should be noted that the computer storage medium here can be a high-speed random access memory (RAM), or it can be non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer storage medium located remotely from the aforementioned processor.

[0229] In one embodiment, the processor 1201 may load and execute one or more instructions stored in the computer storage medium to implement the aforementioned related... Figure 2 and Figure 8 In the workflow deployment method embodiment, the corresponding steps of the method are as follows: In a specific implementation, one or more instructions in the computer storage medium are loaded and executed by the processor 1201, such as... Figure 2 or Figure 8 The steps in the workflow deployment method embodiment shown.

[0230] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the aforementioned actions. Figure 2 or Figure 5 The method embodiment shown. The computer-readable storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0231] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims. Furthermore, this application does not limit the execution order of the various steps in the specific embodiments.

Claims

1. A workflow deployment method, characterized in that, The method includes: Displays a workflow design interface; wherein the workflow design interface includes a workflow editing component and a workflow deployment component; In response to an editing operation on the workflow editing component, a reference node selected from at least one preset node is obtained, and a workflow to be deployed is constructed based on the reference node; In response to a trigger operation on the workflow deployment component, the execution performance data of the reference node in the workflow to be deployed during the execution of data processing tasks is obtained, and the workflow to be deployed is optimized based on the execution performance data corresponding to the reference node to obtain the target workflow; Obtain the dependency data package required for the target node to run in the target workflow, and generate the running data package corresponding to the target workflow based on the dependency data package corresponding to the target node; Based on the runtime data package, a container instance corresponding to the target workflow is created to deploy the target workflow.

2. The method according to claim 1, characterized in that, The number of reference nodes includes at least one, and the optimization of the workflow to be deployed based on the execution performance data corresponding to the reference nodes to obtain the target workflow includes: Based on the execution performance data corresponding to the at least one reference node, the performance bottleneck node in the workflow to be deployed is analyzed and obtained. The performance bottleneck node in the workflow to be deployed is updated to obtain the updated workflow. The updated workflow is then subjected to node pruning to obtain the target workflow.

3. The method according to claim 2, characterized in that, The step of performing node pruning on the updated workflow to obtain the target workflow includes: Based on the data input and data output relationships between every two nodes in the updated workflow, construct a node adjacency table; Select the target endpoint node from the endpoint nodes contained in the updated workflow; Based on the node adjacency list, obtain the dependent nodes of the target endpoint node in the updated workflow; The nodes in the updated workflow, excluding the target endpoint node and the dependent nodes, are pruned to obtain the target workflow.

4. The method according to claim 2, characterized in that, The execution performance data for each reference node includes: the execution time of each reference node in executing the data processing task, and the resource usage interval sequence data obtained based on the amount of computing resources occupied by each reference node in executing the data processing task. The process of analyzing the performance bottleneck nodes in the workflow to be deployed based on the execution performance data corresponding to the at least one reference node includes: Based on the execution duration corresponding to the at least one reference node and the execution order of the nodes corresponding to the at least one reference node, the execution duration sequence data corresponding to the workflow to be deployed is constructed. According to the execution order of the nodes corresponding to the at least one reference node, the resource occupancy interval sequence data corresponding to the at least one reference node are concatenated to obtain the resource occupancy interval sequence data corresponding to the workflow to be deployed; A node performance analysis interface is generated based on the execution duration sequence data corresponding to the workflow to be deployed and the resource usage interval sequence data corresponding to the workflow to be deployed. Display the node performance analysis interface to obtain the performance bottleneck node based on the node performance analysis interface.

5. The method according to claim 1, characterized in that, The step of obtaining the execution performance data of the reference node in the workflow to be deployed during the execution of data processing tasks includes: During the execution of the data processing task by the reference node, the amount of computing resources occupied by the reference node at each execution time point is obtained; Based on each execution time point and the resource quantity corresponding to each execution time point, construct the resource occupancy interval sequence data corresponding to the reference node; From at least two execution time points corresponding to the reference node, obtain the start execution time point when the reference node begins to execute the data processing task, and the end execution time point when the reference node stops executing the data processing task; Based on the start and end execution times corresponding to the reference node, the execution duration of the reference node is obtained; Based on the resource usage interval sequence data and execution time corresponding to the reference node, the execution performance data corresponding to the reference node is obtained.

6. The method according to any one of claims 1 to 5, characterized in that, After optimizing the workflow to be deployed based on the execution performance data corresponding to the reference node to obtain the target workflow, the method further includes: Obtain the executable code segment corresponding to the target node in the target workflow; The interface display code segment contained in the executable code segment is deleted and updated to obtain the target executable code segment; The step of generating the runtime data package corresponding to the target workflow based on the dependency data package corresponding to the target node includes: Based on the dependency data package and target execution code segment corresponding to the target node, a runtime data package corresponding to the target workflow is generated.

7. The method according to any one of claims 1 to 5, characterized in that, Before displaying the workflow design interface, the method further includes: Retrieve the code segment invoked during the registration process of each preset node; The storage path of the acquired code segment is parsed to obtain the source data packet of the acquired code segment; Establish a mapping relationship between each preset node and the corresponding parsed source data packet to obtain at least one mapping relationship; The step of obtaining the dependency data packets required for the target node to run in the target workflow includes: Based on the at least one mapping relationship, the source data packet corresponding to the target node is obtained, and the obtained source data packet is used as the dependency data packet required for the target node to run.

8. The method according to any one of claims 1 to 5, characterized in that, Creating a container instance corresponding to the target workflow based on the runtime data package includes: Obtain a preset image deployment script; wherein, the preset image deployment script contains the image identifier of the image required for each preset node to run; Based on the preset image deployment script, obtain the target image that matches the image identifier corresponding to the target node; Based on the target image and the runtime data package, a container instance corresponding to the target workflow is constructed.

9. The method according to any one of claims 1 to 5, characterized in that, After creating a container instance corresponding to the target workflow based on the runtime data package to deploy the target workflow, the method further includes: Obtain the target node type to which the target node running in the container instance belongs; According to the pre-defined correspondence between node types and parameter mapping rules, obtain the target parameter mapping rule corresponding to the target node type; Based on the target parameter mapping rule, the input parameters in the target node are assigned values ​​to obtain the workflow to be tested; The test task is executed by calling the workflow to be tested in order to obtain the execution test results for the target workflow; If the execution test results indicate that the target workflow passes the test, then the target workflow is deployed to the production environment.

10. A workflow deployment device, characterized in that, The device includes a display unit, an editing response unit, a trigger response unit, an acquisition unit, and a deployment unit, wherein: The display unit is used to display the workflow design interface; wherein, the workflow design interface includes a workflow editing component and a workflow deployment component; The editing response unit is configured to respond to the editing operation of the workflow editing component, obtain a reference node selected from at least one preset node, and construct a workflow to be deployed based on the reference node; The trigger response unit is used to respond to the trigger operation of the workflow deployment component, obtain the execution performance data of the reference node in the workflow to be deployed during the execution of data processing tasks, and optimize the workflow to be deployed based on the execution performance data corresponding to the reference node to obtain the target workflow; The acquisition unit is used to acquire the dependency data packets required for the target node to run in the target workflow, and generate the running data packets corresponding to the target workflow based on the dependency data packets corresponding to the target node; The deployment unit is used to create a container instance corresponding to the target workflow based on the running data package, so as to deploy the target workflow.

11. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the workflow deployment method as described in any one of claims 1 to 9.

12. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the workflow deployment method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes a computer program adapted to be loaded by a processor and executed as a deployment method of the workflow as described in any one of claims 1 to 9.

Images