AI Agent-based workflow decision-making and execution method, device, and storage medium for text graphs.

Abstract

The application provides an AI Agent-based text-to-image flow process decision execution method and device and a storage medium. The method comprises the following steps: analyzing a workflow file of a text-to-image tool, extracting a node set, a data link connection relationship between nodes, a model set relied on by the nodes, and a dependency relationship between the nodes and the models; comparing the node set and the model set with installed nodes and deployed models of a target platform to determine a fault type; determining a fault handling sequence by using an AI Agent with a built-in knowledge base module; performing a self-healing operation by using the AI Agent according to the fault handling sequence; starting a trial operation verification on the workflow after the self-healing operation is completed, and updating the fault type, the handling sequence and the self-healing operation to the knowledge base module. By using the application, cross-environment migration of the text-to-image workflow is achieved, and the migration efficiency and success rate are improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and text-based graph technology, specifically to a text-based graph process decision-making execution method, apparatus, and storage medium based on an AI Agent. Background Technology

[0002] Text-to-image (TPE) technology automatically generates images from text descriptions using artificial intelligence models, and is widely used in creative design, content creation, and other fields. As TPE applications become more complex, node-based TPE tools (such as ComfyUI) are gradually becoming the mainstream choice for professional users. These tools break down the TPE process into interconnected functional nodes, forming a workflow through data links between nodes. Each node can access corresponding model resources (such as basic generation models and fine-tuning models). Users often need to migrate external workflows to their own platform for operation.

[0003] However, due to differences in environment configuration across different platforms, missing nodes or models often occur after workflow migration. Currently, when a node is missing, users need to manually identify and install it; when a model is missing, users need to download the model and configure its path. If multiple resources are missing simultaneously, users also need to rely on experience to determine the processing order. The entire process relies on manual operation, which is inefficient.

[0004] Therefore, how to achieve automated detection, processing order decision-making, and self-healing operations for missing nodes and models during the migration of Wensheng graph workflows, in order to improve migration efficiency and success rate, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] In view of this, it is necessary to provide a method, device and storage medium for decision-making and execution of text-based graphs based on AI Agent, so as to solve the technical problems of the existing technology in the process of migrating text-based graph workflows, such as the need for manual detection of missing nodes and models, reliance on experience for processing order and low efficiency of resource replenishment.

[0006] To address the aforementioned technical problems, in a first aspect, the present invention provides a text-based graph process decision execution method based on an AI Agent, comprising: The workflow file of the Wenshengtu tool is parsed to extract the node set, the data link connection relationship between nodes, the model set that the nodes depend on, and the dependency relationship between nodes and models; The node set and model set are compared with the installed nodes and deployed models of the target platform to determine the fault type, which includes node missing fault and / or model missing fault. An AI Agent with a built-in knowledge base module is used to determine the fault handling sequence based on the fault type, the data link connection relationship between nodes, and the dependency relationship between nodes and the model. The knowledge base module stores historical fault handling records. According to the fault handling sequence, the AI ​​Agent performs a self-healing operation, wherein, for the node missing fault, the missing node is retrieved and downloaded and installed; for the model missing fault, the missing model is retrieved and downloaded, and the model library index is updated. After the self-healing operation is completed, a trial run is initiated to verify the workflow, and the fault type, processing sequence, and self-healing operation are updated to the knowledge base module.

[0007] In one possible implementation, before parsing the workflow file of the text image tool, the process further includes: Configure compliant resource retrieval sources, including official node repositories, open-source code repositories, and model licensing download platforms.

[0008] In one possible implementation, the step of comparing the node set and model set with the installed nodes and deployed models of the target platform to determine the fault type further includes: Compare the version information of each node in the node set with the version information of the nodes already installed on the target platform to determine node version incompatibility faults; and / or, Compare the version information of each model in the model set with the version information of the models already deployed on the target platform to identify model version incompatibility faults; and / or, The model path configuration in the workflow file is compared with the model storage path rules of the target platform to identify path configuration errors.

[0009] In one possible implementation, the step of retrieving and downloading / installing the missing node in response to the node missing fault includes: Based on the name, version, and function information of the missing node, the corresponding download address is retrieved from the preset compliant resource retrieval sources; Verify the validity and security of the download address; Initiate multi-threaded download to download the node file to the specified directory; Install the nodes in the node file and configure the dependencies of the nodes.

[0010] In one possible implementation, the step of retrieving and downloading the missing model and updating the model library index in response to the model missing fault includes: Based on the missing model's type, version, and training parameter information, the corresponding download address is retrieved from the preset compliant resource retrieval sources; For models requiring authorization, prompt the user to enter authorization information and verify it; Start the download and enable the resume function for model files that exceed the preset threshold; After the missing model is downloaded, the index information of the missing model is registered in the model library, and the access path of the missing model is configured.

[0011] In one possible implementation, the step of performing self-healing operations using the AI ​​Agent according to the fault handling sequence further includes: When a node version incompatibility fault is identified, the version compatibility is checked. If the higher version is backward compatible, the higher version is automatically upgraded. If the versions are incompatible, the corresponding version is downloaded and deployed while the original version is retained, and a version switching mechanism is configured. When a path configuration error is identified, the model path configuration in the workflow is corrected in batches based on the path rules of the target platform, and the model call parameters of the corresponding nodes are updated synchronously.

[0012] In one possible implementation, the step of initiating a trial run verification of the workflow after the self-healing operation is completed includes: The workflow is monitored in real time, and node response speed, model call success rate and rendering effect indicators are collected. If the trial run is successful, a self-healing success report will be generated; If a new fault occurs, return to the step that determined the fault type and reprocess it.

[0013] In one possible implementation, before initiating the trial run verification of the workflow after the self-healing operation is completed, the following is also included: Based on the hardware configuration information of the target platform, the node parameters and model calling logic are optimized. The hardware configuration information includes the GPU model and memory size, and the node parameters include the number of inference threads and the video memory usage threshold.

[0014] Secondly, the present invention also provides a text-based graph process decision execution device based on an AI Agent, comprising: The parsing unit is used to parse the workflow file of the Wenshengtu tool, extracting the node set, the data link connection relationship between nodes, the model set that the nodes depend on, and the dependency relationship between nodes and models; The detection unit is used to compare the node set and model set with the installed nodes and deployed models of the target platform to determine the fault type, which includes node missing fault and / or model missing fault. The determining unit is used to use an AI Agent with a built-in knowledge base module to determine the fault handling order based on the fault type, the data link connection relationship between nodes, and the dependency relationship between nodes and the model. The knowledge base module stores historical fault handling records. The migration self-healing unit is used to perform self-healing operations using the AI ​​Agent according to the fault handling sequence, wherein, for the node missing fault, the missing node is retrieved and downloaded and installed; for the model missing fault, the missing model is retrieved and downloaded, and the model library index is updated. The update unit is used to initiate a trial run verification of the workflow after the self-healing operation is completed, and to update the fault type, processing sequence and the self-healing operation to the knowledge base module.

[0015] Thirdly, the present invention also provides an electronic device, including a memory and a processor, wherein, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the AI ​​Agent-based text graph process decision execution method described in any of the above implementations.

[0016] Fourthly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instruction, which, when executed by a processor, can implement the steps in the AIAgent-based text-to-image flow decision execution method described in any of the above implementations.

[0017] The beneficial effects of this invention are: This invention provides an AI Agent-based workflow decision-making and execution method for Wensheng graph tools. By parsing the workflow files of the Wensheng graph tool, it extracts the node set, the data link connections between nodes, the model sets that nodes depend on, and the dependencies between nodes and models. This captures the core information required for workflow migration, avoiding missed or false detections due to incomplete information and improving the accuracy and comprehensiveness of fault identification. The node set and model set are compared with the installed nodes and deployed models on the target platform to determine the fault type, including node missing faults and / or model missing faults, ensuring the accuracy of fault location. Using an AI Agent with a built-in knowledge base module, the fault handling sequence is determined based on the fault type, the data link connections between nodes, and the dependencies between nodes and models. The knowledge base module stores historical fault handling records, enabling intelligent planning in multiple fault scenarios, avoiding duplicate processing or dependency conflicts due to improper ordering, and improving overall migration efficiency. Based on the fault handling sequence, AI... The Agent performs a self-healing operation. For missing node failures, it retrieves and downloads the missing node; for missing model failures, it retrieves and downloads the missing model and updates the model library index, ensuring version matching and correct paths for resources. This completes the resource replenishment of missing nodes and models, improving resource replenishment efficiency and increasing the success rate of workflow migration. After completing the self-healing operation, the workflow is tested and verified, and the failure type, processing order, and self-healing operation are updated in the knowledge base module. This enables the Wenshengtu workflow to be used immediately upon cross-environment migration, improving migration efficiency and success rate. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart of an embodiment of the text-based graph process decision execution method provided by the present invention; Figure 2 The flowchart illustrates the fully automated migration of the text-to-image workflow provided by this invention to the ComfyUI platform. Figure 3 This is a schematic diagram of the structure of the AI ​​Agent-based text graph process decision execution device provided by the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0021] In the description of the embodiments of the present invention, unless otherwise stated, "a plurality of" means two or more.

[0022] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0023] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0024] Before demonstrating the embodiments, the following terms will be explained.

[0025] AI Agent: Refers to an autonomous decision-making and execution entity based on artificial intelligence technology, which integrates capabilities such as perception, reasoning, planning, execution, and learning. In the specific implementation of this embodiment, the AI ​​Agent can be implemented as a software service or an algorithm module, which determines the fault handling sequence and executes self-healing operations by calling pre-stored decision rules or knowledge-based matching algorithms.

[0026] This invention provides a text-based graph process decision execution method, apparatus, and storage medium based on AI Agent, which are described below.

[0027] The text-based image processing decision-making execution method based on AI Agent provided in this application can be executed by an electronic device configured with an AI Agent module. Specifically, the execution entity can be a host, server, or cloud instance that has a node-based text-based image tool platform (such as ComfyUI) installed and integrates AI Agent services. As an option, the AI ​​Agent can run as an independent background service process, interacting with the text-based image tool platform via a network interface; alternatively, the AI ​​Agent can be embedded within the text-based image tool platform as a plugin or built-in module, directly calling the platform's resource management interface. It should be understood that regardless of the deployment method, as long as the parsing, comparison, decision-making, self-healing, and verification steps described in this embodiment are executed, it falls within the protection scope of this application.

[0028] Figure 1 This is a schematic diagram of an embodiment of the text-based graph flow decision execution method based on AI Agent provided by the present invention, as shown below. Figure 1 As shown, the AI ​​Agent-based text-based graph process decision execution method includes: S101. Parse the workflow file of the Wenshengtu tool to extract the node set, the data link connection relationship between nodes, the model set that the nodes depend on, and the dependency relationship between nodes and models.

[0029] In this embodiment, the text-to-image tool can be ComfyUI.

[0030] Workflow files typically use JSON format or a custom workflow format defined by ComfyUI. They describe the node topology, node parameter configuration, and model call information for the entire text graph process.

[0031] The node set includes basic nodes and custom nodes, and the model set includes basic generative models (such as Stable Diffusion 2.1), LoRA fine-tuning models (such as “cartoon character” LoRA), and VAE models, etc.

[0032] The data link connections between nodes reflect the flow of data in the workflow, for example, model loading node → parameter adjustment node → sampling node → image output node.

[0033] The dependency relationship between nodes and models specifies the specific model that each node needs to call at runtime. For example, a LoRA node depends on a specific underlying generative model.

[0034] Specifically, the workflow file of the Wensheng graph tool to be migrated is obtained, and the workflow file is parsed in a structured manner to identify the type, name, version, and input / output ports of each node in the workflow file, thereby constructing a complete node graph structure. By parsing the model name and path fields in the node parameters, the mapping relationship between nodes and models is established, and then the node set, the data link connection relationship between nodes, the set of models that nodes depend on, and the dependency relationship between nodes and models are extracted.

[0035] Understandably, by performing structured parsing of workflow files, extracting node sets, data link connections between nodes, model sets that nodes depend on, and dependencies between nodes and models, the core information required for workflow migration is captured. This enables subsequent fault detection to be accurate down to the specific node name, version, and model type, avoiding missed or false detections due to incomplete information, thereby improving the accuracy and comprehensiveness of fault identification.

[0036] S102. Compare the node set and model set with the installed nodes and deployed models of the target platform to determine the fault type, which includes node missing fault and / or model missing fault.

[0037] The target platform's operating environment includes a list of installed nodes and deployed model files and their storage paths. The node list includes node names and versions.

[0038] Specifically, the extracted node set and model set are compared one by one with the existing operating environment of the target platform to determine whether the current workflow has any fault types on the target platform. Fault types include missing node faults and / or missing model faults. For example, if a custom node named "Advanced Image Upscale" (version v1.2) in the parsed node set is not present in the list of installed nodes on the target platform, it is determined to be a missing node fault; if a "Cartoon Character" LoRA model (version v2.0) in the parsed model set is not present in the list of deployed models on the target platform, it is determined to be a missing model fault.

[0039] Understandably, comparing the node set and model set with the installed nodes and deployed models of the target platform can quickly and accurately identify node and / or model missing faults, while ensuring the accuracy of fault location and providing reliable factual basis for subsequent intelligent decision-making.

[0040] S103. Using an AI Agent with a built-in knowledge base module, the fault handling sequence is determined based on the fault type, the data link connection relationship between nodes, and the dependency relationship between nodes and the model. The knowledge base module stores historical fault handling records.

[0041] Among them, AI Agent is a software entity that can autonomously perceive the environment, analyze problems and perform actions. Its internal knowledge base module stores historical fault handling records, which include the fault type, processing order and specific self-healing operations used when handling similar faults in the past.

[0042] The knowledge base module is the internal storage unit of the AI ​​Agent, used to persistently store historical fault handling records. Each record contains at least the fault type, such as the combination of missing nodes and missing models, the processing order (e.g., processing missing models before missing nodes), and the corresponding migration self-healing operation (e.g., the specific download address and installation steps). The knowledge base can be implemented using relational databases, key-value stores, or vector-based retrieval libraries to support efficient read and update operations.

[0043] Specifically, when a workflow file is found to have multiple faults, the AI ​​Agent's knowledge base module determines the fault handling order based on the currently identified fault types, the data link connections between nodes, and the dependencies between nodes and models. For example, when both node missing faults and model missing faults exist simultaneously, and the missing node is downstream of the missing model in the data link, or the missing node explicitly requires the missing model in its dependencies, the AI ​​Agent prioritizes handling the model missing fault before handling the node missing fault. This fault handling order avoids nodes failing to function properly after installation due to model missingness, thus reducing redundant operations.

[0044] Understandably, by employing an AI Agent with a built-in knowledge base module, the fault handling sequence is determined based on fault type, data link connections between nodes, and dependencies between nodes and the model, enabling intelligent planning in multi-fault scenarios. Compared to the traditional method of manually selecting the handling sequence based on experience, this embodiment can automatically generate the optimal handling sequence based on historical success cases and the actual dependency structure of the current workflow, thereby avoiding duplicate processing or dependency conflicts caused by improper ordering and improving overall migration efficiency.

[0045] S104. According to the fault handling sequence, the AI ​​Agent performs a self-healing operation, wherein, for the node missing fault, the missing node is retrieved and downloaded and installed; for the model missing fault, the missing model is retrieved and downloaded, and the model library index is updated.

[0046] Self-healing operations refer to a series of repair actions automatically performed by the AI ​​Agent in response to the identified fault types. For node missing faults, self-healing operations include retrieving the node download address, verifying the address security, downloading the node file, and installing the node and its dependencies. For model missing faults, self-healing operations include retrieving the model download address, handling authorization verification (if necessary), downloading the model file, updating the model library index, and configuring the access path.

[0047] Specifically, based on the fault type and processing order, the AI ​​Agent performs self-healing operations. These self-healing operations include two branches: for node missing faults, the AI ​​Agent automatically retrieves compliant resources, such as the pre-defined ComfyUI official node repository and GitHub open-source repositories, obtains the download address of the missing node, and then downloads and installs it; for model missing faults, the AI ​​Agent automatically retrieves compliant resources, such as authorized model download platforms like Hugging Face and Civitai, obtains the download address of the missing model, downloads the model, registers its index information in the target platform's model library, and configures the model's access path. All of the above retrieval, download, installation, and configuration processes require no manual user intervention. For example, when the processing order is model first, then node, the AI ​​Agent first completes the self-healing operation for model missing faults, and then performs the self-healing operation for node missing faults.

[0048] Understandably, by using an AI Agent to automatically retrieve, download, and install missing nodes, as well as retrieve, download, and update the index of missing models, the resource completion process that originally required manual completion by users is fully automated according to the determined processing order. This solves the problems of time-consuming and error-prone manual operations, ensures the version matching and path correctness of resources, thereby realizing the resource completion of missing nodes and models, improving the efficiency of resource completion, and increasing the success rate of workflow migration.

[0049] S105. Start a trial run to verify the workflow after the self-healing operation is completed, and update the fault type, processing sequence and the self-healing operation to the knowledge base module.

[0050] Trial operation refers to actually executing the workflow on the target platform to verify whether it can run stably.

[0051] Specifically, after completing all self-healing operations, a trial run is initiated to verify the workflow that has now been filled in with missing nodes and models. During the trial run, the execution status of the workflow is monitored, such as whether nodes respond normally, whether models are successfully invoked, and whether rendering is complete. After the trial run, the fault types, fault handling order, and self-healing operations recorded during this process are updated in the knowledge base module. This allows the AI ​​Agent to directly refer to this record when encountering similar fault combinations in the future, thereby determining the fault handling order more quickly and accurately.

[0052] Understandably, conducting trial runs to verify the workflow after completing the self-healing operation ensures that the migrated workflow is truly executable on the target platform, achieving immediate usability and improving the efficiency of Wenshengtu's process decision-making and execution. Simultaneously, updating the fault type, processing sequence, and self-healing operation to the knowledge base module allows the AI ​​Agent's experience to continuously accumulate and iterate, thereby continuously improving the efficiency and accuracy of decision-making for similar future faults, forming a positive self-optimization loop.

[0053] As a specific application scenario, suppose a user imports an external workflow file into the ComfyUI platform. This workflow contains a custom node called "Advanced Image Upscale" (version v1.2) and a LoRA model called "Cartoon Character" (version v2.0). The target platform does not have this custom node in its list of installed nodes, nor does it have this LoRA model in its list of deployed models. However, this custom node calls the LoRA model at runtime.

[0054] This embodiment first parses the workflow file, extracting the node set (including the aforementioned custom nodes) and model set (including the aforementioned LoRA models), and identifies the dependencies between nodes and models (custom nodes depend on LoRA models). Then, by comparing with the target platform, it determines that there are node missing faults and model missing faults. The AI ​​Agent reads historical fault handling records in the knowledge base and finds that prioritizing the handling of model missing faults before handling node missing faults has a higher success rate in similar scenarios. Combined with the current dependency relationship (nodes depend on models), the processing order is determined to be: handle model missing faults first, then handle node missing faults. Subsequently, the AI ​​Agent automatically retrieves and downloads the "Cartoon Character" LoRA model from the preset model licensing platform and updates the model library index; then it retrieves and downloads the "Advanced ImageUpscale" node from the ComfyUI official node repository and completes the installation. Finally, trial operation verification shows that the workflow can execute normally and generate images. The fault type (node ​​missing, model missing), processing order (model first, then node), and self-healing operation (retrieving the specific source and method of download) are recorded in the knowledge base.

[0055] This method can be applied to node-based text graph tool platforms, such as the ComfyUI platform. Specifically, when a user needs to migrate a workflow created in another environment, such as a ComfyUI instance on another device, to the current target platform, the method described in this embodiment can automatically complete workflow parsing, fault detection, intelligent decision-making, automatic repair, and verification, thereby achieving a smooth workflow migration.

[0056] In summary, the AI ​​Agent-based Wensheng Graph workflow decision-making and execution method provided in this invention, through parsing workflow files, fully extracts the dual dependency information of nodes and models, achieving accurate identification of two core faults: missing nodes and missing models. Utilizing the AI ​​Agent with a built-in knowledge base module, it intelligently plans the optimal fault handling sequence by comprehensively considering fault types, data link connections between nodes, and dependencies between nodes and models, overcoming the problem of repetitive processing caused by disordered sequences in multi-fault scenarios. Furthermore, it automatically executes self-healing operations according to a determined order, completing the retrieval and installation of missing nodes and the download and configuration of missing models, transforming the inefficient traditional mode of relying on manual troubleshooting and manual completion into fully automated execution, improving resource completion efficiency. Finally, trial operation verifies that the workflow can be actually executed, and the processing experience is updated to the knowledge base, forming a closed-loop mechanism for continuous optimization. This enables the Wensheng Graph workflow to be used immediately upon cross-environment migration, improving migration efficiency and success rate.

[0057] In some embodiments of the present invention, before step S101, the method further includes: configuring a compliant resource retrieval source, wherein the compliant resource retrieval source includes the node's official repository, open-source code repository, and model authorization download platform.

[0058] Among them, compliant resource retrieval sources refer to a predefined set of resource repositories used to obtain legitimate and secure download sources for missing nodes or missing models.

[0059] The official node repository includes the ComfyUI official node repository, the open source code repository includes the GitHub open source repository, and the model licensed download platforms include officially authorized model distribution platforms such as the Hugging Face model library and the Civitai model community.

[0060] Specifically, before executing workflow parsing, the environment is pre-configured. Pre-defined compliant resource retrieval sources are configured to include the ComfyUI official node repository, the GitHub open-source repository, and the Hugging Face model library. Simultaneously, security verification parameters are configured, including enabling MD5 hash verification and virus scanning mechanisms to ensure the integrity and security of downloaded resources. Furthermore, multi-threaded download parameters are set, configuring the number of concurrent download threads to 8, and enabling breakpoint resumption for large model files exceeding 100MB to handle download interruptions caused by network instability. These configuration parameters are persistently stored in a configuration file for the AI ​​Agent to access during self-healing operations.

[0061] Understandably, this embodiment provides the AI ​​Agent with a reliable and efficient resource acquisition channel by pre-configuring compliant resource retrieval sources, avoiding self-healing failures caused by unknown sources or invalid addresses during resource download. Simultaneously, the configuration of security verification parameters ensures the integrity of downloaded resources, preventing malicious or corrupted resources from being introduced to the target platform and guaranteeing the platform's operational security. The multi-threaded download and breakpoint resume mechanisms improve the download efficiency of large model files.

[0062] In some embodiments of the present invention, step S102 further includes: comparing the version information of each node in the node set with the version information of the nodes already installed on the target platform to determine a node version incompatibility fault; and / or comparing the version information of each model in the model set with the version information of the models already deployed on the target platform to determine a model version incompatibility fault; and / or comparing the model path configuration in the workflow file with the model storage path rules of the target platform to determine a path configuration error fault.

[0063] The version information refers to the version number identifier corresponding to the node or model, such as node version "v1.2" and model version "2.0", which is used to determine the compatibility of the node or model under different environments.

[0064] Version compatibility assessment refers to evaluating the compatibility between the node or model version used in the source workflow and the version already installed on the target platform, including backward compatibility with higher versions and incompatibility between versions.

[0065] A path configuration error refers to a situation where the model storage path recorded in the workflow file is inconsistent with the model storage path actually used by the target platform.

[0066] The model storage path rules of the target platform refer to the directory structure for storing model files as agreed upon by the target platform.

[0067] Specifically, when comparing the extracted node set with the nodes already installed on the target platform, this embodiment also obtains the version information of each node. For any node in the node set, its version number is compared with the version number of the same-named node already installed on the target platform. If the version numbers are inconsistent, it is determined that there is a node version incompatibility fault.

[0068] Similarly, when comparing the extracted model set with the models already deployed on the target platform, this embodiment obtains the version information of each model. For any model in the model set, its version number is compared with the version number of the same-named model already deployed on the target platform. If the version numbers are inconsistent, it is determined that there is a model version incompatibility fault.

[0069] This embodiment also checks the model path configuration in the workflow file. When parsing the workflow file, the storage path configured for each model node is obtained, and this path is compared with the model storage path rules of the target platform. If the path format, directory level, or root directory does not match the target platform rules, a path configuration error is determined to exist.

[0070] Understandably, by including node version, model version, and model path configuration in the fault detection scope, this embodiment can comprehensively identify various compatibility issues that may occur during workflow migration, providing more complete fault information for subsequent self-healing operations, thereby further improving the success rate and reliability of workflow migration.

[0071] In some embodiments of the present invention, step S104 includes: retrieving the corresponding download address from a preset compliant resource retrieval source based on the name, version, and functional information of the missing node; verifying the validity and security of the download address; starting multi-threaded download to download the node file to a specified directory; installing the node in the node file and configuring the node's dependencies.

[0072] Among them, compliant resource retrieval sources refer to preset and verified channels for obtaining node resources, including but not limited to: ComfyUI official node repository, GitHub open source repository, Comfy Manager resource repository, and third-party authorized node distribution platforms.

[0073] Multi-threaded downloading refers to a technique that establishes multiple network connections simultaneously to download files in parallel, used to improve the download efficiency of large or numerous files.

[0074] Dependencies refer to the supporting components required for a node to run, including but not limited to: other base nodes, specific versions of Python libraries, system dynamic link libraries, or configuration files.

[0075] Specifically, when the AI ​​Agent identifies a missing node, this embodiment performs automated retrieval, verification, download, installation, and configuration operations for the missing node. First, the AI ​​Agent obtains the name, version number, and functional description of the missing node from the fault detection results, using this information as search keywords to search within a preset compliant resource retrieval source. For example, for the missing "Advanced Image Upscale" node (version v1.2), the AI ​​Agent will sequentially access the ComfyUI official node repository and the GitHub open-source repository to find resource entries matching the node name and version.

[0076] Furthermore, after retrieving candidate download addresses, the AI ​​Agent verifies the validity and security of the addresses. Validity verification includes checking whether the link is accessible and whether the resource files are complete; security verification includes checking whether the download source is on a preset whitelist and whether the resource files have been scanned for viruses. For example, for a node repository address obtained from GitHub, the AI ​​Agent will verify whether the repository has sufficient downloads, whether it is an official release, and check whether there are malicious scripts in the repository code.

[0077] After successful verification, the AI ​​Agent initiates multi-threaded downloading, downloading the node files to the target platform's preset node storage directory. For example, for the "Advanced Image Upscale" node, the AI ​​Agent will start 8 parallel threads to download the node compressed package to the "E: / ComfyUI / nodes / custom" directory and automatically decompress it. During the download process, if the network is interrupted, the AI ​​Agent supports resuming interrupted downloads to ensure the integrity of the download task.

[0078] Once the download is complete, the AI ​​Agent automatically performs the installation. The installation process includes: copying the node files to the correct directory location, executing the node's built-in installation script, and configuring the node's dependencies. Dependency configuration refers to automatically detecting and installing other components required for the node to run. For example, the "Advanced Image Upscale" node may depend on the "comfy-ui-image-processing" base node. The AI ​​Agent will check if this base node already exists; if not, it will automatically trigger the corresponding missing node handling process to ensure that the dependencies are fully satisfied.

[0079] Understandably, this embodiment decomposes the handling of missing nodes into multiple automated steps, such as retrieval, verification, download, installation, and dependency configuration, achieving a closed-loop process from fault identification to resource readiness. Compared to manually searching, downloading, and installing nodes, this embodiment significantly improves the efficiency and accuracy of node completion, avoiding secondary failures caused by version mismatches, insecure download sources, or missing dependencies, thus laying a solid foundation for the stable operation of the workflow on the target platform.

[0080] In some embodiments of the present invention, step S104 includes: retrieving the corresponding download address from a preset compliant resource retrieval source based on the type, version, and training parameter information of the missing model; prompting the user to enter authorization information and verifying it for models requiring authorization; starting the download and enabling the breakpoint resume function for model files exceeding a preset threshold; and registering the index information of the missing model in the model library and configuring the access path of the missing model after the missing model has been downloaded.

[0081] Among them, compliant resource retrieval sources refer to pre-defined legal channels used to obtain download addresses of missing models, including but not limited to: officially authorized model platforms (such as Hugging Face and Civitai), open-source model repositories, and third-party resource sites that have been certified for security.

[0082] Authorization verification refers to the process of requesting users to input authorization information before downloading models that require user authentication or paid authorization, and verifying the validity of the user's input account, token, or authorization code.

[0083] Resumable download refers to the ability to resume downloading a model file that exceeds a preset threshold (e.g., 100MB) after an interruption, without needing to re-download the entire file. Model library index refers to a database or directory structure on the target platform used to record information about deployed models, including metadata such as model name, version, storage path, and calling parameters.

[0084] Specifically, when performing self-healing operations for missing model faults, the system first retrieves the corresponding download address from pre-defined compliant resource retrieval sources based on the missing model's type, version, and training parameter information. For example, if the missing model is a "cartoon character" LoRA model, it will sequentially search the Hugging Face platform, Civitai platform, and pre-defined open-source repositories to obtain the official download link for the model. After retrieving the download address, its validity and security are verified, including checking whether the link is accessible and whether it belongs to a pre-defined whitelist domain. For models requiring authorization, the user is prompted to enter authorization information, such as an API key or account token, and the validity of this information is verified by calling the platform interface. After successful verification, the download process is initiated. For model files exceeding a pre-defined threshold, the breakpoint resume function is enabled to ensure the stability and integrity of large file downloads. After the download is complete, the index information of the missing model is registered in the model library, including the model name, version number, storage path, and calling parameters, and its access path is configured so that the corresponding node in the workflow can correctly load the model.

[0085] Understandably, this embodiment avoids the tedious manual search for model download links by automatically retrieving compliant resources and verifying address security, while ensuring the legitimacy and security of the download source. For large model files, the resume function is enabled, improving the stability and reliability of the download process and avoiding retransmission time lost due to network fluctuations. After downloading, the model library index is automatically updated and the access path is configured, enabling nodes in the workflow to seamlessly access deployed models without requiring manual modification of path configurations by the user. This significantly improves the efficiency and success rate of handling model missing failures.

[0086] In some embodiments of the present invention, step S104 further includes: when a node version incompatibility fault is determined, determining the version compatibility, wherein if the higher version is backward compatible, then automatically upgrading to the higher version; if the versions are incompatible, then downloading and deploying the corresponding version while retaining the original version, and configuring a version switching mechanism; when a path configuration error fault is determined, based on the path rules of the target platform, batch correcting the model path configuration in the workflow, and synchronously updating the model call parameters of the corresponding node.

[0087] Version compatibility refers to the ability of different versions of nodes or models to work together in the same environment. Version compatibility includes two scenarios: backward compatibility (a higher version can replace a lower version to work normally) and incompatibility (different versions have functional differences or interface changes and cannot replace each other).

[0088] Path configuration refers to the storage location of model files recorded in the workflow file. Since the model storage directory may differ between different ComfyUI environments, for example, the source platform may use "D: / ComfyUI / models" while the target platform may use "E: / ComfyUI / models", the path needs to be corrected so that the workflow can correctly locate the model files.

[0089] Specifically, when the fault detection module determines that a node version is incompatible, the AI ​​Agent's decision execution module first determines the version compatibility. If the higher version is backward compatible, for example, if node version v2.0 is fully compatible with the functional interfaces of version v1.5, then the node is automatically upgraded to the higher version. If the versions are incompatible, for example, if node version v2.0 modifies the core interface definition, causing the workflow of version v1.5 to be unusable, then the corresponding version that the deployment workflow depends on is downloaded and deployed, while other versions on the target platform are retained, and a version switching mechanism is configured so that users can choose which version to use as needed.

[0090] Furthermore, when a path configuration error is identified, the AI ​​Agent's decision execution module, based on the target platform's preset path rules (e.g., the standard model storage directory is "E: / ComfyUI / models" and the storage structure is "storage directory / resource type / resource name / version"), batch corrects the model path configuration in the workflow file, changing the original path (e.g., "D: / ComfyUI / models / Stable-diffusion / v1.5") to the target path (e.g., "E: / ComfyUI / models / Stable-diffusion / v1.5"), and synchronously updates the model call parameters of the corresponding nodes to ensure that the nodes can correctly load the model.

[0091] Understandably, this embodiment avoids the tedious process of manually checking node versions, finding matching versions, and handling version conflicts by automatically determining version compatibility and performing corresponding upgrades or parallel deployments. Simultaneously, the version switching mechanism ensures flexibility in scenarios where multiple versions coexist. Batch correction of path configurations avoids the inefficient operation of modifying model paths one by one, eliminating the risk of workflow failure due to incorrect paths.

[0092] In some embodiments of the present invention, step S105 includes: real-time monitoring of the workflow's running status, collecting node response speed, model call success rate, and rendering effect indicators; if the trial run is successful, generating a self-healing success report; if a new fault occurs, returning to the step of determining the fault type for reprocessing.

[0093] The self-healing success report is a document that records the entire process of the self-healing operation. Its content includes at least the fault type, processing sequence, self-healing operation details, and trial operation results.

[0094] Specifically, after completing the self-healing operation, a trial run of the workflow is initiated. During the trial run, multiple operational metrics are collected in real time through the built-in monitoring module, including node response speed, model call success rate, and rendering performance metrics. Node response speed refers to the time consumed by each node from receiving input data to outputting a result; model call success rate refers to the percentage of times a result is successfully returned during model loading and inference; rendering performance metrics include, but are not limited to, image resolution, generation frame rate, and whether rendering errors occur.

[0095] As a specific implementation method, the trial run can select typical input prompts and execute them multiple times, for example, setting the execution count to 5 times. During each execution, the response latency of each node, the model call status, and the quality of the final generated image are recorded. If all nodes respond normally during the trial run, the model call success rate reaches 100%, and the rendering result is clear and without anomalies, the trial run is considered successful, and the system automatically generates a self-healing success report. This report details the fault types identified during the migration process (e.g., missing nodes, missing models), the determined processing order (e.g., model first, then nodes), the self-healing operations performed (e.g., downloading and installing nodes from a specified repository, downloading and updating the model index from an authorized platform), and the key indicator values ​​collected during the trial run (e.g., average node response speed ≤ 0.4 seconds, model call success rate 100%, rendering frame rate ≥ 12fps).

[0096] If new faults occur during the trial run, such as a node being successfully installed but throwing an exception at runtime, or a model being successfully downloaded but encountering a version compatibility error when called, the system will not generate a success report. Instead, it will automatically return to the step that determined the fault type, add the newly occurring fault to the fault set, and re-execute subsequent decisions and self-healing operations. This closed-loop mechanism ensures that the workflow can eventually run stably after multiple iterations.

[0097] Understandably, this embodiment uses a trial run verification mechanism to actually execute and test the workflow after the self-healing operation, ensuring the reliability of the migration results and truly achieving "use immediately after migration." Simultaneously, by collecting key operational indicators, it provides quantitative evidence for the successful self-healing report, facilitating user traceability and evaluation of migration quality. The closed-loop design, which automatically returns to reprocessing when new faults occur during trial run, enables the system to cope with multiple faults in complex scenarios, further improving the success rate and robustness of workflow migration.

[0098] In some embodiments of the present invention, before step S105, the method further includes: optimizing node parameters and model calling logic based on the hardware configuration information of the target platform, wherein the hardware configuration information includes GPU model and memory size, and the node parameters include the number of inference threads and the video memory usage threshold.

[0099] Among them, hardware configuration information refers to the physical computing resource parameters of the target platform, including but not limited to the graphics processor model, video memory capacity, memory size, and number of central processing unit cores.

[0100] Node parameters refer to the computing resource parameters that each functional node in the workflow needs to be configured during runtime, such as the number of inference threads, the threshold for video memory usage, and the batch size.

[0101] Model invocation logic refers to the resource scheduling strategy when multiple models are loaded or executed simultaneously in a workflow, such as the model loading order, the timing of memory release, and the priority of resource preemption between models.

[0102] Specifically, before initiating trial operation and verification, node parameters and model calling logic were optimized based on the target platform's hardware configuration information. First, the hardware configuration information of the target platform was obtained through system interfaces, such as reading the GPU model (e.g., NVIDIA GeForce RTX 4070) and its video memory capacity (e.g., 8GB) via the operating system API, and simultaneously obtaining the memory size (e.g., 32GB). Then, node parameters were dynamically adjusted based on the aforementioned hardware configuration information: for example, when the video memory capacity was 8GB, the number of inference threads was set to 6, and the video memory usage threshold was set to 6GB to avoid video memory overflow; when the memory capacity was small, the number of concurrently processing nodes was appropriately reduced. Simultaneously, the model calling logic was optimized, for example, adjusting the model loading order based on model file size and calling frequency, prioritizing the loading of small, frequently called models, or allowing only a limited number of models to occupy video memory simultaneously, avoiding resource conflicts caused by multiple nodes calling the same model at the same time.

[0103] Understandably, this embodiment achieves precise adaptation of computing resources by dynamically optimizing node parameters and model calling logic based on the actual hardware configuration of the target platform. On the one hand, it avoids memory overflow or resource contention caused by improper parameter settings, thus improving the stability of workflow operation; on the other hand, by reasonably configuring the number of inference threads and the model loading order, it fully utilizes hardware performance, reduces node response latency and model loading time, thereby improving the overall smoothness and execution efficiency of the workflow.

[0104] In one specific implementation, such as Figure 2 The diagram shown is a flowchart of the fully automated migration of the Wenshengtu workflow to the ComfyUI platform, including the following steps: Step 201, Preliminary Preparation: Set up the target ComfyUI runtime environment, configure the core runtime parameters of the AI ​​Agent, preset compliant resource retrieval sources, set the standard storage directory and path configuration rules for nodes and models, and configure the security verification parameters, multi-threaded download parameters, and breakpoint resume parameters of the AI ​​Agent.

[0105] Step 202, Workflow Import and Parsing: Obtain the external Wenshengtu workflow file and import it into the target ComfyUI platform. The AI ​​Agent's workflow parsing module automatically identifies the underlying structure of the workflow file, performs structured parsing, and extracts the core information of the workflow.

[0106] Step 203, Fault Detection and Location: The fault detection module of the AI ​​Agent calls the core information in the parsing report, combines it with the existing operating environment of the target ComfyUI platform, performs pre-detection of the imported workflow, automatically identifies the fault type and locates the root cause of the fault.

[0107] Step 204, Fault Handling: The AI ​​Agent's decision execution module automatically generates targeted fault handling strategies based on the fault detection results and recorded fault information, combined with historical fault handling experience in the knowledge base module, and executes fault self-healing operations.

[0108] Step 205, Adaptation Optimization and Trial Run: After the fault handling is completed, the AI ​​Agent's adaptation optimization module optimizes the node parameters and model calling logic based on the hardware configuration of the target ComfyUI platform. After the adaptation is completed, the workflow trial run is automatically started, the running status is monitored in real time, and key indicators are collected.

[0109] Step 206, Run Verification: The verification feedback module analyzes key indicators of the trial run to determine whether the workflow is running stably and whether the rendering results meet expectations. If the run is successful, proceed to step 207; if a new fault occurs, return to step 203 for reprocessing.

[0110] Step 207, Migration Complete: Generate a migration success report, record details of troubleshooting and resource replenishment, and complete the workflow migration.

[0111] To better implement the AI ​​Agent-based text-to-graph flow decision execution method in this invention embodiment, based on the AI ​​Agent-based text-to-graph flow decision execution method, as follows: Figure 3 As shown, this embodiment of the invention also provides a text-based graph process decision execution device based on an AI Agent. The text-based graph process decision execution device 300 based on an AI Agent includes: The parsing unit 301 is used to parse the workflow file of the Wenshengtu tool, extract the node set, the data link connection relationship between nodes, the model set that the nodes depend on, and the dependency relationship between nodes and models; The detection unit 302 is used to compare the node set and model set with the installed nodes and deployed models of the target platform to determine the fault type, which includes node missing fault and / or model missing fault. The determining unit 303 is used to use an AI Agent with a built-in knowledge base module to determine the fault handling order based on the fault type, the data link connection relationship between nodes, and the dependency relationship between nodes and the model. The knowledge base module stores historical fault handling records. The migration self-healing unit 304 is used to perform self-healing operations using the AI ​​Agent according to the fault handling sequence, wherein, for the node missing fault, the missing node is retrieved and downloaded and installed; for the model missing fault, the missing model is retrieved and downloaded, and the model library index is updated. The update unit 305 is used to start a trial run verification of the workflow after the self-healing operation is completed, and to update the fault type, processing sequence and the self-healing operation to the knowledge base module.

[0112] The AI ​​Agent-based text-to-image workflow decision execution device 300 provided in the above embodiments can realize the technical solutions described in the above AI Agent-based text-to-image workflow decision execution method embodiments. The specific implementation principles of each module or unit can be found in the corresponding content in the above AI Agent-based text-to-image workflow decision execution method embodiments, which will not be repeated here.

[0113] Accordingly, this application also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the AI ​​Agent-based text graph flow decision execution method provided in the above-described method embodiments.

[0114] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0115] The above provides a detailed description of the text graph flow decision execution method, apparatus, and storage medium based on AI Agent provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A text-based graph process decision execution method based on AI Agent, characterized in that, include: The workflow file of the Wenshengtu tool is parsed to extract the node set, the data link connection relationship between nodes, the model set that the nodes depend on, and the dependency relationship between nodes and models; The node set and model set are compared with the installed nodes and deployed models of the target platform to determine the fault type, which includes node missing fault and / or model missing fault. An AI Agent with a built-in knowledge base module is used to determine the fault handling sequence based on the fault type, the data link connection relationship between nodes, and the dependency relationship between nodes and the model. The knowledge base module stores historical fault handling records. According to the fault handling sequence, the AI ​​Agent performs a self-healing operation, wherein, for the node missing fault, the missing node is retrieved and downloaded and installed; for the model missing fault, the missing model is retrieved and downloaded, and the model library index is updated. After the self-healing operation is completed, a trial run is initiated to verify the workflow, and the fault type, processing sequence, and self-healing operation are updated to the knowledge base module.

2. The text-based graph flow decision execution method based on AI Agent according to claim 1, characterized in that, Before parsing the workflow file of the text-based image tool, the process also includes: Configure compliant resource retrieval sources, including official node repositories, open-source code repositories, and model licensing download platforms.

3. The text-based graph flow decision execution method based on AI Agent according to claim 1, characterized in that, The step of comparing the node set and model set with the installed nodes and deployed models of the target platform to determine the fault type also includes: Compare the version information of each node in the node set with the version information of the nodes already installed on the target platform to determine node version incompatibility faults; and / or, Compare the version information of each model in the model set with the version information of the models already deployed on the target platform to identify model version incompatibility faults; and / or, The model path configuration in the workflow file is compared with the model storage path rules of the target platform to identify path configuration errors.

4. The text-based graph flow decision execution method based on AI Agent according to claim 1, characterized in that, The process of retrieving, downloading, and installing the missing node in response to the node missing fault includes: Based on the name, version, and function information of the missing node, the corresponding download address is retrieved from the preset compliant resource retrieval sources; Verify the validity and security of the download address; Initiate multi-threaded download to download the node file to the specified directory; Install the nodes in the node file and configure the dependencies of the nodes.

5. The text-based graph flow decision execution method based on AI Agent according to claim 1, characterized in that, The steps for addressing the missing model fault, including retrieving and downloading the missing model and updating the model library index, include: Based on the missing model's type, version, and training parameter information, the corresponding download address is retrieved from the preset compliant resource retrieval sources; For models requiring authorization, prompt the user to enter authorization information and verify it; Start the download and enable the resume function for model files that exceed the preset threshold; After the missing model is downloaded, the index information of the missing model is registered in the model library, and the access path of the missing model is configured.

6. The text-based graph flow decision execution method based on AI Agent according to claim 1, characterized in that, The step of performing self-healing operations using the AI ​​Agent according to the fault handling sequence further includes: When a node version incompatibility fault is identified, the version compatibility is checked. If the higher version is backward compatible, the higher version is automatically upgraded. If the versions are incompatible, the corresponding version is downloaded and deployed while the original version is retained, and a version switching mechanism is configured. When a path configuration error is identified, the model path configuration in the workflow is corrected in batches based on the path rules of the target platform, and the model call parameters of the corresponding nodes are updated synchronously.

7. The text-based graph flow decision execution method based on AI Agent according to claim 1, characterized in that, The process of initiating a trial run verification of the workflow after the self-healing operation is completed includes: The workflow is monitored in real time, and node response speed, model call success rate and rendering effect indicators are collected. If the trial run is successful, a self-healing success report will be generated; If a new fault occurs, return to the step that determined the fault type and reprocess it.

8. The text-based graph flow decision execution method based on AI Agent according to claim 1, characterized in that, Before initiating trial run verification of the workflow after the self-healing operation is completed, the following is also included: Based on the hardware configuration information of the target platform, the node parameters and model calling logic are optimized. The hardware configuration information includes the GPU model and memory size, and the node parameters include the number of inference threads and the video memory usage threshold.

9. A text-based graph process decision-making and execution device based on AI Agent, characterized in that, include: The parsing unit is used to parse the workflow file of the Wenshengtu tool, extracting the node set, the data link connection relationship between nodes, the model set that the nodes depend on, and the dependency relationship between nodes and models; The detection unit is used to compare the node set and model set with the installed nodes and deployed models of the target platform to determine the fault type, which includes node missing fault and / or model missing fault. The determining unit is used to use an AI Agent with a built-in knowledge base module to determine the fault handling order based on the fault type, the data link connection relationship between nodes, and the dependency relationship between nodes and the model. The knowledge base module stores historical fault handling records. The migration self-healing unit is used to perform self-healing operations using the AI ​​Agent according to the fault handling sequence, wherein, for the node missing fault, the missing node is retrieved and downloaded and installed; for the model missing fault, the missing model is retrieved and downloaded, and the model library index is updated. The update unit is used to initiate a trial run verification of the workflow after the self-healing operation is completed, and to update the fault type, processing sequence and the self-healing operation to the knowledge base module.

10. A computer-readable storage medium, characterized in that, Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the AIAgent-based text graph process decision execution method as described in any one of claims 1 to 8.

Images