Script intelligent adaptation and visual arrangement method and system for non-technical personnel

Abstract

The present invention provides a method and system for intelligent script adaptation and visual orchestration for non-technical personnel. The method includes: obtaining the script file uploaded by the user, extracting the dependency library and building an isolated container environment through AST analysis technology; intelligently identifying and converting the file path and database connection in the script to generate standardized API calls; performing input and output analysis based on the converted script, and performing DAG workflow orchestration in a visual manner; encapsulating and standardizing the script components in the workflow at the function level to achieve component reuse and version management; finally, configuring the scheduling strategy and performing automated operation monitoring. The present invention enables non-technical personnel to independently complete the entire process from AI-generated scripts to production-level automated processes through automated processing of script environment dependencies and intelligent path adaptation, combined with visual orchestration and modular reuse, solving the "last mile" problem in script deployment and improving work efficiency.

Description

Technical Field

[0001] The present invention relates to technical fields such as artificial intelligence, automated script processing, visual workflow orchestration, container environment management, and script asset reuse, and in particular to a method and system for intelligent script adaptation and visual orchestration for non-technical personnel. Background Art

[0002] With the rapid development of artificial intelligence and large language model technologies, non-technical personnel can now use AI generation tools to create various scripts, theoretically significantly improving work efficiency. This technology mainly involves script automation, low-code platforms, process orchestration, and containerization technologies.

[0003] In the current technological landscape, common automation tools such as Apache Airflow provide workflow definition and scheduling capabilities, CI / CD tools like Jenkins support continuous integration and deployment of scripts, and interactive environments like Jupyter Notebook allow users to write and execute code within a web interface. These tools have achieved significant success in their respective fields.

[0004] However, when non-technical personnel generate scripts using AI tools, they often encounter environmental dependency issues. For example, the scripts require specific versions of Python libraries to run, or contain hard-coded local file paths and database connection information. Existing technologies such as virtualenv and Docker can create isolated environments, but they require manual identification and configuration of dependencies, and are unable to dynamically resolve dependency version conflicts across scripts. Furthermore, these tools typically require users to master configuration syntax such as YAML and JSON to create task dependencies, lacking intuitive visual debugging capabilities.

[0005] The main defects of the current technical solution are: First, there is a "last mile" problem between the AI-generated scripts and the actual operating environment, which makes it impossible for non-technical personnel to independently complete the entire process from script generation to deployment and operation; second, there is a lack of intelligent path identification and correction capabilities, and it cannot automatically adapt to different storage environments; third, the technical threshold of process orchestration tools is high, and it is difficult for non-technical personnel to realize the visual construction of complex workflows; fourth, the script reuse capability is limited. The scripts developed by technical personnel usually exist in a holistic form and cannot be modularized at the function level, resulting in similar needs still needing repeated development. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for intelligent script adaptation and visual arrangement for non-technical personnel, so as to solve the problem in the prior art that non-technical personnel cannot independently complete the entire process from AI-generated scripts to production-level automation processes.

[0007] To achieve the above objectives, the present invention provides a non-technical person-oriented script intelligent adaptation and visual arrangement method, comprising:

[0008] Obtain the script file uploaded by the user, analyze the dependency library of the script file, and obtain an isolated container environment;

[0009] Using the isolated container environment, statically analyzing and converting the file operation statements and database connection codes in the script file to generate standardized API calls;

[0010] Based on the standardized API calls, perform input and output feature analysis and visual orchestration to form a DAG definition of the workflow;

[0011] Perform function-level encapsulation and interface standardization on the script components in the DAG definition of the workflow to obtain a reusable module and its version management solution;

[0012] Based on the DAG definition of the workflow and combined with the version management solution of the reusable module, the workflow scheduling strategy is configured and automated operation and monitoring are performed, and the execution status and exception handling results of the workflow are output.

[0013] Preferably, the performing dependency library analysis on the script file to obtain an isolated container environment includes:

[0014] Performing abstract syntax tree (AST) analysis on the script file, extracting import statements and from-import statements, and obtaining third-party library reference information;

[0015] Build a dependency graph based on the third-party library reference information and perform version conflict detection. When a version conflict is detected, use a dependency isolation strategy to generate different isolation environment configurations;

[0016] The isolated environment configuration is used to call the container API to build the image and store it in a private warehouse to form the isolated container environment.

[0017] Preferably, the static analysis and conversion of the file operation statements and database connection codes in the script file to obtain standardized API calls includes:

[0018] Process the script file using abstract syntax tree (AST) analysis and regular expression matching to generate a file operation list and a database access list;

[0019] Perform code pattern recognition and classification based on the file operation list and the database access list to form a conversion rule mapping;

[0020] The file operation statements and database connection codes are replaced with platform standard interfaces by utilizing the conversion rule mapping to obtain the standardized API calls.

[0021] Preferably, the input and output feature analysis and visual arrangement are performed based on the standardized API call to form a DAG definition of the workflow, including:

[0022] Analyzing variable definitions and function return values ​​in the standardized API call to generate input and output feature representations of the script component;

[0023] According to the input and output feature representation, node parameters and dependencies are configured by visual drag and drop to form an orchestration result;

[0024] Perform connection validity verification and format conversion on the orchestration result, and output the DAG definition of the workflow.

[0025] Preferably, the configuration of workflow scheduling strategies and automatic operation and monitoring, and output of workflow execution status and exception handling results, include:

[0026] Based on the DAG definition of the workflow and the version management scheme of the reusable module, the workflow configuration information is obtained, and the execution time rule is converted into a timing expression to obtain a scheduling strategy;

[0027] Based on the scheduling strategy, the running status data of the workflow nodes are collected to obtain the execution progress information;

[0028] Based on the execution progress information, anomaly detection and recovery strategy execution are performed to obtain processing results and notification messages.

[0029] Preferably, the method further comprises:

[0030] Obtaining historical execution data of the workflow, extracting time dimension features and resource dimension features, and obtaining a workflow feature vector;

[0031] Based on the workflow feature vector, a density clustering algorithm is used to group the workflows to obtain workflow groups with similar execution features;

[0032] Based on the workflow group, a pseudo-random detection point sequence is generated and a state snapshot is collected to obtain an anomaly score.

[0033] Preferably, the method further comprises:

[0034] Performing a multiplication graph model conversion on the DAG definition of the workflow to obtain a node mapping relationship;

[0035] Based on the node mapping relationship, a spanning tree cover algorithm is used to find the optimal connection path and obtain real-time connection suggestions;

[0036] Based on the real-time connection suggestions, a virtual force model is used to perform layout optimization, thereby obtaining a display effect with reduced line crossing.

[0037] Preferably, the method further comprises:

[0038] Obtaining a DAG definition of the workflow, constructing a graph representation annotated with function labels and attribute tags, and obtaining a multi-level workflow model;

[0039] Based on the multi-level workflow model, pattern recognition is performed using a connected maximum common subgraph algorithm to obtain a shared pattern;

[0040] Based on the sharing mode, the importance scores are calculated and standardized, and a new reusable module is obtained.

[0041] Preferably, the script components in the DAG definition of the workflow are encapsulated at the function level and the interface is standardized to obtain a reusable module and a version management solution thereof, including:

[0042] Function splitting and interface definition are performed based on the script components to obtain standardized function modules;

[0043] Based on the standardized function module, a metadata description of function classification and version information is created to obtain a component market interface;

[0044] Based on the component market interface, version updates are tracked and compatibility impacts are evaluated to obtain upgrade recommendations.

[0045] The present invention also provides a script intelligent adaptation and visual arrangement system for non-technical personnel, comprising:

[0046] A dependency analysis module is used to obtain script files uploaded by users, perform dependency library analysis on the script files, and obtain an isolated container environment;

[0047] A path adaptation module, configured to perform static analysis and conversion on the file operation statements and database connection codes in the script file based on the isolated container environment to obtain standardized API calls;

[0048] A workflow orchestration module is used to perform input and output feature analysis and visual orchestration based on the standardized API calls to obtain a DAG definition of the workflow;

[0049] A module management module is used to perform function-level encapsulation and interface standardization on script components based on the DAG definition of the workflow to obtain reusable modules and their version management solutions;

[0050] The scheduling execution module is used to configure the workflow scheduling strategy and perform automatic operation and monitoring based on the DAG definition of the workflow and in combination with the version management solution of the reusable module, and output the execution status and exception handling results of the workflow.

[0051] The present invention has the following beneficial effects:

[0052] 1. An automated dependency management system based on AST static analysis and container technology can automatically parse script dependencies without manual intervention and build an isolated execution environment, solving the environmental configuration challenges faced by non-technical personnel.

[0053] 2. Combining the rule engine with the AI ​​model's intelligent file path recognition and conversion mechanism, it automatically replaces local file paths and database connections with platform standard APIs to ensure script compatibility in different environments.

[0054] 3. A visual DAG workflow orchestration system that enables zero-code concatenation of complex script processes through drag-and-drop, and automatically generates the configuration code required by the underlying execution engine.

[0055] 4. Function-level script modularization and standardized interface definition methods support version control and compatibility management of script components, enabling efficient reuse of technical assets.

[0056] 5. A full-link codeless solution integrating environment adaptation, path conversion, process orchestration, and automatic scheduling enables non-technical personnel to independently complete the entire process from AI-generated scripts to production-level automated processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 This is a flowchart of a method for intelligent script adaptation and visual arrangement for non-technical personnel provided by an embodiment of the present invention;

[0059] Figure 2 This is a flow chart of performing dependency library analysis on a script file provided by an embodiment of the present invention;

[0060] Figure 3 This is a flowchart of static analysis and conversion of file operation statements and database connection codes provided by an embodiment of the present invention;

[0061] Figure 4This is a workflow visualization orchestration flowchart provided by an embodiment of the present invention;

[0062] Figure 5 This is a flowchart of script component function-level encapsulation and interface standardization processing provided by an embodiment of the present invention;

[0063] Figure 6 This is a flowchart of automated scheduling and monitoring provided by an embodiment of the present invention;

[0064] Figure 7 This is a flowchart of workflow grouping and anomaly detection based on historical data provided by an embodiment of the present invention;

[0065] Figure 8 This is a workflow layout optimization flow chart based on a multiplication graph provided by an embodiment of the present invention;

[0066] Figure 9 This is a flow chart of a modular method based on common subgraph mining provided by an embodiment of the present invention;

[0067] Figure 10 This is a structural block diagram of a script intelligent adaptation and visual arrangement system for non-technical personnel provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0069] like Figure 1 As shown, the embodiment of the present invention provides a script intelligent adaptation and visual arrangement method for non-technical personnel, including the following steps:

[0070] Step S1: Obtain the script file uploaded by the user, perform dependency library analysis on the script file, and obtain an isolated container environment;

[0071] In actual application scenarios, step S1 first receives a Python script file uploaded by the user through a web interface. Users simply drag and drop the script generated by the AI ​​tool into the system's upload area. The system immediately performs file format verification and security checks to ensure that only legitimate script files are received. Once verified, the script file is stored in the distributed file system. Simultaneously, the system creates a script metadata record in the database, containing basic information such as the script ID, name, upload time, and user ID, to establish an index for subsequent dependency analysis and processing.

[0072] Next, the uploaded script is deeply analyzed using Python's AST (Abstract Syntax Tree) library. AST analysis parses Python code into a tree structure, enabling the system to accurately identify import and from...import statements within the code. For example, when analyzing statements such as "import pandas as pd" or "from numpy import array," the system identifies "pandas" and "numpy" as required third-party libraries. Furthermore, regular expressions are used to search for comments and strings within the script, looking for version constraints such as "#requires tensorflow>=2.0.0." This method comprehensively extracts all third-party library dependencies and their version requirements, generating a complete dependency list.

[0073] Based on the extracted dependency list, a dependency graph is constructed to analyze interdependencies and potential version conflicts between libraries. For example, when multiple scripts in a workflow require different versions of the same library (e.g., one script requires pandas 0.25 while another requires pandas 1.3), the system detects this conflict and automatically determines whether to create an isolated environment. In the event of a version conflict, the system employs a dependency isolation strategy, creating independent runtime environment configurations for incompatible script groups, ensuring that each script can execute correctly in the appropriate environment.

[0074] Subsequently, a Dockerfile is automatically generated based on the dependency analysis results. This file defines all the steps required to build a container image. The Dockerfile usually starts with a base Python image and then adds the necessary system dependencies and Python library installation instructions. For example, for scripts that require data processing capabilities, the system will generate installation commands for libraries such as pandas and numpy; for scripts that require machine learning capabilities, installation instructions for scikit-learn or tensorflow will be added. The system automatically executes the build process through Docker API calls to generate a container image that contains all necessary dependencies. The completed image will be pushed to a private repository, and the mapping relationship between the image ID and the script will be recorded in the system database to form a reusable container environment.

[0075] To improve efficiency, a mapping table between dependency configurations and container images is maintained. When a new script is uploaded, the system compares its dependency configuration with the existing environment. If the existing environment is found to already contain all the necessary dependency libraries, it is reused instead of rebuilding the image. This caching and reuse mechanism significantly improves the efficiency of script deployment, reduces system resource consumption, and ensures the consistency and reliability of the environment. In actual use, as the number of scripts in the system increases, the environment reuse rate will gradually increase. Most newly uploaded scripts can find a compatible existing environment, achieving near-instant deployment capabilities.

[0076] Step S2: utilizing the isolated container environment to statically analyze and convert the file operation statements and database connection codes in the script file to generate standardized API calls;

[0077] In step S2, the script file is first subjected to in-depth static analysis within the isolated container environment created in step S1. This analysis combines AST (Abstract Syntax Tree) technology with regular expression matching to comprehensively scan the script for file operations and database connection code. For file operations, the system identifies common Python file handling functions, such as open(), read(), and write(), as well as data file handling functions from the pandas library, such as pd.read_csv(), pd.read_excel(), and pd.to_csv(). The system extracts the file path strings and operation mode (read / write) from these function calls to determine the file access method and location. For example, when detecting code such as "with open('C: / Users / data / input.txt', 'r') as f:," the system identifies this as a local file read operation and records the file path "C: / Users / data / input.txt."

[0078] Similarly, the system also recognizes database connection code within scripts. Different database access libraries have different connection modes, such as SQLAlchemy's "create_engine" function, pymysql's "connect" method, and psycopg2's connection string. The system has designed recognition rules for various common database connection modes, accurately extracting key information such as the database type (such as MySQL, PostgreSQL, and SQLite), connection address, username, password, and database name.

[0079] Based on the generated list, code pattern recognition and classification will be performed. A set of conversion rule libraries for file operations and database connections is maintained, and corresponding conversion strategies are defined for different types of operations. For example, for local file paths, they are converted into platform-standard file access APIs; for database connection strings, they are converted into platform-unified data source configurations. These rules are organized into a conversion rule mapping table, and each operation mode has a corresponding conversion template. When a file reading operation is identified in the script, the appropriate conversion rule is selected based on the file type and reading method, and the hard-coded local path is replaced with the platform path.

[0080] When encountering complex situations not covered by the rule base, the built-in AI model is invoked for intelligent analysis. This AI model is specifically trained to understand the code context and intent, inferring the most appropriate alternatives. For example, when a script contains complex file path construction logic (such as path concatenation and environment variable references), the AI ​​model can analyze the entire logical chain and propose multiple possible alternatives. The system then ranks these alternatives based on confidence, selecting the best alternative for replacement or requesting user confirmation if necessary. This hybrid intelligent approach enables the system to handle a variety of complex file operations and database connection scenarios, going beyond simple pattern matching.

[0081] After applying the transformation rules, the identified file operation statements and database connection codes will be replaced with platform-standard API calls. For example,

[0082] Convert "open('C: / data / file.csv', 'r')" to "get_platform_file('user_files / file.csv', 'r')" or replace direct database connection strings with "get_database_connection('database_alias')". These standard APIs encapsulate the platform's unified access logic for files and databases, ensuring that scripts can correctly access required resources in the platform environment without relying on specific local paths or hard-coded connection information. This conversion makes scripts that originally only run in specific environments portable and can be executed stably in any container environment provided by the platform.

[0083] When performing code conversion, the mapping relationship between the original code and the modified code is saved, and all modifications are visually displayed in the user interface. Users can review each modification and choose to accept or reject specific changes. The system provides a code comparison view, marking the added, deleted, and modified parts with colors, so that users can clearly understand each change. For non-technical personnel, the system also provides simplified explanations, describing the purpose and impact of each change in plain language. The system also provides a complete modification history and supports rollback to previous versions to ensure the controllability and transparency of the code modification process. This interactive conversion process ensures the convenience of automation while retaining user control over key decisions, and is particularly suitable for non-technical personnel to handle AI-generated scripts. Through this step, the system solves the environmental dependency problem in the script, laying the foundation for subsequent stable operation on the platform.

[0084] The AI ​​model in the system is a large language model based on the Transformer architecture, which has been fine-tuned specifically for the code conversion task. The model adopts an encoder-decoder structure. The encoder consists of 12 self-attention layers, each containing 8 attention heads, and the hidden layer dimension is 768. The decoder has a similar structure, but adds a cross-attention mechanism to focus on the key parts of the input code. The model training uses a dataset consisting of 2 million pairs of code conversion samples. These samples cover a variety of conversion modes from local file paths to platform APIs, and from direct database connections to data source abstraction APIs. The training process adopts a teacher forcing strategy, uses a cross-entropy loss function with label smoothing, and the AdamW optimizer (learning rate of 3e-5 and weight decay of 0.01). Distributed training is performed on 8 GPUs, and a total of 5 epochs of training are completed.

[0085] The model was evaluated using three metrics: 87.3% conversion accuracy (exact match score), 93.5% functional isoefficiency (consistency of execution results), and an average edit distance of 4.2. In actual deployment, the model was stored in quantized int8 format, significantly reducing the model size while sacrificing less than 1% accuracy. During the code conversion process, the model receives the original code snippet and its context as input, generates multiple possible conversion results, and calculates a confidence score for each result. The system prioritizes the result with the highest confidence, but when the confidence of all results falls below a threshold (0.85), multiple candidate conversions are presented to the user for selection. This human-machine collaborative approach ensures the accuracy and controllability of code conversion.

[0086] Step S3: Based on the standardized API call, perform input and output feature analysis and visual arrangement to form a DAG definition of the workflow;

[0087] In step S3, the standardized API calls generated in step S2 are first analyzed in depth to understand the input and output characteristics of the script. This analysis process focuses on the functional essence of the script, not just its surface code. The variable definitions, function parameters, return values, and standardized API call patterns in the script are examined to infer the input parameters that the script may require and the output results that will be produced. For example, when it is detected that an unassigned variable is subsequently used at the beginning of the script, the system will identify it as a possible input parameter; when it is found that certain variables are returned or written to a file at the end of the script, they will be marked as output results. The system will also analyze the data flow, trace the entire processing path of the variable from input to output, and identify key data conversion steps and branch points.

[0088] For some complex scripts, automatic inference may not be accurate enough, so users are provided with the ability to manually define interfaces. Through an intuitive interface, users can specify which variables should serve as script input parameters and which should serve as output results. They can also add information such as parameter types, default values, and documentation. A user-friendly form-based interface guides users step by step through interface definition. Users do not need to understand complex programming concepts; they simply answer simple questions such as "What input data does this script require?" and "What results will be produced after running it?" The system automatically converts this information into a technical definition. This human-computer interaction ensures the accuracy of script interface definitions, laying the foundation for the subsequent correct connection of script components in the workflow.

[0089] Through analysis, standardized input and output feature representations are generated for each script, including information such as parameter name, data type, data format, and whether a field is required. These feature representations are stored in JSON format to facilitate internal system processing and interaction with the front-end interface. For example, the feature representation of a data preprocessing script might include a detailed description of the input parameter "data_file" (CSV file type) and the output result "processed_data" (DataFrame type). Based on these feature representations, a visual representation of the script is automatically generated, including the appearance of the graph nodes, the display of the input and output interfaces, and the interactive behavior. This ensures that the visual elements seen by the user accurately reflect the actual functionality and interface requirements of the script.

[0090] Based on these input and output feature representations, a powerful visual orchestration interface is provided. Built using modern web technologies, the interface leverages the React framework and SVG / Canvas drawing capabilities to achieve a smooth drag-and-drop workflow design experience. The interface has a clear layout. On the left is the component library panel, which displays available script components, organized by function and supporting search and filtering. In the center is the workspace, where users can drag and drop components and connect data flows. On the right is the property panel, which displays detailed information and configurable options for the selected component. The entire interface adopts an intuitive visual design, using soft color coding and clear icons, making it easy to understand and operate even for non-technical personnel.

[0091] Users can drag and drop script components from the component library to the workspace. Each component is represented by an intuitive graphical node with the component name and main function displayed on the node. The size, shape and color of the nodes are carefully designed to make it easier for users to identify different types of components. For example, data source components may use blue, conversion components use green, and output components use orange. Small icons are also displayed on the nodes to intuitively indicate the functional type of the component, such as database icons, file icons, or algorithm icons. When the user selects a node, the system displays detailed information and configurable parameters for the component. The user can set these parameters through the form interface without writing any code. The system provides intelligent parameter setting assistance, such as data type checking, value range validation, and automatic completion to help users avoid common errors.

[0092] The system's visual interface supports specifying data flow through connecting lines. Users can create a data transfer relationship between components by simply dragging the output port of one component to the input port of another. During the connection process, the system will provide visual feedback, such as highlighting compatible ports and previewing the connection path. When creating a connection, the compatibility of data types is automatically checked to ensure that only ports of matching types can be connected. For example, a component that outputs a CSV file can be connected to a component that accepts CSV input, but cannot be directly connected to a component that requires JSON input. When a user tries to connect an incompatible port, the system will display a friendly error prompt and suggest possible solutions, such as adding a format conversion component. This real-time verification mechanism greatly reduces errors in workflow design and improves the work efficiency of non-technical personnel.

[0093] In addition to basic nodes and connections, advanced workflow control structures such as conditional branching (if-else logic), looping (foreach loops), and parallel tasks are also supported. These control structures are also represented through intuitive graphical elements, allowing users to implement complex business logic through simple drag-and-drop configuration. For example, users can add a conditional branch node, set the conditional expression "data.rows > 1000" through a drop-down menu or a simplified expression editor, and then connect different processing paths for the true and false branches. The system provides a conditional preview function, allowing users to enter sample data and view the evaluation results of the condition to ensure that the logic meets expectations. These advanced control structures enable even non-technical personnel to create complex workflows, achieving true business automation.

[0094] Once the user completes the workflow design, the entire workflow will be verified for validity, checking for issues such as unconnected required inputs and circular dependencies. Verification results are displayed with clear messages, indicating the specific problem location and suggested fixes. For complex workflows, a "Workflow Checker" function is provided to automatically analyze the structure and data flow of the entire workflow, providing a comprehensive quality report, including potential performance bottlenecks, error risk points, and optimization suggestions. After verification, the system converts the visual workflow created by the user on the interface into a standardized DAG (directed acyclic graph) definition format, typically in JSON or YAML format. This DAG definition details each node (script component) in the workflow, the dependencies between nodes, execution conditions, parameter configuration, and other information, and is a complete technical representation of the workflow. To ensure portability and compatibility, the system uses an open standard format to facilitate integration with other tools and platforms.

[0095] Finally, based on the DAG definition, configuration code suitable for the underlying execution engine (such as Airflow, Prefect, or a custom executor) is automatically generated. This code includes task definitions, dependency settings, and parameter passing logic, ensuring that the workflow can be correctly parsed and run by the execution engine. The generation process takes into account the differences in the characteristics of different execution engines, ensuring that the generated code utilizes the optimal performance of each engine. The entire process is completely transparent to the user, allowing non-technical personnel to create professional-level workflow automation solutions without having to understand the details of the underlying execution engine. Through this visual orchestration approach, the system successfully transforms complex technical tasks into intuitive graphical operations, enabling non-technical personnel to independently create and manage complex data processing and automation workflows.

[0096] Step S4: performing function-level encapsulation and interface standardization on the script components in the DAG definition of the workflow to obtain a reusable module and its version management solution;

[0097] In step S4, the focus is on converting script components in the workflow into reusable standardized modules, enabling the accumulation and sharing of technical assets within the organization. This step begins with an in-depth analysis of the script components in the workflow DAG definition, identifying relatively independent code blocks with reuse potential. The components' functional boundaries, input and output interfaces, and internal logic are examined to assess their independence and versatility. Components that implement a clear, single function and have broad application scenarios are marked as potentially reusable modules. This analysis combines static code features with usage scenario statistics to intelligently identify the most valuable reuse opportunities.

[0098] The system will guide technical personnel (or with the assistance of AI) to refactor these code blocks into independent functional modules, following good software engineering practices such as the single responsibility principle and clear input and output interfaces. The refactoring process focuses on improving the readability, maintainability, and reusability of the code while maintaining the integrity of the original functionality. The system provides code refactoring suggestions and points out potential optimization points, such as parameter merging, logical separation, or naming improvements. For scripts created by non-technical personnel, the system can automatically perform basic refactoring with the assistance of AI, improving code quality and making it more suitable for sharing and reuse in an enterprise environment.

[0099] During the function-level encapsulation process, developers are helped to add standardized function signatures and detailed documentation strings for each function. The function signature clearly defines the parameter list, type annotations, and return value type, making the use of the function clear at a glance. The documentation string contains the function description, parameter description, return value explanation, and usage examples, following the industry-standard documentation format. The system provides a document generation tool that can extract information from code analysis and usage scenarios to assist in creating comprehensive function documentation. This detailed documentation allows non-technical personnel to understand the purpose and usage of the function, greatly reducing the technical threshold. The encapsulated function module retains the original functionality, but has better structure and readability, making it easier to reuse in different workflows.

[0100] After function-level encapsulation, a structured metadata description is created for each script component. This metadata, stored in JSON format, contains rich attribute information, such as functional classification (e.g., data processing, machine learning, report generation), description of applicable scenarios, detailed description of input parameters (including type, format requirements, value range, default value, etc.), output format and structure, performance characteristics (typical execution time, resource consumption, etc.), author and maintainer information, version history and change log, other dependent components or libraries, usage examples, and best practices. This metadata is not only used for component retrieval and display, but also supports the system's intelligent recommendation and compatibility checking.

[0101] The metadata creation process is semi-automated, extracting initial information from code analysis, usage history, and existing documentation, and then guiding users to supplement and improve it. The user interface is intuitive and user-friendly, using a wizard to guide users through the process of filling in key information and providing intelligent suggestions and examples to reduce the burden of document writing. Collaborative editing is also supported, allowing multiple team members to work together to improve component documentation. These metadata support multi-dimensional search and filtering, allowing users to quickly find the components they need. For example, users can locate components that meet specific needs by searching by keyword, browsing by category, or filtering based on input and output types.

[0102] An intuitive component market interface is built based on these standardized function modules and metadata. This interface displays available script components in a manner similar to an app store, with each component having a thumbnail, a brief description, and rating information. The homepage of the component market displays popular components and the latest added components, and recommends related components based on the user's historical usage preferences. Users can browse the component library through category navigation or search functions and view the detailed description page of the component, which displays the component's complete documentation, usage examples, performance indicators, and user reviews. The component market also provides a component comparison function, allowing users to compare the functions and features of multiple similar components side by side and make the best choice.

[0103] Users can directly add selected components from the Component Marketplace to their workflows, automatically handling component import and initial configuration. The Component Marketplace also supports user reviews and comments, allowing users to rate components and share their experiences, provide suggestions for improvements, or report issues. This feedback forms a community knowledge base, helping other users select appropriate components and providing direction for component authors to improve. The system calculates component popularity based on ratings and usage frequency, increasing the visibility of high-quality components and creating a virtuous cycle.

[0104] In terms of version management, a comprehensive component version control mechanism has been implemented. Each component's version change is carefully documented, including modifications, improvements, known issues, and compatibility notes. The system uses semantic versioning to clearly indicate the compatibility relationships between versions. Major version number changes indicate incompatible API modifications, minor version number changes indicate backward-compatible feature additions, and revision number changes indicate backward-compatible bug fixes. This version management convention enables users to quickly understand the scope of a version upgrade.

[0105] Component developers are provided with release tools to guide them through standard version management processes. Developers are required to provide detailed change logs, clearly identifying improvements and potential compatibility issues in the new version. When a new version is released, a series of validation tests are automatically performed to ensure component quality, and a difference report is generated compared to the previous version. When component interfaces change, the compatibility impact of these changes on existing workflows is automatically assessed, and a detailed impact analysis report is generated, identifying potentially affected workflows and any necessary adjustments.

[0106] For incompatible version upgrades, migration suggestions are provided, guiding users on how to adjust their workflows to accommodate the new component version. For example, if a new component version changes a parameter name or adds a mandatory parameter, the system will specifically identify the required modifications and even provide an automatic migration option. Where possible, the system generates adaptation code to automatically handle parameter format conversion or default value filling, minimizing manual adjustments by the user. This intelligent version management ensures that workflows that rely on the component will not be accidentally interrupted by upgrades, significantly reducing the burden of version maintenance.

[0107] Intuitive parameter configuration forms are also generated for each component, based on the parameter definitions in the component metadata. These forms include parameter descriptions, default values, value ranges, and format validation, guiding users in setting parameters correctly and providing real-time previews of parameter effects to reduce the likelihood of configuration errors. For complex parameters, the system provides visual configuration aids such as date selectors, color selectors, and data structure visual editors to further simplify operations for non-technical personnel. The system also supports parameter templates, allowing users to save commonly used parameter combinations as templates for rapid application in different workflows, improving work efficiency.

[0108] Through this function-level encapsulation and interface standardization, the system effectively transforms dispersed script code into a structured, manageable, and reusable technical asset library, significantly improving knowledge accumulation and technology reuse efficiency within the organization. As the asset library continues to grow, users can quickly build complex workflows by combining existing components, significantly reducing duplication of development and improving business responsiveness. This modular approach also promotes the dissemination and standardization of best practices, enhancing technical quality and consistency across the organization.

[0109] Step S5: Based on the DAG definition of the workflow and in combination with the version management solution of the reusable module, the workflow scheduling strategy is configured and automated operation and monitoring are performed, and the execution status and exception handling results of the workflow are output.

[0110] In step S5, the workflow defined in the previous steps is converted into an actual automated process, and comprehensive monitoring and management capabilities are provided. First, the system builds a complete workflow configuration based on the workflow's DAG definition and the version management information of reusable modules. This configuration contains detailed execution parameters, resource requirements, dependencies, and error handling strategies for each node to ensure that the workflow can correctly obtain all required resources and component versions at runtime. The system checks each component version used in the workflow to ensure that they are compatible with each other and resolves possible version conflicts. For some key components, the system will also automatically add version lock settings to prevent unexpected version changes from affecting workflow stability.

[0111] It provides an intuitive timing policy setting interface to support various scheduling needs. The interface design is simple and clear, mainly divided into basic scheduling and advanced scheduling. In the basic scheduling section, users can use a graphical calendar view to select the execution time, such as "9:00 AM every Monday, Wednesday, and Friday" or "5:00 PM on the last weekday of each month." The interface provides quick options for common scheduling modes, such as "Every day," "Every weekday," or "The beginning of the month," which users can apply with a single click. For more complex needs, users can switch to advanced mode and use a more flexible time selector to specify complex recurrence patterns. In either case, the system automatically converts these natural language time rules into standard Cron expressions, such as "09 * * 1,3,5" (9:00 AM every Monday, Wednesday, and Friday).

[0112] To ensure users understand the effects of their settings, the specific times of several expected executions are displayed in a calendar view and time list format. This allows users to visually verify that their schedules meet their expectations, especially for complex scheduling rules such as "the second Wednesday of every month" or "the last working day of every quarter." The system also provides a time zone selection feature to ensure that scheduled times remain consistent across users in different regions, avoiding scheduling errors caused by time zone differences.

[0113] In addition to basic scheduled execution, workflows can also be initiated based on events such as completed file uploads, database updates, or API callbacks. Users can select trigger conditions, specify event sources, and specific event types through simple drop-down menus and tabs. For example, a user can set a workflow to trigger "when the order table in the sales database is updated" or "when a new file is received in a specified FTP directory." A rich set of predefined event types covers common business scenarios, while custom events are also supported to meet special needs. For more technical event configurations, the system provides visual auxiliary tools, such as a database table selector or file path browser, to simplify the configuration process.

[0114] It also allows the setting of conditional execution rules to increase the intelligence of workflow initiation. Users can define preconditions, such as "this workflow will only be executed if the current workflow is successfully completed and the number of data rows generated is greater than 100." The condition setting interface uses an intuitive form. Users can select the condition type (such as workflow status, data characteristics, time window, etc.) and specify the specific condition value through drop-down menus and input boxes. The system supports the combination of multiple conditions, and users can create complex conditional expressions through "and" and "or" logical connectors. For advanced users, the system also provides a conditional expression editor that supports more complex logical definitions. These advanced scheduling features enable automated processes to respond to business needs more intelligently, reduce unnecessary execution, and improve system resource utilization efficiency.

[0115] Once configured, a complete scheduling strategy is generated and submitted to the underlying scheduling engine. The scheduling engine is responsible for launching workflows at specified times or when events occur, and allocating appropriate computing resources. To optimize resource utilization, the system analyzes the workflow's historical operation data, predicting the execution time and resource requirements of each node. Based on these analysis results and the current system load, computing resources are intelligently allocated to avoid resource competition and improve overall execution efficiency. The concept of resource pooling is implemented, allowing different resource quotas to be allocated to different workflows, ensuring that critical business processes receive sufficient resources while preventing a single workflow from occupying too many system resources.

[0116] After the workflow starts executing, comprehensive monitoring begins. The monitoring system utilizes a multi-tiered architecture, encompassing infrastructure monitoring, container monitoring, process monitoring, and application-layer monitoring, providing comprehensive visibility into the workflow's operational status. The system tracks the execution status, runtime, resource usage, and log output of each node in the workflow in real time. Each node's operational metrics are stored in a time-series database, supporting historical trend analysis and anomaly detection. This monitoring information is presented to users through an intuitive visual interface, responsively designed to adapt to display devices of varying sizes, allowing users to easily monitor workflow status anytime, anywhere.

[0117] The nodes on the workflow diagram will display their current status in different colors: nodes waiting to be executed are gray, nodes being executed are blue, nodes that have been successfully completed are green, nodes that have failed are red, and nodes that are paused or skipped are specially marked. The node icon will also display a progress indicator to intuitively reflect the completion percentage of long-running tasks. Users can understand the progress of the entire workflow at a glance and quickly identify problem nodes. The system supports zooming in, out, and panning operations on workflow diagrams, making it easy to view large and complex workflows. For workflows with a large number of nodes, the system also provides node grouping and folding functions, allowing users to focus on specific parts and reduce visual complexity.

[0118] For executing nodes, real-time performance indicator monitoring is provided, such as CPU usage, memory usage, data processing speed, etc., as well as detailed log viewing functions. Performance indicators are displayed in the form of charts, clearly showing resource usage trends and fluctuations. Users can expand nodes to view more details, or click the log button to view the complete execution log. The log viewer supports real-time scrolling, keyword search, and log level filtering to help users quickly locate key information. For nodes that process large amounts of data, the system also provides data sampling previews, allowing users to view the data samples currently being processed and understand the data quality and processing effects. These detailed information helps users understand the execution of the workflow, especially to quickly locate the cause when problems arise.

[0119] It also implements intelligent anomaly detection and automatic recovery mechanisms. During the monitoring process, the system continuously analyzes execution data, looking for possible anomalies, such as abnormally prolonged node execution time, sudden increases in resource usage, and significant deviations in output data volume from historical averages. Anomaly detection is based on a variety of techniques, including statistical analysis, time series forecasting, and machine learning models. A normal behavior baseline is established for each workflow, and then abnormal patterns that deviate from this baseline are identified. For example, it can be detected that a data processing node that usually takes 5 minutes to complete has been running for 15 minutes, but the progress is only 30%, which may indicate that the data processing has encountered a problem.

[0120] When these anomalies are detected, corresponding measures are automatically taken according to the preset policies. Response measures are divided into multiple levels, from light to heavy, including: logging warnings but continuing execution, attempting light intervention (such as reallocating resources), performing recovery operations (such as restarting specific nodes), rolling back to a safe point, or completely terminating the workflow. The specific measures taken depend on the nature and severity of the anomaly and the importance of the workflow. For nodes that fail to execute, the system can automatically retry, and users can configure the number of retries, interval time, and maximum waiting time. When performance problems caused by insufficient resources are detected, the system can dynamically allocate more resources, such as increasing container memory limits or increasing CPU priority. In the event of data anomalies, you can automatically switch to predefined alternative processing paths, such as using cached data or simplifying processing logic.

[0121] For unrecoverable errors, the workflow is promptly terminated and relevant personnel are notified to prevent the error from spreading. The termination process is controlled, attempting to complete key milestones that have already begun to ensure data consistency while canceling unstarted tasks to free up resources. These automatic recovery mechanisms significantly improve workflow reliability and reduce the need for manual intervention, making them particularly suitable for non-technical personnel managing complex automated processes. Even when problems arise, the system autonomously attempts to repair them, requesting manual intervention only when the problem exceeds the capabilities of automated processing.

[0122] In terms of notification management, multi-level notification policy configuration is provided. Users can define different notification rules at the workflow level and the node level, precisely controlling which events require notification and how notifications are sent. Notification levels are divided into multiple levels, such as information, warning, error, and critical error, and users can set different notification methods for each level. Notification channels include a variety of options, such as in-app messages, email, SMS, mobile app push, and integration with enterprise instant messaging tools (such as Slack or WeChat for Work). Users can flexibly combine these channels. For example, normal warnings will only send in-app notifications, while critical errors will be sent via email, SMS, and instant messaging simultaneously.

[0123] It also supports different notification strategies for working hours and non-working hours to avoid disturbing users' rest time in non-emergency situations. Users can define their own working hours and time zones, and the system will adjust notification behavior based on these settings. For example, during working hours, general warnings may be sent via in-app messages, while during non-working hours, only serious errors will trigger notifications, and high-priority channels such as SMS will be used first. The system also implements a notification escalation mechanism. If no one responds to an important alert within a specified time, the system will automatically notify superiors or backup personnel to ensure that critical issues are handled in a timely manner. The content of the notification is also intelligently generated, including a description of the problem, the scope of impact, possible causes, and recommended actions, so that the recipient can quickly understand the situation and take action.

[0124] Complete execution results, performance data, and exception information will be recorded by the system in the database to form the execution history of the workflow. Historical data storage adopts a layered design, with hot data retained in high-performance storage for immediate query, while historical data is gradually moved to archive storage to ensure the cost-effectiveness of long-term data. Users can view execution records for any period in the past through the historical query interface. The interface provides a variety of filtering conditions, such as time range, execution status, keywords, and resource usage. Users can compare execution performance at different times, analyze long-term trends, and identify performance degradation or improvement. A variety of visual charts are provided, such as execution time trend charts, resource usage heat maps, and status distribution pie charts, to help users understand execution patterns and identify anomalies. These historical data can also be exported to CSV or Excel format for external reporting or in-depth analysis.

[0125] Historical data is not only useful for user queries but also provides valuable resources for system optimization. The system continuously learns from this data to improve resource allocation strategies, anomaly detection models, and prediction accuracy. For example, by analyzing historical execution data, the system can more accurately predict workflow runtimes and resource requirements, optimizing scheduling decisions. The system also identifies performance bottlenecks and common failure points in workflows, generates optimization recommendations, and helps users improve workflow design. Through this continuous learning and optimization, the system becomes increasingly intelligent over time, providing more precise monitoring and more efficient resource management.

[0126] Through this comprehensive scheduling, execution, and monitoring system, non-technical personnel can manage complex automated workflows, ensure processes run reliably as planned, promptly identify and resolve issues, and ultimately achieve stable automation of business processes. The system's design philosophy is to hide complex technical details behind an intuitive user interface, allowing users to focus on business objectives rather than technical implementation. This approach significantly lowers the technical threshold for automation, enabling more business personnel to participate in process automation and accelerating an organization's digital transformation.

[0127] In a preferred embodiment of the present invention, Figure 2 As shown, in step S1, the script file is analyzed for dependency libraries to obtain an isolated container environment, including:

[0128] Step S11: performing abstract syntax tree (AST) analysis on the script file, extracting import statements and from-import statements, and obtaining third-party library reference information;

[0129] Specifically, deep AST analysis, based on Python's ast module, is implemented, capable of traversing the entire Python syntax tree. The core implementation logic of AST analysis is as follows: The Python script is first parsed into an AST tree. A custom NodeVisitor class is then implemented to traverse the tree, specifically identifying Import and ImportFrom nodes. For each Import node, the module name is extracted; for ImportFrom nodes, both the module name and the specific imported function or class are recorded. Alias ​​imports (such as "import numpy as np") are also identified and correctly associated with the original library name. In addition to directly parsing import statements, the system also uses regular expressions to identify version requirement information in comments, such as "# requires pandas>=1.0.0" or "# dependency: tensorflow==2.4.0." For complex conditional imports (such as imports within try-except blocks or dynamic imports at runtime), a combination of static analysis and heuristic rules is used to achieve over 95% dependency identification accuracy.

[0130] Step S12: constructing a dependency graph based on the third-party library reference information and performing version conflict detection. When a version conflict is detected, a dependency isolation strategy is used to generate a different isolation environment configuration.

[0131] Specifically, the system implements a dependency model based on a directed graph, using an adjacency list to represent dependencies between libraries. During graph construction, nodes represent libraries, edges represent dependencies, and each edge also contains version constraint information. The system uses a semantic version parser to process version strings and supports complex version constraint expressions, such as ">=1.2.0,<2.0.0". The version conflict detection algorithm is based on a depth-first search, traversing the dependency graph starting from the entry script and maintaining a global version constraint set. When an incompatible version constraint is detected in a library, the source of the conflict and the constraint conditions are recorded to provide a decision basis for subsequent dependency isolation.

[0132] An optimized "connected component analysis" algorithm is used to partition incompatible library groups. This algorithm first represents all mutually incompatible version constraint relationships as an undirected graph, where nodes represent "library + version constraint" and edges represent conflicting relationships. A variant of the Tarjan algorithm is then applied to identify all strongly connected components, each representing a set of libraries that require isolation. This approach isolates the minimum number of library groups in different environments while maximizing library sharing and optimizing image size and build time.

[0133] Step S13: Utilize the isolated environment configuration to call the container API to build the image and store it in a private warehouse to form the isolated container environment.

[0134] Specifically, in step S11, Python's AST library is used to perform abstract syntax tree analysis on the user-uploaded script, identifying import statements and from...import statements and extracting all third-party library references. At the same time, regular expressions are used to analyze comments and strings in the script, looking for possible version requirements (e.g., #requires numpy>=1.20), and a complete list of dependencies and version constraints is output.

[0135] In step S12, a dependency graph is constructed based on the third-party library reference information to detect potential version conflicts. When a conflict is detected (e.g., one script requires pandas < 1.0 while another requires pandas >= 2.0), the system employs a dependency isolation strategy, creating independent runtime environment configurations for each set of incompatible scripts to ensure that all scripts can run in the appropriate environment.

[0136] In step S13, based on the dependency analysis results, a Dockerfile is dynamically generated, starting from the basic Python image, and adding the necessary system dependencies and Python library installation instructions. The image is built through the Docker API call. After the build is completed, the image is pushed to a private warehouse and the mapping relationship between the image ID and the script is recorded to form a reusable container environment. In addition, the system also maintains a mapping table between dependency configuration and container image. When a new script is uploaded, its dependency configuration is compared with the existing environment. If a compatible existing environment is found, it is directly reused to avoid repeated construction, significantly improving script deployment efficiency and saving system resources.

[0137] In a preferred embodiment of the present invention, Figure 3 As shown, in step S2, the file operation statements and database connection codes in the script file are statically analyzed and converted to obtain standardized API calls, including:

[0138] Step S21: Processing the script file using abstract syntax tree (AST) analysis and regular expression matching to generate a file operation list and a database access list;

[0139] Step S22: performing code pattern recognition and classification based on the file operation list and the database access list to form a conversion rule mapping;

[0140] Step S23: using the conversion rule mapping to replace the file operation statements and database connection codes with the platform standard interface to obtain the standardized API call.

[0141] Specifically, in step S21, a combination of AST analysis and regular expression matching is used to scan the script for file operation statements, such as open(), pd.read_csv(), and pd.read_excel(). The file path string and operation mode (read / write) are extracted, and a script file operation list is created. Simultaneously, common database connection patterns in the script are identified, including connection strings for libraries such as SQLAlchemy, pymysql, and psycopg2. Information such as the database type, address, and username is extracted, and a database access list is generated.

[0142] The system's static analyzer combines AST analysis and regular expression techniques. For AST analysis, the system specifically identifies function calls related to file operations, including the built-in open function, os.path module functions, pathlib.Path operations, and file reading and writing functions from common data processing libraries (such as pandas.read_csv and numpy.load). For each identified function call, the system extracts relevant parameter information, specifically the file path string and operation mode. The system can handle complex parameter passing patterns, including keyword arguments, positional arguments, and default arguments.

[0143] For database connection identification, we maintain a rule library covering common database connection modes, including: SQLAlchemy's create_engine function, pymysql / mysqlclient's connect method, psycopg2's connect function, sqlite3's connect function, and MongoDB's MongoClient constructor. Each connection mode has a corresponding parameter extraction rule, which can parse key information such as the server address, port, username, password, and database name from the connection string or parameters.

[0144] In step S22, a conversion rule library for file operations and database connections is maintained. For the operations identified in step S21, the corresponding rules are applied to perform code conversion. For complex cases that cannot be covered by the rules, the system calls a built-in AI model to analyze the context, infer the code intent, generate platform-adapted alternative code, and sort it by confidence, selecting the optimal solution for replacement.

[0145] In step S23, local file paths, such as "C: / data.csv," are converted into platform-unified storage API calls, such as "get_platform_file('user_files / data.csv')," ensuring that the script correctly accesses the file within the platform environment. Furthermore, after applying the path conversion, the system saves the mapping between the original and modified code and visually displays the modified content in the user interface, allowing users to confirm or reject specific modifications. A modification history is provided, supporting rollback to previous versions, ensuring controllability and transparency of code modifications.

[0146] In a preferred embodiment of the present invention, Figure 4 As shown, in step S3, input and output feature analysis and visual arrangement are performed based on the standardized API call to form a DAG definition of the workflow, including:

[0147] Step S31: Analyze the variable definitions and function return values ​​in the standardized API call to generate input and output feature representations of the script component;

[0148] Step S32: According to the input and output feature representation, node parameters and dependencies are configured by visual dragging to form an orchestration result;

[0149] Step S33: performing connection validity verification and format conversion on the orchestration result, and outputting the DAG definition of the workflow.

[0150] Specifically, in step S31, the script's variable definitions, function return values, and file operations are analyzed to infer the script's possible input parameters and output results, generating a script I / O feature description. For cases where automatic inference is not possible, a user-defined interface is provided to ensure that the script can be correctly connected in the workflow.

[0151] In step S32, the front-end implements a drag-and-drop workflow design interface based on React and SVG / Canvas technology, providing a visual representation of nodes (representing scripts) and connections (representing data flows). Users can intuitively drag and drop script components, set execution conditions and parameters, and specify data flow directions using connection lines. The system automatically checks the connection validity.

[0152] In step S33, the visual workflow created by the user on the interface is converted into a standardized DAG definition format (JSON format), which includes node information, dependencies, execution conditions, and parameter configuration. The system verifies the legitimacy of the DAG, detects errors such as circular dependencies, and ensures that the workflow structure is correct. At the same time, based on the DAG definition, the system automatically generates configuration code suitable for the underlying execution engine (such as Airflow), including task definitions, dependency settings, and parameter passing logic. In addition, the system also provides a workflow simulation execution function, which allows users to verify the correctness of data flow and parameter passing without actually running the complete workflow, helping users discover and fix potential problems.

[0153] In a preferred embodiment of the present invention, Figure 5 As shown, in step S4, the script components in the DAG definition of the workflow are encapsulated at the function level and the interface is standardized to obtain a reusable module and its version management solution, including:

[0154] Step S41: performing function splitting and interface definition based on the script component to obtain a standardized function module;

[0155] Step S42: creating metadata descriptions of function classification and version information based on the standardized function module to obtain a component market interface;

[0156] Step S43: Based on the component market interface, track version updates and evaluate compatibility impacts to obtain upgrade suggestions.

[0157] Specifically, in step S41, the technical staff is instructed to split the script into independent functions according to their functions and add standardized function signatures and documentation strings. The processed function modules contain clear parameter definitions, return value descriptions, and usage examples, so that non-technical staff can understand the module functions and usage methods.

[0158] In step S42, a structured metadata description is created for each script component, including functional classification, applicable scenarios, input parameter descriptions, output format, version information, and more. This metadata is stored in JSON format and supports search and filtering, making it easy for users to quickly find the required components. Simultaneously, the system builds a component marketplace interface that visually displays available script components, including functional descriptions, usage frequency, and ratings. Users can browse, search, and compare components, selecting the appropriate one to add to their workflows.

[0159] In step S43, a component version management mechanism is implemented to automatically track component update history and record each change. When a component interface changes, the system automatically assesses compatibility impacts and provides migration suggestions if incompatible, ensuring that workflows that rely on that component are not interrupted by the upgrade. When a new version of a component used in a workflow is detected, the system automatically assesses upgrade risks and provides a visual impact analysis report to help users make upgrade decisions.

[0160] In a preferred embodiment of the present invention, Figure 6 As shown, in step S5, the workflow scheduling strategy is configured and automatically run and monitored, and the execution status and exception handling results of the workflow are output, including:

[0161] Step S51: Based on the DAG definition of the workflow and the version management scheme of the reusable module, the workflow configuration information is obtained, and the execution time rule is converted into a timing expression to obtain a scheduling strategy;

[0162] Step S52: Based on the scheduling strategy, collect the running status data of the workflow nodes to obtain execution progress information;

[0163] Step S53: Based on the execution progress information, perform anomaly detection and recovery strategy execution to obtain processing results and notification messages.

[0164] Specifically, in step S51, the system provides a visual interface for setting up a timed policy, supporting both precise scheduling based on Cron expressions and simplified configuration using natural language (e.g., "every Monday at 9:00 AM"). The system automatically converts the user's configuration into a standard Cron expression and displays the specific execution times for the next few times, ensuring compliance with user expectations.

[0165] In step S52, the system tracks the workflow execution status in real time, collecting the runtime, resource usage, and output logs for each node. The front-end displays the execution progress and status visually, including completed nodes, currently executing nodes, and waiting nodes, allowing users to intuitively understand the workflow execution status.

[0166] In step S53, the system monitors workflow execution for anomalies, such as node failures, timeouts, or insufficient resources. Based on pre-set policies, it automatically attempts recovery, including task retry, resource reallocation, or alternative execution paths, to maximize workflow completion and minimize the need for manual intervention. Furthermore, the system employs a multi-level notification strategy, selecting different notification methods based on event importance to ensure that important information is delivered promptly without causing undue disruption.

[0167] In a preferred embodiment of the present invention, Figure 7 As shown, the method further includes:

[0168] Step S6: Obtain historical execution data of the workflow, extract time dimension features and resource dimension features, and obtain a workflow feature vector;

[0169] Step S7: Based on the workflow feature vector, a density clustering algorithm is used to group the workflows to obtain workflow groups with similar execution features;

[0170] Step S8: Based on the workflow group, a pseudo-random detection point sequence is generated and a state snapshot is collected to obtain an anomaly score.

[0171] Specifically, in step S6, the system extracts multi-dimensional features from the historical execution data of all workflows, including time-dimensional features (execution duration, start time, end time, and timing frequency) and resource-dimensional features (CPU usage, memory usage, I / O operation frequency, and network traffic). The system preprocesses this raw data using sliding window technology to eliminate noise and standardize feature values. It also calculates the time series autocorrelation and cross-correlation of each feature to form a "behavioral fingerprint" for the workflow.

[0172] A multi-level data preprocessing strategy was adopted during the processing of historical execution data. The original execution logs were first cleaned to identify and correct outliers, missing values, and duplicate records to ensure data quality. For time series data, a 24-hour overlapping sliding window with a window step of 4 hours was applied. This configuration effectively captures intraday fluctuation patterns. For each sliding window, the system calculates an exponentially weighted moving average with a decay factor set to 0.85, which smooths short-term noise while retaining trend changes. For resource data such as CPU and memory, multi-dimensional features such as mean, peak value, fluctuation amplitude, and surge frequency are captured; for I / O operations, professional indicators such as read-write ratio, average block size, and queue depth are additionally recorded. The system pays special attention to the burst pattern of resource usage. By analyzing the first-order and second-order derivatives of resource indicators, it identifies phases of rapid growth or decline. These features are crucial for predicting resource bottlenecks. For temporal pattern analysis, the system uses fast Fourier transforms (FFTs) to detect periodicity and autocorrelation functions (ACFs) to analyze temporal dependencies, enabling the identification of multiple periodicities at hourly, daily, weekly, and monthly levels. For workflows with high sporadic nature, the extreme value theory model is also applied to establish the distribution characteristics of extreme values ​​in resource usage. All extracted features are normalized (mean subtracted and divided by standard deviation) and then dimensionality reduced using principal component analysis (PCA), retaining the top N principal components that explain 95% of the total variance. This complex feature engineering process ensures that the workflow feature vector comprehensively and accurately characterizes its operational behavior.

[0173] In step S7, the system uses the DBSCAN (density-based spatial clustering) algorithm to cluster workflow features and identify groups of workflows with similar execution characteristics. The system dynamically adjusts the neighborhood parameter ε to find the optimal clustering effect and evaluates clustering quality using the silhouette coefficient. For each cluster, the system calculates the characteristic center points of the core samples and extracts common features of the cluster, such as typical execution time ranges, resource usage patterns, and failure rates. These clustering results are used for resource planning and anomaly detection and are automatically updated every 24 hours to ensure that the model adapts to dynamic changes in the system.

[0174] The system's density clustering algorithm implements several innovative optimizations. First, in terms of parameter adaptation, the optimal neighborhood radius ε is automatically determined through k-distance graph analysis. Specifically, the system calculates the distance from each sample to its k-th nearest neighbor, sorts these distances, and plots the k-distance graph. The value of ε is then determined by detecting inflection points (using discrete curvature calculation). The minimum number of samples, MinPts, is set to the number of dimensions + 1 (the number of dimensions is the number of principal components after PCA) and is adjusted based on the total number of workflows to ensure that clusters do not become overly subdivided as the number of workflows increases. Regarding distance metric selection, the system implements the Mahalanobis distance, which takes into account inter-feature correlation and provides better discriminative power for high-dimensional data. To process large datasets, a grid-accelerated DBSCAN variant is implemented. This partitions the feature space into grid cells and only computes distances to neighbors within non-empty grid cells, significantly reducing computational effort. The system also implements a distributed version of DBSCAN, which utilizes a graph-based parallel processing framework to partition data across multiple compute nodes, improving processing speed while ensuring accurate results. Cluster quality assessment utilizes a comprehensive approach using multiple metrics: the Silhouette Coefficient assesses cluster closeness and separation, and the Davies-Bouldin Index evaluates inter-cluster similarity. This combination of metrics provides a more comprehensive quality assessment than using any single metric alone. The system also implements an incremental cluster update mechanism. Cluster recalculation is triggered when new workflow data accumulates to a certain threshold or when a significant decrease in cluster quality (a change of more than 10% in any evaluation metric) is detected. This recalculation utilizes an incremental update strategy, reassessing cluster affiliation only for new data and samples in boundary areas, significantly improving update efficiency. For each formed cluster, the system conducts an in-depth analysis of its feature distribution, calculating the core interval (5th to 95th percentile) of each dimension, a feature correlation matrix, and typical time series patterns. These cluster features are stored in a distributed cache system, supporting real-time queries in milliseconds and providing a benchmark for anomaly detection.

[0175] In step S8, the system generates a pseudo-random sequence of checkpoints for each workflow based on a hash function and Poisson distribution, avoiding system load spikes and potentially circumventable vulnerabilities caused by traditional fixed-interval checkpoints. The density of checkpoints is dynamically adjusted based on the workflow's importance, historical stability, and current system load. When each checkpoint is triggered, the system collects a snapshot of the current execution state and compares it with the clustering model's prediction to calculate a deviation score. The system also builds a knowledge base containing various anomaly patterns. Through real-time matching and analysis, it identifies potential anomalies and implements intelligent early warning grading and self-healing based on the scope of impact.

[0176] The system's pseudo-random checkpoint generation mechanism uses a cryptographically secure random number generator, combined with the SHA-256 hash of the workflow ID as a seed, to ensure that the checkpoint sequence is both unpredictable and deterministic (the same workflow will generate a consistent checkpoint sequence under the same conditions). Based on a non-uniform Poisson process model, the system dynamically adjusts the detection intensity λ(t) based on the characteristics of the workflow. Specifically, the calculation of λ(t) takes into account multiple factors: the workflow's importance weight (based on a business impact rating, ranging from 1 to 10), the historical anomaly rate (the frequency of anomalies over the past 30 days), the current system load level, and the time of day. The system sets upper and lower limits for the detection interval to ensure minimum monitoring even for the lowest-priority workflows while avoiding system load caused by excessive monitoring of high-priority workflows. The system also implements an adaptive adjustment mechanism. When the anomaly rate of a particular workflow is detected to increase, the monitoring intensity of that workflow is automatically increased. When the overall system load approaches the warning threshold, the monitoring frequency of workflows with high stability is prioritized and reduced. For state collection, the system implements two modes: lightweight and deep. The lightweight mode collects only basic metrics (CPU, memory, disk I / O, and network traffic), offering low overhead and high frequency. The deep mode additionally collects thread state, stack traces, and detailed log patterns, providing more comprehensive analytical data, but at a higher cost and therefore less frequent. The system also implements a state inference algorithm between detection points. Using a conditional random field (CRF) model, this algorithm infers possible states at unobserved time points based on temporal correlations in historical data, improving monitoring coverage while avoiding the performance impact of continuous high-frequency sampling. Anomaly scoring combines multiple methods: Mahalanobis distance calculation based on common neighborhoods, deviation scoring based on an LSTM prediction model, and scoring from an expert rule engine. The final anomaly score integrates these three scores using a dynamic weighting mechanism, which automatically adjusts based on historical detection accuracy. When the anomaly score exceeds a threshold (dynamically set based on workflow importance), the system triggers a multi-level response mechanism, ranging from simple logging to proactive intervention, ensuring timely resolution of potential issues while avoiding stability risks caused by excessive intervention.

[0177] In a preferred embodiment of the present invention, Figure 8 As shown, the method further includes:

[0178] Step S9: performing a multiplication graph model conversion on the DAG definition of the workflow to obtain a node mapping relationship;

[0179] Step S10: Based on the node mapping relationship, a spanning tree cover algorithm is used to find the optimal connection path and obtain a real-time connection suggestion;

[0180] Step S11: Based on the real-time connection suggestion, a virtual force model is used to perform layout optimization to obtain a display effect that reduces line crossings.

[0181] Specifically, in step S9, the system performs a deep dependency analysis on all script components, constructs a direct dependency graph between components, and converts this initial dependency graph into a multiplicative graph structure. The system first identifies each component's input and output types, data formats, and call patterns. It then uses a hash function to map similar components to adjacent nodes, ensuring that any two components that might need to be connected are no more than logarithmically distant in the graph.

[0182] In the implementation of the multiplicative graph model conversion, the system first constructs an initial dependency graph G = (V, E) for the workflow component set V, where the edge set E represents the direct dependencies between components. To convert this graph into a multiplicative graph structure, the system implements a feature vector representation. Each component is mapped into a multidimensional feature space. The feature vector contains: a function category encoding (using one-hot encoding to represent the primary function, such as data input, transformation, analysis, and output), input and output type signatures (encoding the data type and format, such as CSV, JSON, and DataFrame), component performance characteristics (normalized values ​​of metrics such as average execution time and resource consumption), and usage frequency characteristics (based on historical workflow usage statistics). The system uses locality-sensitive hashing (LSH) to map components with similar characteristics to similar hash buckets. The SimHash algorithm, using 64-bit hash values, is chosen for its implementation. Compared to standard LSH, SimHash is more suitable for preserving similarity in high-dimensional features. The system establishes d = log2(n) + 2 virtual connections for each component (n is the total number of components), connecting it to the components with the closest hash value. To ensure connection quality, the system applies a Hamming distance threshold filter, retaining only connections with a distance less than a preset threshold. This strategy, in experiments, kept the average degree of the graph below 25 while maintaining the shortest path between any two nodes within log2(n) hops. For version dependency management, the system implements a semantic version resolver that supports complex version constraint expressions (e.g., ">=1.2.0,<2.0.0,!=1.3.5"). Version constraints are represented as attributes on directed edges, and the system uses interval arithmetic to verify compatibility between constraints. When multiple constraints apply to a component simultaneously, the system calculates the valid version range by intersecting them. If the intersection is empty, a version conflict exists. For detected conflicts, the system generates conflict resolution suggestions, including version upgrade paths, API compatibility analysis, and potential impact assessment, to help users make the best choice. The converted multiplication graph is stored using an adjacency list structure and indexed for common queries (such as shortest path search and neighborhood search), significantly improving the efficiency of subsequent path discovery algorithms.

[0183] In step S10, the system implements a spanning tree cover algorithm based on the multiplication graph to discover the optimal connection paths between components. The system constructs a minimum spanning tree forest using a modified Boruvka algorithm, which is particularly well-suited for large-scale graph processing in distributed environments. Given user-specified input and desired output, the system automatically recommends the shortest component connection path, minimizing data conversion costs and execution complexity.

[0184] In the implementation of the spanning tree cover algorithm, components are first divided into functional domains using spectral clustering. Spectral clustering uses the eigenvectors of the component graph's Laplacian matrix and applies K-means clustering in the feature space, with the K value automatically determined by the silhouette coefficient. For each functional domain, the system implements a modified Boruvka algorithm to construct a minimum spanning tree. Key optimizations include: parallel processing—leveraging multithreading to simultaneously process independent subtrees, achieving near-linear speedup on large component graphs; edge weight adaptation—edge weights take into account multiple factors and are dynamically adjusted based on user feedback through machine learning; and sparse graph optimization—using an adjacency list representation and a heap-optimized union-find implementation to reduce the algorithm complexity to approximately O(E log V), where E is the number of edges and V is the number of nodes. To handle cross-domain connections, the system evaluates all possible bridging edges between each pair of functional domains and calculates their importance scores. The importance score considers the frequency of use of connected components (number of times they co-appeared in workflows in the past 90 days), data type conversion complexity (based on a predefined type conversion cost matrix), co-occurrence patterns in historical workflows (using an association rule mining algorithm), and centrality metrics in the multiplication graph (calculated using the PageRank algorithm). The top k bridging edges with the highest importance scores (k is the logarithm of the number of functional domains) are selected and, along with the intra-domain spanning trees, form a forest structure covering the entire component graph. During real-time path recommendation, the system leverages the properties of the spanning tree forest to rapidly find the optimal connecting path between components. Specifically, if two components reside in the same tree, a unique path exists between them, which can be found in O(log V) time using the tree's parent pointer array. If they reside in different trees, the system must find the optimal cross-tree bridging path. The algorithm examines all possible bridging combinations and uses dynamic programming to find the optimal path combination. Path scoring comprehensively considers path length, data conversion consistency, execution efficiency, and reliability. An adaptive learning mechanism for path recommendation is also implemented, continuously adjusting path scoring weights and recommendation strategies by recording user acceptance or rejection of recommended paths. Using a reinforcement learning framework, we treat user behavior as a reward signal and optimize our recommendation strategy to maximize long-term acceptance. This continuous learning enables the system to adapt to an organization's unique work patterns and user preferences, providing increasingly accurate route recommendations.

[0185] In step S11, the system implements a congestion-aware automatic layout algorithm based on the multiplication graph. First, the system divides the workflow graph into multiple layers, with a balanced number of nodes in each layer and the highest node connectivity within each layer. It then applies the improved Sugiyama algorithm for initial layout, introducing a congestion weight function to evaluate the connection density of each region. For congested areas, the system uses a local optimization algorithm based on a spring model, applying virtual forces to readjust node positions until congestion is reduced to an acceptable level.

[0186] The system's layout optimization algorithm implements visual workflow optimization based on a physical mechanics model. It first uses a modified Coffman-Graham hierarchical algorithm to convert the workflow DAG into a hierarchical structure. This algorithm is implemented in two stages: the first stage assigns priority numbers to nodes, taking into account the depth and breadth of their dependencies. The second stage assigns nodes to hierarchies based on priority, ensuring that dependent nodes are always located in the preceding layer. The system extends the standard algorithm by adding a weighting factor that takes into account data flow size, arranging paths with high data flow vertically as much as possible and reducing long horizontal connections. After the hierarchical structure is completed, the system applies a modified Sugiyama algorithm for initial node arrangement. This algorithm consists of four steps: loop removal, hierarchical assignment, node sorting within a hierarchical layer, and coordinate assignment. In the node sorting within a hierarchical layer, a weighted median method is used, taking into account link weights and link directions to ensure that related nodes are as close as possible. The core of the layout optimization is a congestion-aware force-directed algorithm, which defines a congestion weight function w(x,y) that represents the link density in the area around the coordinate (x,y). Specifically, a two-dimensional Gaussian kernel function is used to calculate the contribution of links to each spatial point. A fast convolution algorithm is then used to efficiently compute a congestion map for the entire visible area. During the optimization process, the system iteratively calculates the forces acting on each node: inter-node repulsion (inversely proportional to distance and proportional to congestion), link attraction (proportional to distance and proportional to link importance), and layer conservation forces (keeping nodes within their assigned layers). The system updates node positions using a modified Verlet integration method and applies a simulated annealing strategy to gradually reduce the maximum node movement distance, ensuring convergence to a local optimum. For large workflows (over 100 nodes), the system employs a multi-scale optimization strategy: a coarse layout optimization is first performed on a simplified workflow (merging similar nodes), followed by a gradual refinement process, optimizing local details while preserving the overall structure. The system also specifically optimizes link visualization, implementing an intelligent routing algorithm that adaptively selects straight lines, broken lines, orthogonal lines, or Bezier curves for different scenarios. In high-congestion areas, the system prioritizes curved lines to minimize visual overlap; critical paths are highlighted with bold lines or special colors. It also supports line grouping. When multiple parallel lines connect multiple nodes, they are visualized as a single, thick line, which users can expand to view details as needed. This multi-level layout optimization strategy significantly improves the readability of complex workflows, enabling non-technical personnel to intuitively understand data flows and easily edit and adjust them.

[0187] In a preferred embodiment of the present invention, Figure 9 As shown, the method further includes:

[0188] Step S12: obtaining the DAG definition of the workflow, constructing a graph representation annotated with function labels and attribute tags, and obtaining a multi-level workflow model;

[0189] Step S13: Based on the multi-level workflow model, a connected maximum common subgraph algorithm is used to perform pattern recognition to obtain a shared pattern;

[0190] Step S14: Based on the sharing mode, the importance score is calculated and standardized to obtain a new reusable module.

[0191] Specifically, in step S12, the system constructs a multi-level labeled graph representation of the workflow, converting each workflow into a graph structure with rich attribute annotations. Nodes contain not only script component IDs but also multi-dimensional attribute tags such as function labels, input and output types, and execution frequency. Edges also contain information such as data flow type and transformation complexity. The system utilizes a hierarchical storage architecture that supports representing the same workflow at different levels of abstraction, adapting to the needs of discovering subgraphs of varying complexity.

[0192] In implementing the multi-level workflow model, a sophisticated labeled graph data structure was first constructed. Each workflow is represented as a graph G = (V, E, L, A), where the node set V represents the script components, the edge set E represents the data flow, L is a function that maps nodes and edges to functional labels, and A maps them to a detailed attribute set. Node labels contain multidimensional information: functional categories use a hierarchical classification system, such as "data processing / cleaning / missing value handling," supporting functional descriptions up to three levels of precision; input and output types use a unified data type system that defines basic types (such as numeric, text, and time) and composite types (such as tables, trees, and graphs), as well as conversion relationships between types; execution characteristics include average runtime (and its variance), resource usage patterns (CPU-intensive, memory-intensive, or I / O-intensive), and parallelism; quality metrics record historical failure rates, average number of errors, and common error patterns. Edge labeling is also rich: the data flow type specifies the data format and protocol being transmitted; the data magnitude uses a logarithmic scale to represent the range of data volumes; and the transformation complexity indicates the difficulty of transforming the data during transmission, ranging from direct transmission (complexity 0) to structural changes requiring deep transformation (complexity 5). The system collects labeling information through three channels: automatically extracting structured information from component metadata; analyzing historical execution records to extract performance characteristics through statistical methods; and integrating user-added annotations and converting them into system labels through standardization. To support multi-level representation, a hierarchical abstraction algorithm is implemented. The L0 (detailed layer) retains the complete structure of the original graph; the L1 (intermediate layer) merges consecutive nodes of similar functionality using aggregation rules, such as merging multiple consecutive data cleaning steps into a single "data cleaning" node; the L2 (overview layer) further abstracts, retaining only core functional blocks and major data flow paths. This abstraction is achieved through functional similarity analysis. The system uses a semantic similarity algorithm to calculate functional similarity between nodes. Then, a community detection algorithm is applied to identify groups of closely related nodes. For each group, an abstract node is generated, preserving statistical summaries of important attributes within the group. Different layers are linked via bidirectional mapping functions, enabling seamless drill-down from high-level overviews to specific implementation details. Graph data is stored in a dedicated graph database, optimized for efficient graph retrieval: multidimensional indexes are established based on functional labels, attribute characteristics, and topological structures; a caching mechanism is implemented for commonly used query patterns; and an incremental index update strategy is used for frequently updated workflows to avoid the performance overhead of full rebuilds. To ensure data consistency in highly concurrent environments, a transaction processing mechanism based on MVCC (Multi-Version Concurrency Control) is implemented, supporting atomic updates of graph data and query consistency.

[0193] In step S13, the system implements an efficient connected maximum common subgraph algorithm to identify shared patterns in large workflow labeled graphs. This system uses a modified McGregor algorithm combined with a pruning optimization strategy. For large-scale graphs, it introduces a distributed computing framework to partition the graph for parallel processing. The algorithm specifically focuses on node and edge label compatibility. Two nodes are considered mappable only when key labels, such as functional category and input and output types, match. A fuzzy matching mechanism is also implemented to allow for label differences within a certain threshold.

[0194] In implementing the connected maximum common subgraph algorithm, several optimizations were first applied to the basic McGregor algorithm to handle large-scale workflow collections. The standard McGregor algorithm uses a state-space search method to find common subgraphs between two graphs. Its core is a backtracking process that attempts to map nodes from one graph to nodes from the other. Three key optimizations were introduced to the original algorithm: a pruning strategy—which rapidly eliminates search branches that are unlikely to produce larger results by applying an upper bound on the subgraph size. This upper bound is estimated by comparing the number of unprocessed nodes with the current maximum matching size. When the upper bound is less than or equal to the current maximum matching size, the search for that branch is immediately terminated; a node pre-sorting strategy—which prioritizes nodes based on metrics such as degree, label rarity, and centrality, prioritizing exploration of node pairs that are more likely to form large matchings. Experiments show that this heuristic strategy reduces the search space by over 60% on average; and a similarity pre-filtering strategy—which calculates similarity scores for node pairs before backtracking and considers only those pairs that exceed a threshold (typically set to 0.7), significantly reducing the search space. To determine node compatibility, the system implements multi-dimensional similarity calculations: functional category labels use semantic similarity based on a hierarchical classification system, taking into account distance within the category tree; text description labels use semantic embeddings based on BERT and cosine similarity; numeric attributes use normalized Euclidean distance; and input and output types use a predefined type compatibility matrix, which represents the difficulty of converting between different data types. The system also incorporates structural compatibility checks to ensure that the neighbor relationships of mapped nodes remain consistent across the two graphs. For large-scale graph collections, the system employs a multi-stage distributed computing strategy: first, workflow graphs are preliminarily clustered using the LSH (locality-sensitive hashing) technique to group similar graphs; then, within each cluster, an optimized MCCS algorithm is applied in parallel to identify the exact common subgraphs; finally, the results are merged and redundancy is removed. The system uses Apache Spark as the distributed computing framework, enabling partitioned storage and parallel processing of workflow graphs. To ensure that the discovered subgraphs meet connectivity requirements, the system performs connectivity verification after each match, using a breadth-first search to confirm that the mapped nodes form a connected subgraph within the original graph. Furthermore, the system focuses on edge compatibility, checking the consistency of edge types and attributes across different workflows to ensure the semantic coherence of the data flow. To further improve performance, the system implements a multi-level caching strategy: subgraph isomorphism check result caching—avoiding repeated calculations of isomorphism for identical subgraph pairs; intermediate mapping result caching—saving partial mapping results to accelerate the processing of similar graphs; and feature vector caching—storing precomputed graph feature vectors to accelerate similarity calculations. These optimizations enable the system to complete comprehensive pattern mining within an acceptable timeframe in enterprise environments encompassing thousands of workflows.

[0195] In step S14, the system develops a multi-dimensional scoring system to rank the discovered common subgraphs by importance and identify the most valuable workflow patterns. Scoring criteria include subgraph size, frequency of use, business value, performance stability, and structural integrity. The system automatically converts identified high-value common subgraphs into standardized, reusable modules, analyzes the subgraph's boundary conditions, identifies clear input and output interfaces, extracts parameter configuration patterns within the subgraph, and finally generates self-describing documentation for the module, including functional descriptions, typical use cases, and configuration guidelines.

[0196] During the sharing model scoring and conversion process, a multi-dimensional value assessment framework was implemented. The scoring system considers five core dimensions: subgraph size—a weighted and quantified calculation of the number of nodes and edges, with a weighting of 3:1, reflecting the dominance of a node in the workflow; usage frequency—the number of complete appearances in different workflows, as well as the number of partial matches; business value—assessed through the strength of association with key business indicators. The system analyzes the business tags and impact indicators of the workflow to which the subgraph belongs, and quantifies this association using machine learning models; performance stability—the coefficient of variation (standard deviation divided by mean) of time and resource consumption is calculated based on historical execution data. Subgraphs with high stability receive higher scores; structural integrity—assesses the self-containedness of the subgraph as an independent functional unit, including the clarity of input and output interfaces, the degree of encapsulation of internal dependencies, and the degree of minimization of external dependencies.

[0197] like Figure 10 As shown, the embodiment of the present invention also provides a script intelligent adaptation and visual arrangement system for non-technical personnel, including:

[0198] The dependency analysis module 100 is used to obtain the script file uploaded by the user, perform dependency library analysis on the script file, and obtain an isolated container environment;

[0199] A path adaptation module 200 is configured to perform static analysis and conversion on the file operation statements and database connection codes in the script file based on the isolated container environment to obtain standardized API calls;

[0200] The workflow orchestration module 300 is used to perform input and output feature analysis and visual orchestration based on the standardized API call to obtain a DAG definition of the workflow;

[0201] The module management module 400 is used to perform function-level encapsulation and interface standardization on script components based on the DAG definition of the workflow to obtain a reusable module and its version management solution;

[0202] The scheduling execution module 500 is used to configure the workflow scheduling strategy and perform automatic operation and monitoring based on the DAG definition of the workflow and in combination with the version management solution of the reusable module, and output the execution status and exception handling results of the workflow.

[0203] The dependency analysis module 100 implements the function of step S1 in the above method, performs AST analysis on the script file, extracts third-party library reference information, builds a dependency graph, detects version conflicts, generates an isolated environment configuration, and calls the container API to build an image.

[0204] The path adaptation module 200 implements the function of step S2 in the above method, performs static analysis on the script file, identifies file operation statements and database connection codes, generates a file operation list and a database access list, performs code conversion according to a rule library or AI model, and realizes intelligent adaptation of file paths and data sources.

[0205] The workflow orchestration module 300 implements the function of step S3 in the above method, analyzes the input and output characteristics of the script, provides a visual drag-and-drop interface, supports node parameter configuration and dependency setting, generates a standardized DAG definition, and provides workflow simulation verification function.

[0206] The module management module 400 implements the function of step S4 in the above method, encapsulates the script components at the function level, creates standardized interfaces and metadata descriptions, builds a component market interface, and implements version management and compatibility evaluation.

[0207] The scheduling execution module 500 implements the function of step S5 in the above method, provides a timing policy configuration interface, converts user configuration into a standard scheduling expression, monitors the workflow execution status, handles abnormal situations, and provides a multi-level notification mechanism.

[0208] The embodiment of the present invention may further include a workflow analysis module to implement the functions of steps S6 to S8 in the above method, extract features from workflow historical execution data, group workflows using a density clustering algorithm, generate a pseudo-random detection point sequence, and perform anomaly detection and early warning.

[0209] An embodiment of the present invention may further include a layout optimization module to implement the functions of steps S9 to S11 in the above method, convert the workflow DAG into a multiplication graph model, use a spanning tree cover algorithm to find the optimal connection path, and perform layout optimization through a virtual force model to improve visualization effects.

[0210] The embodiment of the present invention may further include a pattern recognition module to implement the functions of steps S12 to S14 in the above method, construct a multi-level workflow model, use the connected maximum common subgraph algorithm for pattern recognition, discover shared patterns and convert them into new reusable modules.

[0211] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A script intelligent adaptation and visual arrangement method for non-technical personnel, characterized in that: include: Obtain the script file uploaded by the user, analyze the dependency library of the script file, and obtain an isolated container environment; Using the isolated container environment, statically analyzing and converting the file operation statements and database connection codes in the script file to generate standardized API calls; Based on the standardized API calls, perform input and output feature analysis and visual orchestration to form a DAG definition of the workflow; Perform function-level encapsulation and interface standardization on the script components in the DAG definition of the workflow to obtain a reusable module and its version management solution; Based on the DAG definition of the workflow and in combination with the version management solution of the reusable module, the workflow scheduling strategy is configured and automatically run and monitored, and the execution status and exception handling results of the workflow are output; The method further comprises: Obtaining historical execution data of the workflow, extracting time dimension features and resource dimension features, and obtaining a workflow feature vector; Based on the workflow feature vector, a density clustering algorithm is used to group the workflows to obtain workflow groups with similar execution features; Based on the workflow group, a pseudo-random detection point sequence is generated and a state snapshot is collected to obtain an anomaly score; The method further comprises: Performing a multiplication graph model conversion on the DAG definition of the workflow to obtain a node mapping relationship; Based on the node mapping relationship, a spanning tree cover algorithm is used to find the optimal connection path and obtain real-time connection suggestions; Based on the real-time connection suggestions, a virtual force model is used to perform layout optimization, thereby obtaining a display effect with reduced line crossing.

2. The method according to claim 1, characterized in that The step of analyzing the dependency library of the script file to obtain an isolated container environment includes: Performing abstract syntax tree (AST) analysis on the script file, extracting import statements and from-import statements, and obtaining third-party library reference information; Build a dependency graph based on the third-party library reference information and perform version conflict detection. When a version conflict is detected, use a dependency isolation strategy to generate different isolation environment configurations; The isolated environment configuration is used to call the container API to build the image and store it in a private warehouse to form the isolated container environment.

3. The method according to claim 1, characterized in that The static analysis and conversion of the file operation statements and database connection codes in the script file to obtain standardized API calls includes: Process the script file using abstract syntax tree (AST) analysis and regular expression matching to generate a file operation list and a database access list; Perform code pattern recognition and classification based on the file operation list and the database access list to form a conversion rule mapping; The file operation statements and database connection codes are replaced with platform standard interfaces by utilizing the conversion rule mapping to obtain the standardized API calls.

4. The method according to claim 1, wherein The input and output feature analysis and visual arrangement are performed based on the standardized API call to form a DAG definition of the workflow, including: Analyzing variable definitions and function return values ​​in the standardized API call to generate input and output feature representations of the script component; According to the input and output feature representation, node parameters and dependencies are configured by visual drag and drop to form an orchestration result; Perform connection validity verification and format conversion on the orchestration result, and output a DAG definition of the workflow.

5. The method according to claim 1, wherein The configuration of workflow scheduling strategies and automatic operation and monitoring, output of workflow execution status and exception handling results, includes: Based on the DAG definition of the workflow and the version management scheme of the reusable module, the workflow configuration information is obtained, and the execution time rule is converted into a timing expression to obtain a scheduling strategy; Based on the scheduling strategy, the running status data of the workflow nodes are collected to obtain the execution progress information; Based on the execution progress information, anomaly detection and recovery strategy execution are performed to obtain processing results and notification messages.

6. The method according to claim 1, characterized in that Also includes: Obtaining a DAG definition of the workflow, constructing a graph representation annotated with function labels and attribute tags, and obtaining a multi-level workflow model; Based on the multi-level workflow model, pattern recognition is performed using a connected maximum common subgraph algorithm to obtain a shared pattern; Based on the sharing mode, the importance scores are calculated and standardized, and a new reusable module is obtained.

7. The method according to claim 1, characterized in that The function-level encapsulation and interface standardization processing of the script components in the DAG definition of the workflow to obtain a reusable module and its version management solution include: Function splitting and interface definition are performed based on the script components to obtain standardized function modules; Based on the standardized function module, a metadata description of function classification and version information is created to obtain a component market interface; Based on the component market interface, version updates are tracked and compatibility impacts are evaluated to obtain upgrade recommendations.

8. A script intelligent adaptation and visual arrangement device for non-technical personnel, characterized in that: include: A dependency analysis module is used to obtain script files uploaded by users, perform dependency library analysis on the script files, and obtain an isolated container environment; A path adaptation module, configured to perform static analysis and conversion on the file operation statements and database connection codes in the script file based on the isolated container environment to obtain standardized API calls; A workflow orchestration module is used to perform input and output feature analysis and visual orchestration based on the standardized API calls to obtain a DAG definition of the workflow; A module management module is used to perform function-level encapsulation and interface standardization on script components based on the DAG definition of the workflow to obtain reusable modules and their version management solutions; A scheduling execution module is used to configure workflow scheduling strategies based on the workflow DAG definition and in combination with the version management solution of the reusable module, and to perform automated operation and monitoring, and output the workflow execution status and exception handling results; The device is further configured to: obtain historical execution data of the workflow, extract time dimension features and resource dimension features, and obtain a workflow feature vector; group the workflows using a density clustering algorithm based on the workflow feature vector to obtain workflow groups with similar execution features; and generate a pseudo-random detection point sequence based on the workflow group and collect state snapshots to obtain anomaly scores; The device is further used to: perform multiplication graph model conversion on the DAG definition of the workflow to obtain a node mapping relationship; Based on the node mapping relationship, a spanning tree cover algorithm is used to find the optimal connection path to obtain real-time connection suggestions; based on the real-time connection suggestions, a virtual force model is used to perform layout optimization to obtain a display effect that reduces line crossing.

Images