Data workbench modular process adaptive management and control method and system

Abstract

The present application relates to a data workbench modular process adaptive management and control method and system, belonging to the technical field of data processing. The method comprises: building a standardized tool access interface system, constructing a semantic mapping knowledge base, and adaptively packaging the tool native interface to generate a general interaction interface; collecting tool instance running state indicators to generate a health degree portrait and constructing a structured tool resource library; building a standardized process node model, generating an initial process topology, and generating a process template after link connectivity and tool compatibility verification; when the process template triggers version update, suspending the execution state of the target task instance in the running state and taking a snapshot of the running context, completing instance state mapping conversion according to the updated template and restoring the running, and realizing task instance live migration. The present application can realize plug-and-play of tools and smooth upgrade of process templates, effectively improving the flexibility and running stability of process management and control.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a modular process adaptive control method and system for data workbench. Background Technology

[0002] In enterprise-level data workbench scenarios characterized by frequent iterations of heterogeneous tools and rapid changes in business process requirements, various software tools are often deployed in a dispersed manner with differing interface standards. Data interaction between tools lacks a unified semantic self-description mechanism, and inherent differences in field types, units of measurement, and business meanings require manual conversion and verification. Particularly noteworthy is the high degree of binding between tool logic and process logic in existing systems. Once a tool is integrated into a process, its calling method and data format are fixed in the process definition. When a tool's interface changes or needs to be replaced with a similar heterogeneous tool, the original process must be recoded, debugged, and deployed, failing to achieve plug-and-play functionality and flexible replacement. Conventional industry optimization approaches are often limited to optimizing interface adaptation code for single scenarios, failing to break the tight coupling between tools and processes at the architectural level. Therefore, they remain unable to escape the inherent limitation of linearly increasing adaptation costs with the number of tools, severely restricting the scalability and flexibility of processes and making it difficult to adapt to dynamically changing business needs.

[0003] Meanwhile, the management mechanism for process templates also has fundamental flaws. Traditional methods fail to effectively isolate template definitions from running task instances. After a template version is updated, task instances already in execution cannot be smoothly migrated to the new version, often forcing the current task to be interrupted and redeployed. This not only disrupts the continuity of business processes but also makes it difficult to incrementally upgrade templates based on actual business feedback and manage multi-version compatibility. The low template reuse rate and the interruption caused by instance upgrades combine to create significant architectural obstacles to the transition from static orchestration to dynamic adaptive control of processes. Summary of the Invention

[0004] Based on this, it is necessary to provide a modular process adaptive management and control method and system for data workbench to address the above-mentioned technical problems. This method and system can solve the technical problems of high integration and adaptation costs and business interruption caused by tight coupling between tools and processes and lack of isolation between templates and running instances in the process orchestration and control scenario of enterprise-level data workbench, and achieve plug-and-play tools and smooth migration of process instances.

[0005] A modular workflow adaptive management and control method for data workbench, the method comprising: Establish a standardized tool access interface system, collect structured attribute information of external tools and generate unique identifiers, obtain the field parameters, data types, measurement units and business meanings of tool input and output through the interface semantic self-description registration mechanism, build a semantic mapping knowledge base and adaptively encapsulate the tool native interfaces to generate a unified and general interaction interface, and collect the running status indicators of each tool instance in real time to generate a health profile and build a structured tool resource library. A standardized process node model is constructed, including start node, task node, gateway node and end node. Based on the input and output data dependencies of each tool in the structured tool resource library, node connection relationships and initial process topology are generated. After link connectivity and tool compatibility verification, a process template is generated. The process template is managed in a multi-version hierarchical manner, and the process template definition and the running task instance are kept independent of each other. When a process template triggers a version update, the execution state of the target task instance in the running state is suspended and a snapshot of the running context is taken. The mapping transformation of the execution instance state is performed according to the node definition of the updated process template. After the transformation is completed, the running state of the target task instance is restored, and the hot migration of the task instance is completed.

[0006] On the other hand, a modular process adaptive control system for data workbench is also provided, including: The structured tool resource library construction module is used to build a standardized tool access interface system, collect structured attribute information of external tools and generate unique identifiers, obtain the field parameters, data types, measurement units and business meanings of tool input and output through the interface semantic self-description registration mechanism, build a semantic mapping knowledge base and adaptively encapsulate the tool's native interface to generate a unified and general interaction interface. At the same time, it collects the running status indicators of each tool instance in real time to generate a health profile and build a structured tool resource library. The process template generation module is used to construct a standardized process node model that includes a start node, task node, gateway node, and end node. Based on the input and output data dependencies of each tool in the structured tool resource library, it generates node connection relationships and an initial process topology. After verifying the link connectivity and tool compatibility, it generates a process template. The process template is managed in a multi-version hierarchical manner, and the process template definition and the running task instance are kept independent of each other. The task instance hot migration module is used to suspend the execution state and runtime context snapshot of the target task instance that is in the running state when the process template triggers a version update. It performs the mapping transformation of the execution instance state according to the node definition of the updated process template, and restores the running state of the target task instance after the transformation is completed, thus completing the hot migration of the task instance.

[0007] Compared with existing technologies, the data workbench modular process adaptive control method and system provided by this invention have the following advantages: 1. By building a standardized tool access interface system and constructing a semantic mapping knowledge base in conjunction with the interface semantic self-description registration mechanism, the native interfaces of tools are adaptively encapsulated to generate unified and universal interaction interfaces. This breaks the tight coupling between tool logic and process logic, eliminates the interaction differences in field types, units of measurement, and business meanings among heterogeneous tools, and allows tool access, replacement, and interface adaptation to be completed without modifying the process definition. This significantly reduces integration and adaptation costs, decouples tools and processes at the architectural level, and improves process scalability and flexibility.

[0008] 2. By constructing a standardized process node model, generating and verifying the process topology based on the dependency relationship between tool input and output data, and cooperating with the multi-version hierarchical management of process templates and the independent design of templates and instances, we standardize process modeling standards, improve process compilation efficiency and template reuse rate, provide architectural support for incremental upgrades and compatibility management of templates, realize the isolation between templates and running instances, and support the evolution of multiple versions of templates.

[0009] 3. When a version update is triggered by the process template, a hot migration mechanism is used to suspend the execution status of the running task instance, take a snapshot of the running context, convert the instance state mapping, and restore the running status. This allows for a smooth transition of the instance to the new version template without interrupting the current task, avoiding interruption of the business process during the version upgrade. At the same time, it takes into account the template iteration requirements and the continuity of business operation, ensuring continuous business operation and achieving smooth instance upgrade. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention, and those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating an adaptive control method for modular workflows on a data workbench in one embodiment. Figure 2 This is a structural block diagram of a modular workflow adaptive control system for a data workbench in one embodiment. Figure 3 This is an internal structural diagram of a computer device in one embodiment.

[0012] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0014] It is understood that the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0015] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0016] Example 1 like Figure 1 As shown, a modular workflow adaptive management method for data workbench is provided, including the following steps: Step 201: Establish a standardized tool access interface system, collect structured attribute information of external tools and generate unique identifiers, obtain the field parameters, data types, measurement units and business meanings of tool input and output through the interface semantic self-description registration mechanism, build a semantic mapping knowledge base and adaptively encapsulate the tool's native interface to generate a unified and general interactive interface, and collect the running status indicators of each tool instance in real time to generate a health profile and build a structured tool resource library.

[0017] Step 202: Construct a standardized process node model that includes a start node, task node, gateway node, and end node. Based on the input and output data dependencies of each tool in the structured tool resource library, generate node connection relationships and initial process topology. After verifying link connectivity and tool compatibility, generate a process template. Perform multi-version hierarchical management on the process template and keep the process template definition and runtime task instance independent of each other.

[0018] Step 203: When the process template triggers a version update, the execution state of the target task instance in the running state is suspended and the running context snapshot is taken. The mapping transformation of the execution instance state is performed according to the node definition of the updated process template. After the transformation is completed, the running state of the target task instance is restored, and the hot migration of the task instance is completed.

[0019] The aforementioned modular process adaptive management and control method for the data workbench establishes a standardized tool access interface system and a semantic mapping knowledge base to uniformly and adaptively encapsulate heterogeneous tool interfaces. This breaks the tight coupling between tools and processes at the architectural level, enabling plug-and-play and flexible replacement of tools. By constructing a standardized process node model and a multi-version hierarchical management mechanism for templates, the template definition and runtime task instances remain independent. Combined with a hot migration mechanism for task instances, this ensures uninterrupted business processes during template version updates. This effectively solves the technical problems of high integration and adaptation costs and business interruption during version upgrades in existing technologies, significantly improving the scalability, flexibility, and operational stability of data workbench process management.

[0020] In step 201, the various external heterogeneous tools to be connected and their structured attribute information are obtained. The standardized access, semantic adaptation and encapsulation, and dynamic health management of tool resources are completed. A structured tool resource library that can be uniformly scheduled is output, providing a standardized tool resource foundation for subsequent process modeling and operation scheduling.

[0021] In the specific implementation of this step, a standardized tool access interface system is first established, collecting structured attribute information of external tools and generating a globally unique identifier for each tool. Tool interface information is collected through an interface semantic self-description registration mechanism. This mechanism is a standardized registration mechanism where tools proactively report their own interface metadata upon access, eliminating the need for manual field-by-field entry. This mechanism obtains the field parameters, data types, measurement units, and business meanings of each tool's input and output, and a semantic mapping knowledge base is constructed based on field mapping rules, unit constraint rules, business meaning constraints, and structural constraint rules.

[0022] The semantic mapping knowledge base is a standardized knowledge base that stores the mapping relationships between field business concepts, unit conversion rules, and data type conversion rules. It is used to automatically resolve data semantic differences between different tools. Specifically, field mapping rules define a standard data type dictionary; field types reported by tools must be mapped to the corresponding standard dictionary, and an alarm is triggered if a match fails. Unit constraint rules establish a standard unit of measurement library, supporting the mapping of different units to standard units through preset conversion formulas. Business meaning constraints construct a business terminology table, associating the meanings of field types reported by tools with standard terms through semantic similarity matching; fields with a matching degree lower than a preset threshold are entered into a manual review queue. Structural constraint rules define standard patterns for data structures, automatically detecting anomalies such as circular references and excessively deep structures through a structure validator.

[0023] After completing the semantic mapping knowledge base construction, the native interfaces of the tools are adaptively encapsulated to generate a unified and universal interaction interface. When the tool interface undergoes version changes, the system can automatically detect adjustments to fields, types, or constraints through a semantic self-description mechanism, triggering real-time regeneration and updates of the adaptation scripts, without requiring manual intervention in the process definition. The unified and universal interaction interface is a standardized call interface that shields the differences between underlying tool interfaces. The process side only needs to use this interface to call heterogeneous tools from different vendors and with different formats. The specific encapsulation process is as follows: obtain the output field information of the upstream tool and the input field information of the downstream tool, input them into the semantic mapping knowledge base for business concept matching, and output the field mapping pairing results and the corresponding standard conversion rule identifiers; based on the field mapping pairing results and the standard conversion rule identifiers, sequentially call the unit conversion rule and the data type conversion rule to generate the corresponding data conversion adaptation script; insert the adaptation script between the upstream and downstream tools' interaction links to complete the adaptive encapsulation of heterogeneous tool interfaces.

[0024] If, when matching business concepts into the semantic mapping knowledge base, no corresponding business concept relationship is found or the conversion rules conflict, dynamic negotiation is performed. Specifically, a field matching negotiation request is initiated to the tool registry center, and the current interaction link between upstream and downstream tools is marked as pending confirmation; candidate field mapping relationships and corresponding conversion rules are generated and pushed to the review end; the formal mapping relationships and conversion rules after review and confirmation are received and stored in the semantic mapping knowledge base; and the data conversion adaptation script is regenerated based on the formal mapping relationships and conversion rules to complete the adaptive encapsulation of heterogeneous tool interfaces.

[0025] While adaptively encapsulating interfaces, the system collects real-time runtime status metrics for each tool instance to generate health profiles and constructs a structured tool resource library. The health profile is a quantitative assessment of the tool instance's runtime status, comprehensively reflecting the tool's service quality and availability. The structured tool resource library is a resource pool that stores all tool instance information, health data, and scheduling rules according to a unified standard, serving as a unified entry point for process-based tool invocation.

[0026] The specific construction process is as follows: Response latency, call success rate, runtime load, and availability status of each tool instance are collected according to a preset time window. Maximum value normalization is then performed on the response latency and runtime load, outputting the corresponding standardized indicator set. Maximum value normalization is a standardization process that maps the original indicator values ​​to the range of 0 to 1, eliminating the dimensional differences between different indicators and enabling unified weighted calculation of all indicators.

[0027] By using pre-set dynamic weighting coefficients, a weighted summation is performed on each indicator within the standardized indicator set to obtain the overall health score for each tool instance. The formula for calculating the overall health score is: H = w1 * (1 - normalized response latency) + w2 * call success rate + w3 * (1 - normalized runtime load) + w4 * available status (0 or 1). In the formula, H represents the overall health score; w1, w2, w3, and w4 are the dynamic weighting coefficients of the corresponding indicators, with the availability status taking values ​​of 0 or 1, where 0 represents unavailable and 1 represents available. Preferably, the dynamic weighting coefficients can be flexibly adjusted according to the system operation strategy. For example, during peak business periods, the latency weight can be reduced and the success rate weight increased to adapt to the operation and maintenance needs of different scenarios.

[0028] The overall health score is associated with the unique identifier, common interaction interface information, and attribute information of the corresponding tool instance to generate a health profile for each tool instance. The health profiles of all tool instances are stored categorized by tool type, and preset routing filtering thresholds, circuit breaker conditions, and degradation processing rules are synchronously written to form a dynamically updatable structured tool resource library. Once the structured tool resource library is built, when a tool call is triggered at a process node, the tool instance is dynamically scheduled based on the structured tool resource library.

[0029] Specifically, upon receiving tool invocation requests from process nodes, the system extracts all tool instances of the corresponding type from the structured tool resource library and filters out a set of candidate instances whose overall health score is higher than a preset routing filtering threshold. Weighted random selection is then performed using the overall health score of each tool instance in the candidate instance set as a weight to determine the target invocation instance. Weighted random selection means that random scheduling is performed using the health score as a weight; instances with higher health scores have a greater probability of being selected, thus achieving load balancing while ensuring service quality.

[0030] Receive execution response data returned by the target call instance, extract the call time and execution success / failure status and write it back to the structured tool resource library, update the comprehensive health score of the corresponding tool instance in real time, and transmit the output results data in the execution response data to the process engine to promote the flow of subsequent process nodes.

[0031] The system counts the number of call failures and failure rate of each tool instance within a preset time window. When the preset circuit breaker criteria are met, the corresponding tool instance is marked as being in a circuit breaker state and updated synchronously to the structured tool resource library. A circuit breaker state refers to the temporary removal of an instance from the available scheduling queue when frequent failures occur, preventing the faulty instance from continuously receiving requests and causing the failure to spread. Preferably, the circuit breaker criteria can be set to a single instance accumulating more than 5 failures or a failure rate exceeding 50% within a preset time window. After an instance enters the circuit breaker state, it enters a half-open state at a preset period, receiving a small number of probe requests to verify service recovery. If the verification is successful, the circuit breaker state is lifted and the instance returns to the available scheduling queue; otherwise, the circuit breaker state remains in effect.

[0032] When the overall health score of all tool instances of a corresponding tool type is lower than the route filtering threshold, a preset degradation processing rule is triggered. Degradation processing can take various forms, such as returning cached data, executing simplified processing logic, directly rejecting requests and pushing alerts, and can be flexibly configured according to the business level.

[0033] This step automatically resolves the differences in field types, measurement units, and business meanings between heterogeneous tools through interface semantic self-description and adaptive encapsulation mechanisms. Tool integration, replacement, and interface adaptation can be completed without modifying the process definition, fundamentally decoupling tool logic from process logic. Through health profiling and intelligent routing, circuit breaking, and degradation mechanisms, the stability and reliability of tool calls are dynamically ensured, achieving plug-and-play functionality and dynamic elastic management of tools, significantly reducing the integration and adaptation costs and operational costs of heterogeneous tools.

[0034] In step 202, the input consists of tool metadata from the structured tool resource library and business process orchestration requirements. This completes the standardized definition of process nodes, automatic generation of process topology, and full lifecycle management of templates. The final output is a reusable and evolvable standardized process template, while maintaining the independence between template definition and running instances.

[0035] In the specific implementation of this step, a standardized process node model is first constructed, including a start node, task nodes, gateway nodes, and an end node. The standardized process node model provides a unified and standardized definition of various nodes in the process, clarifying the functional boundaries and attribute rules of each type of node, ensuring that all processes follow a unified modeling standard. The start node serves as the entry trigger node for the process, carrying the process startup parameters and initialization logic; the task node is the core execution unit of the process, configuring tool associations, responsible persons, and other functional positions to associate with corresponding tools and perform data processing operations; the gateway node controls the branching, aggregation, and flow logic of the process, including two types: exclusive gateways and aggregation gateways. Exclusive gateways are used to select a single execution branch based on conditions, while aggregation gateways are used to aggregate the execution results of multiple parallel branches; the end node serves as the process's closing node, used to synchronize the final state and output results of the process.

[0036] After completing the node model construction, based on the input-output data dependencies of each tool in the structured tool resource library, node connection relationships and an initial process topology are generated. The input and output field sets of each tool are extracted from the structured tool resource library. Using the tool's output fields as data producers and its input fields as data consumers, a directed tool data dependency graph is constructed. The directed tool data dependency graph is a relationship graph with tools as nodes and data flow directions as directed edges, used to depict the data production and consumption relationships between tools.

[0037] Traverse all tool nodes in the directed graph of tool data dependencies. If the output field set of the upstream tool and the input field set of the downstream tool have a non-empty intersection between two tool nodes, then a serial connection relationship is generated between the two tool nodes. If there is no data dependency relationship between the two tool nodes, then a parallel structure relationship is generated between the two tool nodes.

[0038] If the input requirements of a downstream tool are simultaneously met by the outputs of multiple upstream tools, conflict resolution logic is executed. A comprehensive score is calculated based on three dimensions: data freshness, tool health, and path length. The single upstream tool with the highest score is recommended as the data source, or an aggregation gateway node is recommended to aggregate multi-source data before inputting it into the downstream tool.

[0039] The identification tool identifies branching and aggregation scenarios in the directed graph, adding exclusive gateway nodes and aggregation gateway nodes accordingly to generate a directed acyclic initial process topology. A directed acyclic initial process topology refers to a process structure without cyclic dependency links, a prerequisite for the normal sequential execution of the process.

[0040] After generating the initial process topology, manual drag-and-drop adjustments and optimizations are supported. During manual adjustments, the system performs real-time verification to prevent drag-and-drop operations from creating circular dependency links, ensuring that the adjusted topology always satisfies the directed acyclic constraint.

[0041] After topology adjustments are completed, a process template is generated following link connectivity and tool compatibility checks. All upstream and downstream tool nodes with serial connections in the initial process topology are traversed, and the input and output field information corresponding to each tool node is retrieved. Based on a semantic mapping knowledge base, the business concepts of the fields of upstream and downstream tools are verified to ensure mappable relationships, thus completing the tool compatibility check. Tool compatibility checking verifies whether the data fields of upstream and downstream tools can interact normally through semantic mapping, preventing the generation of invalid processes that cannot be executed correctly.

[0042] Perform a link connectivity check on the initial process topology that has completed tool compatibility verification, traversing all nodes and connecting edges to check for any circular dependencies. Link connectivity verification verifies whether the process topology can flow normally from the start node to the end node without broken links or circular dependencies. If a circular dependency is detected or the tool compatibility check fails, the corresponding abnormal node is marked and template generation is prohibited; if both checks pass, the official process template is generated and stored.

[0043] After generating the process template, implement multi-version hierarchical management of the process template, and maintain the independence between the process template definition and the runtime task instance. Obtain the changes to be updated in the process template, compare the changes with the original process template, and determine the type of change.

[0044] If the change type is non-destructive, the version revision number of the process template is incremented. Task instances already in operation will continue to run with the original version of the process template, while newly initiated task instances will use the updated process template. Non-destructive changes refer to changes that do not alter the core execution logic of the process and do not affect the normal execution of existing running instances, such as adding optional nodes, modifying node descriptions, or optimizing gateway conditions.

[0045] If the change type is a destructive change, the major or minor version number of the process template is incremented, triggering a hot migration process for the running task instance. A destructive change refers to a change that alters the core execution logic of the process and affects the normal execution of existing running instances, such as deleting nodes, modifying required parameters, or changing tool dependencies.

[0046] Template definitions and running instances are completely isolated. Template updates will not directly interfere with running task instances, ensuring the independence of business operations. It also supports compatible switching and one-click rollback of multiple template versions, allowing for quick switching to a specified historical version according to business needs, adapting to the process execution requirements of different scenarios.

[0047] This step automatically generates the process topology based on tool data dependencies, significantly reducing the technical threshold for process orchestration and improving process modeling efficiency. Through multi-dimensional conflict resolution rules and a dual-verification mechanism, the legality and executability of the process template are guaranteed from the source. A multi-version hierarchical management mechanism that isolates templates and instances supports continuous iteration and multi-version compatibility of process templates, significantly improving the reusability of process templates and the flexibility of process control.

[0048] In step 203, the input is the new process template triggered by the version update and the target task instance in the running state, to complete the uninterrupted version migration of the running instance, and finally achieve the smooth operation of the task instance under the new version template.

[0049] In the specific implementation of this step, when the process template triggers a version update and the change type is a destructive change, a hot migration operation is performed on the target task instance that is in a running state. Task instance hot migration refers to the technical mechanism of migrating a running task from the old version of the process template to the new version of the process template without interrupting task execution or losing task data and state.

[0050] First, a state suspension operation is performed on the target task instance that is in a running state. State suspension means pausing the instance from receiving new event commands and retaining all current running states to ensure that no new state changes occur during the migration process. The current node execution progress, generated intermediate processing data, node running states, variable values, and associated traceability code information of the target task instance are collected to generate a runtime context snapshot. The runtime context snapshot is a serialized record of all current running states of the task instance, containing all state and data information required for the migration.

[0051] The runtime context snapshot is matched with the node definitions of the updated process template. The instance state is mapped and transformed according to the node arrangement rules of the updated process template to generate transformed context data adapted to the updated process template. If the updated template deletes the original node, the state information of the corresponding node is removed synchronously in the snapshot; if the updated template adds a node, the initial unstarted state is inserted for the new node in the snapshot.

[0052] The transformed context data is loaded into the execution environment corresponding to the updated process template, restoring the running state of the target task instance. If the state restoration is successful, the task instance resources corresponding to the original version of the process template are destroyed, completing the migration. If the state restoration fails, the target task instance is rolled back to its running state before the state was suspended, ensuring that the task is not interrupted or data is lost due to migration failure.

[0053] This step utilizes a hot migration mechanism involving state suspension, snapshot generation, mapping transformation, and atomic commit. When a destructive update occurs to the process template, the instance can smoothly transition to the new version template without interrupting the currently running tasks. This avoids version upgrades disrupting business processes while balancing template iteration needs with business continuity, supporting the continuous and smooth evolution of process templates.

[0054] In one embodiment, the method further includes adaptive approval and traceable task generation. This step follows the formal process template generated in step 202 and is executed during the task creation phase. It is the core step in instantiating the standardized process template into an executable task. In this step, a task creation request is received, a task risk value and an approver's workload score are calculated, the approval chain is dynamically adjusted based on the task risk value and the approver's workload score, and after approval, a process task instance with a nested traceability code is generated, establishing a task-level traceability relationship.

[0055] This step takes as input the user-submitted task creation request and corresponding process template definition, completes task risk quantification assessment, dynamic scheduling of the approval chain, and generation of traceable task instances, and finally outputs a formal task instance with a nested traceability code. The formal task instance, along with the embedded traceability code and permission configuration information, will serve as the unified identity basis for the entire chain of subsequent task issuance, node execution, and result traceability.

[0056] In the specific implementation of this step, after receiving the task creation request, the task risk value is first calculated. The task risk value is obtained by multiplying four risk factors—data sensitivity, task size, historical anomaly rate, and compliance level—by their respective weights and then summing them. Among them, data sensitivity is assigned a score of 1 to 10 based on data classification and grading labels; task size is normalized to a score of 1 to 10 based on the estimated scale of data to be processed; historical anomaly rate is mapped to a score of 1 to 10 based on the historical execution failure rate of the corresponding template and tool; and compliance level is mapped to a score of 1 to 10 based on the business compliance requirements associated with the task.

[0057] The workload score for each candidate approver is calculated simultaneously. The calculation expression for the workload score is as follows: Load = α * (Pending tasks / Standard processing capacity) + β * (Average processing time / SLA commitment time) + γ * On-duty status; In the formula, Load is the load score; α, β, and γ are the weight coefficients of the corresponding indicators; the on-duty status is 0 or 1, with 0 for on-duty and 1 for leave.

[0058] Dynamic approval process scheduling is implemented based on task risk value and approver load score. If the task risk value is higher than the first preset threshold, a joint approval mode is initiated, requiring the consent of all designated approvers for approval. If the task risk value is between the first and second preset thresholds, a hierarchical approval mode is initiated. If the task risk value is lower than the second preset threshold, an either-or approval mode is initiated, requiring the consent of any one approver for approval.

[0059] If the primary approver's workload score exceeds the preset workload threshold, the person with the lowest workload score among members of the same role will be selected as the proxy approver. If the task severity factor exceeds the preset urgency threshold, the approval process will automatically skip unnecessary steps and proceed directly to the final decision-maker.

[0060] After approval, a formal task instance carrying nested traceability codes is generated. The nested traceability codes employ a hierarchical coding rule, with the coding structure being the root task ID plus the hierarchy, branch sequence number, and auto-incrementing sequence number. During execution, each node consumes the input traceability codes and generates its own output traceability codes. The system automatically records the mapping relationship between input and output codes, forming a directed acyclic traceability graph. Simultaneously, corresponding tools, responsible personnel, and granular permissions are assigned to the task instance, completing the formal creation of the task instance.

[0061] This step overcomes the rigidity of fixed approval chains through dynamic approval scheduling based on both risk and load dimensions, balancing approval compliance and processing efficiency. Nested traceability codes enable end-to-end traceability across parent-child tasks, branches, and processes, providing support for subsequent data tracing and compliance audits.

[0062] In one embodiment, the method further includes dual-end synchronous delivery and offline adaptation. This step follows the previously generated formal task instance and, based on the association between nodes and tools in the structured tool resource library constructed in step 201, is executed during the task startup phase. It serves as the connecting link between the task's creation state and execution state. In this step, task information is delivered to associated tools in node order, to-do information is pushed synchronously on both ends, offline pre-caching is performed according to the corresponding level based on network quality, and result data is asynchronously transmitted back after reconnection via a segmented breakpoint resume mechanism. When the tool starts, an encrypted runtime context token is automatically injected, and after successful parsing, it directly enters the task processing state.

[0063] The input for this step is the information of the created formal task instance. It completes the dual-end push of the task, offline adaptation and context awareness startup. Finally, the task execution data and node status are synchronized and sent back to the process engine. The returned result data will enter the subsequent result data management stage, and the returned execution status will trigger the process node flow and exception management logic.

[0064] In the specific implementation of this step, task information is distributed to related tools in node order and simultaneously pushed to the responsible person's workbench and mobile device through the information integration and interaction platform, achieving multi-terminal synchronous delivery of pending tasks. The distributed task information contains a nested traceability code identifier for the corresponding node, which is transmitted throughout the entire task lifecycle.

[0065] When a task is issued, the current network quality is assessed. If the network quality is poor or the user is in offline mode, full caching is performed, pre-caching the task definition, configuration parameters, prerequisite data, and data description information locally. If the network quality is good, lightweight caching is performed, caching only the task metadata, while business data is retrieved in real time as needed.

[0066] In offline mode, users can process tasks and temporarily store results data, which carries a local timestamp and a nested traceability code for the corresponding node. Upon connecting to the network, asynchronous data transmission is performed. Transmitted data in fixed-size blocks, with a checksum generated for each block. Preferably, each 1MB block can be configured, and each block uses an MD5 algorithm to generate a checksum. If transmission is interrupted, the next transmission will resume from the last successfully confirmed data block, enabling breakpoint resumption.

[0067] When sending data back, a conflict check is performed. If the corresponding node on the server is in a pending state, the data is received directly and the node status is updated. If the corresponding node on the server is already in a processing or completed state, the timestamps of both sides are compared. If the offline result is updated, the server data is overwritten; otherwise, the offline result is discarded and an alarm is pushed. Manual intervention to resolve conflicts is supported.

[0068] Upon tool startup, context-aware injection is performed, automatically generating an encrypted runtime context token and passing it to the tool. The runtime context token contains a task identifier, historical processing data, node configuration, permission information, and a nested traceability code. After startup, the tool prioritizes parsing the runtime context token. If parsing is successful, it directly enters the processing state of the corresponding task without requiring manual parameter input. If parsing fails, the tool refuses to start or enters manual mode, and reports an error message.

[0069] This step utilizes offline pre-caching and asynchronous data return mechanisms to ensure normal task processing in weak network and offline scenarios, preventing network fluctuations from impacting business operations. Context-aware injection enables one-click tool startup, eliminating manual parameter entry and improving task execution efficiency and seamless workflow.

[0070] In one embodiment, the method further includes adaptive process anomaly control. This step spans the entire execution cycle of the task instance. Relying on the process topology generated in step 202 and the tool health scheduling capability in step 201, the status of running nodes is adjusted in real time, which is the core control link to ensure stable process execution. The running status of each node is collected in real time and synchronized globally. When a node anomaly is detected, the root cause is located based on the fault causal knowledge graph and a hierarchical self-healing repair is performed. During the node's operation, the remaining execution time is calculated through a predictive model. If the time exceeds the warning threshold, priority scheduling is actively triggered.

[0071] The input for this step is the real-time running status data of each node in the process. It mainly completes intelligent diagnosis of the root causes of anomalies, hierarchical self-healing repair and predictive process advancement. Finally, the processing status and execution logs are synchronously written back to the corresponding system modules, the self-healing results are synchronously updated to update the status of process nodes and the health profile of tools, and the processing logs are connected to the full-link traceability system.

[0072] During the implementation of this step, the four running states of each node—not started, in progress, completed, and abnormal—are collected in real time and synchronized globally. Once a node completes its execution normally, it automatically triggers the start of the next node, propelling the process forward continuously.

[0073] When an abnormal node status change is detected, the causal root cause diagnosis process is initiated. Status logs, API call records, and performance metrics of the abnormal node and its upstream and downstream components within a preset time period are collected, and the observed anomalies are input into the causal inference engine. Based on a pre-built fault causal knowledge graph, the causal inference engine calculates the posterior probability of each potential root cause and outputs the three root causes with the highest probabilities and their corresponding remediation strategies.

[0074] The system performs tiered self-healing repair based on the severity of the anomaly. Level 1 self-healing involves automatic retries using an exponential backoff strategy, suitable for network timeouts and temporary service unavailability. Preferably, retries can be performed at intervals of 1 second, 2 seconds, and 4 seconds, with a maximum of 3 retries. Level 2 self-healing involves bypassing the process, switching to a backup tool instance with similar health, and migrating the task context. This is suitable for single-tool instance crashes, and the switching operation synchronously writes the call results back to the structured tool resource library, updating the overall health score of the corresponding instance. Level 3 self-healing involves resource scheduling, requesting higher priority resource quotas for the process instance, suitable for scenarios with insufficient computing resources. Level 4 self-healing suspends the process, generates a detailed diagnostic report, and pushes it to operations personnel. This is suitable for scenarios with unknown root causes or where the first three levels of self-healing have failed.

[0075] During normal node execution, a predictive propagation mechanism is initiated. Based on the node's historical completion time, current input data volume, and downstream node load, a lightweight regression model is trained to continuously predict the node's remaining execution time. If the predicted remaining time exceeds the warning proportion of the SLA-committed time, a potential delay warning is sent, and priority safeguard actions are proactively triggered to allocate resources to ensure the execution of the critical path.

[0076] All anomaly handling and early warning operations generate processing logs with associated traceability codes, which are synchronously stored in the system log library. The log data is synchronously associated with the full-link data traceability index, forming a complete closed loop with the result data audit logs, supporting subsequent auditing and traceability.

[0077] This step achieves automatic location and hierarchical self-healing of the root causes of anomalies through causal inference, upgrading traditional passive alarms to proactive repair and significantly reducing anomaly handling time. A predictive advancement mechanism intervenes in potential delays in advance, ensuring timely completion of the process and achieving proactive adaptive control throughout the entire process.

[0078] In one embodiment, the method further includes differential control and compliance traceability of output data. This step takes over the execution output of each process node, and, in conjunction with the aforementioned generated nested traceability codes, executes at the completion stage of each node and the end-of-process stage to achieve compliant flow and full lifecycle traceability of output data. In this step, a data evolution differential version management mechanism is used to store node output data, recording only the change sets of adjacent versions; sensitive fields are automatically identified before data transmission, and adaptive desensitization processing is performed in conjunction with the target node's permissions; a cross-node data association index is constructed based on the nested traceability codes to generate a fully auditable traceability log.

[0079] The input for this step is the output data generated by each node. It completes differential version management, compliant desensitization and transmission, and full-chain traceability. The final output is a compliant and traceable output data chain. The traceability index is associated with the log data of each link of approval, execution, and exception handling to form a complete traceability system for the entire life cycle of the task.

[0080] In the specific implementation of this step, after the node execution is completed, the tool automatically uploads the access information and description information of the result data. The result data carries the nested traceability code identifier of the corresponding node and is sequentially transmitted between upstream and downstream nodes as the data flows. A data evolution differential version management mechanism is used to store the result data. The initial version stores a snapshot of the full data, and subsequent versions only store the change set of the previous version. The change set includes a list of newly added record identifiers, a list of deleted record identifiers, and a mapping of the old and new values ​​of the fields of the modified records.

[0081] When it is necessary to revert to a historical version, starting from the initial full snapshot, all changesets are sequentially applied to the target version to restore the complete data of the corresponding version. Preferably, a snapshot generation strategy can be configured to generate a full snapshot every fixed number of versions, balancing storage usage and version reconstruction speed.

[0082] Before data transmission, compliance checks are performed, automatically identifying sensitive fields in the output data based on regular expression matching, keyword matching, and semantic recognition models. Dynamic de-identification strategies are executed according to the security level and data sensitivity level of the target node. If the security level of the target node is lower than the data sensitivity level, de-identification must be performed. De-identification methods include four categories: masking, encryption, replacement, and generalization, which can be flexibly configured according to business scenarios.

[0083] The data anonymization strategy is transmitted synchronously with the data link as metadata. Downstream nodes can view plaintext data when they have the corresponding permissions, and automatically apply the anonymization rules when they do not have the corresponding permissions.

[0084] A cross-node data association index is built based on nested traceability codes. It records the source node traceability code, destination node traceability code, set of transmitted fields, de-identification encryption algorithm, timestamp and operator of each data transmission, and stores it in a blockchain or tamper-proof database to form an immutable audit log. The index is synchronously associated with task approval records, node execution logs and exception handling records, supporting fine-grained traceability of the entire data lifecycle.

[0085] This step significantly reduces storage redundancy of output data and improves version backtracking efficiency through a differential version management mechanism. Adaptive data masking and transmission ensure the compliance of sensitive data flow and prevent cross-node data leakage. Full-link traceability indexing enables fine-grained auditability of data evolution, meeting data security and compliance control requirements.

[0086] It should be understood that, although this embodiment Figure 1 The steps are shown sequentially as indicated by the arrows, but they are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are performed; they can be executed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0087] Example 2 Based on the data workbench modular process adaptive control method in Embodiment 1, this embodiment discloses a data workbench modular process adaptive control system, such as... Figure 2 As shown, the modular workflow adaptive management and control system for the data workbench includes: a structured tool resource library construction module 401, a workflow template generation module 402, and a task instance hot migration module 403, wherein: The structured tool resource library construction module 401 is used to build a standardized tool access interface system, collect structured attribute information of external tools and generate unique identifiers, obtain the field parameters, data types, measurement units and business meanings of tool input and output through the interface semantic self-description registration mechanism, build a semantic mapping knowledge base and adaptively encapsulate the tool's native interface to generate a unified and general interactive interface. At the same time, it collects the running status indicators of each tool instance in real time to generate a health profile and build a structured tool resource library.

[0088] The process template generation module 402 is used to construct a standardized process node model that includes a start node, task node, gateway node and end node. Based on the input and output data dependencies of each tool in the structured tool resource library, it generates node connection relationships and initial process topology. After link connectivity and tool compatibility verification, it generates process templates, performs multi-version hierarchical management of process templates, and keeps process template definitions and runtime task instances independent of each other.

[0089] The task instance hot migration module 403 is used to suspend the execution state and execution context snapshot of the target task instance that is in the running state when the process template triggers a version update, perform the mapping conversion of the execution instance state according to the node definition of the updated process template, restore the running state of the target task instance after the conversion is completed, and complete the hot migration of the task instance.

[0090] In this embodiment, the specific working process and working principle of the structured tool resource library construction module 401, the process template generation module 402, and the task instance hot migration module 403 are the same as those in Embodiment 1, and therefore will not be described again in this embodiment. Each unit module can be implemented entirely or partially through software, hardware, or a combination thereof. Each unit module can be embedded in or independent of the processor in a computer device in hardware form, or it can be stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each of the above unit modules.

[0091] Example 3 like Figure 3 The diagram illustrates a computer device disclosed in this embodiment, including a transmitter, a receiver, a memory, and a processor. The transmitter is used to send instructions and data, the receiver is used to receive instructions and data, the memory is used to store computer execution instructions, and the processor is used to execute the computer execution instructions stored in the memory to implement the method in Embodiment 1 above.

[0092] It is important to note that the aforementioned memory can be either standalone or integrated with the processor. When the memory is set up independently, the terminal device also includes a bus for connecting the memory and the processor.

[0093] Example 4 This embodiment discloses a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the method in Embodiment 1 above.

[0094] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0095] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0096] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A modular workflow adaptive control method for data workbench, characterized in that, The method includes: Establish a standardized tool access interface system, collect structured attribute information of external tools and generate unique identifiers, obtain the field parameters, data types, measurement units and business meanings of tool input and output through the interface semantic self-description registration mechanism, build a semantic mapping knowledge base and adaptively encapsulate the tool native interfaces to generate a unified and general interaction interface, and collect the running status indicators of each tool instance in real time to generate a health profile and build a structured tool resource library. A standardized process node model is constructed, including start node, task node, gateway node and end node. Based on the input and output data dependencies of each tool in the structured tool resource library, node connection relationships and initial process topology are generated. After link connectivity and tool compatibility verification, a process template is generated. The process template is managed in a multi-version hierarchical manner, and the process template definition and the running task instance are kept independent of each other. When a process template triggers a version update, the execution state of the target task instance in the running state is suspended and a snapshot of the running context is taken. The mapping transformation of the execution instance state is performed according to the node definition of the updated process template. After the transformation is completed, the running state of the target task instance is restored, and the hot migration of the task instance is completed.

2. The data workbench modular process adaptive control method according to claim 1, characterized in that, Adaptive encapsulation of the tool's native interfaces, including: Obtain the output field information of upstream tools and the input field information of downstream tools, input the semantic mapping knowledge base for business concept matching, and output the field mapping pairing results and the corresponding standard conversion rule identifiers; Based on the field mapping pairing results and the standard conversion rule identifier, the unit conversion rule and data type conversion rule are called in sequence to generate the corresponding data conversion adaptation script; The adaptation script is inserted between the interaction links of upstream and downstream tools to complete the adaptive encapsulation of heterogeneous tool interfaces and generate a unified and universal interaction interface.

3. The data workbench modular process adaptive control method according to claim 2, characterized in that, When performing business concept matching on the semantic mapping knowledge base, if no business concept relationship is found for the corresponding field or if there is a conversion rule conflict, the following processing is performed: Initiate a field matching negotiation request to the tool registry center and mark the current interaction link between upstream and downstream tools as pending confirmation. The candidate field mapping relationship and corresponding conversion rule are generated and pushed to the review end. The official mapping relationship and conversion rule after review and confirmation are received and stored in the semantic mapping knowledge base. Based on the formal mapping relationship and transformation rules, a data transformation adaptation script is regenerated to complete the adaptive encapsulation of heterogeneous tool interfaces.

4. The data workbench modular process adaptive control method according to claim 1, characterized in that, Real-time collection of runtime status metrics for each tool instance generates health profiles, and a structured tool resource library is built, including: The response latency, call success rate, operating load and availability of each tool instance are collected according to a preset time window. The response latency and operating load are respectively subjected to maximum value normalization processing, and the corresponding standardized indicator set is output. By using pre-set dynamic weighting coefficients, a weighted summation calculation is performed on each indicator in the standardized indicator set to obtain the comprehensive health score of each tool instance. The comprehensive health score is then associated and bound with the unique identifier, general interactive interface information, and attribute information of the corresponding tool instance to generate a health profile of a single tool instance. The health profiles of all tool instances are categorized and stored by tool type, and preset routing filtering thresholds, circuit breaker judgment conditions, and degradation processing rules are synchronously written to form a structured tool resource library.

5. The data workbench modular process adaptive control method according to claim 4, characterized in that, After a structured tool resource library is formed, when a tool call is triggered at a process node, dynamic scheduling of tool instances is performed based on the structured tool resource library, including: Receive tool call requests from process nodes, extract all tool instances of the corresponding type from the structured tool resource library, and filter out a set of candidate instances whose comprehensive health score is higher than the preset route filtering threshold; The target call instance is determined by performing a weighted random selection based on the comprehensive health score of each tool instance in the candidate instance set. The system receives the execution response data returned by the target call instance, extracts the call time and execution success / failure status, writes it back to the structured tool resource library, and updates the comprehensive health score of the corresponding tool instance in real time. At the same time, the system transmits the output result data in the execution response data to the process engine to promote the flow of subsequent process nodes. The number of failed calls and the failure rate of each tool instance are counted according to a preset time window. When the preset circuit breaker judgment condition is met, the corresponding tool instance is marked as circuit breaker and updated synchronously to the structured tool resource library. When the overall health score of all tool instances of the corresponding tool type is lower than the routing filtering threshold, the preset degradation processing rule is triggered.

6. The data workbench modular process adaptive control method according to any one of claims 1 to 5, characterized in that, Based on the input-output data dependencies of each tool in the structured tool resource library, node connection relationships and an initial process topology are generated, including: Extract the input field set and output field set of each tool from the structured tool resource library, and construct a directed graph of tool data dependency with the output field of the tool as the data producer and the input field as the data consumer. Traverse all tool nodes in the directed graph of tool data dependency. If the output field set of the upstream tool and the input field set of the downstream tool have a non-empty intersection between two tool nodes, then generate a serial connection relationship between the two tool nodes; if there is no data dependency relationship between the two tool nodes, then generate a parallel structure relationship between the two tool nodes. Identify the branching and aggregation scenarios in the directed graph of the tool's data dependency, add exclusive gateway nodes and aggregation gateway nodes accordingly, and generate an initial directed acyclic flow topology.

7. The data workbench modular process adaptive control method according to claim 6, characterized in that, After generating the initial directed acyclic flow topology, a flow template is generated after link connectivity and tool compatibility checks, including: Traverse all upstream and downstream tool nodes with serial connection relationships in the initial process topology, retrieve the input and output field information corresponding to each tool node, and verify whether the field business concepts of upstream and downstream tools are mappable based on the semantic mapping knowledge base to complete the tool compatibility verification. Perform link connectivity checks on the initial process topology that has completed tool compatibility verification, and traverse all nodes and connecting edges to check for the existence of circular dependency links; If a circular dependency or tool compatibility check fails, the corresponding abnormal node is marked and template generation is prohibited; if both checks pass, the formal process template is generated and stored.

8. The data workbench modular process adaptive control method according to any one of claims 1 to 5, characterized in that, Implement multi-version hierarchical management of process templates, including: Obtain the changes to the process template to be updated, compare the changes with the original process template, and determine the type of change. If the change type is a non-destructive change, the version revision number of the process template is incremented. Task instances that are already running will continue to run with the original version of the process template, while newly initiated task instances will use the updated process template. If the change type is a destructive change, the major or minor version number of the process template is incremented to trigger the hot migration process of the running task instance.

9. The data workbench modular process adaptive control method according to any one of claims 1 to 5, characterized in that, Complete the hot migration of task instances, including: A status suspension operation is performed on the target task instance that is in the running state, and the current node execution progress, generated intermediate processing data and node running status information of the target task instance are collected to generate a running context snapshot; The runtime context snapshot is matched with the node definition of the updated process template, and the instance state is mapped and transformed according to the node arrangement rules of the updated process template to generate transformed context data adapted to the updated process template. The transformed context data is loaded into the process execution environment corresponding to the updated process template to restore the running state of the target task instance. If the state restoration is successful, the task instance resources corresponding to the original version of the process template are destroyed. If the state restoration fails, the target task instance is rolled back to the running state before the state was suspended.

10. A modular workflow adaptive control system for a data workbench, characterized in that, The system includes: The structured tool resource library construction module is used to build a standardized tool access interface system, collect structured attribute information of external tools and generate unique identifiers, obtain the field parameters, data types, measurement units and business meanings of tool input and output through the interface semantic self-description registration mechanism, build a semantic mapping knowledge base and adaptively encapsulate the tool's native interface to generate a unified and general interaction interface. At the same time, it collects the running status indicators of each tool instance in real time to generate a health profile and build a structured tool resource library. The process template generation module is used to construct a standardized process node model that includes a start node, task node, gateway node, and end node. Based on the input and output data dependencies of each tool in the structured tool resource library, it generates node connection relationships and an initial process topology. After verifying the link connectivity and tool compatibility, it generates a process template. The process template is managed in a multi-version hierarchical manner, and the process template definition and the running task instance are kept independent of each other. The task instance hot migration module is used to suspend the execution state and runtime context snapshot of the target task instance that is in the running state when the process template triggers a version update. It performs a mapping transformation of the execution instance state according to the node definition of the updated process template. After the transformation is completed, the running state of the target task instance is restored, thus completing the hot migration of the task instance.

Images