An operation and maintenance task workflow arrangement system in a distributed graph database

Abstract

The application relates to a distributed graph database operation and maintenance task workflow arrangement system, wherein the system comprises: a workflow arrangement layer for recursively analyzing a user-input workflow specification to generate an executable DAG operation and maintenance task graph; a task management layer for adopting a two-phase commit protocol to ensure the atomic consistency of operation and maintenance tasks in the DAG operation and maintenance task graph in volatile memory and persistent storage, and based on the atomic consistency, performing fault recovery on uncompleted operation and maintenance tasks from the persistent storage in the case of memory data loss; and a conditional task execution layer for creating a secure isolation sandbox for the execution of operation and maintenance tasks in the DAG operation and maintenance task graph, and in the execution process of a current operation and maintenance task, evaluating a task execution result based on a dynamically updated conditional function to determine an execution path of a subsequent operation and maintenance task. Through the application, the problem of how to improve the workflow efficiency of operation and maintenance tasks in a graph database is solved.

Description

Technical Field

[0001] This application relates to the field of graph database technology, and in particular to a workflow orchestration system for operation and maintenance tasks in a distributed graph database. Background Technology

[0002] In modern enterprise applications, distributed graph database systems typically need to manage hundreds of service nodes, involving complex service dependencies and operational processes. As enterprises grow, the demand for large-scale graph data processing also increases, posing growing challenges to the operation and management of distributed graph database systems. Traditional task scheduling solutions suffer from the following technical problems:

[0003] The inability to dynamically select execution paths based on real-time running status leads to low task execution efficiency; complex dependencies between tasks make effective management difficult when serial and parallel tasks are executed in combination; and there is a lack of intelligent exception recovery and state management mechanisms for task execution failures or timeouts. Furthermore, there are problems such as difficulty in effectively managing task resource allocation and load balancing in a distributed environment, and a lack of real-time status tracking and visualization management of the task execution process.

[0004] Currently, no effective solution has been proposed for improving the workflow efficiency of operation and maintenance tasks in graph databases. Summary of the Invention

[0005] This application provides a workflow orchestration system for operation and maintenance tasks in a distributed graph database, which at least addresses the problem of how to improve the workflow efficiency of operation and maintenance tasks in graph databases in related technologies.

[0006] In a first aspect, embodiments of this application provide an operation and maintenance task workflow orchestration system in a distributed graph database, the system comprising a workflow orchestration layer, a task management layer, and a conditional task execution layer;

[0007] The workflow orchestration layer is used to receive the operation and maintenance task workflow specifications input by the user, and to recursively parse the workflow specifications to generate an executable DAG operation and maintenance task graph.

[0008] The task management layer is used to employ a two-phase commit protocol to ensure atomic consistency of each operation and maintenance task in the DAG operation and maintenance task graph in volatile memory and persistent storage. In the event of a failure that causes data loss in memory, the unfinished operation and maintenance tasks are recovered from persistent storage based on the atomic consistency.

[0009] The conditional task execution layer is used to create a secure isolation sandbox for the execution of various operation and maintenance tasks in the DAG operation and maintenance task graph. During the execution of the current operation and maintenance task, the task execution result is evaluated based on a dynamically updated condition function to determine the execution path of subsequent operation and maintenance tasks.

[0010] In some embodiments, the workflow orchestration layer is used to receive the operation and maintenance task workflow specification input by the user, and to abstract the serial branches, parallel branches and conditional branches in the workflow specification into a directed acyclic graph. Then, the Kahn algorithm is used to parse the dependencies and execution order of the operation and maintenance tasks, and to perform topological sorting on the directed acyclic graph to generate an executable DAG operation and maintenance task graph.

[0011] In some embodiments, the workflow orchestration layer is used to introduce multi-version concurrency control through the MVCC version control mechanism to support workflow specification updates during the execution of operation and maintenance tasks.

[0012] If the workflow specification is updated during the execution of an operation and maintenance task, the new version of the DAG operation and maintenance task graph is copied and recursively parsed to generate the new version. Then, atomic update of the version is performed through atomic.SwapPointer, so that the operation and maintenance task being executed continues to use the old version of the DAG operation and maintenance task graph, while subsequent operation and maintenance tasks that have not been executed use the new version of the DAG operation and maintenance task graph.

[0013] In some embodiments, the task management layer is configured to, when the task status of an operation and maintenance task in the DAG operation and maintenance task graph is updated, first write a preparation update log to the distributed storage during the preparation phase of the two-phase commit protocol, and then atomically update the task status of the operation and maintenance task in volatile memory and the persistent storage during the commit phase of the two-phase commit protocol. After the update is successful, the preparation update log is deleted.

[0014] In the event of a failure that results in the loss of memory data, the unfinished maintenance tasks can be recovered from persistent storage based on the atomic update feature.

[0015] In some embodiments, the task management layer is used to support high-concurrency execution of various operation and maintenance tasks in the DAG operation and maintenance task graph through a Go coroutine-based task concurrency execution mechanism.

[0016] In some embodiments, the conditional task execution layer is used to create a Wasm security isolation sandbox through a dynamically loaded Wasm plugin, or a Cgroups security isolation sandbox through a dynamically loaded Go plugin, for the execution of various operation and maintenance tasks in the DAG operation and maintenance task graph.

[0017] In some embodiments, the conditional task execution layer is used to evaluate the task execution result based on dynamically updated conditional functions during the execution of the current operation and maintenance task, so as to determine the execution path of subsequent operation and maintenance tasks. The conditional functions support complex logic combinations and high-order function programming, the update method of the conditional functions is hot update, and the writing languages ​​of the conditional functions include Rust, C++, and TinyGo.

[0018] In some embodiments, the system includes a user interface layer;

[0019] The user interface layer provides multiple interaction methods for users to input operation and maintenance task workflow specifications, including a web console, REST API, and command-line tools.

[0020] In some embodiments, the system includes a storage layer and a resource management layer, the storage layer comprising persistent distributed storage units;

[0021] The storage layer is used to persistently store data related to operation and maintenance tasks, and supports transaction logs and checkpoint mechanisms.

[0022] The resource management layer is used to intelligently schedule hardware resources for the execution of operation and maintenance tasks in order to perform load balancing.

[0023] In some embodiments, the system includes a graph database cluster, which is a cluster of the NebulaGraph distributed graph database.

[0024] Compared to related technologies, this application provides a distributed graph database operation and maintenance task workflow orchestration system. This system includes: a workflow orchestration layer, used to recursively parse user-input workflow specifications to generate an executable DAG operation and maintenance task graph; a task management layer, used to employ a two-phase commit protocol to ensure atomic consistency of each operation and maintenance task in the DAG operation and maintenance task graph across volatile memory and persistent storage. In the event of memory data loss, it performs fault recovery of incomplete operation and maintenance tasks from persistent storage based on atomic consistency; and a conditional task execution layer, used to create a secure isolation sandbox for the execution of each operation and maintenance task in the DAG operation and maintenance task graph. During the execution of the current operation and maintenance task, it evaluates the task execution result based on dynamically updated conditional functions to determine the execution path of subsequent operation and maintenance tasks. This achieves automatic state recovery of operation and maintenance tasks after a fault restart, avoids global rollback, and the sandboxed execution environment effectively prevents malicious code damage. The system uses conditional functions to evaluate the task running status in real time to dynamically select the execution path, improving task execution efficiency and solving the problem of how to improve the workflow performance of operation and maintenance tasks in a graph database. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0026] Figure 1 This is a structural block diagram of an operation and maintenance task workflow orchestration system according to an embodiment of this application;

[0027] Figure 2 This is a schematic diagram of the workflow orchestration layer according to the embodiments of the application;

[0028] Figure 3 This is a flowchart illustrating the task management layer according to the embodiments of the application;

[0029] Figure 4 This is a schematic diagram of the workflow of the conditional task execution layer according to the application embodiment;

[0030] Figure 5 This is a structural block diagram of an operation and maintenance task workflow orchestration system according to a specific embodiment of this application;

[0031] Figure 6 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0033] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0034] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0035] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0036] This application provides an embodiment of an operation and maintenance task workflow orchestration system in a distributed graph database. Figure 1 This is a structural block diagram of the operation and maintenance task workflow orchestration system according to an embodiment of this application, such as... Figure 1 As shown, the system includes a workflow orchestration layer, a task management layer (TaskManager), and a conditional task execution layer (ConditionalTask).

[0037] The workflow orchestration layer is used to receive user-inputted operation and maintenance task workflow specifications and recursively parse the workflow specifications to generate an executable DAG operation and maintenance task graph.

[0038] Specifically, the workflow orchestration layer receives the operation and maintenance task workflow specification (WorkflowSpec) input by the user, and abstracts the serial branches, parallel branches and conditional branches in the workflow specification into a directed acyclic graph. Then, the Kahn algorithm is used to parse the dependencies and execution order of the operation and maintenance tasks, and the directed acyclic graph is topologically sorted to generate an executable DAG operation and maintenance task graph.

[0039] It is also used to introduce multi-version concurrency control through the MVCC version control mechanism to support workflow specification updates during the execution of operation and maintenance tasks: if the workflow specification is updated during the execution of operation and maintenance tasks, the new version of the DAG operation and maintenance task graph is copied and recursively parsed to generate the new version, and then atomic update of the version is performed through atomic.SwapPointer, so that the operation and maintenance tasks being executed continue to use the old version of the DAG operation and maintenance task graph, and the subsequent unexecuted operation and maintenance tasks use the new version of the DAG operation and maintenance task graph.

[0040] It should be noted that existing workflow engines mostly rely on statically defined task graphs when handling dependencies. Although some systems support dynamic adjustment of workflows at runtime, such as by returning a nested WorkflowDAG or using a "rewrite" mechanism to add new dependencies to running tasks, these operations are often heavyweight and lack a lightweight dynamic dependency resolution and execution algorithm tightly integrated with resource scheduling and conditional judgment. In contrast, the workflow orchestration layer in this application implements efficient topology sorting and scheduling combined with heterogeneous resources for scenarios involving mixed serial, parallel, and conditional branching. Figure 2 This is a schematic diagram of the workflow orchestration layer according to the application embodiment, such as... Figure 2 As shown, the workflow orchestrator in the workflow orchestration layer is the brain of the system, responsible for parsing complex workflow definitions into executable task graphs and performing intelligent scheduling. Its technological innovations include: unified modeling based on DAG—abstracting serial, parallel, and conditional branching processes into a unified directed acyclic graph; topological sorting using the Kahn algorithm—efficiently resolving task dependencies and execution order; MVCC version control—supporting dynamic updates to workflow definitions at runtime, employing a copy-on-write mechanism; HEFT heuristic scheduling—combining a data locality-aware heterogeneous resource scheduling algorithm; and incremental dependency resolution—supporting dynamic addition or modification of task dependencies at runtime.

[0041] It should be further explained that, such as Figure 2As shown, the workflow parsing process recursively parses the WorkflowSpec to construct a complete DAG graph. Kahn's algorithm topological sorting calculates the in-degree of all nodes, adds nodes with an in-degree of 0 to the ready queue, processes the ready queue in a loop, executes executable tasks in parallel, and dynamically updates the in-degree of subsequent nodes. The MVCC version control mechanism introduces multi-version concurrency control to support runtime workflow updates: Copy-on-Write—a new version is created when the workflow is modified, without affecting currently executing tasks; Atomic version switching—atomic version updates are achieved through atomic.SwapPointer; Snapshot reading—currently executing tasks continue to use the old version of the DAG, while new tasks use the new version.

[0042] Preferably, the core architecture code of the workflow orchestration layer is as follows:

[0043] type WorkflowEngine struct {

[0044] dagManager *DAGManager / / DAG manager

[0045] topologyManager *TopologyManager / / Topology sorting manager

[0046] versionManager *MVCCVersionManager / / Version Control Manager

[0047] scheduler *HEFTScheduler / / HEFT scheduler

[0048] taskManager *TaskManager / / Task Manager

[0049] logger logx.Logger

[0050] }

[0051] type WorkflowSpec struct {

[0052] ID string / / Unique identifier for workflow

[0053] Version int64 / / Version number

[0054] Type WorkflowType / / "serial" | "parallel" | "conditional" | "dag"

[0055] Rollback bool / / Rollback support

[0056] Tasks []*TaskSpec / / List of subtasks

[0057] Dependencies map[string][]string / / Task dependency graph

[0058] Metadata map[string]interface{} / / Metadata

[0059] }

[0060] type DAGNode struct {

[0061] TaskID string / / Task ID

[0062] TaskSpec *TaskSpec / / Task specification

[0063] Dependencies []string / / Prerequisite dependencies

[0064] Successors []string / / Subsequent tasks

[0065] Status DAGNodeStatus / / Node status

[0066] Priority int / / Priority (upward rank)

[0067] EstCost time.Duration / / Estimated execution time

[0068] }

[0069] type DAGManager struct {

[0070] nodes map[string]*DAGNode / / All nodes

[0071] adjList map[string][] string / / Adjacency list

[0072] inDegree map[string]int / / In-degree statistics

[0073] version int64 / / Current version

[0074] isImmutable bool / / Whether the graph is immutable

[0075] }

[0076] Preferably, the implementation code for topological sorting based on Kahn's algorithm is as follows:

[0077] type TopologyManager struct {

[0078] dagManager *DAGManager

[0079] readyQueue chan *DAGNode

[0080] logger logx.Logger

[0081] }

[0082] / / Kahn Algorithm Implementation

[0083] func (tm *TopologyManager) ExecuteTopologicalSort(dag *DAG) error {

[0084] / / 1. Calculate the in-degree of all nodes

[0085] inDegree := make(map[string]int)

[0086] for nodeID, node := range dag.nodes {

[0087] inDegree[nodeID] = len(node.Dependencies)

[0088] }

[0089] / / 2. Add nodes with an in-degree of 0 to the ready queue.

[0090] readyNodes := make([]*DAGNode, 0)

[0091] for nodeID, degree := range inDegree {

[0092] if degree == 0 {

[0093] readyNodes = append(readyNodes, dag.nodes[nodeID])

[0094] }

[0095] }

[0096] / / 3. Begin the topological sorting process

[0097] processedCount := 0

[0098] totalNodes := len(dag.nodes)

[0099] for len(readyNodes) > 0 {

[0100] / / Process all currently ready nodes in parallel

[0101] currentBatch := readyNodes

[0102] readyNodes = make([]*DAGNode, 0)

[0103] / / Calculate and sort the priorities of the tasks in the current batch.

[0104] tm.sortByPriority(currentBatch)

[0105] / / Execute the current batch of tasks in parallel

[0106] err := tm.executeBatchTasks(currentBatch)

[0107] if err != nil {

[0108] return fmt.Errorf("batch execution failed: %w", err)

[0109] }

[0110] processedCount += len(currentBatch)

[0111] / / Update the in-degree of subsequent nodes

[0112] for _, completedNode := range currentBatch {

[0113] for _, successorID := range completedNode.Successors {

[0114] inDegree[successorID]--

[0115] if inDegree[successorID] == 0 {

[0116] readyNodes = append(readyNodes, dag.nodes[successorID])

[0117] }

[0118] }

[0119] }

[0120] }

[0121] / / 4. Check if all nodes have been processed (detect circular dependencies)

[0122] if processedCount != totalNodes {

[0123] return fmt.Errorf("cycle detected in DAG: processed %d of %d nodes",

[0124] processedCount, totalNodes)

[0125] }

[0126] tm.logger.Infof("Successfully executed DAG with %d nodes",totalNodes)

[0127] return nil

[0128] }

[0129] / / Sort tasks by priority

[0130] func (tm *TopologyManager) sortByPriority(nodes []*DAGNode) {

[0131] sort.Slice(nodes, func(i, j int) bool {

[0132] return nodes[i].Priority > nodes[j].Priority / / Higher priority first

[0133] [[ID=4}}

[0134] }

[0135] / / Execute batch tasks in parallel

[0136] func (tm *TopologyManager) executeBatchTasks(nodes []*DAGNode) error{

[0137] var wg sync.WaitGroup

[0138] errChan := make(chan error, len(nodes))

[0139] for _, node := range nodes {

[0140] wg.Add(1)

[0141] go func(n *DAGNode) {

[0142] defer wg.Done()

[0143] tm.logger.Infof("Executing task: %s (priority: %d)", n.TaskID,n.Priority)

[0144] if err := tm.executeTask(n); err != nil {

[0145] errChan <- fmt.Errorf("task %s failed: %w", n.TaskID, err)

[0146] return

[0147] }

[0148] tm.logger.Infof("Task completed: %s", n.TaskID)

[0149] }(node)

[0150] }

[0151] wg.Wait()

[0152] close(errChan)

[0153] / / Check for errors

[0154] for err := range errChan {

[0155] if err != nil {

[0156] return err

[0157] }

[0158] }

[0159] return nil

[0160] }

[0161] Preferably, the implementation code for MVCC version control is as follows:

[0162] typeMVCCVersionManager struct {

[0163] currentVersion *atomic.Value / / Pointer to the current version of the DAG

[0164] versionHistory map[int64]*DAG / / Version history

[0165] mutex sync.RWMutex

[0166] logger logx.Logger

[0167] }

[0168] / / Copy-on-write update workflow

[0169] func (vm *MVCCVersionManager) UpdateWorkflowCopyOnWrite(modifications*WorkflowModification) error {

[0170] vm.mutex.Lock()

[0171] defer vm.mutex.Unlock()

[0172] / / 1. Get the current version

[0173] currentDAG := vm.currentVersion.Load().(*DAG)

[0174] / / 2. Create a deep copy

[0175] newDAG := vm.deepCopyDAG(currentDAG)

[0176] newVersion := time.Now().UnixNano()

[0177] newDAG.version = newVersion

[0178] / / 3. Application Modification

[0179] if err := vm.applyModifications(newDAG, modifications); err != nil {

[0180] return fmt.Errorf("failed to modify applys: %w", err)

[0181] }

[0182] / / 4. Verify the legitimacy of the new version

[0183] if err := vm.validateDAG(newDAG); err != nil {

[0184] return fmt.Errorf("invalid DAG after modifications: %w", err)

[0185] }

[0186] / / 5. Atomic update version pointer

[0187] vm.currentVersion.Store(newDAG)

[0188] vm.versionHistory[newVersion] = newDAG

[0189] / / 6. Clean up old versions (keep the 3 most recent versions)

[0190] vm.cleanupOldVersions()

[0191] vm.logger.Infof("Workflow updated to version %d with copy-on-write",newVersion)

[0192] return nil

[0193] }}

[0194] / / MARCH DAY

[0195] func(vm *MVCCVersionManager) deepCopyDAY(originalDAY *DAY) *DAY {

[0196] newDAY := &DAY{

[0197] nodes: make(map[string]*DAYNode),

[0198] adjList: make(map[string][]string),

[0199] inDegree : make ( map [ string ] int ) .

[0200] }}

[0201] / / Configure the key

[0202] for nodeID, originalNode := range originalDAY.nodes {

[0203] newNode := &DAGNode{

[0204] TaskID: originalNode.TaskID,

[0205] TaskSpec : originalNode . TaskSpec , / / Configure the original node

[0206] Dependencies: make([]string, len(originalNode.Dependencies)),

[0207] Successors: make([]string, len(originalNode.Successors)),

[0208] Status: originalNode.Status,

[0209] Priority: originalNode.Priority,

[0210] EstCost: originalNode.EstCost,

[0211] }

[0212] copy(newNode.Dependencies, originalNode.Dependencies)

[0213] copy(newNode.Successors, originalNode.Successors)

[0214] newDAG.nodes[nodeID] = newNode

[0215] }

[0216] / / Copy the adjacency list and in-degree information

[0217] for nodeID, neighbors := range originalDAG.adjList {

[0218] newDAG.adjList[nodeID] = make([]string, len(neighbors))

[0219] copy(newDAG.adjList[nodeID], neighbors)

[0220] }

[0221] for nodeID, degree := range originalDAG.inDegree {

[0222] newDAG.inDegree[nodeID] = degree

[0223] }

[0224] return newDAG

[0225] }

[0226] The task management layer is used to ensure atomic consistency of each operation and maintenance task in the DAG operation and maintenance task graph in volatile memory and persistent storage by adopting a two-phase commit protocol. In the event of a failure that causes data loss in memory, the incomplete operation and maintenance tasks are recovered from persistent storage based on atomic consistency.

[0227] Specifically, the task management layer is used to write a preparation update log to the distributed storage during the preparation phase of the two-phase commit protocol when the task status of the operation and maintenance task in the DAG operation and maintenance task graph is updated. Then, during the commit phase of the two-phase commit protocol, the task status of the operation and maintenance task is atomically updated in volatile memory and persistent storage. After the update is successful, the preparation update log is deleted.

[0228] In the event of a failure that results in the loss of memory data, the unfinished maintenance tasks can be recovered from persistent storage based on the atomic update feature.

[0229] It is also used to support high-concurrency execution of various operation and maintenance tasks in the DAG operation and maintenance task graph through a task concurrency execution mechanism based on Go coroutines.

[0230] It should be noted that traditional graph computing systems, such as the early Pregel, while employing checkpointing mechanisms, are typically globally blocking. Recovery may require all nodes to roll back to the previous checkpoint, resulting in a significant waste of computational resources. Although subsequent optimizations have been implemented, such as incremental checkpointing or lightweight checkpointing (LWCP) that decouples vertex updates, these optimizations are often specific to particular computational models. In contrast, the task management layer in this application implements a general atomic state recovery scheme that integrates with complex workflow orchestration and supports hybrid tasks. Figure 3 This is a flowchart illustrating the task management layer according to the application's embodiments, such as... Figure 3 As shown, the asynchronous task manager (TaskManager) in the task management layer is the cornerstone of stable system operation, responsible for task lifecycle management, state persistence, and fault recovery. Its technical innovations include: a task concurrency execution mechanism based on Go coroutines, supporting high-concurrency task processing; using the two-phase commit (2PC) protocol to ensure atomic consistency between memory and persistent state; a coordinated checkpointing mechanism to support distributed fault recovery; the HEFT heuristic scheduling algorithm combined with data locality awareness to achieve intelligent resource allocation; and a confined recovery mechanism to avoid global rollback and improve recovery efficiency.

[0231] Preferably, the core architecture code of the task management layer is as follows:

[0232] type WorkflowEngine struct {

[0233] dagManager *DAGManager / / DAG manager

[0234] topologyManager *TopologyManager / / Topology sorting manager

[0235] versionManager *MVCCVersionManager / / Version Control Manager

[0236] scheduler *HEFTScheduler / / HEFT scheduler

[0237] taskManager *TaskManager / / Task Manager

[0238] logger logx.Logger

[0239] }

[0240] type WorkflowSpec struct {

[0241] ID string / / Unique identifier for workflow

[0242] Version int64 / / Version number

[0243] Type WorkflowType / / "serial" | "parallel" | "conditional" | "dag"

[0244] Rollback bool / / Rollback support

[0245] Tasks []*TaskSpec / / List of subtasks

[0246] Dependencies map[string][]string / / Task dependency graph

[0247] Metadata map[string]interface{} / / Metadata

[0248] }

[0249] type DAGNode struct {

[0250] TaskID string / / Task ID

[0251] TaskSpec *TaskSpec / / Task specification

[0252] Dependencies []string / / Prerequisite dependencies

[0253] Successors []string / / Subsequent tasks

[0254] Status DAGNodeStatus / / Node status

[0255] Priority int / / Priority (upward rank)

[0256] EstCost time.Duration / / Estimated execution time

[0257] }

[0258] type DAGManager struct {

[0259] nodes map[string]*DAGNode / / All nodes

[0260] adjList map[string][] string / / Adjacency list

[0261] inDegree map[string]int / / In-degree statistics

[0262] version int64 / / Current version

[0263] isImmutable bool / / Whether the graph is immutable

[0264] }

[0265] Preferably, the implementation code for the two-phase commit protocol is as follows:

[0266] / / Two-phase commit implementation for state updates

[0267] func (tm *TaskManager) UpdateTaskStateAtomic(taskID string, newStatusTaskStatus, result *TaskResult) error {

[0268] / / Phase 1: Preparation Phase - Writing to Transaction Log

[0269] commitLog := &CommitLog{

[0270] TaskID: taskID,

[0271] FinalStatus: newStatus,

[0272] Result: result,

[0273] Timestamp: time.Now(),

[0274] LogID: utils.GenerateUUID(),

[0275] }

[0276] if err := tm.taskRepo.WriteCommitLog(commitLog); err != nil {

[0277] return fmt.Errorf("prepare phase failed: %w", err)

[0278] }

[0279] / / Phase 2: Commit Phase - Atomic updates to memory and persistent storage

[0280] tm.mutex.Lock()

[0281] defer tm.mutex.Unlock()

[0282] / / Update memory status

[0283] if taskState, exists := tm.runningTasks.Load(taskID); exists {

[0284] state := taskState.(*TaskState)

[0285] state.Status = newStatus

[0286] state.UpdateTime = time.Now()

[0287] state.Result = result

[0288] tm.runningTasks.Store(taskID, state)

[0289] }

[0290] / / Persisting the final state

[0291] if err := tm.taskRepo.CommitFinalState(commitLog); err != nil {

[0292] / / If the submission fails, it will be handled using the recovery mechanism.

[0293] tm.logger.Errorf("Commit phase failed, will be recovered: %v", err)

[0294] return err

[0295] }

[0296] / / Clean up the transaction log

[0297] tm.taskRepo.DeleteCommitLog(commitLog.LogID)

[0298] return nil

[0299] }

[0300] / / Recovery logic during system startup

[0301] func (tm *TaskManager) RecoverFromCommitLogs() error {

[0302] unfinishedLogs, err := tm.taskRepo.GetUnfinishedCommitLogs()

[0303] if err != nil {

[0304] return fmt.Errorf("failed to get unfinished commit logs: %w", err)

[0305] }

[0306] for _, log := range unfinishedLogs {

[0307] tm.logger.Infof("Recovering task %s from commit log", log.TaskID)

[0308] / / Re-execute the commit phase

[0309] if err := tm.taskRepo.CommitFinalState(log); err != nil {

[0310] tm.logger.Errorf("Failed to recover task %s: %v", log.TaskID, err)

[0311] continue

[0312] }

[0313] / / Clean up the recovery logs

[0314] tm.taskRepo.DeleteCommitLog(log.LogID)

[0315] tm.logger.Infof("Successfully recovered task %s", log.TaskID)

[0316] }

[0317] return nil

[0318] }

[0319] Preferably, the implementation code for the coordinated checkpoint mechanism is as follows:

[0320] type CheckpointManager struct {

[0321] nodeManager *NodeManager

[0322] logger logx.Logger

[0323] checkpointDir string

[0324] mutex sync.RWMutex

[0325] }

[0326] / / Initiate a global consistency checkpoint

[0327] func (cm *CheckpointManager) InitiateGlobalCheckpoint(workflowIDstring) error {

[0328] cm.logger.Infof("Initiating global checkpoint for workflow %s",workflowID)

[0329] / / 1. Send a checkpoint readiness signal to all participating nodes

[0330] nodes := cm.nodeManager.GetActiveNodes()

[0331] prepareChan := make(chan error, len(nodes))

[0332] for _, node := range nodes {

[0333] go func(n *NodeInfo) {

[0334] prepareChan <- cm.sendPrepareCheckpoint(n, workflowID)

[0335] }(node)

[0336] }

[0337] / / 2. Wait for all nodes to complete preparation

[0338] for i := 0; i < len(nodes); i++ {

[0339] if err := <-prepareChan; err != nil {

[0340] return fmt.Errorf("node prepare failed: %w", err)

[0341] }

[0342] }

[0343] / / 3. Once all nodes are ready, send a commit checkpoint signal.

[0344] checkpointID := fmt.Sprintf("cp_%s_%d", workflowID, time.Now().Unix())

[0345] commitChan := make(chan error, len(nodes))

[0346] for _, node := range nodes {

[0347] go func(n *NodeInfo) {

[0348] commitChan <- cm.sendCommitCheckpoint(n, checkpointID)

[0349] }(node)

[0350] }

[0351] / / 4. Wait for all nodes to complete their commits.

[0352] for i := 0; i < len(nodes); i++ {

[0353] if err := <-commitChan; err != nil {

[0354] return fmt.Errorf("node commit failed: %w", err)

[0355] }

[0356] }

[0357] / / 5. Record global checkpoints

[0358] return cm.recordGlobalCheckpoint(checkpointID, workflowID)

[0359] }

[0360] / / Limited recovery implementation

[0361] func (cm *CheckpointManager) PerformConfinedRecovery(failedNodeIDstring) error {

[0362] cm.logger.Infof("Performing confined recovery for node %s",failedNodeID)

[0363] / / 1. Get the latest global checkpoint

[0364] latestCheckpoint, err := cm.getLatestGlobalCheckpoint()

[0365] if err != nil {

[0366] return fmt.Errorf("failed to get latest checkpoint: %w", err)

[0367] }

[0368] / / 2. Only restore the state of the failed node.

[0369] if err := cm.restoreNodeFromCheckpoint(failedNodeID,latestCheckpoint); err != nil {

[0370] return fmt.Errorf("failed to restore node: %w", err)

[0371] }

[0372] / / 3. Reschedule running tasks on the failed node

[0373] runningTasks, err := cm.getRunningTasksOnNode(failedNodeID,latestCheckpoint)

[0374] if err != nil {

[0375] return fmt.Errorf("failed to get running tasks: %w", err)

[0376] }

[0377] for _, task := range runningTasks {

[0378] if err := cm.rescheduleTask(task); err != nil {

[0379] cm.logger.Errorf("Failed to reschedule task %s: %v", task.ID, err)

[0380] }

[0381] }

[0382] cm.logger.Infof("Confined recovery completed for node %s",failedNodeID)

[0383] return nil

[0384] }

[0385] Preferably, the implementation code of the HEFT scheduling algorithm is as follows:

[0386] type HEFTScheduler struct {

[0387] nodeManager *NodeManager

[0388] graphDB GraphDBInterface

[0389] logger logx.Logger

[0390] localityWeight float64 / / Data locality weight

[0391] }

[0392] / / Task sorting stage of the HEFT algorithm

[0393] func (hs *HEFTScheduler) CalculateUpwardRank(dag *DAG) map[string]int{

[0394] upwardRank := make(map[string]int)

[0395] visited := make(map[string]bool)

[0396] / / Depth-first traversal to calculate upward ranking

[0397] var dfs func(nodeID string) int

[0398] dfs = func(nodeID string) int {

[0399] if visited[nodeID] {

[0400] return upwardRank[nodeID]

[0401] }

[0402] visited[nodeID] = true

[0403] node := dag.GetNode(nodeID)

[0404] maxSuccessorRank := 0

[0405] for _, successorID := range node.Successors {

[0406] successorRank := dfs(successorID)

[0407] if successorRank > maxSuccessorRank {

[0408] maxSuccessorRank = successorRank

[0409] }

[0410] }

[0411] / / Ranking Up = Estimated Time of Current Task + Weight of Maximum Subsequent Path

[0412] upwardRank[nodeID] = int(node.EstCost.Seconds()) + maxSuccessorRank

[0413] return upwardRank[nodeID]

[0414] }

[0415] / / Calculate the upward ranking of all nodes

[0416] for nodeID := range dag.nodes {

[0417] dfs(nodeID)

[0418] }

[0419] return upwardRank

[0420] }

[0421] / / Processor selection phase of the HEFT algorithm

[0422] func (hs *HEFTScheduler) SelectOptimalNode(task *TaskSpec,availableNodes []*NodeInfo) (*NodeInfo, error) {

[0423] var bestNode *NodeInfo

[0424] var minFinishTime time.Duration = time.Duration(math.MaxInt64)

[0425] for _, node := range availableNodes {

[0426] / / Calculate the estimated execution time of the task on this node

[0427] execTime := hs.estimateExecutionTime(task, node)

[0428] / / Calculate data transfer time

[0429] transferTime := hs.calculateDataTransferTime(task, node)

[0430] / / Calculate data locality weight

[0431] localityBonus := hs.calculateLocalityBonus(task, node)

[0432] / / Total cost = Execution time + Transmission time - Locality bonus

[0433] totalCost := execTime + transferTime - time.Duration(float64(localityBonus)*hs.localityWeight)

[0434] / / Calculate the earliest completion time

[0435] earliestStartTime := hs.getEarliestStartTime(node)

[0436] finishTime := earliestStartTime + totalCost

[0437] if finishTime < minFinishTime {

[0438] minFinishTime = finishTime

[0439] bestNode = node

[0440] }

[0441] }

[0442] if bestNode == nil {

[0443] return nil, fmt.Errorf("no suitable node found for task %s", task.ID)

[0444] }

[0445] hs.logger.Infof("Selected node %s for task %s (finish time: %v)",

[0446] bestNode.ID, task.ID, minFinishTime)

[0447] return bestNode, nil

[0448] }

[0449] / / Data locality awareness

[0450] func (hs *HEFTScheduler) calculateLocalityBonus(task *TaskSpec, node*NodeInfo) time.Duration {

[0451] bonus := time.Duration(0)

[0452] / / Query graph database metadata to obtain data sharding information

[0453] shardInfo, err := hs.graphDB.GetShardInfo(task.DataRequirements)

[0454] if err != nil {

[0455] hs.logger.Warnf("Failed to get shard info: %v", err)

[0456] return bonus

[0457] }

[0458] / / Calculate the proportion of local data

[0459] localDataRatio := hs.calculateLocalDataRatio(shardInfo, node)

[0460] / / Local reward = Local data ratio * Maximum reward time

[0461] maxBonus := time.Duration(10 * time.Second) / / Maximum 10-second bonus

[0462] bonus = time.Duration(float64(maxBonus) * localDataRatio)

[0463] hs.logger.Debugf("Node %s locality bonus for task %s: %v (ratio: %.2f)",

[0464] node.ID, task.ID, bonus, localDataRatio)

[0465] return bonus

[0466] }

[0467] The conditional task execution layer is used to create a secure isolation sandbox for the execution of various operation and maintenance tasks in the DAG operation and maintenance task graph. During the execution of the current operation and maintenance task, the task execution result is evaluated based on dynamically updated conditional functions to determine the execution path of subsequent operation and maintenance tasks.

[0468] Specifically, the conditional task execution layer is used to create Wasm security isolation sandboxes through dynamically loaded Wasm plugins or Cgroups security isolation sandboxes through dynamically loaded Go plugins, for the execution of various operation and maintenance tasks in the DAG operation and maintenance task graph.

[0469] It is also used to evaluate the execution results of the current operation and maintenance task based on dynamically updated condition functions, so as to determine the execution path of subsequent operation and maintenance tasks. The condition functions support complex logic combinations and high-order function programming. The update method of the condition functions is hot update. The writing languages ​​of the condition functions include Rust, C++ and TinyGo.

[0470] It should be noted that,

[0471] Traditional graph computing systems often introduce conditional branches into workflows, typically using simple if-else logic or dedicated branch operators (such as Airflow's BranchPythonOperator). However, this introduces two deep-seated problems: if the conditional logic is provided by the user, the code may contain malicious operations or defects, posing serious security risks if executed directly on the master or worker nodes; traditional methods usually hardcode conditional logic into the workflow definition, lacking the ability to dynamically load, update, and inject new conditional logic at runtime. In contrast, the conditional task execution layer in this application implements dynamic updates and flexible expansion of conditional logic through a pluggable architecture; sandboxed execution effectively reduces the security risks of user-defined code, while also supporting WebAssembly multi-language programming, offering high flexibility. Figure 4 This is a schematic diagram of the workflow of the conditional task execution layer according to the application embodiment, such as... Figure 4As shown, the ConditionalTask ​​executor in the ConditionalTask ​​execution layer is the core component of the system's intelligent decision-making, supporting secure dynamic condition judgment and branch selection. Its technological innovations are: pluggable architecture—supporting runtime dynamic loading of WebAssembly (Wasm) and native Go plugins; sandboxed execution—a secure isolated execution environment based on Linux namespaces and Cgroups; functional programming paradigm—supporting complex logic combinations (AND, OR, NOT) and higher-order functions; hot update mechanism—supporting runtime updates of conditional logic without requiring a system restart; and multi-language support—conditional functions can be written in languages ​​such as Rust, C++, and TinyGo.

[0472] Further explanation is needed regarding the sandboxed execution mechanism. When executing user-defined conditional functions, ConditionalTask ​​places them in a strictly restricted sandbox environment: namespace isolation—creating independent PID, Network, User, and Mount namespaces; resource limitations—limiting CPU, memory, and I / O usage through Cgroups; system call filtering—using seccomp-bpf to restrict executable system calls; and file system isolation—read-only mounting of necessary system files and prohibiting access to sensitive directories. As for dynamic plugin loading: the WebAssembly runtime—using Wasmtime to provide cross-platform, high-performance sandboxed execution; and Go native plugins—dynamically loading .so shared library files using plugin packages.

[0473] Preferably, the core architecture code of the conditional task execution layer is as follows:

[0474] typeConditionalParams struct {

[0475] Host string / / Target host

[0476] Condition ConditionFunc / / Condition evaluation function

[0477] CheckTask *TaskSpec / / Check the task

[0478] TrueTask *TaskSpec / / Execute when the condition is true

[0479] FalseTask *TaskSpec / / Execute when the condition is false

[0480] Description string / / Task description

[0481] PluginPath string / / Plugin path (Wasm or .so)

[0482] SandboxConfig *SandboxConfig / / Sandbox configuration

[0483] ResourceLimits *ResourceLimits / / Resource limits

[0484] }

[0485] / / Conditional function type definition

[0486] type ConditionFunc func(*agent.ShellResult) bool

[0487] / / Sandbox Configuration

[0488] type SandboxConfig struct {

[0489] EnableNetworkIsolation bool / / Network isolation

[0490] EnableFileSystemIsolation bool / / File system isolation

[0491] AllowedSyscalls []string / / Allowed system calls

[0492] TimeoutSeconds int / / Execution timeout

[0493] }

[0494] / / Sandbox Actuator Interface

[0495] type SandboxExecutor interface {

[0496] Execute(ctx context.Context, pluginPath string, input []byte) ([]byte, error)

[0497] CreateSandbox(config SandboxConfig) (*Sandbox, error)

[0498] DestroySandbox(sandbox *Sandbox) error

[0499] }

[0500] Preferably, the implementation code for the sandboxed execution mechanism of the conditional task is as follows:

[0501] / / Sandbox implementation based on Cgroups and Namespaces

[0502] type CgroupSandbox struct {

[0503] cgroupPath string

[0504] sandboxDir string

[0505] config SandboxConfig

[0506] logger logx.Logger

[0507] }

[0508] func (s *CgroupSandbox) Execute(ctx context.Context, pluginPathstring, input []byte) ([]byte, error) {

[0509] sandboxID := utils.GenerateUUID()

[0510] / / 1. Create and configure Cgroup

[0511] cgroupName := fmt.Sprintf("sandbox-%s", sandboxID)

[0512] if err := s.createCgroup(cgroupName); err != nil {

[0513] return nil, fmt.Errorf("failed to create cgroup: %w", err)

[0514] }

[0515] defer s.cleanupCgroup(cgroupName)

[0516] / / 2. Set resource limits

[0517] if err := s.configureCgroupLimits(cgroupName); err != nil {

[0518] return nil, fmt.Errorf("failed to configure cgroup limits: %w", err)

[0519] }

[0520] / / 3. Create a sandbox process

[0521] cmd := exec.CommandContext(ctx, pluginPath)

[0522] / / Set up a separate namespace

[0523] cmd.SysProcAttr = &syscall.SysProcAttr{

[0524] Cloneflags: syscall.CLONE_NEWPID | / / Process namespace

[0525] syscall.CLONE_NEWNET | / / Network namespace

[0526] syscall.CLONE_NEWUSER | / / User namespace

[0527] syscall.CLONE_NEWNS | / / Mount namespace

[0528] syscall.CLONE_NEWUTS | / / UTS namespace

[0529] syscall.CLONE_NEWIPC, / / IPC namespace

[0530] / / User namespace mapping (running as a non-root user)

[0531] UidMappings: []syscall.SysProcIDMap{

[0532] {ContainerID: 0, HostID: 65534, Size: 1}, / / Mapped to the nobody user

[0533] },

[0534] GidMappings: []syscall.SysProcIDMap{

[0535] {ContainerID: 0, HostID: 65534, Size: 1}, / / Mapped to the nobody group

[0536] },

[0537] }

[0538] / / 4. Configure seccomp system call filters

[0539] if err := s.configureSeccomp(cmd); err != nil {

[0540] return nil, fmt.Errorf("failed to configure seccomp: %w", err)

[0541] }

[0542] / / 5. Set input / output

[0543] cmd.Stdin = bytes.NewReader(input)

[0544] var stdout, stderr bytes.Buffer

[0545] cmd.Stdout = &stdout

[0546] cmd.Stderr = &stderr

[0547] / / 6. Start the process and add it to the Cgroup

[0548] if err := cmd.Start(); err != nil {

[0549] return nil, fmt.Errorf("failed to start sandbox process: %w", err)

[0550] }

[0551] if err := s.addProcessToCgroup(cgroupName, cmd.Process.Pid); err !=nil {

[0552] cmd.Process.Kill()

[0553] return nil, fmt.Errorf("failed to add process to cgroup: %w", err)

[0554] }

[0555] / / 7. Wait for execution to complete

[0556] if err := cmd.Wait(); err != nil {

[0557] s.logger.Errorf("Sandbox execution failed: %v, stderr: %s", err,stderr.String())

[0558] return nil, fmt.Errorf("sandbox execution failed: %w", err)

[0559] }

[0560] return stdout.Bytes(), nil

[0561] }

[0562] / / Create Cgroup

[0563] func (s *CgroupSandbox) createCgroup(name string) error {

[0564] cgroupPath := filepath.Join(" / sys / fs / cgroup", name)

[0565] / / Create Cgroup directory

[0566] if err := os.MkdirAll(cgroupPath, 0755); err != nil {

[0567] return fmt.Errorf("failed to create cgroup directory: %w", err)

[0568] }

[0569] s.cgroupPath = cgroupPath

[0570] return nil

[0571] }

[0572] / / Configure Cgroup resource limits

[0573] func (s *CgroupSandbox) configureCgroupLimits(name string) error {

[0574] cgroupPath := s.cgroupPath

[0575] / / Set memory limit (default 64MB)

[0576] memoryLimit := s.config.MemoryLimitMB

[0577] if memoryLimit == 0 {

[0578] memoryLimit = 64

[0579] }

[0580] if err := s.writeCgroupFile(cgroupPath, "memory.limit_in_bytes",

[0581] fmt.Sprintf("%d", memoryLimit*1024*1024)); err != nil {

[0582] return err

[0583] }

[0584] / / Set CPU limit (default 0.1 cores)

[0585] cpuQuota := s.config.CPUQuotaPercent

[0586] if cpuQuota == 0 {

[0587] cpuQuota = 10 / / 10%

[0588] }

[0589] if err := s.writeCgroupFile(cgroupPath, "cpu.cfs_quota_us",

[0590] fmt.Sprintf("%d", cpuQuota*1000)); err != nil {

[0591] return err

[0592] }

[0593] if err := s.writeCgroupFile(cgroupPath, "cpu.cfs_period_us", "100000"); err != nil {

[0594] return err

[0595] }

[0596] / / Set I / O limits

[0597] if err := s.writeCgroupFile(cgroupPath, "blkio.throttle.read_bps_device",

[0598] "8:0 10485760"); err != nil { / / 10MB / s read limit

[0599] return err

[0600] }

[0601] return nil

[0602] }

[0603] Preferably, the WebAssembly plugin provides higher-level sandbox protection at runtime, and its implementation code is as follows:

[0604] type WasmSandbox struct {

[0605] engine *wasmtime.Engine

[0606] store *wasmtime.Store

[0607] config WasmConfig

[0608] logger logx.Logger

[0609] }

[0610] func (ws *WasmSandbox) Execute(ctx context.Context, wasmPath string,input []byte) ([]byte, error) {

[0611] / / 1. Configure the Wasm engine

[0612] config := wasmtime.NewConfig()

[0613] config.SetCraneliftOptLevel(wasmtime.OptLevelNone) / / Disable optimization for improved security

[0614] config.SetConsumeFuel(true) / / Enable fuel consumption mechanism

[0615] engine := wasmtime.NewEngineWithConfig(config)

[0616] store := wasmtime.NewStore(engine)

[0617] / / 2. Set resource limits

[0618] store.AddFuel(ws.config.MaxInstructions) / / Maximum number of instructions

[0619] / / 3. Load the Wasm module

[0620] wasmBytes, err := os.ReadFile(wasmPath)

[0621] if err != nil {

[0622] return nil, fmt.Errorf("failed to read wasm file: %w", err)

[0623] }

[0624] module, err := wasmtime.NewModule(engine, wasmBytes)

[0625] if err != nil {

[0626] return nil, fmt.Errorf("failed to compile wasm module: %w", err)

[0627] }

[0628] / / 4. Creating restricted import functions

[0629] imports := ws.createLimitedImports(store)

[0630] / / 5. Instantiating the module

[0631] instance, err := wasmtime.NewInstance(store, module, imports)

[0632] if err != nil {

[0633] return nil, fmt.Errorf("failed to instantiate wasm module: %w", err)

[0634] }

[0635] / / 6. Retrieve the exported conditional function

[0636] conditionFunc := instance.GetFunc(store, "evaluate_condition")

[0637] if conditionFunc == nil {

[0638] return nil, fmt.Errorf("condition function not found in wasm module")

[0639] }

[0640] / / 7. Set a timeout for execution

[0641] resultChan := make(chan []byte, 1)

[0642] errorChan := make(chan error, 1)

[0643] go func() {

[0644] defer func() {

[0645] if r := recover(); r != nil {

[0646] errorChan <- fmt.Errorf("wasm execution panic: %v", r)

[0647] }

[0648] }()

[0649] / / Execute the conditional function

[0650] result, err := ws.executeConditionFunction(store, conditionFunc, input)

[0651] if err != nil {

[0652] errorChan <- err

[0653] return

[0654] }

[0655] resultChan <- result<able>

[0656] }()

[0657] / / 8. Wait for the result or timeout

[0658] select {

[0659] case result := <-resultChan:

[0660] return result, nil<able>

[0661] case err := <-errorChan:

[0662] return nil, err

[0663] case <- ctx.Done():

[0664] return nil, fmt.Errorf("wasm execution timeout")

[0665] }

[0666] }

[0667] / / Create restricted import functions

[0668] func (ws *WasmSandbox) createLimitedImports(store *wasmtime.Store)[]*wasmtime.Extern {

[0669] var imports []*wasmtime.Extern

[0670] / / Only provide safe logging functions

[0671] logFunc := wasmtime.NewFunc(store, wasmtime.NewFuncType(

[0672] []*wasmtime.ValType{wasmtime.NewValType(wasmtime.KindI32)},

[0673] []*wasmtime.ValType{},

[0674] ), func(caller *wasmtime.Caller, args []wasmtime.Val) ([]wasmtime.Val, *wasmtime.Trap) {

[0675] / / Secure logging implementation to prevent malicious output

[0676] ws.logger.Infof("Wasm log: %d", args[0].I32())

[0677] return []wasmtime.Val{}, nil

[0678] })

[0679] imports = append(imports, logFunc.AsExtern())

[0680] / / Other safe import functions can be added as needed.

[0681] return imports

[0682] }

[0683] Preferably, the implementation code for the dynamic plugin hot update mechanism is as follows:

[0684] type PluginManager struct {

[0685] pluginStore map[string]*Plugin

[0686] versionStore map[string]int64

[0687] updateChan chan PluginUpdateEvent

[0688] mutex sync.RWMutex

[0689] logger logx.Logger

[0690] }

[0691] / / Hot update plugin

[0692] func (pm *PluginManager) UpdatePlugin(pluginID string, newVersion []byte) error {

[0693] pm.mutex.Lock()

[0694] defer pm.mutex.Unlock()

[0695] / / 1. Verify the new plugin

[0696] if err := pm.validatePlugin(newVersion); err != nil {

[0697] return fmt.Errorf("plugin validation failed: %w", err)

[0698] }

[0699] / / 2. Create a new version

[0700] newVersionID := time.Now().Unix()

[0701] tempPath := fmt.Sprintf(" / tmp / plugin_%s_v%d", pluginID, newVersionID);

[0702] if err := os . WriteFile ( tempPath , newVersion , 0644 ); err != nil {

[0703] return fmt . Errorf ( " failed to write plugin file : %w " , error )

[0704] }}

[0705] / / 3. Enclosure

[0706] oldPlugin := pm.pluginStore[pluginID]

[0707] newPlugin := &Plugin{

[0708] ID: pluginID,

[0709] Version: newVersionID,

[0710] Path: tempPath,

[0711] LoadTime : time . Now ( )

[0712] }}

[0713] pm.pluginStore[pluginID] = newPlugin

[0714] pm.versionStore[pluginID] = newVersionID

[0715] / / 4. Default value

[0716] if oldPlugin != null { {

[0717] to func() {

[0718] time.Sleep(30 * time.Second) / / Wait for the running task to complete

[0719] os.Remove(oldPlugin.Path)

[0720] pm.logger.Infof("Cleaned up old plugin version: %s", oldPlugin.Path)

[0721] }()

[0722] }

[0723] pm.logger.Infof("Plugin %s updated to version %d", pluginID,newVersionID)

[0724] return nil

[0725] }

[0726] In some specific embodiments, Figure 5 This is a structural block diagram of an operation and maintenance task workflow orchestration system according to a specific embodiment of this application, such as... Figure 5 As shown, the system includes a user interface layer, a storage layer, a resource management layer, and a graph database cluster;

[0727] The user interface layer provides multiple interaction methods for users to input operation and maintenance task workflow specifications. These interaction methods include a web console, a REST API, and command-line tools.

[0728] The storage layer contains persistent distributed storage units; the storage layer is used to persistently store data related to operation and maintenance tasks, and supports transaction logs and checkpoint mechanisms; the resource management layer is used to intelligently schedule hardware resources for the execution of operation and maintenance tasks to achieve load balancing.

[0729] The graph database cluster is a cluster of NebulaGraph distributed graph databases, containing Meta nodes, Graph nodes, and Storage nodes.

[0730] This embodiment provides an electronic device including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the system embodiments described above.

[0731] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0732] Optionally, the electronic device may further include a processor, memory, network interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements an operation and maintenance task workflow orchestration system. The display screen may be a liquid crystal display (LCD) or an e-ink display. The input device may be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0733] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0734] Furthermore, in conjunction with the operation and maintenance task workflow orchestration system in the above embodiments, this application embodiment can provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements any of the operation and maintenance task workflow orchestration systems in the above embodiments.

[0735] In one embodiment, Figure 6 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 6 As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 6 As shown, the electronic device includes a processor, a network interface, internal memory, and non-volatile memory connected via an internal bus. The non-volatile memory stores the operating system, computer programs, and a database. The processor provides computing and control capabilities, the network interface communicates with external terminals via a network, the internal memory provides an environment for the operation of the operating system and computer programs, the computer programs are executed by the processor to implement an operational task workflow orchestration system, and the database stores data.

[0736] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0737] Those skilled in the art will understand that all or part of the processes in the systems described in the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0738] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0739] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A workflow orchestration system for operation and maintenance tasks in a distributed graph database, characterized in that, The system includes a workflow orchestration layer, a task management layer, and a conditional task execution layer; The workflow orchestration layer is used to receive the operation and maintenance task workflow specifications input by the user, and to abstract the serial branches, parallel branches and conditional branches in the workflow specifications into a directed acyclic graph. Then, the Kahn algorithm is used to parse the dependencies and execution order of the operation and maintenance tasks, and to perform topological sorting on the directed acyclic graph to generate an executable DAG operation and maintenance task graph. The task management layer is used to employ a two-phase commit protocol to ensure atomic consistency of each operation and maintenance task in the DAG operation and maintenance task graph in volatile memory and persistent storage. In the event of a failure that causes data loss in memory, the unfinished operation and maintenance tasks are recovered from persistent storage based on the atomic consistency. The conditional task execution layer is used to create a secure isolation sandbox for the execution of various operation and maintenance tasks in the DAG operation and maintenance task graph. During the execution of the current operation and maintenance task, the task execution result is evaluated based on a dynamically updated conditional function to determine the execution path of subsequent operation and maintenance tasks. The workflow orchestration layer is also used to introduce multi-version concurrency control through the MVCC version control mechanism to support workflow specification updates during the execution of operation and maintenance tasks: if the workflow specification is updated during the execution of operation and maintenance tasks, the new version of the DAG operation and maintenance task graph is copied and recursively parsed to generate the new version, and then atomic update of the version is performed through atomic.SwapPointer, so that the operation and maintenance tasks being executed continue to use the old version of the DAG operation and maintenance task graph, and the subsequent unexecuted operation and maintenance tasks use the new version of the DAG operation and maintenance task graph; The task management layer is specifically used to, when the task status of the operation and maintenance task in the DAG operation and maintenance task graph is updated, firstly, during the preparation phase of the two-phase commit protocol, write a preparation update log to the distributed storage, and then, during the commit phase of the two-phase commit protocol, atomically update the task status of the operation and maintenance task in volatile memory and the persistent storage. After the update is successful, delete the preparation update log. In the event of a failure that results in the loss of memory data, the unfinished maintenance tasks can be recovered from persistent storage based on the atomic update feature.

2. The system according to claim 1, characterized in that, The task management layer is used to support high-concurrency execution of various operation and maintenance tasks in the DAG operation and maintenance task graph through a Go coroutine-based task concurrency execution mechanism.

3. The system according to claim 1, characterized in that, The conditional task execution layer is used to create a Wasm security isolation sandbox through a dynamically loaded Wasm plugin, or a Cgroups security isolation sandbox through a dynamically loaded Go plugin, for the execution of various operation and maintenance tasks in the DAG operation and maintenance task graph.

4. The system according to claim 3, characterized in that, The conditional task execution layer is used to evaluate the task execution result based on dynamically updated conditional functions during the execution of the current operation and maintenance task, so as to determine the execution path of subsequent operation and maintenance tasks. The conditional functions support complex logic combinations and high-order function programming. The update method of the conditional functions is hot update. The writing languages ​​of the conditional functions include Rust, C++ and TinyGo.

5. The system according to claim 1, characterized in that, The system includes a user interface layer; The user interface layer provides multiple interaction methods for users to input operation and maintenance task workflow specifications, including a web console, REST API, and command-line tools.

6. The system according to claim 1, characterized in that, The system includes a storage layer and a resource management layer, wherein the storage layer contains persistent distributed storage units; The storage layer is used to persistently store data related to operation and maintenance tasks, and supports transaction logs and checkpoint mechanisms. The resource management layer is used to intelligently schedule hardware resources for the execution of operation and maintenance tasks in order to perform load balancing.

7. The system according to claim 1, characterized in that, The system includes a graph database cluster, which is a cluster of the NebulaGraph distributed graph database.

Images