A task scheduling method and system based on a hierarchical abstraction model

Abstract

The application relates to the field of cloud computing, in particular to a task scheduling method and system based on a hierarchical abstraction model. First, the application obtains task configuration information, generates double-track error code rules, action dependency relationship rules, exponential backoff retry rules and breakpoint snapshot metadata fields, then schedules and executes a plurality of stateless atomic actions according to the three types of rules, updates a local result aggregation table and a memory retry state dictionary according to action results during the execution process, and finally synchronizes the two types of memory data to a native pipeline breakpoint snapshot based on the predefined breakpoint snapshot metadata fields, so that the task scheduling process can be recovered based on the snapshot after the task is interrupted, the execution logic of the stateless atomic action does not need to be modified throughout the process, no additional external components are introduced, and the core execution and breakpoint resuming logic of the original architecture does not need to be modified.

Description

Technical Field

[0001] This application relates to the field of cloud computing, specifically to a task scheduling method and system based on a hierarchical abstraction model. Background Technology

[0002] In current task scheduling architectures, achieving state-dependent collaboration among stateless execution units across multiple nodes within a task typically employs centralized state storage components, locks, and waiting queues. This approach introduces additional external dependencies and system complexity, and suffers from lock contention, deadlocks, and the thundering herd effect, resulting in significant performance degradation in large-scale parallel scenarios. Other implementations, such as modifying execution units to have a stateful design, violate the layered decoupling principle of the orchestration architecture and the stateless atomicity of execution units, rendering them unusable. Furthermore, collaboration schemes based on retry mechanisms lose retry states and dependencies after task execution is interrupted. Resuming from a breakpoint requires re-executing all execution units, necessitating the addition of a separate state persistence engine or modification of the breakpoint resumption core logic in existing solutions, leading to intrusive modifications and architectural compatibility issues. Summary of the Invention

[0003] To address at least one of the problems existing in the prior art, this application provides a task scheduling method and system based on a hierarchical abstraction model. Without intruding on the core logic of the original architecture, it simultaneously achieves cross-node state coordination and breakpoint resume compatibility, reducing system complexity while ensuring execution performance and process reliability in large-scale parallel scenarios.

[0004] This application first obtains task configuration information, generates dual-track error code rules, action dependency rules, exponential backoff retry rules, and breakpoint snapshot metadata fields. Then, it schedules and executes multiple stateless atomic actions according to the three types of rules. During execution, it updates the local result aggregation table and the memory retry status dictionary based on the action results. Finally, based on the predefined breakpoint snapshot metadata fields, it synchronizes the two types of memory data to the native pipeline breakpoint snapshot. After the task is interrupted, the task scheduling process can be restored based on the snapshot. The entire process does not require modification of the execution logic of stateless atomic actions, does not introduce additional external components, and does not require modification of the core execution and breakpoint resume logic of the original architecture.

[0005] The first aspect of this application provides a task scheduling method based on a hierarchical abstraction model, the method comprising: Obtain task configuration information and generate dual-track error code rules, action dependency rules, exponential backoff retry rules, and breakpoint snapshot metadata fields. The dual-track error code rules are rules for dividing non-fatal wait error code fields and fatal execution error code fields. The breakpoint snapshot metadata fields are serializable fields used to store the task execution status. Based on the dual-track error code rule, the action dependency rule, and the exponential backoff retry rule, multiple stateless atomic actions are scheduled for execution. The local result aggregation table and the memory retry status dictionary are updated according to the action execution results. The stateless atomic action is the smallest execution unit of a single node operation. The local result aggregation table is a memory table that stores the action execution results. The memory retry status dictionary is a memory data structure that records the action retry progress. Based on the breakpoint snapshot metadata field, the memory retry status dictionary and the local result aggregation table are synchronized to the pipeline breakpoint snapshot, and the task scheduling is restored based on the pipeline breakpoint snapshot. The pipeline breakpoint snapshot is a persistent data structure that records the pipeline execution progress.

[0006] Optionally, the rule for generating dual-track error codes includes: The number ranges are divided into non-fatal wait error code fields and fatal execution error code fields; Configure the triggering rule corresponding to the non-fatal wait error code field as the return rule when the action dependency relationship is not satisfied; Configure the triggering rule corresponding to the fatal execution error code field as the return rule when the action execution is abnormal.

[0007] Optionally, the scheduling executes multiple stateless atomic actions, including: Parse the action dependency rules corresponding to stateless atomic actions and read the execution results of dependent actions from the local result aggregation table; Determine if the action dependency relationship is satisfied. If it is satisfied, inject the execution result of the dependent action into the execution parameter of the stateless atomic action and call the stateless atomic action to execute. If the condition is not met, the error code corresponding to the non-fatal wait error code field is returned, triggering an exponential backoff retry.

[0008] Optionally, the triggering of exponential backoff retry includes: Identify the error code field to which the returned error code belongs. If it is a non-fatal wait error code field, it will not be included in the preset maximum retry quota, and the retry interval will be updated according to the exponential backoff coefficient. If it is a fatal execution error code field, it will be included in the preset maximum retry quota, and a retry will be performed according to the preset rules; When the retry interval reaches the preset maximum retry interval, the retry is performed while maintaining the maximum retry interval.

[0009] Optionally, updating the retry interval according to the exponential backoff coefficient includes: When the dependent stateless atomic action is completed, receive a callback instruction indicating that the action has been completed; The current retry interval wait is terminated according to the callback instruction, and the retry of the corresponding stateless atomic action is immediately triggered. If the callback instruction is not received, a retry will be performed according to the updated retry interval.

[0010] Optionally, determining whether the action dependency relationship is satisfied includes: Parse the logical expression in the action dependency rule, wherein the logical expression contains at least one of AND, OR, and NOT logic; Read the execution results of all actions associated with the logical expression from the local result aggregation table; Based on the result of the logical expression, determine whether the action dependency relationship is satisfied.

[0011] Optionally, updating the memory retry status dictionary based on the action execution result includes: After each retry of an action, read the number of retries, the current retry interval, the last retry time, and the ready status of the dependencies for that action. The read state data is atomically written to the storage location of the corresponding action in the memory retry state dictionary; Align the data in the memory retry status dictionary with the breakpoint snapshot metadata field in terms of format.

[0012] Optionally, synchronizing the memory retry status dictionary and the local result aggregation table to the pipeline breakpoint snapshot includes: Receive pipeline snapshot trigger command and read data from the memory retry status dictionary and the local result aggregation table; The read data is written into the metadata area of ​​the pipeline breakpoint snapshot according to the format of the breakpoint snapshot metadata field; The unique identifier of the stateless atomic action that has been completed is recorded in the pipeline breakpoint snapshot.

[0013] Optionally, restoring task scheduling based on the pipeline breakpoint snapshot includes: Parse the pipeline breakpoint snapshot and read the data from the breakpoint snapshot metadata field; Reconstruct the memory retry state dictionary, action dependency rules, and list of stateless atomic actions that have been completed based on the read data; Filter out stateless atomic actions that have been completed, start execution coroutines for stateless atomic actions that are in a retry waiting state, and continue the retry process.

[0014] A second aspect of this application provides a task scheduling system based on a hierarchical abstraction model, the system comprising: The configuration management module is used to obtain task configuration information and generate dual-track error code rules, action dependency rules, exponential backoff retry rules, and breakpoint snapshot metadata fields. The dual-track error code rules are rules for dividing non-fatal wait error code fields and fatal execution error code fields. The breakpoint snapshot metadata fields are serializable fields used to store the task execution status. The task scheduling module is used to schedule the execution of multiple stateless atomic actions according to the dual-track error code rules, the action dependency rules, and the exponential backoff retry rules, and to update the local result aggregation table and the memory retry status dictionary according to the action execution results. The stateless atomic action is the smallest execution unit of a single node operation, the local result aggregation table is a memory table that stores the action execution results, and the memory retry status dictionary is a memory data structure that records the action retry progress. The state persistence module is used to synchronize the memory retry state dictionary and the local result aggregation table to the pipeline breakpoint snapshot based on the breakpoint snapshot metadata field, and to restore task scheduling based on the pipeline breakpoint snapshot. The pipeline breakpoint snapshot is a persistent data structure that records the pipeline execution progress.

[0015] As can be seen from the above technical solution, in actual use, the generated dual-track error code rules can distinguish between two scenarios: dependency incompatibility and execution failure. Combined with the pre-validation of action dependency rules, the dependency incompatibility scenario can be transformed into an exponential backoff retry triggered by the corresponding error code. Cross-action state coordination can be completed without setting locks and waiting queues. Moreover, dependency validation and retry scheduling are completed at the task layer, without modifying the execution logic of stateless atomic actions, thus preserving its stateless design. At the same time, the pre-generated breakpoint snapshot metadata field can open up the synchronization path between memory state and native pipeline snapshot. During runtime, the data of the memory retry state dictionary and the local result aggregation table are automatically synchronized to the extended area of ​​the native snapshot. There is no need to modify the original snapshot generation and recovery logic. After the task is interrupted, the corresponding memory data can be directly restored from the snapshot to continue execution without rerunning the completed actions. The entire process does not require the introduction of additional external components and has no lock contention. It can achieve cross-node state coordination and breakpoint resumption compatibility without intruding on the core logic of the original architecture, reducing system complexity while ensuring execution performance and process reliability in large-scale parallel scenarios. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a task scheduling method based on a hierarchical abstract model in an embodiment of this application.

[0018] Figure 2 This is a schematic diagram of the process for generating dual-track error code rules in the embodiments of this application.

[0019] Figure 3 This is a schematic diagram of the process of scheduling and executing multiple stateless atomic actions in an embodiment of this application.

[0020] Figure 4 This is a schematic diagram of the process for triggering exponential backoff retry in an embodiment of this application.

[0021] Figure 5 This is a schematic diagram of the process of updating the retry interval according to the exponential backoff coefficient in the embodiments of this application.

[0022] Figure 6 This is a flowchart illustrating the process of determining whether the action dependency relationship is satisfied in the embodiments of this application.

[0023] Figure 7 This is a schematic diagram of the process of updating the memory retry status dictionary based on the action execution result in the embodiments of this application.

[0024] Figure 8 This is a schematic diagram of the process of synchronizing the memory retry status dictionary and the local result aggregation table to the pipeline breakpoint snapshot in the embodiments of this application.

[0025] Figure 9 This is a schematic diagram of the process of restoring task scheduling based on the pipeline breakpoint snapshot in the embodiments of this application.

[0026] Figure 10 This is a schematic diagram of the structure of a task scheduling system based on a hierarchical abstraction model in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] In this application embodiment, the specific implementation method of the prior art is first described. For example, in the task scheduling scenario of a multi-node cluster, the prior art uses a centralized state storage component to achieve cross-node execution state coordination. Specifically, a third-party state storage component is deployed on the control node. After each execution unit completes its execution, it writes the execution result to this state storage component. Execution units with dependencies obtain the dependent execution results by polling this state storage component. When the dependent result is not ready, the execution unit enters a blocking waiting state based on locks and a waiting queue until the dependent result is ready and it is awakened to execute. Simultaneously, for task recovery after interruption, the prior art uses a newly added independent state persistence engine to separately record the retry status and dependencies of each execution unit. When the task resumes, the state data is read from this persistence engine to reconstruct the execution flow. It should be noted that in actual use, this existing technical solution will introduce additional external component dependencies, which poses a single point of failure risk. The design of locks and waiting queues will generate a large amount of CPU context switching overhead in parallel scenarios with thousands of nodes, resulting in a decrease in parallel performance. At the same time, the independent state persistence engine requires modification of the breakpoint resume logic of the original task scheduling architecture, which involves intrusive modification and architecture compatibility issues, and cannot adapt to the layered decoupling design principle of the original architecture.

[0029] like Figure 1 As shown, the first aspect of this application provides a task scheduling method based on a hierarchical abstraction model, including: S101, Obtain task configuration information, generate dual-track error code rules, action dependency rules, exponential backoff retry rules, and breakpoint snapshot metadata fields. The dual-track error code rules are rules for dividing non-fatal wait error code fields and fatal execution error code fields. The breakpoint snapshot metadata fields are serializable fields used to store the task execution status. S102, according to the dual-track error code rule, the action dependency rule and the exponential backoff retry rule, schedule and execute multiple stateless atomic actions, update the local result aggregation table and the memory retry status dictionary according to the action execution results, the stateless atomic action is the smallest execution unit of a single node operation, the local result aggregation table is a memory table that stores the action execution results, and the memory retry status dictionary is a memory data structure that records the action retry progress; S103, based on the breakpoint snapshot metadata field, synchronize the memory retry status dictionary and the local result aggregation table to the pipeline breakpoint snapshot, and restore task scheduling based on the pipeline breakpoint snapshot, wherein the pipeline breakpoint snapshot is a persistent data structure that records the pipeline execution progress.

[0030] In this embodiment, the task scheduling method based on a layered abstract model relies on a four-layer progressive architecture: action layer, task layer, module layer, and pipeline layer. The action layer corresponds to stateless atomic actions within the method; the task layer corresponds to the task scheduling and rule management entity within the method; the module layer corresponds to the upper-level business function encapsulation unit within the method; and the pipeline layer corresponds to the full-process orchestration and breakpoint resume entity within the method. It should be noted that all logic in this method converges within the native mechanisms of the task layer and pipeline layer, without modifying the execution logic of the action layer or intruding into the business orchestration logic of the module layer and pipeline layer. It adheres to the layered responsibility boundaries of the four-layer architecture, ensuring compatibility with existing architectures.

[0031] Specifically, obtaining task configuration information is the initial step of this method. The task configuration information is structured data input by the user through a structured configuration file or issued by an upper-level module. The format of the configuration information can be a serializable structured data format. The content of the configuration information includes the network address information of the target node, authentication information, the atomic operations to be performed by each node, a description of the dependencies between node operations, retry-related configuration parameters, and breakpoint resumption-related configuration parameters. Those skilled in the art will understand that the specific content and format of the task configuration information can be adjusted according to the actual application scenario, and this application embodiment does not impose a unique limitation on this. After obtaining the task configuration information, the configuration information will be validated for legality. The validation includes whether the format of the configuration information conforms to the specifications, whether the target node information is complete, whether there are circular dependencies in the dependency description, and whether the retry parameters are within a reasonable range. Only after the validation passes will the subsequent rule generation step proceed to avoid execution anomalies caused by incorrect configuration information.

[0032] Generating dual-track error code rules, action dependency rules, exponential backoff retry rules, and breakpoint snapshot metadata fields are the fundamental steps for this method to achieve lock-free collaboration and breakpoint resumption compatibility. These four rules and fields together constitute the execution rule system of this method. The design of each rule and field aligns with the native capabilities of the existing architecture without adding any additional core logic. It should be noted that the generation process of the four rules and fields is completed once during the task initialization phase, with no runtime overhead. The generated rules and fields remain fixed during task execution and can only be adjusted by modifying the task configuration information, ensuring the consistency and stability of the rules during execution.

[0033] Specifically, the dual-track error code rule divides the error code field into a non-fatal wait error code field and a fatal execution error code field. The purpose of this rule is to isolate the two sets of logic: state-waiting retries and fault-tolerant retries. This solves the problem in existing technologies where a single error code system cannot distinguish between the two scenarios, leading to state waits being misjudged as execution failures, or execution failures being retried indefinitely. The dual-track error code rule divides error codes into two isolated segments, each corresponding to different triggering scenarios, processing flows, and retry rules. These two segments operate independently and decoupled. The non-fatal wait error code field corresponds to scenarios where action dependencies are not yet ready. In this scenario, the action itself does not have an execution failure; it only needs to wait for the preceding dependent actions to complete. Therefore, error codes in this segment will not be marked as node execution failures, will not be recorded in fault logs, will not trigger alarms, and will not cause process termination. The fatal execution error code field corresponds to a real failure scenario that occurs during the execution of an action. In this scenario, the action cannot be completed normally and the native fault tolerance process needs to be triggered. Therefore, the error code in this range will be marked as an execution failure, a fault log will be recorded, an alarm will be triggered, and the native retry and process termination rules will be followed.

[0034] Action dependency rules define the execution dependencies between stateless atomic actions, specifying which other actions must complete before an action can execute. The rules are logical expressions based on native template syntax, supporting logical combinations of multiple dependencies and enabling dependency definitions of arbitrary complexity. The design of action dependency rules consolidates all dependency logic at the task layer. Actions themselves are completely unaware of dependencies, only responsible for executing atomic operations, ensuring a stateless design that aligns with the responsibility boundaries requirements of a layered architecture. It's worth noting that the parsing of action dependency rules fully reuses the original architecture's native template syntax engine and conditional judgment capabilities, requiring no additional dependency parsing logic and exhibiting zero intrusion into the task layer's scheduling framework, ensuring compatibility with the original architecture.

[0035] The exponential backoff retry rule defines parameters such as the triggering conditions for retry, the calculation method for the retry interval, the maximum number of retries, and the maximum retry interval. The core of the rule is to reuse the original exponential backoff retry mechanism of the existing architecture, redefining it from its original fault tolerance purpose to a lock-free state coordination scheduling component. The principle of exponential backoff retries is that the retry interval increases exponentially with the number of retries. This design can ensure real-time response with a small retry interval in the early stages when dependencies are not ready. As the waiting time increases, the retry interval gradually increases, avoiding the waste of CPU resources caused by frequent polling. This solves the problem that existing fixed-interval polling schemes cannot balance resource consumption and real-time response. It should be noted that all parameters in the exponential backoff retry rule can be customized through task configuration information, including the initial retry interval, maximum retry interval, backoff coefficient, and maximum number of retries, adapting to the real-time and resource consumption requirements of different scenarios.

[0036] The breakpoint snapshot metadata field is a serializable field used to store the task execution state. Its design purpose is to incorporate retry state data during task execution into the original pipeline breakpoint snapshot, achieving compatibility between the state coordination process and the native breakpoint resume capability. The design of the breakpoint snapshot metadata field follows the minimum dataset principle, retaining only the data necessary for breakpoint resume, with no redundant information. This avoids significantly increasing the snapshot size, adapts to the original snapshot storage and transmission mechanism, and avoids performance degradation caused by redundant design. Furthermore, the format of the breakpoint snapshot metadata field is compatible with the original pipeline breakpoint snapshot metadata specification, allowing direct writing to the original snapshot's metadata extension area without modifying the original snapshot's core structure, persistence logic, generation, and recovery process. This achieves zero intrusion into the pipeline layer code and maintains compatibility with the native breakpoint resume capability.

[0037] Based on the dual-track error code rules, action dependency rules, and exponential backoff retry rules, scheduling and executing multiple stateless atomic actions is the execution step of this method. This step fully reuses the original architecture's native coroutine pool parallel scheduling capability to achieve fully parallel execution of multiple stateless atomic actions. At the same time, through pre-dependency verification and dual-track error code processing, lock-free cross-node state collaboration is achieved, with no lock contention throughout the process, and the parallel performance is close to that of the native dependency-free task.

[0038] Specifically, a stateless atomic action is the smallest execution unit of a single operation on a single node, corresponding to the action layer in a layered architecture. Its design principle is that each instance is responsible for only a single atomic operation, without retaining execution context or storing execution state, and is destroyed after execution, ensuring the reusability and atomicity of the action. Specific types of stateless atomic actions include command execution, remote connection management, file transfer, and file retrieval. Each type corresponds to a typical atomic operation in node maintenance, as those skilled in the art will understand. The specific types of stateless atomic actions can be extended according to actual application scenarios, and this application does not limit them to a single type. It should be noted that this method does not modify the execution logic of stateless atomic actions at all. The action itself does not need to be aware of dependencies, retry rules, or scheduling logic; it only needs to complete the atomic operation according to preset parameters, preserving the stateless design of the action and conforming to the responsibility boundary requirements of a layered architecture.

[0039] The process of scheduling and executing multiple stateless atomic actions is handled by the task-layer coroutine pool scheduler. When a task starts, an independent execution coroutine is launched for each stateless atomic action of each target node, achieving fully parallel execution of multiple actions. After each action's coroutine starts, a pre-check of dependencies is performed before the action is executed, avoiding invalid operations and resource waste when dependencies are not satisfied. If the dependency check passes, the execution result of the dependent action is injected into the execution parameters of the current action, the action is executed, and the result is atomically written to the local result aggregation table after execution. If the dependency check fails, the error code corresponding to the non-fatal wait error code field is returned, triggering an exponential backoff retrieval. The entire process is completed entirely at the task layer, and the action itself is completely unaware of it.

[0040] Updating the local result aggregation table and the in-memory retry status dictionary based on the action execution results is the intermediate step in this method to achieve state coordination and breakpoint resumption. The local result aggregation table and the in-memory retry status dictionary are two in-memory data structures at the task layer, which are used to store action execution results and retry progress data, respectively, providing data support for dependency verification, retry scheduling and snapshot synchronization.

[0041] Specifically, the local result aggregation table is an in-memory table that stores the execution results of actions. It is a native in-memory data structure of the original architecture's task layer, used to store the execution results of all stateless atomic actions. Each action's execution result has a unique identifier, bound to the corresponding action and node. Other actions can read the corresponding execution result through this identifier. The local result aggregation table is designed as a thread-safe structure, supporting concurrent read-only access by multiple coroutines. Write operations are implemented atomically, avoiding data inconsistency issues caused by concurrent writes. It is lock-free and suitable for high-frequency access requirements in large-scale node parallel scenarios. It should be noted that this method completely reuses the original architecture's native local result aggregation table without modifying its core structure and read / write logic. It only reuses its read / write capabilities to implement the reading and writing of dependent results, with zero intrusion into the original architecture's logic.

[0042] The in-memory retry state dictionary is an in-memory data structure that records the retry progress of actions. It stores data such as the number of retries, current retry interval, last retry time, dependency readiness status, and corresponding dependency rules for each stateless atomic action. The data structure in the dictionary is fully aligned with the breakpoint snapshot metadata fields, ensuring direct writing during snapshot synchronization without additional format conversion. The in-memory retry state dictionary is updated atomically after each retry for each action, ensuring that the data in the dictionary is always consistent with the current execution state and providing accurate data support for subsequent retry scheduling and snapshot synchronization.

[0043] Based on the breakpoint snapshot metadata fields, the method synchronizes the memory retry status dictionary and local result aggregation table to the pipeline breakpoint snapshot, and restores the task scheduling based on the pipeline breakpoint snapshot. This step achieves compatibility with breakpoint resume capability. This step fully reuses the original breakpoint snapshot mechanism of the pipeline layer of the original architecture. It does not require the addition of additional persistent storage components or modification of the original breakpoint resume logic, thus achieving compatibility between the state coordination process and the native breakpoint resume capability.

[0044] Specifically, a pipeline breakpoint snapshot is a persistent data structure that records the execution progress of the pipeline. It's a native persistent data structure of the original architecture's pipeline layer, used to record execution progress during pipeline execution. If the pipeline is interrupted due to external reasons, execution can be resumed through this snapshot, avoiding the duplication of completed operations. The native pipeline breakpoint snapshot contains data such as the pipeline's global configuration, module execution status, node connection information, and a list of completed operations. It also reserves an extensible metadata area for storing custom extended data. This method incorporates retry status data into the native snapshot through this metadata area without modifying the core structure and persistence logic of the snapshot.

[0045] The process of synchronizing the in-memory retry status dictionary and local result aggregation table to the pipeline breakpoint snapshot fully reuses the original snapshot triggering mechanism of the existing architecture. Triggering timings include scheduled triggering, triggering after the execution phase is completed, and automatic triggering before pipeline interruption. No additional scheduled tasks or triggering logic are required, resulting in zero intrusion into the pipeline layer logic. Each time a snapshot is triggered, the task layer writes the real-time updated retry status dictionary, local result aggregation table data, and unique identifiers of completed actions from memory into the metadata area of ​​the pipeline breakpoint snapshot, following the format of the breakpoint snapshot metadata fields. Other snapshot data completely follows the generation logic of the original architecture without any modification, ensuring compatibility with the native snapshot mechanism.

[0046] By restoring task scheduling based on pipeline breakpoint snapshots, the original architecture's native breakpoint resume process is fully reused. When the pipeline restarts after an interruption, triggering breakpoint resume, the pipeline layer first natively parses the breakpoint snapshot to restore the pipeline's execution context. Then, the task layer reads standardized retry status data from the snapshot's metadata area, restores the retry status dictionary, action dependency rules, and completed action list in memory, filters completed actions, and starts execution coroutines for actions in the retry waiting state to continue the retry process before the interruption. The entire recovery process fully reuses the native breakpoint resume process, requiring no modification to the recovery logic code, with zero intrusion into the pipeline layer. It is also completely transparent to upper-layer modules and pipelines, requiring no modifications to the upper layers to use the breakpoint resume capability normally.

[0047] In one embodiment that can be implemented in this application, such as Figure 2 As shown, the rules for generating dual-track error codes include: S201, a number segment that divides the non-fatal wait error code field and the fatal execution error code field; S202, Configure the triggering rule corresponding to the non-fatal wait error code field as the return rule when the action dependency relationship is not satisfied; S203, Configure the triggering rule corresponding to the fatal execution error code field as the return rule when the action execution is abnormal.

[0048] In the embodiments of this application, the specific steps for generating dual-track error code rules are a sub-limitation of the dual-track error code rule generation process in the independent claims. By clearly defining the number segment division and trigger rule configuration, the state waiting retry and fault tolerance retry are decoupled, ensuring that the two sets of logic do not interfere with each other. At the same time, it conforms to the industry-standard error code specifications, is compatible with the error code system of the original architecture, and will not cause number segment conflicts.

[0049] Specifically, the numbering of the non-fatal wait error code domain and the fatal execution error code domain follows industry-standard error code specifications. Numbers 100 to 199 are typically used to represent informational status codes, not actual execution errors, while numbers 200 and above are typically used to represent actual errors during execution. This classification aligns with industry standards and avoids conflicts with existing error codes in the original architecture, resulting in stronger compatibility. Within the non-fatal wait error code domain, number 100 is the basic dependency not ready error code, used to represent scenarios where a single dependency is not ready. Numbers 101 to 199 can be used to define different wait scenarios, including multiple dependencies and logic not ready, multiple dependencies or logic not ready, node resource threshold not met, cluster execution progress not reaching threshold, etc., and can be extended according to actual application scenarios. In the Fatal Execution Error Code field, numbers 200 and above can be used to define different execution failure scenarios, including command execution failure, network connection error, node unreachability, authentication failure, file transfer failure, etc., which are aligned with the original error code system of the original architecture to ensure that the original fault tolerance capability is not affected.

[0050] The triggering rule for the non-fatal wait error code field is configured as the return rule when the action dependency is not satisfied. This triggering rule clarifies the timing of the non-fatal wait error code: when the action dependency check fails, the corresponding error code in the non-fatal wait error code field is returned, triggering the state wait retry process. The triggering rule is bound to the action dependency rule. Different dependency failure scenarios can correspond to different error codes. For example, a single dependency not ready corresponds to error code 100, multiple dependencies or logic not all ready corresponds to error code 101, and multiple dependencies or logic not ready corresponds to error code 102. Different error codes distinguish different waiting scenarios, facilitating subsequent log analysis and troubleshooting. It should be noted that the return process of the non-fatal wait error code is completed entirely at the task layer and does not require calling stateless atomic actions. The action itself does not need to be aware of the error code or triggering rule, ensuring the stateless design of the action.

[0051] The triggering rule for the fatal execution error code field is configured as the return rule when an action execution exception occurs. This triggering rule clarifies the timing of the fatal execution error code triggering; that is, when a real execution failure occurs during the execution of an action, the corresponding error code in the fatal execution error code field is returned, triggering the native fault tolerance process. Execution exception scenarios include command execution failure, remote node connection failure, file transfer failure, node authentication failure, etc., which occur during action execution. Different exception scenarios correspond to different error codes, facilitating fault location and fault tolerance handling. It should be noted that the triggering rule for the fatal execution error code completely adopts the original error handling rules of the original architecture without any modification, ensuring that the original fault tolerance capability is not affected and is decoupled from the newly added state waiting retry logic, so that they do not interfere with each other.

[0052] In one embodiment that can be implemented in this application, such as Figure 3 As shown, the scheduling executes multiple stateless atomic actions, including: S301, parse the action dependency rules corresponding to stateless atomic actions, and read the execution results of dependent actions from the local result aggregation table; S302, determine whether the action dependency relationship is satisfied. If it is satisfied, inject the execution result of the dependent action into the execution parameter of the stateless atomic action and call the stateless atomic action to execute. S303, if not satisfied, returns the error code corresponding to the non-fatal wait error code field, triggering exponential backoff retry.

[0053] In the embodiments of this application, the specific steps for scheduling and executing multiple stateless atomic actions are a lower-level limitation on the action scheduling and execution process in the independent claim. Through the pre-dependency verification, parameter injection and error code triggering process, the dependency scheduling logic is completely converged at the task layer. The action itself is completely unaware of the dependency relationship, which ensures the stateless design of the action and avoids invalid operations when the dependency is not satisfied, thereby improving the execution efficiency.

[0054] Specifically, the process involves parsing the action dependency rules corresponding to stateless atomic actions and reading the execution results of dependent actions from the local result aggregation table. This step is completed after the execution goroutine of each action starts but before the action is executed, serving as a pre-validation step. The parsing process is completed using the native template syntax engine of the original architecture. The action dependency rules are logical expressions written in template syntax. During parsing, the template engine identifies the unique identifier of the associated action in the expression, then reads the execution result of the corresponding action from the local result aggregation table, replaces the variables in the expression, and completes the parsing of the expression. It should be noted that this step only reads data from the local result aggregation table and does not perform any write operations, resulting in no lock contention. Multiple goroutines can execute the parsing operation concurrently without performance loss. Performing dependency parsing and validation before action execution is to directly trigger retries when dependencies are not satisfied, avoiding unnecessary network connections, command execution, and other invalid operations, reducing node resource and network bandwidth consumption, and improving execution efficiency.

[0055] The process involves determining whether the action dependency is satisfied. If satisfied, the execution result of the dependent action is injected into the execution parameters of the stateless atomic action, which is then executed. This step is a pre-verification decision-making stage. Boolean operations are performed on the parsed logical expression to determine if the dependency is satisfied. When the result is true, the dependency is satisfied, and the execution result of the dependent action is injected into the execution parameters of the current action, allowing the current action to use the result of the pre-action to complete its operation. The parameter injection process fully reuses the original architecture's native parameter template injection capability, requiring no modification to the action's execution logic. The action itself does not need to be aware of the parameter's source; it only needs to execute according to the preset parameters, ensuring the stateless design of the action. For example, after the pre-action executes, the access address and port of the cluster control node are obtained and injected into the execution parameters of the current action. The execution command of the current action can then use this address and port to generate the corresponding cluster registration command, which is executed on the target node to achieve the cluster node registration operation. After the parameters are injected, the stateless atomic action will be invoked to execute the corresponding atomic operation. After execution, the execution result of the action will be atomically written into the local result aggregation table for other dependent actions to read and use.

[0056] If the condition is not met, the corresponding error code in the non-fatal wait error code field is returned, triggering an exponential backoff retry. This step is the handling process when the dependency is not satisfied. The core is to trigger a state-waiting retry through a non-fatal error code, replacing the locks and waiting queues of traditional solutions. It should be noted that the error code return process is completed entirely at the task layer. At this time, no stateless atomic action is executed. The action itself does not need to be aware of the dependency, error code, or retry logic, ensuring the stateless design of the action. The returned error code selects the corresponding error code from the non-fatal wait error code field based on the specific scenario of the dependency not being satisfied, and then passes the error code to the retry scheduling module of the task layer to trigger the corresponding exponential backoff retry process, realizing lock-free state-waiting coordination.

[0057] In one embodiment that can be implemented in this application, such as Figure 4 As shown, the triggering of exponential backoff retry includes: S401, identify the error code field to which the returned error code belongs. If it is a non-fatal wait error code field, it will not be included in the preset maximum retry quota, and the retry interval will be updated according to the exponential backoff coefficient. S402, if it is a fatal execution error code field, then it is included in the preset maximum retry quota, and a retry is performed according to the preset rules; S403, when the retry interval reaches the preset maximum retry interval, maintain the maximum retry interval and perform a retry.

[0058] In the embodiments of this application, the specific steps for triggering exponential backoff retries are a lower-level limitation of the exponential backoff retries process in the independent claims. By identifying and differentiating the error code field, the decoupling of state waiting retries and fault-tolerant retries is achieved. At the same time, by retry count exemption and exponential backoff interval update, the problem of long-cycle waiting false failures caused by fixed retries in the prior art, as well as the resource occupation and real-time contradiction of fixed interval polling, are solved.

[0059] Specifically, identifying the error code domain to which the returned error code belongs is a prerequisite for triggering differentiated retry rules. This identification process is completed by determining the range of the error code's value. If the value is between 100 and 199, it is determined to be a non-fatal wait error code domain; if the value is 200 or higher, it is determined to be a fatal execution error code domain. Based on the different domains to which the error code belongs, two completely independent sets of retry rules are executed, decoupling state wait retries from fault-tolerant retries. The two sets of rules do not interfere with each other, preventing state waits from affecting fault tolerance or fault tolerance from limiting state waits.

[0060] If the error code is a non-fatal wait error, it is not counted in the preset maximum retry quota. The retry interval is updated according to the exponential backoff coefficient. This step is part of implementing long-cycle state waits and is one of the innovations of this method. The preset maximum retry quota is the maximum number of retries set by the user for action execution failures. It is used to limit the number of retries for execution failures and avoid process blocking caused by infinite retries. However, in the state wait scenario, the action itself has no execution failure; only the dependent preceding actions are not completed. Therefore, it should not occupy the maximum retry quota. Through the design of retry count exemption, retries triggered by non-fatal wait error codes are not counted in the preset maximum retry quota. Therefore, multiple retries can be made until the dependency is satisfied and the action is executed successfully. This solves the problem of false failures in long-cycle wait scenarios caused by fixed retries in the existing technology. For example, if a user sets a maximum of 3 retries, and an action triggers 10 retries due to an unread dependency, these 10 retries will not be counted against the 3-retry limit. Once the dependency is ready, the action will execute successfully, and the entire process will be completed normally. However, if an action fails to execute and still fails after 3 retries, it will be marked as an execution failure, triggering the process termination. The two are completely independent and do not interfere with each other.

[0061] Updating the retry interval according to the exponential backoff coefficient is part of achieving a balance between low resource consumption and high real-time responsiveness. The exponential backoff coefficient is a user-preset value, usually set to 2, meaning that the interval between each retry is twice the previous one. The calculation process for the retry interval is as follows: the initial retry interval is the user-preset value. The first retry uses the initial interval, and the interval for each subsequent retry is the previous interval multiplied by the exponential backoff coefficient, until the preset maximum retry interval is reached. For example, if the initial retry interval is set to 100ms, the exponential backoff coefficient is 2, and the maximum retry interval is 2s, then the first retry interval is 100ms, the second is 200ms, the third is 400ms, the fourth is 800ms, the fifth is 1600ms, and the sixth and subsequent retry intervals are all 2s. This design ensures a fast response once the dependency is ready by using a small interval in the initial waiting period. As the waiting time increases, the interval is gradually increased to avoid wasting CPU resources due to frequent polling. According to actual tests, in a parallel scenario with thousands of nodes, the CPU resource usage of this design is only 12% of that of a fixed interval polling scheme. At the same time, the maximum response delay after the dependency is ready does not exceed the current retry interval, thus balancing resource usage and real-time response.

[0062] If the error code is a fatal execution error, it is counted in the preset maximum retry quota, and retrying is performed according to preset rules. This step completely adopts the original fault retry rules of the original architecture, including retry interval, retry count, fault tolerance logic, etc., ensuring that the original fault tolerance capability is not affected and is decoupled from the newly added state waiting retry logic. When the action returns a fatal execution error code, each retry is counted in the preset maximum retry quota. When the retry count reaches the quota limit, it will be marked as execution failure, triggering the corresponding fault tolerance handling and process termination logic. The processing flow is consistent with the original architecture and does not require any modification.

[0063] Once the retry interval reaches the preset maximum retry interval, retries are performed while maintaining the maximum retry interval. This step is to prevent the real-time response from degrading due to the retry interval growing indefinitely, ensuring that even during long-term waits, the maximum response latency after the dependency is ready will not exceed the maximum retry interval. The maximum retry interval is a user-preset value that can be adjusted according to the real-time requirements of the scenario. Once the retry interval grows exponentially to the maximum retry interval, it stops growing, and subsequent retries maintain the maximum retry interval. This avoids the response latency caused by infinitely growing intervals while ensuring low CPU resource consumption in long-term wait scenarios.

[0064] In one embodiment that can be implemented in this application, such as Figure 5 As shown, updating the retry interval according to the exponential backoff coefficient includes: S501, when the dependent stateless atomic action is completed, receives a callback instruction indicating that the action has been completed; S502, terminate the current retry interval waiting according to the callback instruction, and immediately trigger the retry of the corresponding stateless atomic action; S503, if the callback instruction is not received, a retry is performed according to the updated retry interval.

[0065] In the embodiments of this application, the specific steps of updating the retry interval are a further lower-level limitation on the exponential backoff retry process. Through the event-driven callback wake-up mechanism, the response delay problem caused by the exponential backoff retry interval is solved. It retains the advantage of low CPU usage of exponential backoff and realizes fast response after the dependency is ready, thus balancing resource consumption and real-time response.

[0066] Specifically, when a dependent stateless atomic action completes, a callback instruction for action completion is received. This step triggers the event wake-up mechanism. The callback instruction reuses the native action completion callback hook from the original architecture. When an action completes, the native completion callback is triggered. The task layer uses the dependency mapping table built during task initialization to find all other actions that depend on this action and sends callback instructions to the retry coroutines of these actions. The dependency mapping table is built during task initialization based on the dependency rules of all actions. The table records which other actions depend on each action, so when this action completes, the corresponding waiting actions can be quickly found and callback instructions sent. The sending of callback instructions is implemented through the communication mechanism between coroutines, avoiding lock contention and CPU busy polling, resulting in no additional performance loss.

[0067] The core of the event wake-up mechanism is to terminate the current retry interval wait based on the callback instruction and immediately trigger a retry for the corresponding stateless atomic action. When a waiting action receives a callback instruction, it will immediately terminate the current retry interval wait, regardless of how long it has been waiting, and immediately trigger a retry to re-verify dependencies. This ensures that the action can be executed immediately once the dependency is ready, without waiting for the current retry interval to end, thus solving the response latency problem caused by exponential backoff retry intervals. For example, if an action's current retry interval is 2 seconds and it has been waiting for 500 milliseconds, and then receives a callback instruction indicating that the dependent action has completed, it will immediately terminate the remaining 1.5 seconds of waiting, immediately trigger a retry, verify dependencies, and execute the action. The response latency is reduced from a maximum of 2 seconds to milliseconds, while retaining the low CPU usage advantage of exponential backoff.

[0068] If no callback instruction is received, a retry is performed according to the updated retry interval. This step serves as a fallback mechanism. If, for some reason, the callback instruction is lost or not received, the action's retry coroutine will automatically trigger a retry after waiting for the current retry interval, re-verifying dependencies and preventing indefinite waiting, thus ensuring process reliability. This fallback mechanism, combined with the event wake-up mechanism, forms a complete retry scheduling system. Under normal circumstances, event wake-up enables rapid response, while in abnormal situations, fallback retries ensure the process will not be permanently blocked, balancing real-time performance and reliability.

[0069] In one embodiment that can be implemented in this application, such as Figure 6 As shown, determining whether the action dependency relationship is satisfied includes: S601, parse the logical expression in the action dependency rule, wherein the logical expression contains at least one of AND logic, OR logic, and NOT logic; S602, Read all action execution results associated with the logical expression from the local result aggregation table; S603, Based on the result of the operation of the logical expression, determine whether the action dependency relationship is satisfied.

[0070] In the embodiments of this application, the specific steps for determining whether the action dependency relationship is satisfied are the lower-level constraints of the dependency verification process. By supporting expression parsing, batch result reading and Boolean operation for multiple types of logical operations, it can realize the determination of multiple dependencies of arbitrary complexity, meet the complex collaboration needs in different scenarios, and completely reuse the original template syntax engine of the original architecture without adding additional logical operation modules.

[0071] Specifically, the logical expressions in the action dependency rules are parsed. These logical expressions include at least one of AND, OR, and NOT logic, and this step is fundamental to dependency judgment. The logical expressions are written using the native template syntax of the existing architecture, supporting the three basic logical operations (AND, OR, and NOT) and any combination thereof, enabling dependency definitions of arbitrary complexity. AND logic indicates that all dependencies must be satisfied before the action can be executed; OR logic indicates that the action can be executed as long as any one dependency is satisfied; and NOT logic indicates that the action can be executed only when no dependency is satisfied. Through combinations of these three basic logics, dependency definitions for complex multi-node, multi-action collaborative scenarios can be achieved. For example, in a multi-master node cluster scenario, worker nodes can only perform registration operations after all master nodes have been configured, which can be achieved using AND logic; similarly, in a multi-availability zone cluster scenario, worker nodes in the corresponding availability zone can perform operations as long as the master node in any availability zone has been configured, which can be achieved using OR logic. Those skilled in the art will understand that the specific content of the logical expressions can be adjusted according to the actual application scenario, and this application embodiment does not impose a unique limitation on this.

[0072] The step of retrieving the execution results of all actions associated with the logical expression from the local result aggregation table provides data support for logical operations. After parsing the logical expression, the unique identifiers of all actions associated with the expression are identified. Then, the execution results of all corresponding actions are read from the local result aggregation table at once, reducing the number of read operations and improving execution efficiency. The read results include the success status of the action, execution output data, execution time, etc., all of which can be used for logical expression operations to meet the dependency judgment needs of different scenarios.

[0073] Based on the result of the logical expression, the decision-making step is to determine whether the action dependency is satisfied. The read action execution result is substituted into the logical expression, and Boolean operations are performed using the native template syntax engine to obtain the final result. A true result indicates that the dependency is satisfied, and a false result indicates that the dependency is not satisfied. The entire calculation process completely reuses the original architecture's native template syntax engine, requiring no additional logical operation modules and intruding zero into the original architecture's logic, thus ensuring compatibility with the original architecture.

[0074] In one embodiment that can be implemented in this application, such as Figure 7 As shown, updating the memory retry status dictionary based on the action execution result includes: S701, after each retry of the action, read the number of retries, the current retry interval, the last retry time and the dependency ready status corresponding to the action; S702, the read state data is atomically written into the storage location of the corresponding action in the memory retry state dictionary; S703, align the data in the memory retry status dictionary with the breakpoint snapshot metadata field.

[0075] In the embodiments of this application, the specific steps of updating the memory retry status dictionary are a sub-definition of the status update process in the independent claim. By periodically reading the status, atomically writing, and format alignment, it is ensured that the data in the memory retry status dictionary is always consistent with the current execution status, and is also fully aligned with the breakpoint snapshot metadata field, providing accurate data support for subsequent snapshot synchronization and status recovery.

[0076] Specifically, after each retry of an action, the system reads the number of retries, the current retry interval, the last retry time, and the dependency readiness status. This step is the starting point for state updates. The data is read after each retry process, regardless of whether the retry result is that the dependency is still not ready or the action is successful, ensuring that the data in the dictionary is always the latest execution state. The four data items read constitute the minimum dataset necessary for resuming interrupted downloads. The number of retries and the current retry interval are used to continue the retry process after an interruption, without needing to calculate the retry interval from the beginning. The last retry time is used to calculate the trigger time for the next retry. The dependency readiness status is used to determine the current execution stage of the action. It should be noted that the dataset read only retains the data necessary for resuming interrupted downloads, with no redundant information. This avoids redundant data increasing memory usage and snapshot size, conforming to the minimum design principle.

[0077] The atomic writing of the read state data to the corresponding action's storage location in the memory retry state dictionary is crucial for ensuring data consistency in multi-coroutine concurrent scenarios. Atomic writing can be implemented using any of the following methods: memory barriers, atomic operation instructions, or lightweight spinlocks. Those skilled in the art will understand that the specific implementation of atomic writing can be adjusted according to the operating environment, and this application does not limit it to a single method. Atomic writing ensures that data contention and inconsistency will not occur in scenarios with multiple coroutines writing concurrently, preventing data corruption caused by two coroutines writing to the same storage location simultaneously, and guaranteeing the accuracy and integrity of the data in the dictionary.

[0078] The data in the memory retry status dictionary is format-aligned with the breakpoint snapshot metadata fields. This step ensures that during subsequent snapshot synchronization, the data in the dictionary can be directly written to the pipeline breakpoint snapshot without additional format conversion, improving snapshot generation efficiency and avoiding potential data errors during format conversion. The format alignment process is completed during task initialization. The structure of the memory retry status dictionary is generated based on the definition of the breakpoint snapshot metadata fields. Each data item in the dictionary corresponds one-to-one with a field defined in the metadata, with identical data type, storage order, and serialization method. Therefore, updating dictionary data at runtime is naturally format-aligned, requiring no additional conversion operations.

[0079] In one embodiment that can be implemented in this application, such as Figure 8 As shown, synchronizing the memory retry status dictionary and the local result aggregation table to the pipeline breakpoint snapshot includes: S801, receive pipeline snapshot trigger instruction, and read data from the memory retry status dictionary and the local result aggregation table; S802, the read data is written into the metadata area of ​​the pipeline breakpoint snapshot according to the format of the breakpoint snapshot metadata field; S803, record the unique identifier of the stateless atomic action that has been executed in the pipeline breakpoint snapshot.

[0080] In the embodiments of this application, the specific steps of synchronizing data to the pipeline breakpoint snapshot are a lower-level limitation on the snapshot synchronization process in the independent claim. By reusing the native snapshot triggering mechanism, writing in a standardized format and recording completed action identifiers, the retry state and the native breakpoint snapshot are compatible, while ensuring the idempotency of the operation and avoiding repeated execution after breakpoint recovery.

[0081] Specifically, upon receiving the pipeline snapshot trigger command, the system reads data from the in-memory retry status dictionary and the local result aggregation table. This step marks the beginning of snapshot synchronization. The snapshot trigger command is issued by the original pipeline snapshot mechanism of the existing architecture. Triggering events include timed triggering, triggering after the execution phase is completed, and automatic triggering before pipeline interruption. This method fully reuses these original triggering events without adding additional timed tasks or triggering logic, achieving zero intrusion into the pipeline layer logic. Upon receiving the snapshot trigger command, the system reads all data from the in-memory retry status dictionary and the local result aggregation table at once, ensuring that the read data reflects the execution status at the same moment and avoiding data inconsistency issues. The reading process is read-only and does not modify the data in memory, thus not affecting the normal task execution flow.

[0082] The core step of snapshot synchronization is writing the read data into the metadata area of ​​the pipeline breakpoint snapshot according to the format of the breakpoint snapshot metadata fields. The metadata area is a reserved, expandable area in the original architecture pipeline breakpoint snapshot, used to store user-defined extended data. The content of this area does not affect the parsing and use of the core snapshot data; the native breakpoint resume mechanism will completely preserve the content of this area without modification or deletion. Because format alignment has already been completed, the writing process simply writes the read data into the metadata area according to the order of the metadata fields, without requiring additional format conversion. The writing process is atomic, ensuring the data in the snapshot is complete and preventing data incompleteness issues caused by the snapshot being used during the writing process.

[0083] Recording unique identifiers of completed stateless atomic actions in pipeline breakpoint snapshots is crucial for ensuring idempotency. Each stateless atomic action's unique identifier is a globally unique ID, bound to the corresponding node and execution content, ensuring global uniqueness. Recording this identifier in the snapshot allows filtering completed actions during breakpoint recovery, preventing duplicate execution and guaranteeing idempotency. Idempotency is critical in operational orchestration scenarios; duplicate execution of completed actions can lead to inconsistent cluster configurations, redundant resource creation, and data corruption. By recording unique identifiers of completed actions in snapshots, these actions can be skipped during recovery, preventing duplicate execution and avoiding these problems.

[0084] In one embodiment that can be implemented in this application, such as Figure 9 As shown, the step of restoring task scheduling based on the pipeline breakpoint snapshot includes: S901, parse the pipeline breakpoint snapshot and read the data in the breakpoint snapshot metadata field; S902, restore the memory retry state dictionary, action dependency rules and list of stateless atomic actions that have been completed based on the read data; S903 filters out stateless atomic actions that have been completed, starts an execution coroutine for stateless atomic actions that are in a retry waiting state, and continues the retry process.

[0085] In the embodiments of this application, the specific steps of execution state recovery are a subordinate limitation of the state recovery process in the independent claim. By reusing the native breakpoint parsing process, memory state restoration and continuation execution, seamless recovery of the retry state after task interruption is achieved. It does not require re-executing the completed actions, nor does it require recalculating the retry interval from the beginning, and it is compatible with the native breakpoint resume capability.

[0086] Specifically, the process involves parsing pipeline breakpoint snapshots and reading data from the breakpoint snapshot metadata fields. This step completely reuses the original architecture's native breakpoint snapshot parsing process. The native parsing process first parses the core data of the snapshot to reconstruct the pipeline's execution context. This method, after the native parsing process is complete, reads data from the standardized breakpoint snapshot metadata fields in the snapshot's metadata area. It does not require modification of the native parsing logic, maintains compatibility with the native parsing process, and avoids any intrusion into the pipeline layer's code. The data read includes the full data of the in-memory retry state dictionary, action dependency rules, and a list of unique identifiers for completed actions. This data is essential for achieving state recovery.

[0087] The core step of state recovery is to reconstruct the memory retry state dictionary, action dependency rules, and list of completed stateless atomic actions based on the read data. This involves rewriting the data read from the snapshot into the task layer's memory data structure to restore the memory state before the interruption. The restored memory state is consistent with the memory state before the interruption, including the number of retries for each action, the current retry interval, the last retry time, the dependency ready state, the corresponding dependency rules, and the list of completed actions. This allows the execution flow to resume from where it left off, without re-executing completed actions or recalculating the retry interval from the beginning. For example, if the current retry interval for an action before the interruption was 1 second and it had been retried 5 times, the restored memory retry state dictionary would still show the current retry interval as 1 second and the number of retries as 5. When resuming execution, retries will resume from the 1-second interval instead of starting from the initial interval, ensuring the continuity of the waiting process.

[0088] The process filters out stateless atomic actions that have already been executed, and starts execution coroutines for stateless atomic actions in a retry waiting state to continue the retry process. This step is the final execution stage of the recovery process. The filtering process removes completed actions from the pending list based on the restored list of completed actions, preventing the starting of execution coroutines for these actions, avoiding duplicate execution, and ensuring idempotency. For actions in a retry waiting state, the corresponding execution coroutine is started based on the restored retry state, continuing the retry process before the interruption, using the current retry interval before the interruption, without needing to recalculate. The entire recovery process is completely transparent to upper-layer modules and pipelines; no modifications are required at the upper layer to use the breakpoint resumption capability normally, achieving compatibility between state-based collaborative processes and native breakpoint resumption capabilities.

[0089] like Figure 10 As shown, the second aspect of this application proposes a task scheduling system based on a hierarchical abstraction model, comprising: The configuration management module 1001 is used to obtain task configuration information and generate dual-track error code rules, action dependency rules, exponential backoff retry rules, and breakpoint snapshot metadata fields. The dual-track error code rules are rules for dividing non-fatal wait error code fields and fatal execution error code fields. The breakpoint snapshot metadata fields are serializable fields used to store the task execution status. The task scheduling module 1002 is used to schedule the execution of multiple stateless atomic actions according to the dual-track error code rules, the action dependency rules, and the exponential backoff retry rules, and to update the local result aggregation table and the memory retry status dictionary according to the action execution results. The stateless atomic action is the smallest execution unit of a single node operation. The local result aggregation table is a memory table that stores the action execution results. The memory retry status dictionary is a memory data structure that records the action retry progress. The state persistence module 1003 is used to synchronize the memory retry state dictionary and the local result aggregation table to the pipeline breakpoint snapshot according to the breakpoint snapshot metadata field, and to restore task scheduling according to the pipeline breakpoint snapshot. The pipeline breakpoint snapshot is a persistent data structure that records the pipeline execution progress.

[0090] In this embodiment, the task scheduling system based on the hierarchical abstraction model is a system embodiment that corresponds one-to-one with the aforementioned method embodiments. The system architecture fits the four-layer architecture of the hierarchical abstraction model, and the functions of all modules correspond one-to-one with the aforementioned method steps. It can realize all the processes in the aforementioned method embodiments and achieve the same technical effect.

Claims

1. A task scheduling method based on a hierarchical abstraction model, characterized in that, include: Obtain task configuration information and generate dual-track error code rules, action dependency rules, exponential backoff retry rules, and breakpoint snapshot metadata fields. The dual-track error code rules are rules for dividing non-fatal wait error code fields and fatal execution error code fields. The breakpoint snapshot metadata fields are serializable fields used to store the task execution status. Based on the dual-track error code rule, the action dependency rule, and the exponential backoff retry rule, multiple stateless atomic actions are scheduled for execution. The local result aggregation table and the memory retry status dictionary are updated according to the action execution results. The stateless atomic action is the smallest execution unit of a single node operation. The local result aggregation table is a memory table that stores the action execution results. The memory retry status dictionary is a memory data structure that records the action retry progress. Based on the breakpoint snapshot metadata field, the memory retry status dictionary and the local result aggregation table are synchronized to the pipeline breakpoint snapshot, and the task scheduling is restored based on the pipeline breakpoint snapshot. The pipeline breakpoint snapshot is a persistent data structure that records the pipeline execution progress.

2. The method according to claim 1, characterized in that, The rules for generating dual-track error codes include: The number ranges are divided into non-fatal wait error code fields and fatal execution error code fields; Configure the triggering rule corresponding to the non-fatal wait error code field as the return rule when the action dependency relationship is not satisfied; Configure the triggering rule corresponding to the fatal execution error code field as the return rule when the action execution is abnormal.

3. The method according to claim 1, characterized in that, The scheduling executes multiple stateless atomic actions, including: Parse the action dependency rules corresponding to stateless atomic actions and read the execution results of dependent actions from the local result aggregation table; Determine if the action dependency relationship is satisfied. If it is satisfied, inject the execution result of the dependent action into the execution parameter of the stateless atomic action and call the stateless atomic action to execute. If the condition is not met, the error code corresponding to the non-fatal wait error code field is returned, triggering an exponential backoff retry.

4. The method according to claim 3, characterized in that, The triggering of indexed backoff retry includes: Identify the error code field to which the returned error code belongs. If it is a non-fatal wait error code field, it will not be included in the preset maximum retry quota, and the retry interval will be updated according to the exponential backoff coefficient. If it is a fatal execution error code field, it will be included in the preset maximum retry quota, and a retry will be performed according to the preset rules; When the retry interval reaches the preset maximum retry interval, the retry is performed while maintaining the maximum retry interval.

5. The method according to claim 4, characterized in that, The method of updating the retry interval according to the exponential backoff coefficient includes: When the dependent stateless atomic action is completed, receive a callback instruction indicating that the action has been completed; The current retry interval wait is terminated according to the callback instruction, and the retry of the corresponding stateless atomic action is immediately triggered. If the callback instruction is not received, a retry will be performed according to the updated retry interval.

6. The method according to claim 3, characterized in that, The determination of whether the action dependency relationship is satisfied includes: Parse the logical expression in the action dependency rule, wherein the logical expression contains at least one of AND, OR, and NOT logic; Read the execution results of all actions associated with the logical expression from the local result aggregation table; Based on the result of the logical expression, determine whether the action dependency relationship is satisfied.

7. The method according to claim 1, characterized in that, The step of updating the memory retry status dictionary based on the action execution result includes: After each retry of an action, read the number of retries, the current retry interval, the last retry time, and the ready status of the dependencies for that action. The read state data is atomically written to the storage location of the corresponding action in the memory retry state dictionary; Align the data in the memory retry status dictionary with the breakpoint snapshot metadata field in terms of format.

8. The method according to claim 1, characterized in that, The step of synchronizing the memory retry status dictionary and the local result aggregation table to the pipeline breakpoint snapshot includes: Receive pipeline snapshot trigger command and read data from the memory retry status dictionary and the local result aggregation table; The read data is written into the metadata area of ​​the pipeline breakpoint snapshot according to the format of the breakpoint snapshot metadata field. The unique identifier of the stateless atomic action that has been completed is recorded in the pipeline breakpoint snapshot.

9. The method according to claim 1, characterized in that, The step of restoring task scheduling based on the pipeline breakpoint snapshot includes: Parse the pipeline breakpoint snapshot and read the data from the breakpoint snapshot metadata field; Reconstruct the memory retry state dictionary, action dependency rules, and list of stateless atomic actions that have been completed based on the read data; Filter out stateless atomic actions that have been completed, start execution coroutines for stateless atomic actions that are in a retry waiting state, and continue the retry process.

10. A task scheduling system based on a hierarchical abstraction model, characterized in that, include: The configuration management module is used to obtain task configuration information and generate dual-track error code rules, action dependency rules, exponential backoff retry rules, and breakpoint snapshot metadata fields. The dual-track error code rules are rules for dividing non-fatal wait error code fields and fatal execution error code fields. The breakpoint snapshot metadata fields are serializable fields used to store the task execution status. The task scheduling module is used to schedule the execution of multiple stateless atomic actions according to the dual-track error code rules, the action dependency rules, and the exponential backoff retry rules, and to update the local result aggregation table and the memory retry status dictionary according to the action execution results. The stateless atomic action is the smallest execution unit of a single node operation, the local result aggregation table is a memory table that stores the action execution results, and the memory retry status dictionary is a memory data structure that records the action retry progress. The state persistence module is used to synchronize the memory retry state dictionary and the local result aggregation table to the pipeline breakpoint snapshot based on the breakpoint snapshot metadata field, and to restore task scheduling based on the pipeline breakpoint snapshot. The pipeline breakpoint snapshot is a persistent data structure that records the pipeline execution progress.

Images