Workflow execution method and system based on distributed architecture

Abstract

The invention discloses a workflow execution method and system based on a distributed architecture, and the method comprises the following steps: S1, when a workflow node executes a request, loading node strategy configuration, checking whether a forcing rule belongs to forcing independent execution or forcing embedded execution, if yes, entering step S3, and if not, entering step S2; s2, initializing decision scores, calculating and accumulating / deducting the scores layer by layer to form a final decision score, and determining an independent execution mode or an embedded execution mode; and S3, after the independent execution mode or the embedded execution mode is determined, performing workflow execution according to the corresponding mode path. According to the method, an embedded execution mode and an independent execution mode are introduced for execution of workflow nodes, the execution mode is dynamically decided, and meanwhile, resource limitation and resource permission constraint under the embedded execution mode and the independent execution mode are realized in a mode of introducing a strategy configuration file.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of workflow software systems, and particularly relates to a workflow execution method and system based on a distributed architecture. BACKGROUND

[0002] A workflow system is the core infrastructure for modern enterprises to realize business process automation, data process, and other business needs. It breaks down complex business logic into a series of interdependent executable nodes (tasks) and schedules them according to predefined rules, greatly improving production efficiency and system reliability.

[0003] However, as the complexity and scale of business increase, traditional workflow systems gradually reveal their limitations when facing real business challenges: 1. Single execution mode, lack of flexibility and adaptability: Traditional workflow engines, such as Apache Airflow, Zeebe, and Temporal, either embed all nodes in the main process for execution or uniformly use independent processes or containers for execution. The embedded execution mode has the advantages of low latency and high efficiency, but has the problem of insufficient resource isolation. When a node consumes a large amount of memory or CPU resources, it can easily affect the stability of the entire workflow engine. The independent execution mode provides good resource isolation, but introduces additional network communication overhead and resource scheduling complexity, which is too heavy for lightweight nodes.

[0004] 2. Insufficient distributed capabilities, coarse resource control granularity: To solve the single-point performance problem, some workflow systems evolve into a distributed architecture, deploying the engine on multiple server nodes. However, this distribution usually stops at the "engine level" load balancing, and node execution is still strongly bound to the engine process. System administrators can limit the resources of the entire engine process, but cannot perform fine-grained resource quota on specific nodes within the engine that are executing concurrently from different business processes (for example, limit the "transcoding node" of business A to use at most 0.5 core CPU and 256MB memory). This leads to the "noisy neighbor" problem, where high-load business processes will squeeze the resources of low-load businesses, making it impossible to achieve true multi-tenant resource isolation.

[0005] 3. Weak security isolation mechanism, difficult to cope with complex execution environment: In modern enterprises, workflow nodes may be developed by different teams, or even execute third-party or user dynamically submitted code scripts. In the traditional embedded execution mode, all codes share the context of the engine process, which means that malicious or vulnerable code of one node can easily access the data of other nodes, or even perform high-risk operations such as file reading and writing, network attacks, etc. on the host machine. Existing solutions usually rely on coding specifications and code reviews, lack of mandatory and reliable technical isolation means, and have high security risks. Although some systems try to use independent processes or containers to execute all tasks, this introduces new problems: for a large number of simple tasks, the overhead (time and resources) of creating a container is much greater than the execution overhead of the task itself, resulting in a significant decrease in system throughput and a significant increase in latency, which is not worth the cost.

[0006] 4. Static scheduling strategy, lack of dynamic awareness of task characteristics and system state: The scheduling strategy of existing systems is usually static or based on simple rules (such as round robin, least connection number). When deciding which task to assign to which worker node, they neither consider the resource demand characteristics of the task itself (CPU-intensive or I / O-intensive), nor care about the real-time load state of the current worker node (CPU, memory usage, task queue backlog). This "blind" scheduling leads to uneven system load, with some nodes overloaded and others idle, resulting in low overall resource utilization and inability to actively take degradation strategies (such as stripping high-load tasks to isolated environments) to protect the stability of core services when the system is at risk of overload. SUMMARY

[0007] The purpose of the present application is to overcome the shortcomings of the prior art and provide a workflow execution method and system based on a distributed architecture, which introduces "embedded execution mode" and "independent execution mode" for the execution of workflow nodes, dynamically decides the execution mode, and simultaneously realizes resource restriction and resource permission constraints in "embedded execution mode" and "independent execution mode" through the introduction of a strategy configuration file.

[0008] The purpose of the present application is achieved by the following technical solution: a workflow execution method based on a distributed architecture, comprising the following steps: S1. When a workflow node execution request is received, load the node strategy configuration and check whether the mandatory rule belongs to mandatory independent execution or mandatory embedded execution: If it is mandatory independent execution, select independent execution mode and enter step S3; If it is mandatory embedded execution, select embedded execution mode and enter step S3; If it is neither mandatory independent execution nor mandatory embedded execution, enter step S2; S2. Initialize the decision score, calculate and accumulate / deduct the score layer by layer to form the final decision score and determine the independent execution mode or the embedded execution mode; S3. After determining the standalone execution mode or the embedded execution mode, execute the workflow according to the corresponding mode path.

[0009] A workflow execution system based on a distributed architecture includes: The strategy configuration and enforcement rule determination unit is used to load the node strategy configuration and check whether the enforcement rule belongs to forced independent execution or forced embedded execution when the workflow node executes the request. The decision score superposition and execution mode determination unit is used to initialize the decision score when the mandatory rule does not belong to mandatory independent execution or mandatory embedded execution, calculate and accumulate / deduct the score layer by layer to form the final decision score and determine the independent execution mode or the embedded execution mode. The workflow execution unit is used to execute the workflow according to the corresponding mode path after determining the independent execution mode or the embedded execution mode.

[0010] The beneficial effects of this invention are: this invention introduces "embedded execution mode" and "independent execution mode" for the execution of workflow nodes. During runtime, the execution mode is dynamically determined according to the algorithm flow described below. At the same time, by introducing a strategy configuration file, resource restrictions and resource permission constraints are implemented under "embedded execution mode" and "independent execution mode". Attached Figure Description

[0011] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0012] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the following description.

[0013] like Figure 1 As shown, a workflow execution method based on a distributed architecture includes the following steps: S1. When a workflow node executes a request, it loads the node policy configuration and checks whether the enforcement rule belongs to forced independent execution or forced embedded execution: If forced to execute independently, select the independent execution mode and proceed to step S3; If embedded execution is forced, select the embedded execution model and proceed to step S3; If it is neither mandatory independent execution nor mandatory embedded execution, then proceed to step S2; After the workflow instance initialization is completed, the process of executing the nodes in the workflow according to the flow design of the workflow is started. When a specific workflow node is executed, the system first executes the "load node strategy configuration" step (if not loaded yet): reads the strategy configuration file corresponding to the node X of this execution. The strategy configuration file declares the resource constraints and permission permissions of the node in two execution modes (embedded execution and remote execution) in a structured configuration manner.

[0014] The resource constraints of the embedded execution module include: the configured upper limit of memory, the maximum CPU execution time Tcpu_max, and the maximum wall execution time Twall_max. The resource constraints of the independent execution model include: the configured upper limit of memory, the maximum number of shared CPU cores, and the maximum number of exclusive CPU cores. The permission permissions under the embedded execution and independent execution model include: the file directory allowed to read, the file directory allowed to write, the domain name or IP allowed to access, and the environment variable allowed to access. For example, in the embedded execution mode, the resource quota of node X is that the maximum heap memory (V8) upper limit is 256 MB, the young generation upper limit is 64 MB, the maximum CPU execution time (Tcpu_max) is 10 seconds, and the total wall time upper limit (Twall_max) is 30 seconds. The container resource limit in the independent execution mode is that the memory upper limit is 512 MB, the CPU weight (cpuShares) is 512, and the available CPU core number (cpus) upper limit is 0.5 (i.e. at most 0.5 CPU core is used).

[0015] In addition to the resource quota, the resource permissions of node X can also be configured, for example: The set of file paths allowed to read includes / data / images / , the set of file paths allowed to write includes / data / thumbnails / , the network domain name allowed to access includes test.domain.com, and the environment variable key allowed includes TEST_ENV_NAME.

[0016] Each node can independently configure the strategy file, which will be directly input to subsequent steps such as "mode decision", "configuration synthesis", "container start / thread configuration", etc.

[0017] After loading the strategy, the system enters the "check mandatory rule" judgment: whether the current node type belongs to the pre-defined "mandatory independent execution list" (such as OCR recognition, big data calculation, dynamic code execution, etc.). If the result is "yes" (belongs to mandatory independent execution), the independent execution mode is directly selected, and the subsequent running path of independent execution is directly entered; the process ends at this branch and no longer scores and assesses other levels.

[0018] If it does not belong to the pre-defined "mandatory independent execution list", it is judged whether the current workflow node type belongs to the pre-defined mandatory embedded execution list; if so, the embedded execution mode is directly selected, and the running path of embedded execution is entered; the process ends at this branch and no longer scores and assesses other levels.

[0019] If not, step S2 is entered.

[0020] S2. Initialize the decision score, calculate and add / subtract the score layer by layer to form the final decision score and determine the independent execution mode or embedded execution mode; If the "check mandatory rule" judgment result is "no", the system initializes the decision score: Score = 0 (hereinafter referred to as Score or Score_independent). Then calculate and add / subtract the score layer by layer in the following four-layer evaluation order to form the final decision score; The first layer: resource demand evaluation (CPU intensity determination and score) Determine the priority of the node CPU intensity source: the system checks and determines the CPU intensity source of the node in strict order (the priority must be strictly in the following order): (a) Whether the intensity has been explicitly set in the node configuration during workflow design; if so, use the setting, with the highest priority; (b) Otherwise, check the node preset attribute (intensity declared by the node developer during development); if it exists, use the preset, with the second highest priority; (c) If both of the above do not exist, perform statistical determination at runtime, with the statistical source being the sampling of the CPU cumulative usage time of the Cgroup where the node is located by Linux Cgroups (v1 or v2).

[0021] Runtime statistical method (only enabled when the current two items are not specified): Record wall clock time Tstart_wall when the node starts executing the wall, and read the CPU cumulative usage of the Cgroup where the node process resides (Cgroup v1 reads from cpuacct.usage, Cgroup v2 reads usage_usec from cpu.stat), denoted as CPUstart.

[0022] (a) Before the node executes: Record the current time, denoted as Tstart_wall ， Read the CPU cumulative usage of the Cgroup where the node process resides, Cgroup v1 reads from cpuacct.usage, Cgroup v2 reads usage_usec from cpu.stat. Denoted as CPUstart .

[0023] (b) After the node executes: Record the current time, denoted as Tend_wall ， Again, read the CPU cumulative usage of the Cgroup where the node process resides, Cgroup v1 reads from cpuacct.usage, Cgroup v2 reads usage_usec from cpu.stat. Denoted as CPUend.

[0024] (c) Calculate the difference: The total duration of the node execution is , and the total duration of the CPU consumed by the node is .

[0025] (d) Calculate the CPU intensity: The CPU intensity D is defined as the ratio of the CPU duration to the total execution duration. Considering that the node may use multi-core parallel computing, it is necessary to normalize the wall duration by multiplying the number of CPU cores Ncores available to the node. The calculation formula is: Dnormalized=Tcpu / (Twall×Ncores ) Where: Ncores is the number of CPU cores allocated to the Cgroup.

[0026] The value range of Dnormalized is usually between [0,1]. The closer the value is to 1, the more CPU is in a "full load" state during the node's running time, which is typical of compute-intensive. The closer the value is to 0, the more the node spends most of the time waiting for I / O, sleeping, or being preempted by other processes, which is typical of I / O-intensive or idle-type.

[0027] Intensity level determination and scoring rules: By setting two thresholds, Thresholdhigh and Thresholdlow, the nodes can be divided into three levels: High Intensity: if Dnormalized≥ Thresholdhigh.

[0028] Medium Intensity: if Thresholdlow≤ Dnormalized < Thresholdhigh.

[0029] Low Intensity: if Dnormalized < Thresholdlow.

[0030] In practice, it is usually initially set Thresholdhigh = 0.75, Thresholdlow = 0.25. This means: If a node has more than 75% of the time spent on computation for each CPU core assigned to it during the execution of a workflow task cycle, it is high intensity.

[0031] If between 25% and 75%, it is medium intensity.

[0032] If less than 25%, it is low intensity.

[0033] If the intensity source is "set during workflow design" or "node preset", the system will also apply the above score rules according to the set level.

[0034] After completing the intensity determination and score accumulation, the second layer of evaluation is entered.

[0035] Second layer: System state evaluation (system load score) Index collection: The system monitors the current load state of the workflow engine in real time, including CPU usage, memory usage, task queue backlog, and P95 execution delay. The load evaluation adopts a weighted scoring mechanism, considering the deviation of the current value and historical baseline of each index. When the system load exceeds the preset threshold, the decision algorithm tends to choose the independent running mode to reduce the pressure on the main process.

[0036] The system monitors the overall load state of the workflow engine in real time and evaluates it through a quantitative system load score model. The score ranges from 0 to 1, with a higher score indicating a higher system load and a closer to overload state.

[0037] The load evaluation model considers the following four key indicators: CPU usage (Mcpu): CPU usage of the engine master process and its server.

[0038] Memory usage (Mmem): Memory usage of the engine master process and its server.

[0039] Task queue backlog (Mqueue): Ratio of the number of pending nodes to the number of worker threads.

[0040] P95 execution latency (Mlatency): P95 execution time of the last N (e.g. 1000) completed embedded nodes.

[0041] Metric normalization: Since the dimensions of each metric are different, they first need to be normalized to the interval [0, 1], where 0 represents the ideal state and 1 represents the critical or overloaded state. Set a "warning threshold" (Twarn) and a "critical threshold" (Tcrit) for each metric Mi.

[0042] For example: for CPU usage, set Twarn_cpu=60%, Tcrit_cpu=85%. If the current CPU is 70%, its normalized score is (70-60) / (85-60)=0.4.

[0043] Weighted sum: Assign a weight Wi to each normalized metric score Ni, representing the importance of the metric in evaluating system load. The sum of all weights is 1 (∑Wi=1).

[0044] Where i takes cpu, mem, queue, latency, then: Ni is Ncpu, Nmem, Nqueue, Nlatency, representing the metric scores of CPU usage, memory usage, task queue backlog, and P95 execution latency, respectively. Wi is Wcpu, Wmem, Wqueue, Wlatency, representing the weights of CPU usage, memory usage, task queue backlog, and P95 execution latency, respectively. System total load score Sload calculation formula: Sload=WcpuNcpu+WmemNmem+WqueueNqueue+WlatencyNlatency The value of Sload ranges from 0 to 1, with a higher value indicating that the system is closer to overload.

[0045] Convert system load score to decision score and accumulate: S core += Sload × 100 So when Sload =0.7, the decision score will contribute +70 points, after the second layer, enter the third layer evaluation.

[0046] Third layer: Network latency cost evaluation (Latency penalty) For nodes that may choose to run independently, the system will try to estimate the end-to-end latency cost (Tnetwork) introduced by network communication. This cost is a key negative indicator of whether to adopt embedded mode.

[0047] The latency cost is composed of the following parts, the calculation formula is: Tnetwork=Thandshake+(Tserialize+Tdeserialize)+Ttransfer+Tremote_queue Where: Connection establishment delay (Thandshake): The time required for the client to establish a TCP connection and TLS handshake with the remote executor. In most cases when connection pool is available, the connection can be obtained from the connection pool, and this cost is approximately equal to 0. If a new connection needs to be established, this item is equal to the one-way network round-trip time (RTT) overhead, which is obtained by periodically probing the target network path to obtain a dynamic average value (the system measures this by maintaining a background ping).

[0048] Serialization / deserialization delay (Tserialize+Tdeserialize), where: Tserialize represents the serialization time of the remote execution request and the serialization time of the remote execution result; Tdeserialize represents the deserialization time of the remote execution request and the deserialization time of the remote execution result; This delay is related to data size, data complexity, and serialization framework. Through active sampling, the data size of the node is sampled and counted in real time, and a linear regression model of data size -> serialization / deserialization time is established.

[0049] Data transmission delay (Ttransfer): The time required for data transmission in the network. This item is mainly determined by the data volume and effective network bandwidth.

[0050] Wherein EffectiveBandwidth is a variable set according to the actual physical environment when the system is initialized.

[0051] Tremote_queue: the time for a task to wait in the queue of a remote executor after arriving at the remote executor due to the busy of the executor. This item is calculated by the node itself. After receiving a request, if the task enters the queue, the time of entering the queue is recorded, and the waiting time of the task in the queue is calculated after the task is dequeued, and the result is returned to the workflow engine together with the execution result.

[0052] Delay penalty rule (two-level penalty): If the estimated Tnetwork> 50 ms (a significant delay perception threshold), a penalty is applied to the decision score .

[0053] On the basis of the above conditions, if the data volume < 1 MB and Tnetwork> 50 ms (i.e. small data volume but high network delay), an additional penalty is applied (the total penalty from the network at this time is ).

[0054] After completing the third layer, enter the fourth layer decision.

[0055] Fourth layer: execution mode decision layer.

[0056] Decision threshold setting: set the decision threshold Threshold_decision = 60.

[0057] Final comparison: if the final cumulative score S core ≥ Thresholddecision, select independent execution mode; otherwise, select embedded (embedded) execution mode. After selection, the corresponding execution path and resource / permission configuration synthesis are performed.

[0058] S3. After determining the independent execution mode or the embedded execution mode, perform the workflow execution according to the corresponding mode path: Execution after selecting independent execution mode (containerized execution path) If it is determined to be an independent execution mode, or it is determined to be an independent execution mode after scoring, the system enters the independent execution path. The specific process of this path is as follows: Configuration synthesis and container parameter translation: Read each parameter of the resource constraint of the independent execution mode in the policy configuration file and translate it into a container startup parameter; Read the permission permission in the policy configuration file: Translate the file directory allowed to read, the file directory allowed to write into the container mount and network policy; Translate the domain name or IP allowed to access into the container-level network access whitelist / security group or DNS / routing rule, and inject the environment variable allowed to access into the container environment variable; Select or generate Seccomp profile according to the permission requirements to specify the call whitelist; Container security and resource isolation: The container runtime implements hard quotas (CPU, memory, I / O) through Linux kernel Cgroups (v1 / v2); process / network / mount point isolation through namespaces (PID, Network, Mount, etc.); and minimizes the kernel attack surface by loading custom system call whitelist through Seccomp. The root file system of the container can be mounted in read-only mode according to the policy, and the necessary data directory can be mounted in read-only or read-write mode according to the policy requirements.

[0059] Container startup command synthesis and scheduling: Calculate the hash value (SHA-256) of the complete container startup command text, and use the hash value and label as the image / startup command index to find the existing container node cluster with the same label in the cluster; if there is a matching container node cluster, send an execution request to the cluster; if there is no match, initialize the corresponding container node cluster with the configuration through the container runtime API and send a request for execution. The scheduling, network access, volume mounting, etc. of the container are completed in this step, and after completion, the containerized execution is entered and the running state is monitored until the end of execution (flow termination).

[0060] Selection of embedded execution mode after execution (Worker thread execution path) If it is forcibly determined as embedded execution mode, or it is determined as embedded execution mode after scoring, the system enters the embedded execution path. The specific implementation and control flow of this path are as follows: Read the resource constraints of the embedded execution module in the strategy configuration file: the configured memory upper limit, Tcpu_max, Twall_max, etc. as the running parameters when creating a worker thread (using worker_threads in Node.js) and external monitoring thresholds. For example, when creating a worker thread, limit the heap memory upper limit at the V8 layer (the example value is translated as: set the V8 heap memory upper limit for the worker thread to maxOldGenerationSizeMb = 256 MB, maxYoungGenerationSizeMb = 64 MB), so that when the memory attempts to exceed this limit, an exception is thrown by V8 to cause the thread to crash, thereby achieving hard isolation of memory.

[0061] CPU and monitoring of execution hang (soft control and watchdog): Implement CPU usage and hang detection using event loop monitoring and heartbeat mechanism. The specific implementation steps are as follows: (1) Initialization: Retrieve Tcpu_max and Twall_max in the strategy configuration when creating a worker in the main thread, and set two thresholds based on these values. The main thread starts a total wall time timer (e.g. through setTimeout) for this worker, with a duration of Twall_max; if the timer expires and the task is still not completed, trigger the timeout fuse and terminate the execution of the worker.

[0062] (2) Worker side heartbeat: Start a high-priority heartbeat timer in the worker that is decoupled from the business logic. This heartbeat periodically (e.g. every second or at the interval specified by the strategy) reads the CPU execution time statistics file of the current thread in the Linux kernel; calculate the CPU time increment of the current thread since the last heartbeat, and send a heartbeat packet including the field cpuTime (i.e. cumulative CPU usage time T_cpu_current) to the main thread through the message channel.

[0063] (3) Main thread arbitration: The main thread resets a heartbeat timeout timer (e.g. 5 seconds) when it receives each heartbeat packet; if the heartbeat timeout timer expires, it is determined that the event loop may be blocked (hang) and the fuse is triggered. The main thread checks the cumulative CPU time T_cpu_current in the heartbeat packet after receiving it, and if T_cpu_current > Tcpu_max, it triggers the CPU over-limit fuse. Through this mechanism, memory overflow is guaranteed by the V8 layer, and CPU abuse and event loop hang are guaranteed by the external monitoring mechanism.

[0064] Dynamic adaptation of permissions based on runtime characteristics: In the embedded mode, the system probes the current runtime type (Deno, Node.js, or Bun) before creating the worker and takes different permission adaptation strategies based on the probing results: (1) If it is a Deno runtime, the policy permission is accurately translated into a Deno permission object and passed in when creating a worker (for example, the permission to read / data / images / is converted to the Deno permission object read: [" / data / images / "]). The Deno runtime enforces the lock at the child thread level and cannot be bypassed by code, thereby achieving the highest level of embedded mode security.

[0065] (2) If it is Node.js or Bun, the "best effort" proxy isolation is adopted: a bootstrap script is injected when the worker starts, and the script initializes the permission manager and receives the permission policy allocated by the main thread. By rewriting the key functions of high-risk modules (such as node:fs and node:child_process) using Object.defineProperty or Proxy, the operation parameters are checked before execution to see if they are in the allowed list. If not, an "access denied" exception is thrown. This method can prevent most non-compliant calls, but there is still a risk of intentional bypass through underlying process.binding, so it is called the best effort mode.

[0066] Worker acquisition and recycling: The system acquires a worker that meets the policy requirements from the thread pool (if not, it is newly created), injects the above resource restrictions, heartbeat, and permission policy, and starts execution until the task is completed or the monitoring mechanism triggers the fuse. After execution, the worker is recycled and the execution statistics (including actual Tcpu, Twall, serialized data size, network interaction statistics, etc.) are reported for subsequent runtime statistics and model training.

[0067] A workflow execution system based on a distributed architecture, comprising: A policy configuration and mandatory rule determination unit for loading node policy configuration when a workflow node execution request is received, and checking whether the mandatory rule belongs to mandatory independent execution or mandatory embedded execution; A decision score superposition and execution mode determination unit for initializing the decision score when the mandatory rule does not belong to mandatory independent execution or mandatory embedded execution, calculating and accumulating / deducting the score layer by layer to form the final decision score and determine the independent execution mode or embedded execution mode; A workflow execution unit is configured to execute the workflow according to the corresponding mode path after determining the independent execution mode or the embedded execution mode.

[0068] The above description is preferred embodiments of the present application, it should be understood that the present application is not limited to the forms disclosed herein, should not be seen as excluding other embodiments, but can be used in other combinations, modifications and environments, and can be modified within the scope of the concepts described herein, by the above teachings or related art or knowledge. The modifications and changes made by those skilled in the art without departing from the spirit and scope of the present application shall be within the scope of the appended claims of the present application.

Claims

1. A method for workflow execution based on a distributed architecture, characterized in that: The method comprises the following steps: S1. When the workflow node execution request is received, load the node policy configuration, and check whether the mandatory rule belongs to mandatory independent execution or mandatory embedded execution: If it is mandatory independent execution, select independent execution mode, and go to step S3; If it is mandatory embedded execution, select embedded execution mode, and go to step S3; If it neither belongs to mandatory independent execution nor mandatory embedded execution, go to step S2; S2. Initialize the decision score, calculate and accumulate / deduct the score layer by layer to form the final decision score and determine the independent execution mode or embedded execution mode; S3. After determining the independent execution mode or embedded execution mode, perform the workflow according to the corresponding mode path.

2. The workflow execution method based on a distributed architecture according to claim 1, characterized in that: The S1 comprises: S101. When the workflow node execution request is received, load the node policy configuration, including the policy configuration file corresponding to the workflow node, wherein the policy configuration file declares the resource constraints and permission permissions of the node in two execution modes in a structured configuration manner, and the two execution modes include embedded execution mode and independent execution mode; The resource constraints of the embedded execution mode include: configured upper limit of memory, maximum CPU execution time Tcpu_max, and maximum wall execution time Twall_max; The resource constraints of the independent execution mode include: configured upper limit of memory, maximum shared CPU core number, and maximum exclusive CPU core number; The permission permissions in the embedded execution mode and the independent execution mode both include: allowed file directory for reading, allowed file directory for writing, allowed domain name or IP for accessing, and allowed environment variable for accessing; S102. Determine whether the current workflow node type belongs to the pre-defined mandatory independent execution list: If yes, directly select the independent execution mode, and go to the running path of the independent execution; If no, go to step S103; S103. Determine whether the current workflow node type belongs to the pre-defined mandatory embedded execution list: If yes, directly select the embedded execution mode, and go to the running path of the embedded execution; If no, go to step S2.

3. The workflow execution method based on distributed architecture according to claim 1, characterized in that: The S2 comprises: S201. Initialize the decision score: Score = 0; S202. First layer evaluation: determine the priority of the node CPU intensity source, and accumulate the decision score; S203. Second layer evaluation: monitor the load state of the current workflow engine in real time, determine the load score, and convert it into the decision score for accumulation; S204. Third layer evaluation: estimate the end-to-end delay cost introduced by network communication, and accumulate the decision score penalty to obtain the final decision score; S205. Fourth layer decision: compare the final decision score with the decision threshold Threshold_decision, if the final accumulated score Score ≥ Threshold_decision, select the independent execution mode, otherwise, select the embedded execution mode.

4. The workflow execution method based on a distributed architecture according to claim 3, characterized in that: The S202 comprises: A1, determine the priority of the node CPU intensity source: A11, whether the intensity has been explicitly set in the node configuration during workflow design, including high intensity, medium intensity, and low intensity; If yes, use the setting, with the highest priority; if not, go to step A12; A12, check the node preset properties, including high intensity, medium intensity, and low intensity; if present, use the preset, with the second priority; A13, if both of the above do not exist, then make a statistical determination at runtime, the runtime statistical method includes: (a) Before the node executes: record the current time, denoted as Tstart_wall ， Read the cumulative CPU usage time of the Cgroup where the node process is located, denoted as CPUstart; (b) After the node executes: Record the current time, denoted as Tend_wall ， Again read the cumulative CPU usage time of the Cgroup where the node process is located, denoted as CPUend; (c) calculating the difference value: the total execution time of the node is , and the total CPU consumption time of the node value execution is ; (d) Calculate CPU intensity: CPU intensity D is defined as the ratio of CPU time to total execution time, considering that the node uses multi-core parallel computing, the wall time needs to be multiplied by the number of CPU cores Ncores available to the node for normalization, the calculation formula is: Dnormalized=Tcpu / (Twall×Ncores ) Where: Ncores is the number of CPU cores allocated to the Cgroup; A2, intensity level determination and score rules: By setting two thresholds Thresholdhigh and Thresholdlow, the node is divided into three levels: High intensity: if Dnormalized≥Thresholdhigh; Medium intensity: if Thresholdlow≤Dnormalized<Thresholdhigh; Low intensity: if Dnormalized<Thresholdlow; A3, decision score accumulation: If the intensity source is "workflow design setting" or "node preset", the system also applies the above score rules according to the set level.

5. The workflow execution method based on a distributed architecture according to claim 3, characterized in that: The S203 includes: B1, index collection: Real-time monitoring of the current workflow engine load state, including CPU usage, memory usage, task queue backlog, and P95 execution delay in current execution; B2, since the dimensions of each index are different, they need to be normalized to the [0, 1] interval first, where 0 represents the ideal state and 1 represents the critical or overload state; set a warning threshold Twarn and a critical threshold Tcrit for each index Mi; B3, weighted sum: Assign a weight Wi to each normalized index score Ni, representing the importance of the index in evaluating system load, and the sum of all weights is 1; Where i takes cpu, mem, queue, and latency, then: Ni is Ncpu, Nmem, Nqueue, and Nlatency, representing the index scores of CPU usage, memory usage, task queue backlog, and P95 execution delay, respectively; Wi is Wcpu, Wmem, Wqueue, and Wlatency, representing the weights of CPU usage, memory usage, task queue backlog, and P95 execution delay, respectively; Total load score Sload calculation formula: Sload = Wcpu Ncpu + Wmem Nmem + Wqueue Nqueue + Wlatency Nlatency The value of Sload ranges from [0, 1], and the higher the value, the closer the system is to overload; Convert the system load score to a decision score and accumulate it: S core += Sload x 100.

6. The workflow execution method based on a distributed architecture according to claim 3, characterized in that: The S204 comprises: Estimate the end-to-end delay cost introduced by the network communication of the node, which is composed of the following parts, and the calculation formula is: Tnetwork = Thandshake + (Tserialize + Tdeserialize) + Ttransfer + Tremote_queue Wherein, Thandshake is the time required for the client to establish a TCP connection and TLS handshake with the remote executor; Tserialize + Tdeserialize is the serialization / deserialization delay, wherein: Tserialize represents the sum of serialization time of remote execution request and serialization time of remote execution result; Tdeserialize represents the sum of deserialization time of remote execution request and deserialization time of remote execution result; Ttransfer is the time required for data transmission in the network; Tremote_queue is the time for the task to enter the queue due to the busy of the executor after arriving at the remote executor; Delay penalty rules: (1) If the estimated Tnetwork > 50 ms, then apply a penalty to the decision score ; (2) On the basis of (1), if the data volume < 1 MB and Tnetwork > 50 ms, then an additional penalty .

7. The workflow execution method based on distributed architecture according to claim 1, characterized in that: In the S3, if it is forced to be in independent execution mode or judged to be in independent execution mode, it enters the independent execution path, comprising: C1, configuration synthesis and container parameter translation: Read each parameter of the resource constraint of the independent execution mode in the policy configuration file and translate it into the container startup parameter; Read the permission permission in the policy configuration file: Translate the file directory allowed to read and the file directory allowed to write into the container mounting and network policy; Translate the domain name or IP allowed to access into the container-level network access whitelist / security group or DNS / routing rule, and inject the environment variable allowed to access into the container environment variable; Select or generate a Seccomp configuration file according to the permission requirement to specify the call whitelist; C2, container security and resource isolation: The container runtime implements hard quota through the Linux kernel Cgroups; Through the namespace, process / network / mount point isolation is realized; Load a custom system call whitelist through Seccomp to minimize the kernel attack surface; The root file system of the container is mounted in read-only mode according to the policy, and the necessary data directory is mounted in read-only or read-write mode according to the policy requirement; C3, container startup command synthesis and scheduling: Calculate the hash value of the complete container startup command text, and use the hash value and label as the image / startup command index to find the container node cluster with the same Label in the cluster; If there is a matching container node cluster, send an execution request to the cluster; If not, initialize the container node cluster corresponding to the configuration through the container runtime API and send a request for execution; The scheduling of the container, network access, and volume mounting are all completed by this step. After completion, the containerized execution is entered and the running state is monitored within the container until the execution ends.

8. The workflow execution method based on a distributed architecture according to claim 1, characterized in that: In the S3, if it is forcibly determined as the embedded execution mode or it is determined as the embedded execution mode through scoring, the embedded execution path is entered, including: D1, reading the resource constraints of the embedded execution module in the policy configuration file, taking the configured upper limit of memory, Tcpu_max and Twall_max as the running parameters when creating the Worker thread and the external monitoring threshold; D2, monitoring of CPU and execution of false death: The event loop monitoring and heartbeat mechanism are adopted to realize the CPU usage and false death detection, and the specific implementation steps are as follows: D21, initialization: retrieving Tcpu_max and Twall_max in the policy configuration when creating the Worker in the main thread, and setting two thresholds according to the same; the main thread starts a total wall time timer for the Worker, and the time length is Twall_max; if the timer expires and the task is not ended, the timeout fuse is triggered and the execution of the Worker is terminated; D22, heartbeat on the Worker side: starting a high-priority heartbeat timer decoupled from the business logic inside the Worker; the heartbeat periodically reads the CPU execution time statistics file of the current thread in the Linux kernel, calculates the CPU time increment of the current thread since the last heartbeat, and sends the heartbeat packet including the field cpuTime to the main thread through the message channel; D23, main thread decision: the main thread resets a heartbeat timeout timer when receiving each heartbeat packet; if the heartbeat timeout timer expires, it is determined that the event loop is blocked and the fuse is triggered; the main thread checks the cumulative CPU time T_cpu_current in the heartbeat packet after receiving it, and if T_cpu_current > Tcpu_max, the CPU over-limit fuse is triggered; D3, dynamic adaptation of permissions based on runtime characteristics: In the embedded mode, the system detects the current runtime type before creating the Worker, and adopts different permission adaptation strategies based on the detection results: D31, if it is a Deno runtime, the policy permission is accurately translated into a Deno permission object and passed into the Worker when it is created; the Deno runtime is forcibly locked at the child thread level and cannot be bypassed by code, thereby realizing the highest level of embedded mode security; D32, if it is Node.js or Bun, the "best effort" proxy isolation is adopted: a bootstrap script is injected when the Worker is started, which initializes the permission manager and receives the permission policy allocated by the main thread; by rewriting the key functions of high-risk modules, it checks whether the operation parameters are in the allowed list before execution; if not, an "access denied" exception is thrown; D4, Worker acquisition and recycling: Get a worker from the thread pool that meets the policy requirements, if not, create a new one, inject the above resource limits, heartbeat and permission policy, and start execution until the task is completed or triggered by the monitoring mechanism; After the execution is completed, the worker is recycled and the execution statistics are reported for subsequent runtime statistics and model training.

9. A workflow execution system based on a distributed architecture, employing the method of any one of claims 1 to 8, characterized in that: Comprise: A policy configuration and mandatory rule determination unit is configured to load node policy configuration when a workflow node execution request is received, and to check whether the mandatory rule belongs to mandatory independent execution or mandatory embedded execution. A decision score superposition and execution mode determination unit is configured to initialize a decision score when the mandatory rule does not belong to mandatory independent execution or mandatory embedded execution, to calculate and accumulate / deduct the score layer by layer, to form a final decision score, and to determine an independent execution mode or an embedded execution mode. A workflow execution unit is configured to perform workflow execution according to the corresponding mode path after determining the independent execution mode or the embedded execution mode.

Images