Intelligent command based on workflow engine algorithm and data processing method thereof

Abstract

The application discloses an intelligent command based on a workflow engine algorithm and a data processing method thereof, and relates to the technical field of intelligent command. The method comprises the following steps: a system receives a task through a user interface and generates the task, adopts an earliest deadline first algorithm to schedule the task according to slackness, intelligently allocates geographical position and state priority resources, optimizes a multi-node workflow path based on dynamic programming, monitors execution progress and alarms in real time, and stores full life cycle data in a relational database and a non-relational database; an improved C4.5 decision tree model based on weighted entropy is further introduced to replace the earliest deadline first algorithm, historical task data is cleaned, classified and trained, emergency degree prediction and dynamic priority adjustment are realized, scheduling strategies are adaptively optimized, and emergency response efficiency and resource utilization are significantly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent command, in particular to an intelligent command based on a workflow engine algorithm and a data processing method thereof. BACKGROUND

[0002] The intelligent command and dispatch technology field focuses on how to model, analyze and optimize complex task processes through information means, integrate various resource information, and use specific algorithm strategies to ensure that command and dispatch activities can be accurately, quickly and orderly carried out under multiple tasks, multiple resources and multiple constraint conditions, so as to meet the key needs of timeliness, accuracy and efficiency of command and dispatch in many industry scenarios such as police research and judgment attack, emergency rescue, traffic and logistics deployment. It is different from adjacent technical fields such as general information system management or pure data processing, and is a specific technical field for intelligent operation of command and dispatch processes.

[0003] In the intelligent command and dispatch technology field, the existing command and dispatch systems mostly adopt traditional task allocation and management modes, but these technical solutions have the following significant defects:

[0004] Task scheduling is not flexible: the existing command and dispatch systems often use task scheduling methods based on fixed rules or simple order. Many systems allocate resources and execute tasks in the order of task arrival, lacking dynamic adjustment capability. This method does not consider complex factors such as the urgency of tasks, execution time and resource demand, resulting in some important and urgent tasks being delayed, which affects overall efficiency and timely completion of tasks. For example, in emergency rescue tasks, if resource allocation is not timely, the best opportunity for rescue may be missed, causing serious consequences.

[0005] Fixed priority, lack of flexibility: some command and dispatch systems use scheduling strategies based on fixed priority, i.e. setting fixed priority levels for different types of tasks in advance, and executing them according to priority. This method ensures the execution order of tasks to some extent, but also has the problem of lack of flexibility. Especially in the case of limited resources, fixed priority scheduling may lead to intense resource competition, and some tasks may be stalled due to waiting for resources, thereby reducing the response speed and overall efficiency of the system. Resources cannot be dynamically adjusted according to actual conditions, resulting in low resource utilization and inability to meet the needs of high-priority tasks in a timely manner.

[0006] Task path optimization is insufficient: when dealing with complex task flows, existing technologies often face the problem of being unable to effectively optimize task execution paths. Existing scheduling schemes are usually difficult to deal with workflows with multiple task nodes and complex dependency relationships, and cannot accurately calculate the optimal path from the starting point to the target, whether from the perspective of the shortest execution time or the lowest cost. Ultimately, the path cannot achieve the best effect. Especially in scenarios involving multi-node dependency relationships (such as logistics distribution, production scheduling, etc.), traditional solutions often fail to comprehensively optimize task paths, resulting in resource waste and low task execution efficiency. SUMMARY

[0007] The purpose of the present application is to provide an intelligent command and dispatch method based on workflow engine algorithm, which optimizes task scheduling, resource allocation and path selection, improves task execution efficiency, resource utilization, and enhances the adaptability and flexibility of the system in complex scenarios.

[0008] The technical solution of the present application is to provide an intelligent command based on workflow engine algorithm and its data processing method, which comprises:

[0009] S1, task receiving and creating: the system receives task information, and the task management module automatically generates corresponding tasks according to the specific information of the tasks. The task management module submits the tasks to the workflow engine module for subsequent processing;

[0010] S2, task scheduling: the workflow engine module uses the earliest deadline first algorithm to schedule the tasks, and calculates the slack of each task L i :

[0011] ;

[0012] Among them, D i is the deadline of the task, A i is the arrival time of the task, C i is the execution time of the task, and the subscript i indicates the task number; the workflow engine module preferentially selects the task with the smallest slack for scheduling and execution;

[0013] S3, resource allocation: the workflow engine module passes the resource requirements of the task to the resource management module. The resource management module dynamically adjusts and intelligently selects appropriate resources for allocation according to the priority of the task, the type of the task and the available state of the resource, optimizes the utilization of the resource and avoids resource conflicts;

[0014] S4, workflow path selection and optimization: the workflow engine module constructs the task as a directed acyclic graph, and adopts a dynamic programming algorithm to calculate the longest path of the task flow, so as to select the optimal execution path to ensure that the execution time or cost of the task is minimized, and the specific formula of path calculation is: ; wherein, d v is the longest path length from the starting node to the task node v , d u is the longest path length from the starting node to the task node u , the task node u is the node between the starting node and the task node v , w u, v ) is the weight of the task node u to the task node v , representing the execution time or resource consumption metric value from the task node u to the task node v , E is the set of dependency relationships between task nodes;

[0015] S5, task execution and monitoring: after the task is assigned resources, the task begins to execute, and the task management module monitors the execution progress of the task in real time, the monitoring and alarm module tracks the task execution situation, and timely alarm notification is sent;

[0016] S6, data storage: the system selects a suitable database according to the type of data to store all task information, execution log, resource allocation information, and task progress data;

[0017] S7, after a period of time after the above steps S1 to S6, the system introduces a scheduling method based on machine learning to replace the earliest deadline first algorithm of step S2 to continue to work, the new scheduling method combines historical task data to construct a scheduling model, predicts the urgency of the task through the construction of the decision model, and dynamically adjusts the execution order of the task accordingly.

[0018] In any of the above technical solutions, further, step S7 specifically includes:

[0019] S71, collect historical task data from the data storage module, including task arrival time, deadline, execution time, task type, resource consumption, actual completion time, task delay, and task slack; clean, remove outliers and normalize the historical task data to construct a training sample data set in a unified format S ;

[0020] S72, set a task slack threshold according to experience L 0 ​, according to the task slack, whether the task is delayed two indicators of training sample data set S In the sample annotation, form three categories of task data: category 1, task slack L i > L 0 and delayed; category 2, task slack L i < L 0 and delayed; category 3, the task does not occur delay;

[0021] S73, build C4.5 decision tree, introduce weight factor α j Definition of data set S Weighted entropy E mod ( S ), the weighted entropy E mod ( S ) to optimize the standard C4.5 algorithm, weighted entropy E mod ( S ) expression is:

[0022] ;

[0023] Where, the subscript j Three categories of data in the data set, p j The proportion of the data set S Belongs to the first j Category, weight factor α 1 > α 2 > α 3 , the task is more urgent and delayed to give higher weight;

[0024] Next, the system uses the improved formula for each task attribute A Information gain Gain mod ( S, A ) calculation:

[0025] ;

[0026] Where, S n The attribute A Take value n When the subset; then, by calculating the improved information gain rate of each attribute, the amount of split information SplitInfo (A

[0027]

[0028] Final definition of weighted information gain rate GainRatio mod A

[0029]

[0030] S74, the completed decision tree model will evaluate the priority of the newly received task in real time in the system scheduling stage, dynamically schedule and sort according to the task category output by the model, and allocate in combination with the available resources;

[0031] S75, the system periodically collects newly generated task execution data, updates the data set S , and re-trains the scheduling model based on the latest data.

[0032] In any of the above technical solutions, further, the task management module interacts with the system through the user interface module, providing task creation, submission, monitoring and result query functions.

[0033] In any of the above technical solutions, further, the resource management module realizes intelligent allocation of resources through the following logic:

[0034] According to the requirements of the task, the resources are preferentially allocated to high-priority tasks; in the case of limited resources, the available resources closest to the task execution location are preferentially allocated; and whether the resource is allocated to the current task is determined according to the busy state of the resource.

[0035] In any of the above technical solutions, further, the monitoring and alarm module monitors the running state of the system in real time, including the task execution situation, the resource usage situation and the performance indicators of the system; once the task timeout, resource anomaly or other problems are found, the system will timely issue an alarm and provide an exception report, helping the administrator to respond and handle in time, and ensuring the smooth completion of the task.

[0036] In any of the above technical solutions, further, the data storage module guarantees the security and efficient query of data through various storage strategies, supports the storage mode of relational database, non-relational database or distributed file system.

[0037] In any of the above technical solutions, further, the data storage module records and traces the life cycle of each task, including the detailed information of task creation, scheduling, resource allocation and execution progress.

[0038] The beneficial effects of the present application are:

[0039] ​​​​​The technical solution in the application introduces the earliest deadline first algorithm to dynamically schedule tasks. The earliest deadline first algorithm prioritizes the task with the smallest slack based on the time characteristics of the task (such as arrival time, deadline, etc.). This strategy breaks through the limitations of traditional fixed priority scheduling and simple sequential scheduling, and can reasonably allocate resources according to actual task requirements, ensure that urgent tasks are executed first, and avoid task delays. For example, in an emergency rescue scenario, the system can dynamically adjust the execution order of tasks according to the urgency of the tasks, ensuring that critical tasks such as the allocation of rescue supplies are not delayed.

[0040] In scenarios involving complex task flows, the application introduces a workflow path selection algorithm based on dynamic programming. This algorithm effectively solves the task path optimization problem in the prior art by modeling the task flow (i.e., constructing the task as a directed acyclic graph). The dynamic programming algorithm can calculate the longest path of task execution, optimize the task execution path from multiple dimensions such as execution time and cost, ensure that the task is completed in the optimal way, maximize the utilization efficiency of resources, and reduce resource waste in task execution. For example, in a logistics distribution scenario, the system can optimize the distribution route, reduce transportation time and cost, and improve logistics efficiency.

[0041] In the preferred implementation of the application, the application combines a workflow engine and a resource management module to intelligently real-time allocate resources according to task requirements and resource states. This not only optimizes resource utilization, but also avoids resource idling and waste. Through accurate resource allocation algorithms, the system can dynamically adjust resource allocation according to real-time task requirements, so that resources can be optimally utilized during task execution. This method greatly improves the efficiency of task completion, especially in large-scale and complex task scheduling scenarios.

[0042] In the preferred implementation of the application, the system introduces an improved C4.5 decision tree model based on weighted entropy to clean, normalize, and weight classify historical rescue task data including task slack, delay flag, resource consumption, etc., and construct a dynamic scheduling prediction model. When a new task arrives, it evaluates its urgency in real time and dynamically adjusts the scheduling priority. The model is automatically retrained periodically through execution feedback to form a closed-loop self-learning and adaptive optimization, greatly improving emergency response speed, task punctuality rate, and resource utilization efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0043] The advantages of the above and additional aspects of the application will become apparent and easily understood in connection with the description of embodiments with reference to the following drawings, in which:

[0044] Fig. 1This is a block diagram of the overall architecture of intelligent command based on a workflow engine algorithm and a data processing method thereof according to an embodiment of the present invention;

[0045] Fig. 2 This is a flowchart of a task scheduling algorithm based on earliest deadline priority and an intelligent command and data processing method based on a workflow engine algorithm according to an embodiment of the present invention;

[0046] Fig. 3 The present invention is a schematic diagram of a workflow path selection algorithm based on dynamic programming of an intelligent command based on a workflow engine algorithm and a data processing method thereof according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features therein can be combined with each other without conflict.

[0048] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0049] like Figs. 1 to 3 As shown, this embodiment provides an intelligent command and data processing method based on a workflow engine algorithm, the method comprising:

[0050] S1. Task Receiving and Creating: First, the system receives task information through the user interface module. The task management module automatically generates the corresponding task based on the task's specific information (such as its nature, execution time, and deadline). Task information includes, but is not limited to, the task's arrival time, execution time, task priority, and deadline. After a task is created, the task management module submits it to the workflow engine module for scheduling and processing. The workflow engine module schedules the task based on actual requirements and passes the task information to other modules for subsequent processing.

[0051] S2. Task Scheduling: The task management module schedules tasks using the earliest deadline first (EDF) algorithm through the workflow engine module. The slack of each task L i The calculation formula is:

[0052] ;

[0053] in, D iis the deadline of the task, A i is the arrival time of the task, C i is the execution time of the task; the workflow engine module prioritizes tasks with the smallest slack for scheduling and execution based on the above calculation results. This scheduling strategy ensures that time-sensitive tasks are prioritized for execution, thereby avoiding

[0054] task delays and significantly improving the on-time completion rate of tasks, especially in time-critical task scenarios such as emergency rescue.

[0055] S3, Resource Allocation: After task scheduling, the workflow engine module passes the resource requirements of the task to the resource management module. The resource management module dynamically adjusts and allocates resources based on factors such as task priority, resource availability, and resource geographic location. The resource management module implements intelligent resource allocation through the following logic:

[0056] According to the requirements of the task, resources are preferentially allocated to high-priority tasks; in the case of limited resources, the closest available resources to the task execution location are preferentially allocated; and according to the state of the resources (such as idle, busy, maintenance, etc.), it is determined whether the resources are allocated to the current task.

[0057] In this way, the system can maximize the efficiency of resource use, avoid waste and idling of resources, improve resource utilization and reduce operating costs.

[0058] S4, Workflow Path Selection and Optimization: For task flows involving multiple task nodes and dependencies, the workflow engine module constructs the task as a directed acyclic graph and uses a dynamic programming algorithm to calculate the optimal execution path of the task flow. The selection of the workflow path is based on the execution time and resource consumption of each task node, and the specific path selection is calculated according to the following recursive formula to calculate the longest path:

[0059] ;

[0060] where, d v is the longest path length from the starting node to task node v , d u is the longest path length from the starting node to task node u , u is the node between the starting point and task node v , w ( u, v ) is the weight of task node u to task node v , representing the weight of the edge between task node u and task nodev the execution time or resource consumption metric value of the task node, E is a set of dependency relationships between task nodes.

[0061] S5, task execution and monitoring: after the task is allocated resources, the task starts to execute, and the task management module tracks the execution progress of the task in real time through the monitoring system, including the completion status, execution time, resource consumption and other information of the task. During the execution of the task, the system updates the execution status of the task according to the predetermined time interval (such as every 10 minutes), and records the execution log.

[0062] The monitoring and alarm module monitors the running state of the system in real time, including the task execution, resource usage and performance indicators of the system; once the task is found to be overdue, the resource is abnormal or other problems, the system will timely issue an alarm and provide an exception report to help the administrator respond and handle in time to ensure the smooth completion of the task.

[0063] S6, data storage: all task information, execution log, resource allocation information, task progress and other data are stored through the data storage module. According to the type of data, the system selects different types of databases (such as relational databases, non-relational databases or distributed file systems) for storage to ensure the safety, queryability and efficiency of the data. The data storage module ensures that the system can record and trace the life cycle of each task, including task creation, scheduling, resource allocation, execution progress and other detailed information, providing a solid foundation for subsequent task analysis and decision-making of the system.

[0064] S7, after a period of time after the above steps S1 to S6, in order to further improve the rationality and intelligence of task scheduling, the system introduces a scheduling method based on machine learning to replace the EDF algorithm of step S2, constructs a scheduling model combined with historical task data, predicts the urgency of the task through the construction of the decision model, and dynamically adjusts the execution order of the task according to the prediction, so as to realize higher scheduling efficiency and system response ability; specific steps include:

[0065] S71, collect historical task data from the data storage module, including task arrival time, deadline, execution time, task type, resource consumption, actual completion time, task delay and task slack; clean, remove outliers and normalize the historical task data to construct a training sample data set in a unified format S .

[0066] S72, set the task slack threshold according to experience L 0 , according to the task slack, whether the task is delayed or not two indicators to mark the samples in the training sample data set S , form three kinds of task data:

[0067] Category 1: Task slackness L i > L 0 and delay occurs.

[0068] Category 2: Task slackness L i < L 0 and delay occurs.

[0069] Category 3: Task does not delay.

[0070] S73, build C4.5 decision tree, introduce weight factor α j Define data set S Weighted entropy E mod ( S ), weighted entropy E mod ( S ) is optimized to the standard C4.5 algorithm, and the weighted entropy E mod ( S ) expression is:

[0071] ;

[0072] Wherein, the subscript j Indicates three kinds of data in the data set, p j Indicates the proportion of the data set S Belongs to the first j Class, weight factor α 1 > α 2 > α 3 , give higher weight to the task of emergency and delay occurs.

[0073] Next, the system uses the improved formula to calculate the information gain A Gain mod ( S, A ) of each task attribute

[0074] ;

[0075] Wherein, S n Indicates the subset when the attribute A Value is n Then, by calculating the improved information gain rate of each attribute, the split information quantity​SplitInfo A

[0076]

[0077] Final definition of weighted information gain rate GainRatio mod A

[0078]

[0079] The construction of the decision tree not only considers the distribution characteristics of the data, but also integrates the delay consequences and the urgency of the slackness into the model training through weights, so that the model is more inclined to prioritize the scheduling of potential high-risk tasks when predicting.

[0080] S74, the constructed decision tree model will evaluate the priority of the newly received task in real time in the system scheduling stage, dynamically schedule and sort according to the task category output by the model, and allocate in combination with the available resources.

[0081] For tasks predicted as “low slackness and prone to delay”, the system will significantly improve their scheduling priority to reduce the risk of critical task delay.

[0082] S75, the system periodically collects newly generated task execution data, updates the data set S , and re-trains the scheduling model based on the latest data.

[0083] In order to better illustrate the embodiments of the present application, the application of the present application is specifically described in combination with the city emergency rescue command and dispatching scene.

[0084] When a large-scale fire accident occurs in a city, the city emergency command center needs to coordinate multiple departments and resources for rapid response. At this time, the intelligent command and dispatching system based on the workflow engine algorithm of the present application can realize efficient task scheduling and resource allocation according to the urgency of the task and the availability of the resources. The following is the specific implementation process:

[0085] S1, task receiving and creation: when citizens call the alarm phone or report the fire through other channels, the staff of the command center enters the alarm information into the system through the user interface module. The task management module automatically generates the corresponding emergency rescue task. The parameters of the task include: fire type, fire location, estimated execution time (time required to extinguish the fire), and deadline (time considering the spread of the fire and the maximum loss that may be caused).

[0086] ​​​​​​S2, task scheduling: After generating the above tasks, the task management module submits the fire rescue task to the workflow engine module for scheduling. The workflow engine module uses the earliest deadline first algorithm to schedule the fire rescue task. The slack of each task L i is calculated as:

[0087] ;

[0088] where, D i is the deadline of the task, A i is the arrival time of the task, C i is the execution time of the task; assuming the deadline of the fire rescue task is 15:00, the task arrival time is 14:30, and the execution time is 30 minutes, the slack is calculated to be 0 minutes, which is very urgent, and the workflow engine will prioritize the scheduling of this task to avoid the spread of the fire.

[0089] S3, resource allocation: After task scheduling, the workflow engine module passes the task requirements to the resource management module. The resource management module allocates resources according to the priority of the task, the available state of the resource, and the geographical location. For example: dispatch the nearest fire truck, allocate sufficient number of fire personnel and rescue equipment, and prioritize the allocation of professional equipment according to the type of fire.

[0090] The resource management module intelligently allocates resources based on the idle state of the resources and the requirements of the task. If a fire truck is executing other tasks or is in maintenance, the system will automatically select other available fire trucks for task scheduling.

[0091] S4, workflow path selection and optimization: When dealing with multiple tasks and dependencies, the workflow engine will construct the task as a directed acyclic graph. Assuming that the fire rescue task contains multiple subtasks such as personnel assembly, equipment preparation, traffic diversion, etc., the workflow engine module will calculate the optimal path based on the dynamic programming algorithm.

[0092] For example, in the fire rescue task, the starting node is "fire alarm reception", and the following task nodes include "fire personnel assembly", "fire truck dispatch", "go to the fire scene", etc. Assuming that the fire truck needs to pass through a traffic congestion section, the dynamic programming algorithm will adjust the optimal path according to the traffic conditions to select the path that can reach the fire scene in the shortest time. The dynamic programming algorithm calculates the optimal path based on the following formula:

[0093] ;

[0094] where, d vis the longest path length from the start node to the task node v , d u is the longest path length from the start node to the task node u , the task node u is the start node and the task node v , w ( u, v ) is the execution resource of the task node u to the task node v , the execution resource being a time parameter, E is the set of dependency relationships between task nodes.

[0095] S5, Task Execution and Monitoring: After the task begins to execute, the task management module monitors the task progress in real time to ensure that the task is completed according to the predetermined plan. The monitoring and alarm module will monitor the use of resources in real time, such as the remaining fuel of the fire truck, the consumption of fire equipment, etc.

[0096] Suppose during the rescue process, the fuel of the fire truck is about to run out, the monitoring and alarm module will promptly issue an alarm notification. The system will request the resource management module to allocate additional fuel or replace the fire truck according to the alarm information, ensuring that the rescue task is not affected.

[0097] S6, Data Storage: During the execution of the task, all task information, execution logs, resource allocation, task progress, etc. will be stored through the data storage module. The data storage module selects appropriate databases for storage according to the different types of data to ensure data security and query efficiency. After the task is completed, the system stores the task execution data into the database for subsequent analysis and evaluation, ensuring the traceability of the entire life cycle of the task.

[0098] S7, Dispatching Optimization and Model Self-learning: After frequent occurrence of fire rescue tasks or after the system has been running for a period of time, the system automatically introduces a dispatching optimization module based on machine learning to replace the original EDF scheduling algorithm. This module extracts historical fire rescue task data (including task arrival time, deadline, execution time, resource usage, task completion, whether delayed, slack, etc.) from the data storage module, and cleans and classifies the task data.

[0099] The system divides historical tasks into three categories according to "slack" and "delay situation", and constructs a weighted C4.5 decision tree model, in which tasks that are delayed and urgent are given higher weights. Through indicators such as weighted entropy and improved information gain rate, the model learns the key attributes that affect task scheduling effectiveness.

[0100] When a new task arrives, the model can predict its urgency and dynamically adjust the scheduling priority accordingly.

[0101] The model is regularly updated and trained by new task data, forming a closed-loop "task-execution-evaluation-optimization" mechanism, realizing self-optimization and continuous evolution of the scheduling strategy, and further improving the efficiency of urban emergency response.

[0102] In summary, the present application proposes an intelligent command based on a workflow engine algorithm and a data processing method thereof, comprising:

[0103] S1, task receiving and creating: the system receives task information, and the task management module automatically generates corresponding tasks according to the specific information of the tasks; the task management module submits the tasks to the workflow engine module for subsequent processing.

[0104] S2, task scheduling: the workflow engine module adopts the earliest deadline first algorithm to schedule the tasks, and calculates the slack of each task L i :

[0105] ;

[0106] wherein, D i is the deadline of the task, A i is the arrival time of the task, C i is the execution time of the task; the workflow engine module preferentially selects the task with the smallest slack for scheduling and execution.

[0107] S3, resource allocation: the workflow engine module transmits the resource demand of the task to the resource management module; the resource management module dynamically adjusts and intelligently selects appropriate resources for allocation according to the priority of the task, the type of the task and the available state of the resource, optimizes the utilization rate of the resource and avoids resource conflicts.

[0108] S4, workflow path selection and optimization: the workflow engine module constructs the task as a directed acyclic graph, and adopts a dynamic programming algorithm to calculate the longest path of the task flow, so as to select the optimal execution path to ensure that the execution time or cost of the task is the smallest; the specific formula of path calculation is: ;

[0109] wherein, d v is the longest path length from the starting node to the task node v , d u is the longest path length from the starting node to the task node u , and the task node u is the node between the starting point and the task node v , w ​u, v ) is a task node u to the task node v is a weight of the task node u to the task node v is an execution time or resource consumption metric value of the task node E is a set of dependency relationships between task nodes.

[0110] S5, task execution and monitoring: after the task is allocated resources, the task starts to execute, and the task management module monitors the execution progress of the task in real time; the monitoring and alarm module tracks the task execution situation and timely sends an alarm notification.

[0111] S6, data storage: the system selects a suitable database according to the type of data to store all task information, execution logs, resource allocation information and task progress data.

[0112] S7, after a period of time of the above steps S1 to S6, the system introduces a scheduling method based on machine learning to replace the EDF algorithm of step S2; the new scheduling method combines historical task data to construct a scheduling model, predicts the urgency of the task through the construction of the decision model, and dynamically adjusts the execution order of the task accordingly.

[0113] The steps in the application can be adjusted in sequence, combined and deleted according to actual needs.

[0114] The units in the device of the application can be combined, divided and deleted according to actual needs.

[0115] Although the application has been disclosed in detail with reference to the accompanying drawings, it should be understood that the description is only exemplary and is not intended to limit the application of the application. The protection scope of the application is defined by the appended claims, and can include various modifications, modifications and equivalent schemes made to the application without departing from the protection scope and spirit of the application.

Claims

1. An intelligent command and data processing method based on workflow engine algorithm, characterized in that: The method comprises: S1. Task reception and creation: The system receives task information, and the task management module automatically generates corresponding tasks based on the specific information of the task. The task management module submits the task to the workflow engine module for subsequent processing; S2. Task Scheduling: The workflow engine module uses the earliest deadline first algorithm to schedule tasks and calculate the slack of each task. L i : ； in, D i is the deadline for the task, A i is the arrival time of the task, C i Is the execution time of the task, i Indicates the task sequence number; the workflow engine module prioritizes the task with the smallest slack for scheduling and execution; S3. Resource Allocation: The workflow engine module transmits the resource requirements of the task to the resource management module. The resource management module dynamically adjusts and intelligently selects appropriate resources for allocation based on the task priority, task type, and resource availability, optimizing resource utilization and avoiding resource conflicts. S4. Workflow path selection and optimization: The workflow engine module constructs tasks as a directed acyclic graph and uses a dynamic programming algorithm to calculate the longest path of the task flow, thereby selecting the optimal execution path to ensure the minimum execution time or cost of the task. The specific formula for path calculation is: ; in, d v From the start node to the task node v The longest path length, d u From the start node to the task node u The longest path length, task node u It is the starting point and task node v The nodes between w ( u,v ) is the task node u To the task node v The weight represents the weight of the task node u To the task node v Execution time or resource consumption metrics, E It is a set of dependencies between task nodes; S5. Task execution and monitoring: After resources are allocated, the task begins execution. The task management module monitors the progress of the task in real time. The monitoring and alarm module tracks the task execution status and issues alarm notifications in a timely manner. S6. Data storage: The system selects an appropriate database based on the data type to store all task information, execution logs, resource allocation information, and task progress data; S7. After performing steps S1 to S6 for a period of time, the system introduces a scheduling method based on machine learning to replace the earliest deadline first algorithm in step S2 and continue to work. The new scheduling method combines historical task data to build a scheduling model, and predicts the urgency of the task by building a decision model, and dynamically adjusts the task execution order accordingly.

2. The intelligent command and data processing method based on workflow engine algorithm according to claim 1, characterized in that: The step S7 specifically includes: S71. Collect historical task data from the data storage module, including task arrival time, deadline, execution time, task type, resource consumption, actual completion time, task delay, and task slack; clean, remove outliers, and normalize the historical task data to construct a training sample data set in a unified format. S ; S72. Establishing task slack thresholds based on experience L 0 , the training sample data set is analyzed based on two indicators: task slack and whether the task is delayed S The samples in are labeled to form three types of task data: Category 1, task slack L i > L 0 and delays occur; Category 2, task slack L i < L 0 and a delay occurred; Category 3, the task did not experience a delay; S73. Build a C4.5 decision tree and introduce weight factors α j Defining a Dataset S The weighted entropy of E mod ( S ), weighted entropy E mod ( S ) Optimize the standard C4.5 algorithm, weighted entropy E mod ( S ) is expressed as: ； Among them, the corner mark j Represents the three types of data in the dataset, p j Representation dataset S Belong to the j Class ratio, weight factor α 1 > α 2 > α 3 , giving higher weight to urgent and delayed tasks; Next, the system uses the improved formula to calculate the attributes of each task A Information gain Gain mod ( S,A ) to calculate: ； in, S n Representation attributes A The value is n Then, by calculating the improved information gain rate of each attribute, the split information amount is obtained SplitInfo ( A ): ； Finally, the weighted information gain rate is defined GainRatio mod ( A )for: ； S74. The constructed decision tree model will evaluate the priority of newly received tasks in real time during the system scheduling phase, dynamically schedule and sort the tasks according to the task categories output by the model, and allocate them based on the availability of resources. S75. The system regularly collects newly generated task execution data and updates the data set S , and retrain the scheduling model based on the latest data.

3. The intelligent command and data processing method based on workflow engine algorithm according to claim 1, characterized in that: The task management module interacts with the system through the user interface module, providing task creation, submission, monitoring and result query functions.

4. The intelligent command and data processing method based on workflow engine algorithm according to claim 1, characterized in that: The resource management module implements intelligent resource allocation through the following logic: Based on the requirements of the task, resources are allocated to high-priority tasks first; when resources are limited, available resources closest to the task execution location are allocated first; whether resources are allocated to the current task is determined based on the busy status of the resources.

5. The intelligent command and data processing method based on workflow engine algorithm according to claim 1, characterized in that: The monitoring and alarm module monitors the system's operating status in real time, including task execution, resource usage, and system performance indicators. Once a task timeout, resource anomaly, or other problem is detected, the system will promptly issue an alarm and provide an exception report to help administrators respond and handle the problem in a timely manner to ensure the smooth completion of the task.

6. The intelligent command and data processing method based on workflow engine algorithm according to claim 1, characterized in that: The data storage module ensures data security and efficient query through a variety of storage strategies, and supports storage methods such as relational databases, non-relational databases, or distributed file systems.

7. The intelligent command and data processing method based on workflow engine algorithm according to claim 1, characterized in that: The data storage module records and traces the life cycle of each task, including detailed information on task creation, scheduling, resource allocation, and execution progress.

Images