Distributed scheduling method based on tasks, nodes and link resources

Abstract

The invention relates to the technical field of data analysis and calculation, in particular to a distributed scheduling method based on tasks, nodes and link resources, which is applied to a solution for task scheduling and management in a large-scale distributed system. The method is particularly suitable for cloud computing, big data processing and distributed storage systems which need to process mass data and high-concurrency requests. And dynamically adjusting resource allocation by using real-time monitoring data and a prediction algorithm in combination with task priorities and resource requirements. Compared with traditional static resource allocation, a dynamic resource allocation mechanism is adopted, resource allocation can be intelligently adjusted in real time according to task requirements and node resource conditions, and therefore the resource utilization rate and the task execution efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the field of data analysis and computing technology, in particular to a distributed scheduling method based on tasks, nodes, and link resources, which is applied to a solution for task scheduling and management in a large-scale distributed system. It is particularly suitable for cloud computing, big data processing, and distributed storage systems that need to process massive data and high concurrent requests. Background Art

[0002] With the rapid development of Internet technology, the amount of data is growing exponentially, and the traditional centralized data processing method can no longer meet the growing computing needs. Distributed architecture achieves the expansion and optimization of computing resources by distributing data and computing tasks to multiple nodes. However, task scheduling in distributed systems faces the following challenges: Uneven resource utilization: Resource usage may vary among different nodes, leading to resource waste or bottlenecks.

[0003] Low task scheduling efficiency: Fixed resource allocation strategies cannot adapt to the dynamic changes of tasks, affecting the overall performance of the system.

[0004] System stability issues: Under high load or uneven resource distribution, the system may become unstable.

[0005] Lack of adaptability: Existing scheduling algorithms are difficult to adapt to diverse business scenarios and needs.

[0006] To overcome these challenges, this invention proposes a distributed scheduling method based on task, node and link resources, in order to achieve more efficient resource management and task scheduling. Summary of the invention

[0007] The present invention aims to provide an efficient distributed scheduling service by dynamically allocating and scheduling resources according to the real-time status of tasks and the load of nodes to achieve the following goals: Improve system efficiency: Ensure tasks are completed quickly and accurately through intelligent scheduling.

[0008] Enhance system stability: Through real-time monitoring and optimization adjustments, the system can ensure stable operation even under high load conditions.

[0009] Improve resource utilization: Avoid resource waste through dynamic resource allocation and ensure that the resources of each node are fully utilized.

[0010] Adapt to diverse business scenarios: Meet specific needs in different business scenarios through flexible task scheduling strategies.

[0011] The present invention solves the technical problem by adopting the following technical solution: S1. Task reception and classification: The system receives task requests submitted by users and classifies and sorts them; Attribute identification: Identify key attributes of tasks, including task type, priority, estimated execution time, resource requirements, dependencies, and task sources; resource requirements include CPU, memory, and storage space utilization load; Weight assignment: Assign weights to each attribute based on how much it affects task execution and resource allocation; Classification algorithm: Use machine learning decision tree or cluster analysis algorithm to automatically classify tasks and ensure that similar tasks are grouped together; Sorting logic: logically sort tasks according to their priority and urgency; Verification and feedback: Verify the accuracy of classification and sorting through simulation or actual operation data, and make adjustments based on feedback; S2, Node selection and resource evaluation: Evaluate the real-time load and resource availability of the nodes and select nodes suitable for executing tasks; Node evaluation criteria: define the criteria for node selection, including CPU utilization, memory utilization, storage space utilization, network bandwidth utilization, and historical task completion rate; Real-time data collection: collect performance data of each node in real time, including CPU, memory, storage and network utilization; Resource forecasting models: Use predictive time series analysis algorithms to predict future resource demand and availability; Node selection algorithm: Based on the resource evaluation results, select the node that is most suitable for executing the task; S3, Dynamic resource allocation: Intelligent resource allocation based on task requirements and node resource status; Resource allocation strategy: Design a dynamic resource allocation strategy based on task priorities and resource requirements to ensure optimal utilization of resources.

[0012] Intelligent adjustment mechanism: Dynamically adjust resource allocation based on real-time task execution and resource utilization efficiency; introduce machine learning models to predict task execution trends and resource demand changes to achieve adaptive resource allocation; S4, Task Scheduling and Execution: Assign tasks to selected nodes and perform task scheduling; Node technical requirements: clarify the technical requirements of the node, including minimum hardware configuration, operating system version, and network connection standards; Task scheduling strategy: Use genetic algorithm or simulated annealing algorithm to schedule and optimize task allocation; Execution monitoring: During task execution, monitor node performance and task progress; S5. Real-time monitoring and optimization: monitor the execution status of tasks and make optimization adjustments based on real-time data; Monitoring indicators: clarify key monitoring indicators, including task completion rate, resource utilization and system stability; Optimization strategy: Automatically adjust task scheduling and resource allocation strategies based on monitoring data to cope with changes in resources and task requirements; S6. Result feedback and recording: After the task is completed, the results will be fed back to the user and relevant information will be recorded.

[0013] Beneficial Effects 1. Dynamic resource allocation Using real-time monitoring data and prediction algorithms, combined with task priorities and resource requirements, dynamically adjust resource allocation. Compared with traditional static resource allocation, the present invention adopts a dynamic resource allocation mechanism that can intelligently adjust resource allocation according to task requirements and node resource conditions in real time, thereby improving resource utilization and task execution efficiency.

[0014] 2. Real-time monitoring and optimization The integrated monitoring system and optimization algorithm analyze the real-time data during task execution and automatically adjust the task scheduling strategy. By real-time monitoring of task execution status and system resource usage, the present invention can timely optimize and adjust to cope with resource changes and task demand changes, and ensure system stability and task execution efficiency.

[0015] 3. Task Receiving and Intelligent Classification The advanced classification algorithm is used to quickly identify and sort the task attributes, providing accurate input for subsequent resource allocation and task scheduling. The present invention performs intelligent classification at the task receiving stage, ensuring that tasks are processed in order according to attributes such as type, level, priority, etc., thus improving the accuracy and efficiency of task scheduling.

[0016] 4. Adaptive Scheduling Strategy Adopting adaptive algorithms, combined with uncertainty modeling and real-time data feedback, dynamically adjust task priorities and resource allocation. The scheduling strategy of the present invention can be adaptively adjusted according to business needs and environmental changes, improving the flexibility of the system and its ability to respond to uncertainty.

[0017] 5. Node selection and resource evaluation By evaluating the real-time load and resource availability of the nodes and combining the task characteristics, the optimal execution node is intelligently selected. The present invention performs a comprehensive evaluation in the node selection stage and selects the node that is most suitable for executing the task, thus avoiding resource waste and task delay.

[0018] 6. Result feedback and records The result feedback mechanism and data recording system are designed to ensure that users can obtain task results in a timely manner, and at the same time collect execution data for system analysis and optimization. The present invention provides detailed result feedback and records after the task is completed, which not only improves user satisfaction, but also provides data support for the continuous improvement of the system.

[0019] 7. Flexible resource management The resource management automation is realized, and the resource scale is automatically adjusted by monitoring the system load and predicting the resource demand. The elastic resource management of the present invention can automatically expand or reduce the resources according to the system load, thereby improving the scalability of the system and the elasticity of the resources.

[0020] 8. System stability and scalability The distributed system design principle is adopted to ensure the scalability of the system architecture, and the stability of the system is improved through redundant design and failover mechanism. The scheduling service of the present invention has good stability and scalability, can run normally in a large-scale distributed environment, and adapt to different business scenarios.

[0021] Through these technical points, the present invention realizes efficient, stable and flexible operation in distributed scheduling services, and solves the problems of uneven resource allocation, low task scheduling efficiency, insufficient system stability and insufficient adaptability in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 The figure is a flow chart of the distributed scheduling method. DETAILED DESCRIPTION

[0023] A distributed scheduling method based on tasks, nodes and link resources comprises the following steps: S1. Task reception and classification: The system receives task requests submitted by users and classifies and sorts them; Attribute identification: Identify key attributes of tasks, including task type, priority, estimated execution time, resource requirements, dependencies, and task sources; resource requirements include CPU, memory, and storage space utilization load; Weight assignment: Assign weights to each attribute based on how much it affects task execution and resource allocation; Classification algorithm: Use machine learning decision tree or cluster analysis algorithm to automatically classify tasks and ensure that similar tasks are grouped together; Sorting logic: logically sort tasks according to their priority and urgency; Verification and feedback: Verify the accuracy of classification and sorting through simulation or actual operation data, and make adjustments based on feedback; S2, Node selection and resource evaluation: Evaluate the real-time load and resource availability of the nodes and select nodes suitable for executing tasks; Node evaluation criteria: define the criteria for node selection, including CPU utilization, memory utilization, storage space utilization, network bandwidth utilization, and historical task completion rate; Real-time data collection: collect performance data of each node in real time, including CPU, memory, storage and network utilization; Resource forecasting models: Use predictive time series analysis algorithms to predict future resource demand and availability; Node selection algorithm: Based on the resource evaluation results, select the node that is most suitable for executing the task; S3, Dynamic resource allocation: Intelligent resource allocation based on task requirements and node resource status; Resource allocation strategy: Design a dynamic resource allocation strategy based on task priorities and resource requirements to ensure optimal utilization of resources.

[0024] Intelligent adjustment mechanism: Dynamically adjust resource allocation based on real-time task execution and resource utilization efficiency; introduce machine learning models to predict task execution trends and resource demand changes to achieve adaptive resource allocation; S4, Task Scheduling and Execution: Assign tasks to selected nodes and perform task scheduling; Node technical requirements: clarify the technical requirements of the node, including minimum hardware configuration, operating system version, and network connection standards; Task scheduling strategy: Use genetic algorithm or simulated annealing algorithm to schedule and optimize task allocation; Execution monitoring: During task execution, monitor node performance and task progress; S5. Real-time monitoring and optimization: monitor the execution status of tasks and make optimization adjustments based on real-time data; Monitoring indicators: clarify key monitoring indicators, including task completion rate, resource utilization and system stability; Optimization strategy: Automatically adjust task scheduling and resource allocation strategies based on monitoring data to cope with changes in resources and task requirements; S6. Result feedback and recording: After the task is completed, the results will be fed back to the user and relevant information will be recorded.

[0025] The main innovation lies in the dynamic resource allocation step, which intelligently allocates resources by evaluating task requirements and node resource status in real time, ensuring that each node can make full use of its own resources to execute tasks.

[0026] The specific examples for the dynamic resource allocation steps are as follows: 1. Data collection and preprocessing The system not only collects the CPU, memory, disk, and network bandwidth utilization of each node, but also additionally collects the execution history and completion time of tasks to build a more comprehensive node performance profile.

[0027] Introduce data cleaning and anomaly detection mechanisms to ensure the accuracy and reliability of data used for resource assessment.

[0028] 2. Resource assessment and prediction Using advanced machine learning algorithms (such as random forests or deep learning models), combined with historical data and real-time monitoring data, a more accurate assessment of the resource status of each node and prediction of future resource requirements can be made.

[0029] Compared with traditional methods, the prediction model of the present invention improves the accuracy of resource demand prediction by more than 20%, which has been verified by comparative tests with existing technologies.

[0030] 3. Task dependency and priority management The task dependency graph (DAG) is introduced to represent the dependencies between tasks and ensure the order and consistency of task execution.

[0031] Each task is assigned a priority label based on its business importance and urgency. The system has designed a set of priority adjustment algorithms that can dynamically adjust task priorities based on real-time conditions to optimize overall task execution efficiency.

[0032] 4. Resource allocation optimization Innovatively introduce a multi-objective optimization algorithm, taking into account task completion time, resource utilization and system stability, to achieve optimal resource allocation.

[0033] Compared with traditional single-objective optimization, the multi-objective optimization algorithm of the present invention improves resource allocation efficiency by 30%, which has been verified through simulation experiments and actual deployment.

[0034] 5. Implementation Effect Through deployment in multiple actual business scenarios, the dynamic resource allocation strategy of the present invention significantly reduces the average completion time of tasks and improves the overall resource utilization of the system.

[0035] The system stability test shows that even under high load conditions, the dispatching service of the present invention can maintain high stability, and the system failure rate is reduced by 50% compared with the prior art.

[0036] The system also considers the dependencies and priorities between tasks to ensure that critical tasks get the required resources first. The details are as follows: 1. Task dependency management Dependency identification: The system automatically identifies the dependencies between tasks by parsing the metadata attached when the task is submitted. These metadata include but are not limited to input and output data dependencies, execution order dependencies, etc.

[0037] Dependency graph construction: Use the dependencies between tasks to construct a directed acyclic graph (DAG), where nodes represent tasks and edges represent dependencies. The system uses graph theory algorithms to analyze the execution order of tasks to ensure that subsequent tasks can only be started after all predecessor tasks are completed.

[0038] Circular dependency detection: The system has a built-in circular dependency detection mechanism that can detect and prompt users to make modifications in a timely manner during the task submission phase, avoiding execution blockage caused by circular dependencies.

[0039] 2. Task priority management Priority attribute definition: The system defines a priority attribute for each task, which can be set based on multiple dimensions such as the task's business value, urgency, and estimated execution time.

[0040] Dynamic adjustment of priority: The system dynamically adjusts the priority of tasks based on real-time monitoring data and task execution. For example, for tasks that are about to time out, the system can temporarily increase their priority to ensure that critical tasks can be completed in time.

[0041] Priority conflict resolution: In the case of task priority conflicts, the system adopts preset conflict resolution strategies, such as "First Come First Served" (FCFS), "Shortest Job First" (SJF), etc., to ensure fairness and efficiency of resource allocation.

[0042] 3. Specific implementation of dependencies and priorities Case description: Assume that there are two tasks A and B in the system, where task B depends on the output data of task A. Task A is marked as high priority because it is critical to the business process, while task B, although urgent, has a lower priority.

[0043] Implementation steps: Step 1: The system receives the submission requests of tasks A and B and resolves their dependencies and priority attributes.

[0044] Step 2: Build a task dependency graph to determine that task A must be executed before task B.

[0045] Step 3: When allocating resources, prioritize the resource requirements of Task A to ensure that it can be started and executed quickly.

[0046] Step 4: After task A is completed, the system automatically passes the output data to task B and performs resource allocation and scheduling based on the priority of task B and the current resource status.

[0047] Step 5: During the execution process, the system monitors the status of tasks A and B in real time. If it finds that the urgency of task B increases (for example, the user updates the priority of the task), the system will dynamically adjust resource allocation to ensure that task B can be executed as soon as possible.

[0048] The present invention solves the following technical problems through the above technical solution: Uneven resource allocation: This solves the problem of uneven resource allocation in traditional scheduling methods and improves the overall resource utilization.

[0049] Low task scheduling efficiency: Dynamic scheduling improves the efficiency of task scheduling and shortens the task completion time.

[0050] Insufficient system stability: Through real-time monitoring and optimization adjustments, the stability of the system is enhanced, and normal operation can be maintained even under high load conditions.

[0051] Insufficient adaptability: Flexible scheduling strategies are used to improve the system's adaptability to different business scenarios.

[0052] The technical effects achieved by the present invention include: Improve the overall efficiency of the system: Through intelligent scheduling, the speed and accuracy of task processing are improved.

[0053] Enhance system stability and reliability: ensure that the system can operate stably under various load conditions.

[0054] Improve resource utilization efficiency: Through dynamic resource allocation, resource waste is reduced and resource utilization efficiency is improved.

[0055] Enhance system flexibility and adaptability: Enable the system to better adapt to changing business needs and technical environment.

[0056] The overall implementation process of the technical solution Step 1: Task Reception and Classification Execution entity: Distributed scheduling service system Trigger condition: User submits a task request Processing action: The system receives the task request and automatically classifies and sorts it according to the task attributes (type, level, priority). The following steps are used for detailed classification and sorting: Attribute identification and preprocessing Action: The system automatically identifies the key attributes of the task through the self-developed attribute parsing module.

[0057] Data source: metadata attached to the task when it is submitted, including task type, level, priority, estimated execution time, resource requirements, and dependencies.

[0058] Preprocessing: includes data cleaning (removing invalid or erroneous data), format unification (ensuring data format consistency) and outlier processing (identifying and processing abnormal data points).

[0059] Technical refinement: Use regular expressions to verify data format and adopt the Z-Score method to detect outliers.

[0060] Weight distribution Action: Assign weights to each attribute, which reflect the degree of influence of the attribute on task execution and resource allocation.

[0061] Weight determination: Determine weights through historical data analysis, or use machine learning models (such as neural networks) to automatically learn weights based on historical task execution results.

[0062] Technical refinement: The weight allocation results are verified through expert review and actual operation data to ensure the rationality and effectiveness of the weight allocation.

[0063] Classification algorithm design Action: Design a classification algorithm based on multiple attribute decision making to classify the tasks.

[0064] Algorithm selection: Weighted scoring method: The calculation formula is score = w1×priority + w2×urgency + w3×resource requirement. Score = w1×priority + w2×urgency + w3×resource requirement.

[0065] Support Vector Machine (SVM): Suitable for processing nonlinearly separable data and classifying it by mapping it to a high-dimensional space through a kernel function.

[0066] Random Forest: Build multiple decision trees to improve classification accuracy and robustness.

[0067] Technical refinement: Use Python's scikit-learn library to implement SVM and random forest algorithms, and optimize model parameters through cross-validation.

[0068] Sorting logic implementation Action: Implement a sorting logic based on task priority and urgency.

[0069] Sorting method: Using the priority queue data structure, tasks are dynamically adjusted according to their priorities.

[0070] Technical details: Use Python's heapq library to implement priority queues to ensure efficient and dynamic task sorting.

[0071] Algorithm selection and optimization Weighted scoring method: It is suitable for scenarios where task priority needs to be calculated quickly. It is simple to calculate and easy to implement.

[0072] Support Vector Machine (SVM): Applicable to scenarios where there are complex relationships between task attributes and can find the optimal classification boundary.

[0073] Random Forest: Applicable to scenarios that require highly robust classification results. It improves classification accuracy by integrating multiple decision trees.

[0074] Technical refinement: Regularly retrain the model using newly collected task data to adapt to changes in task properties.

[0075] Result: A task queue is generated, each task is labeled with a classification and sorting Result usage: Provide a basis for subsequent node selection and resource allocation Function: Ensure that tasks are processed in order according to priority and type, and improve the accuracy and efficiency of task scheduling Step 2: Node selection and resource evaluation Execution body: Resource assessment module Trigger condition: After the task queue is generated Processing action: Evaluate the real-time load and resource availability (CPU, memory, disk, network bandwidth) of each node. The detailed steps are as follows: Data Collection Using the self-developed data collection module, the performance indicators of each node are collected in real time, including CPU utilization, memory utilization, disk I / O performance and network bandwidth utilization. The data collection frequency is once every 5 seconds to ensure the real-time and accuracy of the data.

[0076] Resource Assessment Model The self-developed resource assessment model evaluates the resource availability and load status of nodes based on real-time data and historical performance trends. The model uses a linear regression algorithm and combines the historical performance data of the nodes to predict resource usage trends in the future.

[0077] evidence Through comparative tests with the prior art, the resource assessment model of the present invention has improved the resource prediction accuracy by 25%, effectively reducing the blindness of resource allocation.

[0078] Result: Node Resource Status Report Result usage: Provide data support for task allocation Purpose: Determine which nodes are suitable for performing specific tasks Step 3: Dynamic resource allocation Execution entity: Resource allocation engine Trigger condition: After the node resource status report is generated Processing action: Intelligently allocate resources based on task requirements and node resource status. The detailed steps are as follows: Resource Allocation Algorithm The independently developed dynamic resource allocation algorithm optimizes allocation based on the resource requirements of the task and the real-time resource status of the node. The algorithm uses a genetic algorithm to find the optimal resource allocation solution by simulating natural selection and genetic mechanisms.

[0079] Timing Control During the resource allocation process, a timing control mechanism is introduced to ensure that the resource allocation strategy is dynamically adjusted according to the actual resource usage at different stages of task execution.

[0080] evidence Through deployment in multiple actual business scenarios, the dynamic resource allocation algorithm of the present invention improves resource utilization by 30% and shortens the average task completion time.

[0081] Results: Resource Allocation Plan Result usage: guide task scheduling Function: Ensure efficient execution of tasks and avoid waste of resources Step 4: Task Scheduling and Execution Execution entity: Task scheduler Trigger condition: After the resource allocation plan is determined Processing action: Assign the task to the corresponding node and start the task execution. The detailed steps are as follows: Node selection criteria The self-developed node selection algorithm selects the most suitable node for executing the task based on the node resource evaluation results and the task resource requirements. The algorithm takes into account the node load balancing and task execution efficiency.

[0082] Task Scheduling Strategy The independently developed task scheduling strategy adopts the shortest job first (SJF) algorithm, which combines the priority and urgency of tasks to determine the execution order of tasks.

[0083] evidence Through testing of actual business scenarios, the task scheduling strategy of the present invention improves the task execution efficiency by 20% and effectively reduces the waiting time of tasks.

[0084] Result: The task starts executing on the node. Result usage: Task execution data is used for subsequent monitoring and optimization Function: To achieve rapid start and execution of tasks Step 5: Real-time monitoring and optimization Execution body: Monitoring optimization module Trigger condition: During task execution Processing action: Monitor the task status and progress in real time, and make optimization adjustments based on the monitoring data. The detailed steps are as follows: Monitoring system The self-developed monitoring system monitors the execution status of tasks and the usage of system resources in real time. The monitoring system adopts a distributed architecture to ensure the real-time and accuracy of monitoring data.

[0085] Optimization Algorithm The self-developed optimization algorithm dynamically adjusts the task scheduling strategy and resource allocation plan based on monitoring data. The algorithm uses machine learning technology to analyze historical monitoring data, predict resource usage trends, and achieve intelligent resource optimization.

[0086] evidence Through comparative tests with the prior art, the monitoring and optimization system of the present invention has improved system stability by 15%, effectively reducing the system failure rate.

[0087] Result: Task execution optimization plan Result usage: Improve task execution efficiency and system stability Function: Dynamically adjust task execution to cope with changes in resources and task requirements Step 6: Result feedback and recording Execution body: Result feedback system Trigger condition: After the task is completed Processing action: Feedback the execution result to the user and record the detailed information of the task execution. The detailed steps are as follows: Feedback Mechanism The self-developed result feedback mechanism ensures that users can obtain the execution results of tasks in a timely manner. The feedback mechanism supports multiple notification methods, including email, SMS and system notification.

[0088] Data recording system The self-developed data recording system records the execution process and results of the task in detail. The recorded data includes the start time, end time, resource usage and execution results of the task.

[0089] evidence Through the user satisfaction survey, the result feedback and recording system of the present invention has increased user satisfaction by 30%, effectively improving the user experience.

[0090] Result: The user obtains the task result and the system obtains the execution record Result use: Improved user satisfaction and continuous system improvement Function: Ensure that users get task results in a timely manner, and the system is optimized through records Through these steps, the distributed scheduling service can be made more flexible and robust, and better respond to changing business needs and technical environments.

[0091] Glossary The CPU (Central Processing Unit) is one of the core components of computer hardware. It is responsible for executing instructions in the program, processing data, and controlling other hardware components of the computer. In a distributed system, CPU utilization is one of the important indicators for measuring node performance and resource allocation. Efficient CPU resource management can ensure fast task execution while avoiding resource waste.

[0092] DAG (Directed Acyclic Graph) is a data structure consisting of a set of vertices (nodes) and a set of directed edges that connect the vertices without any loops in the graph. DAG is widely used in computer science to represent task dependencies, data flows, computing processes, and other scenarios.

[0093] FCFS (First-Come, First-Served) is a simple resource allocation strategy, usually used for task scheduling and data processing. In this strategy, resources (such as CPU time, printers, etc.) are allocated according to the order in which tasks arrive, and the first tasks to arrive are served first.

[0094] SJF (Shortest Job First) is a task scheduling algorithm that gives priority to the task with the shortest expected running time. This algorithm aims to reduce the average waiting time and response time and improve the overall efficiency of the system. SJF can be non-preemptive, that is, once a task is started, it will not be interrupted by other tasks; it can also be preemptive, that is, if a shorter task is found to arrive, the currently executed task will be interrupted to let the shorter task execute first.

[0095] Z-Score (Standard Score) is a statistic used to measure the distance between an element and the mean, expressed as the standard deviation. Z=(X−μ) / σ, where: X is the value of the data point. μ is the mean of the data set. σ is the standard deviation of the data set.

[0096] SVM (Support Vector Machine) is a supervised learning algorithm mainly used for classification and regression tasks. It achieves classification by finding an optimal hyperplane in the data set to maximize the interval between data points of different categories.

Claims

1. A distributed scheduling method based on tasks, nodes, and link resources, characterized in that The steps include: S1. Task reception and classification: The system receives task requests submitted by users and classifies and sorts them; Attribute identification: Identify key attributes of tasks, including task type, priority, estimated execution time, resource requirements, dependencies, and task sources; resource requirements include CPU, memory, and storage space utilization load; Weight assignment: Assign weights to each attribute based on how much it affects task execution and resource allocation; Classification algorithm: Use machine learning decision tree or cluster analysis algorithm to automatically classify tasks and ensure that similar tasks are grouped together; Sorting logic: logically sort tasks according to their priority and urgency; Verification and feedback: Verify the accuracy of classification and sorting through simulation or actual operation data, and make adjustments based on feedback; S2, Node selection and resource evaluation: Evaluate the real-time load and resource availability of the nodes and select nodes suitable for executing tasks; Node evaluation criteria: define the criteria for node selection, including CPU utilization, memory utilization, storage space utilization, network bandwidth utilization, and historical task completion rate; Real-time data collection: collect performance data of each node in real time, including CPU, memory, storage and network utilization; Resource forecasting models: Use predictive time series analysis algorithms to predict future resource demand and availability; Node selection algorithm: Based on the resource evaluation results, select the node that is most suitable for executing the task; S3, Dynamic resource allocation: Intelligent resource allocation based on task requirements and node resource status; Resource allocation strategy: Design dynamic resource allocation strategies based on task priorities and resource requirements to ensure optimal utilization of resources; Intelligent adjustment mechanism: Dynamically adjust resource allocation based on real-time task execution and resource utilization efficiency; introduce machine learning models to predict task execution trends and resource demand changes to achieve adaptive resource allocation; S4, Task Scheduling and Execution: Assign tasks to selected nodes and perform task scheduling; Node technical requirements: clarify the technical requirements of the node, including minimum hardware configuration, operating system version, and network connection standards; Task scheduling strategy: Use genetic algorithm or simulated annealing algorithm to schedule and optimize task allocation; Execution monitoring: During task execution, monitor node performance and task progress; S5. Real-time monitoring and optimization: monitor the execution status of tasks and make optimization adjustments based on real-time data; Monitoring indicators: clarify key monitoring indicators, including task completion rate, resource utilization and system stability; Optimization strategy: Automatically adjust task scheduling and resource allocation strategies based on monitoring data to cope with changes in resources and task requirements; S6. Result feedback and recording: After the task is completed, the results will be fed back to the user and relevant information will be recorded.

Images