Distributed resource scheduling method and system based on multiple agents

Abstract

The invention relates to the field of data processing, and discloses a distributed resource scheduling method and system based on multiple agents, a task management module generates task cycle information, calls corresponding node verification information in a node resource database, and sends the node verification information to a resource scheduling container of a corresponding task cycle sequence number; connecting a resource scheduling container corresponding to the task cycle serial number with a task execution node, wherein the task execution node collects currently available resource information of the task execution node through a resource management module; the task execution node sends current available resource information collected by the resource management module to the resource scheduling container corresponding to the task cycle sequence number, and the resource scheduling container sends the resource information passing verification and the preliminary allocation information to the task execution node; and the task execution node collects resource release information after the task is completed through the resource management module, and if the resource release information is all released, resource scheduling of the distributed resource task cycle is completed. According to the invention, dynamic allocation of resources and optimal scheduling of tasks are realized.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a distributed resource scheduling method and system based on multi-agents. Background Art

[0002] In the current information age, distributed systems have become the mainstream architecture for processing large-scale data and tasks. Distributed systems improve the system's processing power, scalability, and fault tolerance by distributing tasks and resources to multiple nodes. However, as the scale of the system expands, the effective scheduling and management of resources has become a huge challenge. Traditional centralized resource scheduling methods often have problems such as slow response, low efficiency, and poor scalability when facing large-scale, dynamically changing distributed environments.

[0003] In a distributed system, resource scheduling refers to the reasonable allocation and scheduling of resources based on task requirements and the current resource status of the system to ensure efficient execution of tasks. Resource scheduling needs to consider multiple factors, such as resource availability, performance, task priority, task execution time, etc. Traditional resource scheduling methods usually use a centralized scheduler, which manages and allocates resources in a unified manner. However, this centralized method has the following major problems when facing large-scale distributed systems:

[0004] Single-point bottleneck: The centralized scheduler becomes the bottleneck of the system. When the system scale expands, the load of the scheduler increases sharply, resulting in slower response and affecting the overall performance of the system.

[0005] Poor scalability: The centralized scheduler is difficult to adapt to the dynamic changes in the system scale. When new nodes or resources need to be added, major modifications and adjustments to the scheduler are often required.

[0006] Lack of flexibility: Centralized schedulers usually adopt fixed scheduling strategies, which are difficult to flexibly adjust according to the actual situation of the system and task requirements.

[0007] Poor fault tolerance: Once a centralized scheduler fails, the resource scheduling of the entire system will be affected, resulting in task execution failure or delay.

[0008] In order to solve the above problems, the distributed resource scheduling method came into being. The distributed resource scheduling method realizes the distributed management and scheduling of resources by distributing the scheduling function to multiple nodes. Each node independently performs resource scheduling and decision-making based on its own resources and task status, thereby improving the response speed, efficiency and scalability of the system.

[0009] However, existing distributed resource scheduling methods still have some problems. For example, some methods lack an effective resource verification mechanism, resulting in unreasonable resource allocation or resource conflicts; some methods cannot monitor the release of resources in real time, resulting in resource leakage or waste; some methods lack an effective exception handling mechanism during task execution, and cannot promptly alarm and handle problems once they occur. Summary of the invention

[0010] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a distributed resource scheduling method based on multi-agents, comprising the following steps:

[0011] Step 1: The task management module generates task cycle information and sends it to the cloud data server. The cloud data server creates a corresponding resource scheduling container according to the task cycle number in the task cycle information, and calls the corresponding node verification information in the node resource database according to the task execution node in the task cycle information, and sends it to the resource scheduling container corresponding to the task cycle number;

[0012] Step 2: At the start of the task cycle, the cloud data server connects the resource scheduling container corresponding to the task cycle number to the task execution node, and the task execution node collects the resource information currently available to the task execution node through the resource management module;

[0013] Step 3: The task execution node sends the currently available resource information collected by the resource management module to the resource scheduling container corresponding to the task cycle number for resource verification. If the resource verification passes, the preliminary allocation information of the corresponding resource is generated. At the resource verification deadline, the resource scheduling container sends the verified resource information and preliminary allocation information to the task execution node;

[0014] Step 4: The task execution node verifies the actual allocation of resources based on the verified resource information and preliminary allocation information. If there is an allocation anomaly, the task execution node sends an alarm message to the task management module; if there is no allocation anomaly, it enters step 5 at the end of the task cycle;

[0015] Step 5: The task execution node collects resource release information after task completion through the resource management module, and verifies the released resource information based on the verified resource information and preliminary allocation information. If all resources are released normally, proceed to step 7; otherwise, proceed to step 6;

[0016] Step 6: Based on the verified resource information, identify the resources that are not released normally. The task execution node sends the resource information that is not released normally to the resource scheduling container of the corresponding task cycle number. The resource scheduling container calls the detailed resource information of the corresponding node from the node resource database and sends it to the task execution node. The monitoring module identifies the resource usage in the system based on the detailed resource information. If the corresponding unreleased resources are identified, a release notification message is issued and the process returns to step 5. Otherwise, the task execution node sends an alarm message to the task management module.

[0017] Step seven, complete the resource scheduling of the distributed resource task cycle.

[0018] Furthermore, the task cycle information includes the task execution node, the task cycle start and end time, the task cycle sequence number, and the task execution node address information; the task cycle start and end time include the resource verification deadline and the task cycle end time.

[0019] Furthermore, according to the task execution node in the task cycle information, the corresponding node verification information is called in the node resource database and sent to the resource scheduling container corresponding to the task cycle number, wherein the node verification information is the preset resource configuration information and resource performance parameters.

[0020] Furthermore, the resource verification is performed in the resource scheduling container corresponding to the task cycle number. If the resource verification passes, the preliminary allocation information of the corresponding resource is generated, including:

[0021] The resource management module collects the currently available resource configuration information and the corresponding resource performance parameters, and verifies the resource configuration information in the resource scheduling container corresponding to the task cycle number. If the resource configuration information verification passes, the available resource configuration corresponding to the resource configuration information is the preliminary allocation information of the corresponding resources.

[0022] Furthermore, the verification of the released resource information according to the verified resource information and the preliminary allocation information includes:

[0023] The resource management module collects the resource configuration information released after the task is completed, and matches and verifies it in the verified resource information according to the resource configuration information of the released resources. If the resource configuration information of the released resources is consistent with the resource configuration information in the preliminary allocation information of the corresponding resources, there are no resources that are not released normally; otherwise, there are resources that are not released normally.

[0024] A distributed resource scheduling system based on multi-agents, applying the distributed resource scheduling method based on multi-agents, including a task management module, a task execution node, a cloud data server, a communication module, a resource management module and a monitoring module;

[0025] The communication module, resource management module and monitoring module are respectively connected to the task execution node; the cloud data server is connected to the communication module; and the task management module is connected to the cloud data server.

[0026] The beneficial effects of the present invention are: improving the accuracy and reliability of resource scheduling, realizing dynamic allocation of resources and optimal scheduling of tasks. In the resource scheduling process, a resource verification mechanism is introduced to verify and compare the currently available resource information, thereby ensuring the accuracy and reliability of resource allocation. At the same time, the system operation and resource usage are monitored and analyzed in real time through the monitoring module, resource anomalies and system failures are discovered and handled in a timely manner, and the reliability of resource scheduling is further improved.

[0027] The present invention introduces a resource release verification mechanism in the resource release process to verify and compare the released resource information, ensuring that all resources can be released normally after the task is completed, avoiding resource leakage and waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flowchart of a distributed resource scheduling method based on multi-agent;

[0029] Figure 2 Schematic diagram of the principle of a distributed resource scheduling system based on multi-agent. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings, but the protection scope of the present invention is not limited to the following.

[0031] The features and properties of the present invention are further described in detail below in conjunction with the embodiments.

[0032] like Figure 1 As shown, the distributed resource scheduling method based on multi-agent includes the following steps:

[0033] Step 1: The task management module generates task cycle information and sends it to the cloud data server. The cloud data server creates a corresponding resource scheduling container according to the task cycle number in the task cycle information, and calls the corresponding node verification information in the node resource database according to the task execution node in the task cycle information, and sends it to the resource scheduling container corresponding to the task cycle number;

[0034] Step 2: At the start of the task cycle, the cloud data server connects the resource scheduling container corresponding to the task cycle number to the task execution node, and the task execution node collects the resource information currently available to the task execution node through the resource management module;

[0035] Step 3: The task execution node sends the currently available resource information collected by the resource management module to the resource scheduling container corresponding to the task cycle number for resource verification. If the resource verification passes, the preliminary allocation information of the corresponding resource is generated. At the resource verification deadline, the resource scheduling container sends the verified resource information and preliminary allocation information to the task execution node;

[0036] Step 4: The task execution node verifies the actual allocation of resources based on the verified resource information and preliminary allocation information. If there is an allocation anomaly, the task execution node sends an alarm message to the task management module; if there is no allocation anomaly, it enters step 5 at the end of the task cycle;

[0037] Step 5: The task execution node collects resource release information after task completion through the resource management module, and verifies the released resource information based on the verified resource information and preliminary allocation information. If all resources are released normally, proceed to step 7; otherwise, proceed to step 6;

[0038] Step 6: Based on the verified resource information, identify the resources that are not released normally. The task execution node sends the resource information that is not released normally to the resource scheduling container of the corresponding task cycle number. The resource scheduling container calls the detailed resource information of the corresponding node from the node resource database and sends it to the task execution node. The monitoring module identifies the resource usage in the system based on the detailed resource information. If the corresponding unreleased resources are identified, a release notification message is issued and the process returns to step 5. Otherwise, the task execution node sends an alarm message to the task management module.

[0039] Step seven, complete the resource scheduling of the distributed resource task cycle.

[0040] The task cycle information includes the task execution node, the task cycle start and end time, the task cycle sequence number, and the task execution node address information; the task cycle start and end time include the resource verification deadline and the task cycle end time.

[0041] According to the task execution node in the task cycle information, the corresponding node verification information is called in the node resource database and sent to the resource scheduling container corresponding to the task cycle number, wherein the node verification information is the preset resource configuration information and resource performance parameters.

[0042] The resource verification is performed in the resource scheduling container corresponding to the task cycle sequence number. If the resource verification passes, the preliminary allocation information of the corresponding resource is generated, including:

[0043] The resource management module collects the currently available resource configuration information and the corresponding resource performance parameters, and verifies the resource configuration information in the resource scheduling container corresponding to the task cycle number. If the resource configuration information verification passes, the available resource configuration corresponding to the resource configuration information is the preliminary allocation information of the corresponding resources.

[0044] The step of verifying the released resource information based on the verified resource information and the preliminary allocation information includes:

[0045] The resource management module collects the resource configuration information released after the task is completed, and matches and verifies it in the verified resource information according to the resource configuration information of the released resources. If the resource configuration information of the released resources is consistent with the resource configuration information in the preliminary allocation information of the corresponding resources, there are no resources that are not released normally; otherwise, there are resources that are not released normally.

[0046] like Figure 2 As shown, a distributed resource scheduling system based on multi-agents, applying the distributed resource scheduling method based on multi-agents, includes a task management module, a task execution node, a cloud data server, a communication module, a resource management module and a monitoring module;

[0047] The communication module, resource management module and monitoring module are respectively connected to the task execution node; the cloud data server is connected to the communication module; and the task management module is connected to the cloud data server.

[0048] Specifically, the present invention provides a distributed resource scheduling method and system based on multi-agent, aiming to achieve dynamic allocation of resources and optimal scheduling of tasks through the collaborative work of multiple agents, while ensuring the accuracy and reliability of resource allocation, as well as effective monitoring and management of resource release.

[0049] The multi-agent-based distributed resource scheduling method of the present invention comprises the following steps:

[0050] Step 1: The task management module generates task cycle information and sends it to the cloud data server. The task cycle information includes the task execution node, the start and end time of the task cycle, the task cycle sequence number, and the task execution node address information. The cloud data server creates a corresponding resource scheduling container based on the task cycle sequence number in the task cycle information. The resource scheduling container is a logical unit for storing and processing resource scheduling information related to a specific task cycle. Then, the cloud data server calls the corresponding node verification information in the node resource database based on the task execution node in the task cycle information, and sends it to the resource scheduling container corresponding to the task cycle sequence number. The node verification information is the preset resource configuration information and resource performance parameters, which are used for verification and comparison during the resource scheduling process.

[0051] Step 2: At the start of the task cycle, the cloud data server connects the resource scheduling container corresponding to the task cycle number to the task execution node. The task execution node collects currently available resource information through the resource management module, including resource configuration information, resource performance parameters, etc. This resource information is the resource situation that the task execution node can provide at the current moment and is the basis for resource scheduling.

[0052] Step three, the task execution node sends the currently available resource information collected by the resource management module to the resource scheduling container corresponding to the task cycle number for resource verification. Resource verification is a key link to ensure the accuracy and reliability of resource allocation. In the resource scheduling container, the currently available resource information is verified according to the preset node verification information. If the resource verification passes, the preliminary allocation information of the corresponding resources is generated. The preliminary allocation information is a resource allocation plan preliminarily determined based on the current available resource situation and task requirements. At the resource verification deadline, the resource scheduling container sends the verified resource information and preliminary allocation information to the task execution node.

[0053] Step 4: The task execution node verifies the actual allocation of resources based on the verified resource information and preliminary allocation information. This step is to ensure the feasibility and accuracy of the resource allocation plan in actual execution. If there are allocation anomalies, such as insufficient resources, resource conflicts, etc., the task execution node will send an alarm message to the task management module so that timely measures can be taken to make adjustments. If there are no allocation anomalies, then at the end of the task cycle, proceed to the next step.

[0054] In step 5, the task execution node collects resource release information after the task is completed through the resource management module. Resource release information is the resource configuration information released by the task execution node after the task is completed. Then, the task execution node verifies the released resource information based on the verified resource information and preliminary allocation information. This step is to ensure that all resources can be released normally after the task is completed to avoid resource leakage and waste. If all resources are released normally, proceed to step 7; otherwise, proceed to step 6.

[0055] Step six, based on the verified resource information, identify the resources that have not been released normally. The task execution node sends the resource information that has not been released normally to the resource scheduling container of the corresponding task cycle number. The resource scheduling container calls the detailed resource information of the corresponding node from the node resource database and sends it to the task execution node. The detailed resource information includes the specific usage of the resources, historical records, etc., for further analysis and judgment. The monitoring module identifies the resource usage in the system based on the detailed resource information. If the corresponding unreleased resources are identified, a release notification message is issued to notify the task execution node to release the resources, and return to step five for re-verification. If the unreleased resources are still not identified or the resources cannot be released, the task execution node sends an alarm message to the task management module so that further measures can be taken.

[0056] Step 7: Complete the resource scheduling of the distributed resource task cycle. In this step, the task execution node executes the task according to the resource scheduling result and releases the resources normally after the task is completed, thus completing the resource scheduling process of a task cycle.

[0057] Among them, according to the task execution node in the task cycle information, the corresponding node verification information is called in the node resource database and sent to the resource scheduling container of the corresponding task cycle number. The specific implementation method is as follows: After the cloud data server receives the task cycle information sent by the task management module, it searches for the corresponding node verification information in the node resource database according to the identification information of the task execution node. The node verification information includes preset resource configuration information and resource performance parameters, which are used for verification and comparison during the resource scheduling process. Then, the cloud data server sends the called node verification information to the resource scheduling container of the corresponding task cycle number for use in the subsequent resource verification process.

[0058] The resource verification is performed in the resource scheduling container corresponding to the task cycle number. If the resource verification passes, the preliminary allocation information of the corresponding resources is generated. The specific implementation method is as follows: the task execution node sends the collected currently available resource information to the resource scheduling container corresponding to the task cycle number. After receiving the resource information, the resource scheduling container verifies the resource information according to the preset node verification information. The verification content includes the accuracy of the resource configuration information, the compliance of the resource performance parameters, etc. If the resource verification passes, the resource scheduling container generates the preliminary allocation information of the corresponding resources according to the currently available resource situation and task requirements. The preliminary allocation information includes the resource allocation plan, allocation quantity, allocation time, etc., which is used to guide the subsequent actual resource allocation process.

[0059] The method of verifying the released resource information based on the verified resource information and preliminary allocation information is as follows: after the task is completed, the task execution node collects the resource release information after the task is completed through the resource management module. Then, the task execution node verifies the released resource information based on the verified resource information and preliminary allocation information. The verification content includes whether the released resource configuration information is consistent with the resource configuration information in the preliminary allocation information, whether the number of released resources meets expectations, etc. If the released resource information is consistent with the preliminary allocation information, and the number of released resources meets expectations, there are no resources that are not released normally; otherwise, there are resources that are not released normally, and further processing and analysis are required.

[0060] The distributed resource scheduling system based on multi-agents applies the distributed resource scheduling method based on multi-agents, including a task management module, a task execution node, a cloud data server, a communication module, a resource management module and a monitoring module.

[0061] The task management module is responsible for generating task cycle information and sending it to the cloud data server. The task management module is the control center of the system, responsible for the task scheduling and resource management of the entire system. It generates task cycle information based on the system's operating conditions and task requirements, and sends it to the cloud data server for subsequent processing.

[0062] The task execution node is the execution unit of the system, responsible for executing specific tasks and processing resource scheduling requests. The task execution node collects currently available resource information through the resource management module, and allocates and releases resources according to the resource scheduling information sent by the cloud data server. At the same time, the task execution node is also connected to the communication module, resource management module and monitoring module to realize information transmission and interaction.

[0063] The cloud data server is the data storage and processing center of the system, responsible for storing and processing data and information related to resource scheduling. After receiving the task cycle information sent by the task management module, the cloud data server creates a corresponding resource scheduling container according to the task cycle sequence number, and calls the node verification information in the node resource database for verification and comparison. Then, the cloud data server sends the verified resource information and preliminary allocation information to the task execution node to guide it to allocate and release resources.

[0064] The communication module is responsible for realizing the information transmission and interaction between the task execution node and the cloud data server. The communication module adopts reliable communication protocols and technologies to ensure the accurate transmission and timely arrival of information. Through the communication module, the task execution node can receive the resource scheduling information sent by the cloud data server, and send the collected resource information to the cloud data server for verification and processing.

[0065] The resource management module is responsible for collecting and managing the resource information of the task execution node. The resource management module collects the currently available resource information in real time by monitoring the resource usage of the task execution node, and sends it to the cloud data server for verification and comparison. At the same time, the resource management module is also responsible for allocating and releasing the resources of the task execution node according to the resource scheduling information sent by the cloud data server.

[0066] The monitoring module is responsible for monitoring the operation of the system and the use of resources. The monitoring module collects and analyzes various information and data in the system to understand the operation status and resource use of the system in real time. When resource anomalies or system failures are found, the monitoring module promptly issues an alarm message to notify relevant personnel to handle and repair.

Claims

1. A distributed resource scheduling method based on multi-agents, characterized in that: The steps include: Step 1: The task management module generates task cycle information and sends it to the cloud data server. The cloud data server creates a corresponding resource scheduling container according to the task cycle number in the task cycle information, and calls the corresponding node verification information in the node resource database according to the task execution node in the task cycle information, and sends it to the resource scheduling container corresponding to the task cycle number; Step 2: At the start of the task cycle, the cloud data server connects the resource scheduling container corresponding to the task cycle number to the task execution node, and the task execution node collects the resource information currently available to the task execution node through the resource management module; Step 3: The task execution node sends the currently available resource information collected by the resource management module to the resource scheduling container corresponding to the task cycle number for resource verification. If the resource verification passes, the preliminary allocation information of the corresponding resource is generated. At the resource verification deadline, the resource scheduling container sends the verified resource information and preliminary allocation information to the task execution node; Step 4: The task execution node verifies the actual allocation of resources based on the verified resource information and preliminary allocation information. If there is an allocation anomaly, the task execution node sends an alarm message to the task management module; if there is no allocation anomaly, it enters step 5 at the end of the task cycle; Step 5: The task execution node collects resource release information after the task is completed through the resource management module, and verifies the released resource information based on the verified resource information and preliminary allocation information. If all resources are released normally, proceed to step 7; Otherwise, go to step 6; Step 6: Based on the verified resource information, identify the resources that are not released normally. The task execution node sends the resource information that is not released normally to the resource scheduling container of the corresponding task cycle number. The resource scheduling container calls the detailed resource information of the corresponding node from the node resource database and sends it to the task execution node. The monitoring module identifies the resource usage in the system based on the detailed resource information. If the corresponding unreleased resources are identified, a release notification message is issued and the process returns to step 5. Otherwise, the task execution node sends an alarm message to the task management module. Step seven, complete the resource scheduling of the distributed resource task cycle.

2. The multi-agent based distributed resource scheduling method according to claim 1, characterized in that: The task cycle information includes the task execution node, the task cycle start and end time, the task cycle sequence number, and the task execution node address information; the task cycle start and end time include the resource verification deadline and the task cycle end time.

3. The multi-agent-based distributed resource scheduling method according to claim 2, characterized in that: According to the task execution node in the task cycle information, the corresponding node verification information is called in the node resource database and sent to the resource scheduling container corresponding to the task cycle number, wherein the node verification information is the preset resource configuration information and resource performance parameters.

4. The multi-agent based distributed resource scheduling method according to claim 3, characterized in that: The resource verification is performed in the resource scheduling container corresponding to the task cycle sequence number. If the resource verification passes, the preliminary allocation information of the corresponding resource is generated, including: The resource management module collects the currently available resource configuration information and the corresponding resource performance parameters, and verifies the resource configuration information in the resource scheduling container corresponding to the task cycle number. If the resource configuration information verification passes, the available resource configuration corresponding to the resource configuration information is the preliminary allocation information of the corresponding resources.

5. The multi-agent based distributed resource scheduling method according to claim 4, characterized in that: The step of verifying the released resource information based on the verified resource information and the preliminary allocation information includes: The resource management module collects the resource configuration information released after the task is completed, and matches and verifies it in the verified resource information according to the resource configuration information of the released resources. If the resource configuration information of the released resources is consistent with the resource configuration information in the preliminary allocation information of the corresponding resources, there are no resources that are not released normally; otherwise, there are resources that are not released normally.

6. A distributed resource scheduling system based on multi-agents, characterized in that: The distributed resource scheduling method based on multi-agents described in any one of claims 1 to 5 is applied, comprising a task management module, a task execution node, a cloud data server, a communication module, a resource management module and a monitoring module; The communication module, resource management module and monitoring module are respectively connected to the task execution node; the cloud data server is connected to the communication module; and the task management module is connected to the cloud data server.

Images