Multi-agent based distributed resource scheduling method and system

Abstract

The application relates to the field of data processing and discloses a distributed resource scheduling method and system based on multiple agents, a task management module generates task period information, corresponding node verification information is called in a node resource database, and is sent to a resource scheduling container corresponding to a task period serial number; the resource scheduling container corresponding to the task period serial number is connected with a task execution node, the task execution node collects current available resource information of the task execution node through a resource management module; the task execution node sends the current available resource information collected by the resource management module to the resource scheduling container corresponding to the task period serial number, the resource scheduling container sends verified resource information and preliminary allocation information to the task execution node; the task execution node collects resource release information after task completion through the resource management module, and if all resources are released, the resource scheduling of the distributed resource task period is completed. The application realizes dynamic allocation of resources and optimized scheduling of tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, in particular to a distributed resource scheduling method and system based on multi-agent. BACKGROUND

[0002] In the current information age, distributed systems have become the mainstream architecture for processing large-scale data and tasks. Distributed systems improve the processing capacity, scalability, and fault tolerance of the system by distributing tasks and resources across multiple nodes. However, as the system scale expands, effective scheduling and management of resources becomes a significant challenge. Traditional centralized resource scheduling methods often have slow response, low efficiency, poor scalability, and other issues when faced with large-scale, dynamic distributed environments.

[0003] In distributed systems, resource scheduling refers to the rational allocation and scheduling of resources based on the needs of tasks and the current state of system resources, to ensure efficient execution of tasks. Resource scheduling needs to consider multiple factors, such as resource availability, performance, task priority, and task execution time. Traditional resource scheduling methods usually use a centralized scheduler to manage and allocate resources uniformly. However, this centralized approach has the following main problems when faced with large-scale distributed systems:

[0004] Single point bottleneck: The centralized scheduler becomes a bottleneck for the system, and when the system scale expands, the load on the scheduler increases dramatically, leading to slower response and affecting the overall performance of the system.

[0005] Poor scalability: The centralized scheduler is difficult to adapt to dynamic changes in system scale, and when new nodes or resources need to be added, significant modifications and adjustments to the scheduler are often required.

[0006] Lack of flexibility: The centralized scheduler usually uses a fixed scheduling strategy, making it difficult to adjust flexibly according to the actual situation of the system and the needs of the tasks.

[0007] Poor fault tolerance: Once the centralized scheduler fails, the resource scheduling of the entire system will be affected, leading to task execution failure or delay.

[0008] To solve the above problems, distributed resource scheduling methods have emerged. Distributed resource scheduling methods distribute the scheduling function across multiple nodes, achieving distributed management and scheduling of resources. Each node independently schedules and decides resources based on its own resources and task state, thereby improving the response speed, efficiency, and scalability of the system.

[0009] However, the existing distributed resource scheduling method still has some problems. For example, some methods lack effective resource verification mechanism, resulting in unreasonable resource allocation or resource conflict; some methods cannot monitor the release of resources in real time, resulting in resource leakage or waste; some methods lack effective exception handling mechanism during task execution, and cannot alarm and handle in time once problems occur. SUMMARY

[0010] The purpose of the present application is to overcome the shortcomings of the prior art, provide a distributed resource scheduling method based on multi-agent, comprising the following steps:

[0011] Step one, the task management module generates task period information and sends it to the cloud data server. The cloud data server creates a corresponding resource scheduling container according to the task period sequence number in the task period information, and calls the corresponding node verification information in the node resource database according to the task execution node in the task period information, and sends it to the resource scheduling container corresponding to the task period sequence number;

[0012] Step two, the cloud data server connects the resource scheduling container corresponding to the task period sequence number with the task execution node at the starting time of the task period, and the task execution node collects the current available resource information of the task execution node through the resource management module;

[0013] Step three, the task execution node sends the current available resource information collected by the resource management module to the resource scheduling container corresponding to the task period sequence number for resource verification. If the resource verification is passed, the preliminary allocation information of the corresponding resource is generated. At the resource verification deadline, the resource scheduling container sends the verified resource information and preliminary allocation information to the task execution node;

[0014] Step four, the task execution node verifies the actual allocation of resources according to the verified resource information and preliminary allocation information. If there is allocation exception, the task execution node sends alarm information to the task management module; if there is no allocation exception, it enters step five at the end of the task period;

[0015] Step five, the task execution node collects the resource release information after the task is completed through the resource management module, and verifies the released resource information according to the verified resource information and preliminary allocation information. If all resources are released normally, it enters step seven; otherwise, it enters step six;

[0016] Step six, according to the verified resource information, identify the resources that are not normally released, the task execution node sends the information of the resources that are not normally released to the resource scheduling container corresponding to the task period sequence number, the resource scheduling container calls the detailed resource information of the corresponding node from the node resource database and sends it to the task execution node, the monitoring module identifies the resource usage in the system according to the detailed resource information, if the corresponding unreleased resource is identified, the release notification information is sent out, and returns to step five; otherwise, the task execution node sends alarm information to the task management module;

[0017] Step seven, complete the resource scheduling of the distributed resource task period.

[0018] Further, the task period information includes task execution node, task period start and end time, task period sequence number, task execution node address information; the task period start and end time includes resource verification deadline and task period end time.

[0019] Further, according to the task execution node in the task period information, the corresponding node verification information is called from the node resource database and sent to the resource scheduling container corresponding to the task period sequence number, wherein the node verification information is the preset resource configuration information and resource performance parameter.

[0020] Further, the resource verification is carried out in the resource scheduling container corresponding to the task period sequence number, if the resource verification is passed, the preliminary allocation information of the corresponding resource is generated, including:

[0021] The resource management module collects the current available resource configuration information and the corresponding resource performance parameter, carries out resource configuration information verification in the resource scheduling container corresponding to the task period sequence number, if the resource configuration information verification is passed, the available resource configuration of the corresponding resource configuration information is the preliminary allocation information of the corresponding resource.

[0022] Further, the released resource information is verified according to the verified resource information and the preliminary allocation information, including:

[0023] The resource management module collects the resource configuration information released after the task is completed, matches and verifies the released resource configuration information in the verified resource information, if the resource configuration information of the released resource is consistent with the resource configuration information in the preliminary allocation information of the corresponding resource, there is no resource that is not normally released; otherwise, there is a resource that is not normally released.

[0024] The distributed resource scheduling system based on multiple agents applies the distributed resource scheduling method based on multiple agents, including task management module, task execution node, cloud data server, communication module, resource management module and monitoring module;

[0025] The communication module, the resource management module and the monitoring module are connected with the task execution node respectively; the cloud data server is connected with the communication module; the task management module is connected with the cloud data server.

[0026] The application has the advantages that the accuracy and reliability of resource scheduling are improved, and dynamic allocation of resources and optimized scheduling of tasks are realized. In the process of resource scheduling, a resource verification mechanism is introduced to verify and compare the current available resource information, ensuring the accuracy and reliability of resource allocation. At the same time, the running condition and resource usage of the system are monitored and analyzed in real time by the monitoring module, and resource abnormalities and system failures are found and handled in time, further improving the reliability of resource scheduling.

[0027] In the process of resource release, a resource release verification mechanism is introduced to verify and compare the released resource information, ensuring that all resources can be normally released after the completion of the task, avoiding resource leakage and waste BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 It is a flowchart of the distributed resource scheduling method based on multiple agents.

[0029] Figure 2 It is a principle diagram of the distributed resource scheduling system based on multiple agents. DETAILED DESCRIPTION

[0030] The technical solutions of the application will be further described in detail below with reference to the accompanying drawings, but the protection scope of the application is not limited to the following,

[0031] The features and performance of the application will be further described in detail below with reference to the embodiments,

[0032] As shown in the figure, the distributed resource scheduling method based on multiple agents includes the following steps: Figure 1 Step one, the task management module generates task period information and sends it to the cloud data server. The cloud data server creates a corresponding resource scheduling container according to the task period sequence number in the task period information, and calls the corresponding node verification information in the node resource database according to the task execution node in the task period information, and sends it to the resource scheduling container corresponding to the task period sequence number.

[0033] Step two, the cloud data server connects the resource scheduling container corresponding to the task period sequence number with the task execution node at the starting time of the task period, and the task execution node collects the current available resource information of the task execution node through the resource management module.

[0034] ​

[0035] Step three, the task execution node sends the current available resource information collected by the resource management module to the resource scheduling container corresponding to the task period sequence number, performs resource verification, and if the resource verification passes, generates preliminary allocation information of the corresponding resources, and at the resource verification deadline, the resource scheduling container sends the resource information and preliminary allocation information that pass the verification to the task execution node;

[0036] Step four, the task execution node verifies the actual allocation of resources according to the verified resource information and preliminary allocation information, and if there is an allocation exception, the task execution node sends an alarm information to the task management module; if there is no allocation exception, at the end of the task period, step five is entered;

[0037] Step five, the task execution node collects the resource release information after the task is completed through the resource management module, and verifies the released resource information according to the verified resource information and preliminary allocation information, and if all resources are normally released, step seven is entered; otherwise, step six is entered;

[0038] Step six, according to the verified resource information, identify the resources that are not normally released, the task execution node sends the resource information that is not normally released to the resource scheduling container corresponding to the task period sequence number, the resource scheduling container calls the detailed resource information of the corresponding node from the node resource database and sends it to the task execution node, the monitoring module identifies the resource usage in the system according to the detailed resource information, and if the corresponding unreleased resource is identified, a release notification information is sent, and the process returns to step five; otherwise, the task execution node sends an alarm information to the task management module;

[0039] Step seven, the resource scheduling of the distributed resource task period is completed.

[0040] The task period information includes the task execution node, the task period start and end time, the task period sequence number, and the task execution node address information; the task period start and end time includes the resource verification deadline and the end time of the task period.

[0041] The node verification information in the node resource database corresponding to the task execution node in the task period information is pre-set resource configuration information and resource performance parameters.

[0042] The resource verification in the resource scheduling container corresponding to the task period sequence number, if the resource verification passes, generates preliminary allocation information of the corresponding resources, includes:

[0043] The resource management module collects the currently available resource configuration information and corresponding resource performance parameters, and verifies the resource configuration information in the resource scheduling container of the corresponding task cycle number. If the resource configuration information is verified, the available resource configuration of the corresponding resource configuration information is the initial allocation information of the corresponding resource.

[0044] The verification of released resource information based on verified resource information and preliminary allocation information includes:

[0045] The resource management module collects the resource configuration information released after the task is completed. It then matches and verifies the resource configuration information of the released resources with the verified resource information. If the resource configuration information of the released resources is consistent with the resource configuration information in the initial allocation information of the corresponding resources, then there are no resources that have not been released normally; otherwise, there are resources that have not been released normally.

[0046] like Figure 2 As shown, the distributed resource scheduling system based on multi-agent systems, which applies the aforementioned distributed resource scheduling method based on multi-agent systems, includes a task management module, a task execution node, a cloud data server, a communication module, a resource management module, and a monitoring module.

[0047] The communication module, resource management module, and monitoring module are respectively connected to the task execution node; the cloud data server is connected to the communication module; and the task management module is connected to the cloud data server.

[0048] Specifically, this invention provides a distributed resource scheduling method and system based on multiple agents, aiming to achieve dynamic allocation of resources and optimized scheduling of tasks through the collaborative work of multiple agents, while ensuring the accuracy and reliability of resource allocation, as well as effective monitoring and management of resource release.

[0049] The distributed resource scheduling method based on multiple agents of the present invention includes the following steps:

[0050] Step one: The task management module generates task cycle information and sends it to the cloud data server. This task cycle information includes the task execution node, task cycle start and end times, task cycle sequence number, and task execution node address. The cloud data server creates a corresponding resource scheduling container based on the task cycle sequence number in the task cycle information. The resource scheduling container is a logical unit used to store and process resource scheduling information related to a specific task cycle. Then, based on the task execution node in the task cycle information, the cloud data server retrieves the corresponding node verification information from the node resource database and sends it to the resource scheduling container with the corresponding task cycle sequence number. The node verification information consists of preset resource configuration information and resource performance parameters, used for verification and comparison during resource scheduling.

[0051] Step two, the cloud data server connects the resource scheduling container corresponding to the task period sequence number with the task execution node at the starting moment of the task period. The task execution node collects the current available resource information through the resource management module, including resource configuration information, resource performance parameters, etc. These resource information is the resource situation that the task execution node can provide at the current moment, which is the basis for resource scheduling.

[0052] Step three, the task execution node sends the current available resource information collected by the resource management module to the resource scheduling container corresponding to the task period sequence number for resource verification. Resource verification is a key link to ensure the accuracy and reliability of resource allocation. In the resource scheduling container, the current available resource information is verified according to the preset node verification information. If the resource verification is passed, the preliminary allocation information of the corresponding resource is generated. The preliminary allocation information is a preliminary determination of the resource allocation scheme according to the current available resource situation and task demand. At the resource verification deadline, the resource scheduling container sends the verified resource information and preliminary allocation information to the task execution node.

[0053] Step four, the task execution node verifies the actual allocation of resources according to the verified resource information and preliminary allocation information. This step is to ensure the feasibility and accuracy of the resource allocation scheme in actual execution. If there is an allocation exception, such as insufficient resources, resource conflicts, etc., the task execution node sends an alarm information to the task management module to take timely measures for adjustment. If there is no allocation exception, then at the end of the task period, it enters the next step.

[0054] Step five, the task execution node collects the resource release information after the task is completed through the resource management module. The resource release information is the resource configuration information released by the task execution node after the task is completed. Then, the task execution node verifies the released resource information according to the verified resource information and preliminary allocation information. This step is to ensure that all resources can be normally released after the task is completed, avoiding resource leakage and waste. If all resources are normally released, it enters step seven; otherwise, it enters step six.

[0055] Step six, according to the verified resource information, identify the resources that are not normally released. The task execution node sends the information of the resources that are not normally released to the resource scheduling container corresponding to the task cycle sequence number. The resource scheduling container calls the detailed resource information of the corresponding node from the node resource database and sends it to the task execution node. The detailed resource information includes the specific use of the resource, historical records, etc., for further analysis and judgment. The monitoring module identifies the resource usage in the system according to the detailed resource information. If the corresponding unreleased resource is identified, a release notification is sent to the task execution node to release the resource and return to step five for further verification. If the unreleased resource is still not identified or the resource cannot be released, the task execution node sends an alarm information to the task management module for further measures.

[0056] Step seven, complete the resource scheduling of the distributed resource task cycle. In this step, the task execution node executes the task according to the resource scheduling result, and normally releases the resource after the task is completed, thereby completing the resource scheduling process of a task cycle.

[0057] Among them, according to the task execution node in the task cycle information, the corresponding node verification information is called from the node resource database and sent to the resource scheduling container corresponding to the task cycle sequence number, and the specific implementation is as follows: After the cloud data server receives the task cycle information sent by the task management module, the corresponding node verification information is found in the node resource database according to the identification information of the task execution node. The node verification information includes the preset resource configuration information and the resource performance parameter, which is used for verification and comparison in the resource scheduling process. Then, the cloud data server sends the called node verification information to the resource scheduling container corresponding to the task cycle sequence number, so as to be used in the subsequent resource verification process.

[0058] The resource verification in the resource scheduling container corresponding to the task cycle sequence number is carried out, and if the resource verification is passed, the preliminary allocation information of the corresponding resource is generated, and the specific implementation is as follows: The task execution node sends the collected current available resource information to the resource scheduling container corresponding to the task cycle sequence number. After the resource scheduling container receives the resource information, it verifies the resource information according to the preset node verification information. The verification content includes the accuracy of the resource configuration information and the compliance of the resource performance parameter. If the resource verification is passed, the resource scheduling container generates the preliminary allocation information of the corresponding resource according to the current available resource situation and the task demand. The preliminary allocation information includes the allocation scheme, the allocation quantity, the allocation time, etc. of the resource, which is used to guide the subsequent resource actual allocation process.

[0059] The released resource information is verified according to the verified resource information and the preliminary allocation information, and the specific implementation manner is as follows: after the task is completed, the task execution node collects the resource release information after the task is completed through the resource management module. Then, the task execution node verifies the released resource information according to the verified resource information and the preliminary allocation information. The verification content includes whether the released resource configuration information is consistent with the resource configuration information in the preliminary allocation information, whether the number of released resources meets the expectation, etc. If the released resource information is consistent with the preliminary allocation information, and the number of released resources meets the expectation, there is no resource that is not normally released; otherwise, there is a resource that is not normally released, and further processing and analysis are needed.

[0060] The distributed resource scheduling system based on multiple agents applies the distributed resource scheduling method based on multiple agents, and includes a task management module, a task execution node, a cloud data server, a communication module, a resource management module, and a monitoring module.

[0061] The task management module is responsible for generating task cycle information and sending it to the cloud data server. The task management module is the control center of the system and is responsible for task scheduling and resource management of the entire system. It generates task cycle information according to the running situation and task demand of the system and sends it to the cloud data server for subsequent processing.

[0062] The task execution node is the execution unit of the system and is responsible for executing specific tasks and processing resource scheduling requests. The task execution node collects current available resource information through the resource management module and allocates and releases resources according to the resource scheduling information sent by the cloud data server. At the same time, the task execution node is also connected with the communication module, the resource management module, and the monitoring module to realize information transmission and interaction.

[0063] The cloud data server is the data storage and processing center of the system and is responsible for storing and processing data and information related to resource scheduling. After receiving the task cycle information sent by the task management module, the cloud data server creates a corresponding resource scheduling container according to the task cycle serial number and calls the node verification information in the node resource database for verification and comparison. Then, the cloud data server sends the verified resource information and preliminary allocation information to the task execution node to guide it to allocate and release resources.

[0064] The communication module is responsible for realizing information transmission and interaction between the task execution node and the cloud data server. The communication module uses reliable communication protocols and technologies to ensure accurate transmission and timely arrival of information. Through the communication module, the task execution node can receive the resource scheduling information sent by the cloud data server and send the collected resource information to the cloud data server for verification and processing.

[0065] The resource management module is responsible for collecting and managing the resource information of the task execution nodes. The resource management module monitors the resource usage of the task execution nodes, collects the current available resource information in real time, and sends it to the cloud data server for verification and comparison. At the same time, the resource management module is also responsible for allocating and releasing the resources of the task execution nodes according to the resource scheduling information sent by the cloud data server.

[0066] The monitoring module is responsible for monitoring the running situation and resource usage of the system. The monitoring module collects and analyzes various information and data in the system to understand the running state and resource usage of the system in real time. When an abnormal resource or system failure is found, the monitoring module timely sends an alarm information to notify the relevant personnel to handle and repair.

Claims

1. A multi-agent based distributed resource scheduling method, characterized in that, The method comprises the following steps: step one, a task management module generates task period information and sends the information to a cloud data server, the cloud data server creates a corresponding resource scheduling container according to a task period serial number in the task period information, and calls corresponding node check information in a node resource database according to a task execution node in the task period information and sends the information to the resource scheduling container corresponding to the task period serial number; step two, the cloud data server connects the resource scheduling container corresponding to the task period serial number with the task execution node at the start time of the task period, and the task execution node collects current available resource information of the task execution node through a resource management module; step three, the task execution node sends the current available resource information collected by the resource management module to the resource scheduling container corresponding to the task period serial number for resource verification, and if the resource verification is passed, generates preliminary allocation information of the corresponding resources, and at the resource verification deadline, the resource scheduling container sends the verified resource information and the preliminary allocation information to the task execution node; step four, the task execution node verifies the actual allocation of the resources according to the verified resource information and the preliminary allocation information, and if there is an allocation exception, the task execution node sends an alarm information to the task management module; if there is no allocation exception, the task execution node enters step five at the end time of the task period; step five, the task execution node collects resource release information after the task is completed through the resource management module, and verifies the released resource information according to the verified resource information and the preliminary allocation information, and if all the resources are normally released, the task execution node enters step seven; Otherwise, the task execution node enters step six; step six, the task execution node identifies the resources that are not normally released according to the verified resource information, and sends the information of the resources that are not normally released to the resource scheduling container corresponding to the task period serial number, the resource scheduling container calls detailed resource information of the corresponding node from the node resource database and sends the information to the task execution node, a monitoring module identifies the resource usage in the system according to the detailed resource information, and if the corresponding unreleased resources are identified, the monitoring module sends a release notification information and returns to step five; otherwise, the task execution node sends an alarm information to the task management module; step seven, the resource scheduling of the distributed resource task period is completed.

2. The multi-agent based distributed resource scheduling method of claim 1, wherein, The task period information comprises a task execution node, a task period start and end time, a task period serial number and task execution node address information; the task period start and end time comprises a resource verification deadline and an end time of the task period. 3.The multi-agent based distributed resource scheduling method of claim 2, wherein, The node check information in the node resource database is preset resource configuration information and resource performance parameters.

4. The multi-agent based distributed resource scheduling method of claim 3, wherein, The resource verification is performed in the resource scheduling container corresponding to the task cycle serial number, and if the resource verification passes, the preliminary allocation information of the corresponding resource is generated, including that the resource management module collects the current available resource configuration information and the corresponding resource performance parameters, performs resource configuration information verification in the resource scheduling container corresponding to the task cycle serial number, and if the resource configuration information verification passes, the available resource configuration corresponding to the resource configuration information is the preliminary allocation information of the corresponding resource.

5. The multi-agent based distributed resource scheduling method of claim 4, wherein, The released resource information is verified according to the verified resource information and the preliminary allocation information, including that the resource management module collects the resource configuration information released after the task is completed, matches and verifies the released resource configuration information in the verified resource information, and if the released resource configuration information is consistent with the resource configuration information in the preliminary allocation information of the corresponding resource, there is no resource that is not normally released; otherwise, there is a resource that is not normally released.

6. A multi-agent based distributed resource scheduling system, characterized in that, The multi-agent based distributed resource scheduling method according to any one of claims 1-5 comprises a task management module, a task execution node, a cloud data server, a communication module, a resource management module and a monitoring module; the communication module, the resource management module and the monitoring module are connected with the task execution node respectively; the cloud data server is connected with the communication module; and the task management module is connected with the cloud data server.

Images