Multi-Agv cooperative scheduling system for realizing load balancing

Abstract

The invention discloses a multi-AGV cooperative scheduling system for realizing load balancing. The multi-AGV cooperative scheduling system comprises a task management module, a state monitoring module, a load balancing scheduling module, an AGV cooperative scheduling module, an edge server optimization module, a task compression module, a task parallel processing module and a performance evaluation module. Wherein the load balancing scheduling module adopts a mode of combining first-stage weighted polling initial allocation and second-stage optimal allocation based on multi-agent deep reinforcement learning, the AGV collaborative scheduling module adopts an AxTD3 method to divide an Actor network into two parallel sub-networks, and joint training is carried out by sharing a Critic network. The edge server optimization module takes execution time deviation as a state, constructs a lightweight state vector in combination with a server isomerism index, and introduces federated learning and multi-Critic network architecture. According to the invention, adaptive collaborative scheduling of multiple AGVs in an edge computing environment is realized, and the system load balancing capability, the resource utilization efficiency and the task processing response speed are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of edge computing technology, and more specifically, to a multi-Agv collaborative scheduling system for load balancing. Background Technology

[0002] With the rapid development of edge computing technology, the three-tier architecture of mobile devices-edge nodes-remote cloud has become the mainstream mode for handling computationally intensive tasks. In this architecture, Automated Guided Vehicles (AGVs), as mobile computing carriers, can offload collected tasks to edge servers for processing. However, how to achieve reasonable task allocation and dynamic resource scheduling in complex environments with multiple AGVs and edge nodes remains a key challenge. Traditional scheduling methods often employ static rules or simple optimization algorithms, which, while achieving good load balancing or low latency in specific scenarios, struggle to balance computing resource utilization and system energy consumption, lack adaptive adjustment capabilities, and cannot cope with dynamically changing load environments.

[0003] In existing technologies, CN114201303A discloses a fixed-path AGV task unloading optimization method in an industrial IoT environment, which uses a model-free reinforcement learning method to optimize task unloading scheduling. However, there is room for improvement in terms of algorithm convergence speed and training stability. CN120893740A proposes a mobile-fixed charging station collaborative scheduling system based on reinforcement learning. Although it achieves dynamic scheduling of charging resources, its load prediction model is limited by the dimension of input features, and the prediction accuracy needs to be improved. In addition, existing scheduling schemes based on deep reinforcement learning mostly focus on optimizing task unloading energy consumption, and do not fully consider the number of edge servers, resource operating status, and multi-node collaborative mechanisms, making it difficult to achieve global load balancing and efficient resource utilization in heterogeneous computing environments. Summary of the Invention

[0004] To address the aforementioned technical problems in related technologies, this invention proposes a multi-Agv collaborative scheduling system for load balancing, which can overcome the above-mentioned shortcomings of the prior art.

[0005] To achieve the above-mentioned technical objectives, the technical solution of the present invention is implemented as follows: A multi-Agv collaborative scheduling system for load balancing; This multi-Agv collaborative scheduling system for load balancing includes: The task management module is used to receive and decompose task requests and store task information in the task queue. The status monitoring module is used to monitor the resource usage status, task processing efficiency, and AGV working status of the edge servers in the system in real time. The load balancing scheduling module includes a first stage and a second stage. The first stage uses a weighted round-robin load balancing algorithm to initially allocate tasks to each AGV. The second stage uses a multi-agent deep reinforcement learning framework, with each AGV as an independent agent. Through centralized training and distributed execution, the task allocation strategy is collaboratively optimized based on the real-time load of the edge server and the network status. The AGV collaborative scheduling module is used to achieve collaborative scheduling among multiple AGVs based on reinforcement learning algorithms. It dynamically adjusts task allocation and execution order according to the current status and task status of each AGV.

[0006] Furthermore, the system also includes: The edge server optimization module defines the state as the deviation ratio between the actual execution time and the expected time of each server, and constructs a lightweight state vector by combining server heterogeneity indicators. Through federated learning, the scheduling strategy is continuously optimized while protecting data privacy. Multiple Critic networks are used to estimate the value of the output actions of the split Actor subnetwork in the AGV collaborative scheduling module.

[0007] Furthermore, the system also includes: The task compression module simplifies task scheduling by encoding the number of tasks dequeued from the edge server and the scheduling direction, thereby reducing the complexity of action selection. The parallel task processing module is used to split tasks in the task queue according to priority and resource requirements, and allocate them according to the resource availability of each edge server to achieve parallel task processing.

[0008] Furthermore, the system also includes: The performance evaluation module is used to calculate the load balancing level, task processing efficiency and resource utilization of the system in real time, and continuously optimize the scheduling strategy based on the above indicators.

[0009] Furthermore, in the load balancing scheduling module, the deep reinforcement learning algorithm used in the second stage is the DQN algorithm, whose network input is the current system state and output is the optimized task allocation scheme.

[0010] Furthermore, the AGV collaborative scheduling module adopts the AxTD3 method to divide the original Actor network into two parallel sub-networks, which respectively handle the AGV path planning and task unloading decisions, and perform value evaluation through a shared Critic network, and jointly train using a hierarchical reinforcement learning framework.

[0011] Furthermore, the edge server heterogeneity metrics include at least one of CPU architecture and memory bandwidth.

[0012] Furthermore, the task management module is implemented using a RESTful interface, and the task information includes at least one of the following: task type, task size, task priority, and task source.

[0013] Furthermore, the status monitoring module implements a distributed lock through Zookeeper and uses the Prometheus monitoring framework to collect at least one of the following metrics: CPU utilization, memory utilization, network bandwidth utilization, task completion rate, and average task processing time.

[0014] The beneficial effects of this invention are as follows: By constructing a two-stage load balancing scheduling mechanism that includes weighted round-robin initial allocation and multi-agent deep reinforcement learning-optimized allocation, combined with a multi-AGV collaborative scheduling module based on the AxTD3 method, adaptive optimization of task allocation strategy and multi-agent collaborative decision-making are achieved. By lightweight state modeling of edge server execution time deviation and heterogeneity indicators, and by introducing federated learning and a multi-critic network architecture, the complexity of the state-action space is effectively reduced, and the stability and privacy protection capabilities of policy training are improved. Thus, while taking into account the heterogeneity of edge nodes, the overall load balancing degree, resource utilization efficiency, and task processing response speed of the system are significantly improved, fully realizing efficient, stable, and intelligent scheduling in multi-AGV collaborative scenarios. Detailed Implementation

[0015] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0016] A multi-Agv collaborative scheduling system for load balancing according to an embodiment of the present invention includes: The task management module is used to receive and decompose task requests and store task information in the task queue. The status monitoring module is used to monitor the resource usage status, task processing efficiency, and AGV working status of the edge servers in the system in real time. The load balancing scheduling module includes a first stage and a second stage. The first stage uses a weighted round-robin load balancing algorithm to initially allocate tasks to each AGV. The second stage uses a multi-agent deep reinforcement learning framework, with each AGV as an independent agent. Through centralized training and distributed execution, the task allocation strategy is collaboratively optimized based on the real-time load of the edge server and the network status. The AGV collaborative scheduling module is used to achieve collaborative scheduling among multiple AGVs based on reinforcement learning algorithms. It dynamically adjusts task allocation and execution order according to the current status and task status of each AGV.

[0017] According to an embodiment of the present invention, a multi-Agv collaborative scheduling system for load balancing is provided. In a specific embodiment, the system further includes: The edge server optimization module defines the state as the deviation ratio between the actual execution time and the expected time of each server, and constructs a lightweight state vector by combining server heterogeneity indicators. Through federated learning, the scheduling strategy is continuously optimized while protecting data privacy. Multiple Critic networks are used to estimate the value of the output actions of the split Actor subnetwork in the AGV collaborative scheduling module.

[0018] According to an embodiment of the present invention, a multi-Agv collaborative scheduling system for load balancing is provided. In a specific embodiment, the system further includes: The task compression module simplifies task scheduling by encoding the number of tasks dequeued from the edge server and the scheduling direction, thereby reducing the complexity of action selection. The parallel task processing module is used to split tasks in the task queue according to priority and resource requirements, and allocate them according to the resource availability of each edge server to achieve parallel task processing.

[0019] According to an embodiment of the present invention, a multi-Agv collaborative scheduling system for load balancing is provided. In a specific embodiment, the system further includes: The performance evaluation module is used to calculate the load balancing level, task processing efficiency and resource utilization of the system in real time, and continuously optimize the scheduling strategy based on the above indicators.

[0020] According to an embodiment of the present invention, a multi-Agv collaborative scheduling system for load balancing is provided. In a specific implementation, the deep reinforcement learning algorithm used in the second stage of the load balancing scheduling module is the DQN algorithm, whose network input is the current state of the system and whose output is the optimized task allocation scheme.

[0021] According to an embodiment of the present invention, a multi-AGV collaborative scheduling system for load balancing is provided. In a specific implementation, the AGV collaborative scheduling module adopts the AxTD3 method to divide the original Actor network into two parallel sub-networks, which respectively handle the path planning and task unloading decisions of the AGVs, and perform value evaluation through a shared Critic network. The system is jointly trained using a hierarchical reinforcement learning framework.

[0022] According to an embodiment of the present invention, a multi-Agv collaborative scheduling system for load balancing is provided. In a specific implementation, the edge server heterogeneity index includes at least one of CPU architecture and memory bandwidth.

[0023] According to an embodiment of the present invention, a multi-Agv collaborative scheduling system for load balancing is provided. In a specific implementation, the task management module is implemented using a RESTful interface, and the task information includes at least one of the following: task type, task size, task priority, and task source.

[0024] According to an embodiment of the present invention, a multi-Agv collaborative scheduling system for load balancing is provided. In a specific implementation, the status monitoring module implements a distributed lock through Zookeeper and uses the Prometheus monitoring framework to collect at least one of the following metrics: CPU utilization, memory utilization, network bandwidth utilization, task completion rate, and average task processing time.

[0025] To facilitate understanding of the above technical solutions of the present invention, the following detailed description of the above technical solutions of the present invention is provided through specific embodiments.

[0026] Example 1 This embodiment provides a multi-AGV collaborative scheduling system for achieving load balancing, including a task management module, a status monitoring module, a load balancing scheduling module, an AGV collaborative scheduling module, an edge server optimization module, a task compression module, a task parallel processing module, and a performance evaluation module.

[0027] The task management module receives and decomposes task requests, and stores task information in a task queue. Task requests include information such as task type, task size, task priority, and task source. The task queue is sorted according to task priority and resource requirements.

[0028] The status monitoring module is used to monitor the resource usage status, task processing efficiency, and AGV operating status of each edge server in the system in real time. Resource usage status includes CPU utilization, memory utilization, and network bandwidth utilization; task processing efficiency includes task completion rate and average task processing time; AGV operating status includes current position, current load, and fault status.

[0029] The load balancing scheduling module comprises a first stage and a second stage. The first stage employs a weighted round-robin load balancing algorithm to initially allocate tasks to each AGV based on the resource usage status and task processing efficiency of each edge server. The second stage utilizes a multi-agent deep reinforcement learning framework based on DQN, where each AGV acts as an independent agent, collaboratively optimizing the task allocation scheme through centralized training and distributed execution. This stage identifies task allocations that cause resource conflicts on the edge servers and optimizes them using the DQN algorithm. The DQN network input is the current system state, and the output is the optimized task allocation scheme.

[0030] The AGV collaborative scheduling module uses a reinforcement learning algorithm to achieve collaborative scheduling among multiple AGVs, dynamically adjusting task allocation and execution order based on the current state and task status of each AGV. This module employs the AxTD3 method, dividing the original Actor network into two parallel sub-networks to handle AGV path planning and task unloading decisions respectively. Each sub-network has an independent structure, small input dimension, and small parameter size, facilitating efficient training and deployment. The Actor network input is the current state of each AGV, and the output is the action selection of each AGV. The Critic network uses the same MLP structure as the Actor network to evaluate the quality of actions; the two sub-networks share hidden layer parameters to reduce the number of parameters. During training, an empirical replay technique is used with a sampling rate of 64.

[0031] The edge server optimization module, designed for large-scale systems, uses the deviation ratio between the actual execution time and the expected execution time of each edge server as its state. It constructs a lightweight state vector by combining server heterogeneity indicators, including at least one of CPU architecture and memory bandwidth. Scheduling actions are encoded using the total number of tasks and the destination edge server, forming a binary vector. This module continuously optimizes the scheduling strategy while protecting data privacy through federated learning. It also employs multiple Critic networks to estimate the value of the output actions of the split Actor sub-networks in the AGV collaborative scheduling module, thereby enhancing strategy stability. In this embodiment, the Critic network uses a structure with four hidden layers and 64 neurons per layer.

[0032] The task compression module simplifies task scheduling by encoding the number of tasks dequeued from the edge server and the scheduling direction to generate a unique integer code, thereby reducing the complexity of action selection. This embodiment uses Protocol Buffers to implement task compression.

[0033] The parallel task processing module splits tasks in the task queue according to priority and resource requirements, and allocates them based on the resource availability of each edge server to achieve parallel task processing. This module employs a priority-based task splitting strategy, setting priorities based on task size and urgency, and using a FIFO (First-In, First-Out) strategy to allocate resources, ensuring timely processing of high-priority tasks.

[0034] The performance evaluation module calculates the system's load balancing, task processing efficiency, and resource utilization in real time, and continuously optimizes the scheduling strategy based on these indicators. Load balancing is the ratio of the system's total load to its maximum capacity; task processing efficiency is the ratio of the number of completed tasks to the number of pending tasks; and resource utilization is the ratio of the system's total resource usage to available resources. This embodiment uses Grafana to display real-time performance indicators, supporting data visualization analysis and trend prediction.

[0035] Example 2 This embodiment describes the working process of the above-mentioned multi-AGV collaborative scheduling system, as follows: Step 1: Initialize system parameters, set the system operating environment, and establish task queues and edge server resource models; Step 2: The task management module receives task requests and stores the task information in the task queue; Step 3: The status monitoring module monitors the resource usage status, task processing efficiency, and AGV working status of each edge server in the system in real time; Step 4: The load balancing scheduling module performs initial task allocation and optimization, including: Step 4.1: Based on the resource usage status and task processing efficiency of each edge server, use a weighted round-robin load balancing algorithm to perform initial task allocation; Step 4.2: Identify the edge servers causing resource conflicts; Step 4.3: Train the network using the DQN algorithm to optimize the task allocation scheme. The input is the current system state, and the output is the optimized task allocation scheme. Step 5: The AGV collaborative scheduling module performs multi-AGV collaborative scheduling, including: Step 5.1: Obtain the current status and task information of each AGV; Step 5.2: Using the AxTD3 method, the Actor network is divided into two parallel sub-networks, which are trained separately. Step 5.3: The Actor network input is the current state of each AGV, and the output is the action selection of each AGV; Step 6: The task compression module compresses and encodes the task; Step 7: The parallel processing module implements parallel processing of tasks; Step 8: The performance evaluation module calculates the system's load balancing, task processing efficiency, and resource utilization. Step 9: Based on the performance evaluation results, if the system performance does not meet the requirements, return to step 4 to continue optimization; otherwise, terminate the process.

[0036] Example 3 This embodiment provides a specific implementation method for each module in the above system.

[0037] The task management module is implemented using a RESTful interface. It receives task requests from upper-layer applications, parses the task content, and stores it in the database. Task information includes fields such as task type, task size, task priority, and task source.

[0038] The status monitoring module implements a distributed lock through Zookeeper to ensure mutual exclusion of data access; it uses the Prometheus monitoring framework to collect various system metrics, including CPU utilization, memory utilization, network bandwidth utilization, task completion rate, and average task processing time.

[0039] In the load balancing scheduling module, the first stage employs a weighted round-robin algorithm to dynamically adjust task allocation weights based on the resource usage and task processing efficiency of each edge server. The second stage uses the DQN algorithm, which employs a bidirectional long short-term memory network with 256 neurons in the hidden layer, a learning rate of 0.0001, and a discount factor of 0.95. The network input is the current system state, and the output is the optimized task allocation scheme.

[0040] In the AGV collaborative scheduling module, the Actor network adopts an MLP structure, with the current state of each AGV as the input and the action selection as the output; the Critic network adopts the same MLP structure and is used to evaluate the value of the action; the two sub-networks share hidden layer parameters to reduce the number of parameters; during training, the experience replay technique is used with a sampling rate of 64.

[0041] In the edge server optimization module, the execution time deviation of each edge server is used as the state to reduce the communication overhead between the state dimension and the edge servers; the task actions are encoded as binary vectors to represent the task type and the destination edge server; a Critic network structure with 4 hidden layers and 64 neurons in each layer is used to estimate the value of the output actions of the split Actor subnetwork in the AGV collaborative scheduling module; at the same time, it supports continuous policy optimization across edge nodes through federated learning.

[0042] The task compression module uses Protocol Buffers to compress tasks and generates unique integer codes based on the number of tasks dequeued from the edge server and the scheduling direction.

[0043] The parallel task processing module adopts a priority-based task splitting strategy, setting priorities according to task size and urgency; it uses a FIFO strategy to allocate resources, prioritizing the timely processing of high-priority tasks.

[0044] The performance evaluation module uses custom metrics, including load balancing, task processing efficiency, and resource utilization; it displays real-time performance metrics through Grafana, supporting data visualization analysis and trend prediction.

[0045] In summary, by utilizing the technical solutions described above, a two-stage load balancing scheduling mechanism comprising weighted round-robin initial allocation and multi-agent deep reinforcement learning-optimized allocation is constructed. Combined with a multi-AGV collaborative scheduling module based on the AxTD3 method, adaptive optimization of task allocation strategies and multi-agent collaborative decision-making are achieved. Through lightweight state modeling of edge server execution time deviation and heterogeneity indicators, and the introduction of federated learning and a multi-critic network architecture, the complexity of the state-action space is effectively reduced, improving the stability of policy training and privacy protection capabilities. Thus, while considering the heterogeneity of edge nodes, the overall load balancing, resource utilization efficiency, and task processing response speed of the system are significantly improved, fully realizing efficient, stable, and intelligent scheduling in multi-AGV collaborative scenarios.

[0046] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-Agv collaborative scheduling system for load balancing, characterized in that, include: The task management module is used to receive and decompose task requests and store task information in the task queue. The status monitoring module is used to monitor the resource usage status, task processing efficiency, and AGV working status of the edge servers in the system in real time. The load balancing scheduling module includes a first stage and a second stage. The first stage uses a weighted round-robin load balancing algorithm to initially allocate tasks to each AGV. The second stage uses a multi-agent deep reinforcement learning framework, with each AGV as an independent agent. Through centralized training and distributed execution, the task allocation strategy is collaboratively optimized based on the real-time load of the edge server and the network status. The AGV collaborative scheduling module is used to achieve collaborative scheduling among multiple AGVs based on reinforcement learning algorithms. It dynamically adjusts task allocation and execution order according to the current status and task status of each AGV.

2. The multi-Agv collaborative scheduling system for load balancing according to claim 1, characterized in that, The system also includes: The edge server optimization module defines the state as the deviation ratio between the actual execution time and the expected time of each server, and constructs a lightweight state vector by combining server heterogeneity indicators. Through federated learning, the scheduling strategy is continuously optimized while protecting data privacy. Multiple Critic networks are used to estimate the value of the output actions of the split Actor subnetwork in the AGV collaborative scheduling module.

3. A multi-Agv collaborative scheduling system for load balancing according to claim 1, characterized in that, The system also includes: The task compression module simplifies task scheduling by encoding the number of tasks dequeued from the edge server and the scheduling direction, thereby reducing the complexity of action selection. The parallel task processing module is used to split tasks in the task queue according to priority and resource requirements, and allocate them according to the resource availability of each edge server to achieve parallel task processing.

4. A multi-Agv collaborative scheduling system for load balancing according to claim 1, characterized in that, The system also includes: The performance evaluation module is used to calculate the load balancing level, task processing efficiency and resource utilization of the system in real time, and continuously optimize the scheduling strategy based on the above indicators.

5. A multi-Agv collaborative scheduling system for load balancing according to claim 1, characterized in that, In the load balancing scheduling module, the deep reinforcement learning algorithm used in the second stage is the DQN algorithm, whose network input is the current system state and output is the optimized task allocation scheme.

6. A multi-Agv collaborative scheduling system for load balancing according to claim 1, characterized in that, The AGV collaborative scheduling module adopts the AxTD3 method, which divides the original Actor network into two parallel sub-networks to handle AGV path planning and task unloading decisions respectively, and performs value evaluation through a shared Critic network. It is jointly trained using a hierarchical reinforcement learning framework.

7. A multi-Agv collaborative scheduling system for load balancing according to claim 2, characterized in that, The edge server heterogeneity metrics include at least one of CPU architecture and memory bandwidth.

8. A multi-Agv collaborative scheduling system for load balancing according to claim 1, characterized in that, The task management module is implemented using a RESTful interface, and the task information includes at least one of the following: task type, task size, task priority, and task source.

9. A multi-Agv collaborative scheduling system for load balancing according to claim 1, characterized in that, The status monitoring module implements a distributed lock through Zookeeper and uses the Prometheus monitoring framework to collect at least one of the following metrics: CPU utilization, memory utilization, network bandwidth utilization, task completion rate, and average task processing time.