Multi-scene unmanned cluster-oriented task allocation and hardware resource dynamic scheduling optimization system and method

Abstract

The application provides a task allocation and hardware resource dynamic scheduling optimization system and method for multi-scene unmanned clusters, relates to the technical field of unmanned cluster control, and comprises the following steps: collecting dynamic operation data of the unmanned cluster; an enhanced learning training module constructs a weighted summation type reward function to guide the iterative training of an intelligent agent and obtain an optimal decision model; a multi-target task allocation algorithm module calls the model to construct an energy consumption and timeliness double-target optimization function and solve a task allocation scheme; and a hardware scheduling module dynamically allocates hardware resources according to the hierarchical evaluation results of the hardware resources. The application realizes the collaborative optimization of energy consumption and timeliness, controls the large-scale cluster decision response time within 100 ms, significantly improves the hardware resource utilization rate, and is suitable for multi-scene task scheduling of unmanned aerial vehicles, unmanned vehicles, unmanned ships and the like.

Description

Technical Field

[0001] This application relates to the field of unmanned swarm control technology, and in particular to a system and method for task allocation and dynamic scheduling optimization of hardware resources for unmanned swarms in multiple scenarios. Background Technology

[0002] With the rapid development of unmanned technologies (drones, unmanned vehicles, unmanned ships, etc.), unmanned swarms, with their advantages of strong collaborative operation capabilities, wide coverage, and high fault tolerance, have been widely applied in various fields such as transportation, emergency rescue, environmental monitoring, and military reconnaissance. One of the core technologies of unmanned swarms is task allocation and resource scheduling, the performance of which directly determines the task execution efficiency, overall energy consumption, and operational stability of the unmanned swarm.

[0003] Currently, existing unmanned swarm task allocation and resource scheduling technologies have the following shortcomings: 1. Single optimization objective: Most existing technologies only focus on optimizing a single objective, either pursuing the fastest task execution time or the lowest energy consumption, and cannot achieve coordinated optimization of energy consumption and timeliness. As a result, when unmanned swarms perform long-term, high-intensity tasks, they either consume too much energy and have insufficient battery life, or their timeliness deviation is too large and they cannot meet the task requirements. 2. Poor real-time performance of large-scale cluster scheduling: Existing task allocation algorithms mostly use traditional optimization algorithms (such as genetic algorithms and particle swarm optimization algorithms), which have many iterations and long response times. When the scale of the unmanned cluster increases (such as more than 50 devices), the scheduling delay increases significantly and cannot meet the real-time task scheduling requirements. 3. Weak algorithm adaptability: Most existing task allocation algorithms are static algorithms, which cannot adaptively adjust according to the dynamic operating status of unmanned swarms (such as equipment failure, sudden changes in energy consumption, and task additions), and they are not trained and optimized by combining intelligent algorithms such as reinforcement learning, resulting in limited decision accuracy and adaptability. 4. Disconnect between task allocation and hardware scheduling: In existing technologies, task allocation algorithms and hardware resource scheduling are mostly designed independently and do not achieve linkage and coordination, resulting in low utilization of hardware resources. Even if the task allocation scheme is reasonable, improper allocation of hardware resources may affect the task execution efficiency and energy consumption control effect.

[0004] To address the aforementioned technical challenges, there is an urgent need to design an optimization system that combines reinforcement learning to achieve dual objectives of energy consumption and time efficiency optimization, supports real-time scheduling of large-scale clusters, and coordinates task allocation with hardware scheduling. At the same time, it is essential to clearly protect the core algorithms and hardware modules, providing technical support for the efficient and stable operation of unmanned clusters. Summary of the Invention

[0005] The purpose of this invention is to address the technical problems of inability to coordinate energy consumption and timeliness optimization, poor real-time performance of large-scale cluster scheduling, and low hardware resource utilization in existing unmanned cluster task scheduling, and to provide a task allocation and dynamic hardware resource scheduling optimization system and method for unmanned clusters in multiple scenarios.

[0006] The above-mentioned objective of this application is achieved through the following technical solution: The status monitoring module is used to collect dynamic operation data of the unmanned cluster in real time based on task parameters. The dynamic operation data includes the remaining hardware resources of each unmanned device, the current task load, real-time energy consumption, task execution time deviation, and the status of the unassigned task queue. The reinforcement learning training module is used to construct a reinforcement learning environment. It uses dynamic running data as the environment state space, the decision unit of the multi-objective task allocation algorithm as the agent, and the task allocation ratio and hardware resource allocation ratio of each unmanned device as the action space. It also constructs a weighted summation reward function to guide the agent to iterative training until a convergent optimal decision model is obtained. The multi-objective task allocation algorithm module is connected to the reinforcement learning training module. It is used to call the optimal decision model, combine task parameters to construct an optimization function with the dual objectives of minimizing energy consumption and minimizing timeliness deviation, and solve the task allocation scheme under the conditions of satisfying task allocation constraints, hardware resource constraints and timeliness constraints. The hardware scheduling module, in conjunction with the multi-objective task allocation algorithm module, is used to dynamically allocate hardware resources and obtain hardware resource allocation instructions based on the task allocation scheme and the hierarchical evaluation results of the hardware resource status of each unmanned device. The output execution module is used to convert task allocation schemes and hardware resource allocation instructions into execution instructions that can be recognized by unmanned equipment and then issue them for execution.

[0007] Optionally, the key task parameters include: task type, task priority, task execution time limit, task energy consumption threshold, task allocation ratio of unmanned equipment, and hardware resource allocation ratio of unmanned equipment. The status monitoring module collects dynamic operation data every 10 milliseconds. After collection, the data is standardized and fed back to the reinforcement learning training module, the multi-objective task allocation algorithm module, and the hardware scheduling module.

[0008] Optionally, step S2 includes: The reward function expression is: ,in This is a reverse-quantification of energy consumption reward based on the overall task execution energy consumption. This is a reverse-quantification of time-efficiency reward item based on the deviation in task execution timeliness. , These are the adjustable weighting coefficients for energy consumption optimization goals and timeliness optimization goals, respectively; In the weighted summation reward function, the energy consumption reward item Designed using reverse quantization: setting a baseline energy consumption value. When the actual energy consumption Positive rewards are given when the value is below the benchmark. The lower the score, the greater the reward value; conversely, a negative penalty is imposed if the score exceeds the target. (Time-limited reward items) Designed using reverse quantization: setting a preset time limit When the actual execution time Positive rewards will be given within the preset time limit. The smaller the deviation, the greater the reward value; exceeding the time limit results in a negative penalty. , These are energy consumption reward coefficient and timeliness reward coefficient; weighting coefficient. and satisfy Users can flexibly adjust according to application scenarios that prioritize energy saving or time efficiency.

[0009] Optionally, step S4 includes: The optimal decision model used by the multi-objective task allocation algorithm module is trained based on the deep Q-network algorithm or the Q-learning algorithm, and the model parameters are updated online adaptively by receiving real-time running data from the state monitoring module.

[0010] Optionally, the multi-objective task allocation algorithm module adopts a task grouping and parallel solution strategy, dividing the task queue into multiple sub-task groups and performing allocation and solution in parallel using multiple threads.

[0011] Optionally, the task grouping and parallel solution strategy controls the decision response time to within 100 milliseconds, adapting to large-scale cluster scheduling scenarios with 100 or more unmanned devices.

[0012] Optionally, the hardware scheduling module includes: The resource monitoring unit receives dynamic operating data from the status monitoring module and categorizes the dynamic operating data of each unmanned device into three levels: normal, strained, and overloaded, based on resource occupancy rate, to obtain a graded evaluation result. The normal level is defined as a resource occupancy rate of less than or equal to 60%, the strained level is defined as a resource occupancy rate of greater than 60% and less than or equal to 80%, and the overload level is defined as a resource occupancy rate of greater than 80%. The resource allocation unit is used to dynamically allocate hardware resources according to the task allocation scheme and the hierarchical evaluation results, and obtain the hardware resource allocation status and scheduling adjustment results: allocate resources to normal level equipment according to the scheme, assist in scheduling redundant hardware resources of idle equipment in the same partition for tense level equipment, and immediately adjust the task allocation ratio for overload level equipment and transfer unexecuted tasks to idle equipment. The scheduling feedback unit is used to feed back the hardware resource allocation status and scheduling adjustment results to the status monitoring module and the multi-objective task allocation algorithm module in real time.

[0013] A method for optimizing task allocation and dynamic scheduling of hardware resources in multi-scenario unmanned clusters, the method comprising: The dynamic operation data of the unmanned cluster is collected in real time based on task parameters. The dynamic operation data includes the remaining hardware resources of each unmanned device, the current task load, real-time energy consumption, task execution time deviation, and the status of the unassigned task queue. A reinforcement learning environment is constructed, with dynamic running data as the environment state space, decision-making units of multi-objective task allocation algorithms as agents, and task allocation ratios and hardware resource allocation ratios of each unmanned device as action space. A weighted summation reward function is constructed to guide the agent to iterative training until a convergent optimal decision model is obtained. The optimal decision model is invoked, and an optimization function with the dual objectives of minimizing energy consumption and minimizing timeliness deviation is constructed in combination with the task parameters. The task allocation scheme is solved under the conditions of satisfying the task allocation constraints, hardware resource constraints and timeliness constraints. Based on the task allocation plan and the hierarchical evaluation results of the hardware resource status of each unmanned device, hardware resources are dynamically allocated, and hardware resource allocation instructions are obtained. The task allocation scheme and hardware resource allocation instructions are converted into execution instructions that can be recognized by unmanned equipment and then issued for execution.

[0014] The beneficial effects of the technical solution provided in this application are: 1. Achieving coordinated optimization of energy consumption and timeliness: This invention designs a multi-objective task allocation algorithm based on reinforcement learning, constructs a dual-objective optimization function for energy consumption and timeliness, and combines iterative training of reinforcement learning agents to solve the deficiency of single optimization objective in existing technologies. It can minimize the overall energy consumption of unmanned swarms, extend the swarm's endurance, and improve the economy and stability of task execution while ensuring the timeliness of task execution.

[0015] 2. Enhanced real-time scheduling capability for large-scale clusters: The multi-objective task allocation algorithm of this invention is based on the optimal decision model obtained by reinforcement learning training, which reduces the number of iterations in the solution process. At the same time, it adopts task grouping and parallel solution strategies to control the scheduling response time of large-scale clusters within a preset threshold, which can stably support real-time task scheduling of more than 100 unmanned devices and solve the problem of high latency in large-scale cluster scheduling in existing technologies.

[0016] 3. Strong algorithm adaptability: The reinforcement learning training module supports online adaptive updates of the model, and can dynamically fine-tune the decision model parameters based on real-time data fed back by the state monitoring module, adapting to the dynamic operating state of the unmanned swarm (such as equipment failure, task addition, and sudden changes in energy consumption). Compared with traditional static algorithms, the decision accuracy and adaptability are significantly improved.

[0017] 4. Achieving Collaboration Between Task Allocation and Hardware Scheduling: This invention deeply integrates the multi-objective task allocation algorithm with the hardware scheduling module. The hardware scheduling module dynamically allocates hardware resources based on the task allocation scheme and the device hardware status, avoiding the problem of task allocation and hardware scheduling being disconnected, improving the utilization rate of hardware resources, and ensuring that the task allocation scheme can be implemented efficiently.

[0018] 5. Clear scope of protection and strong practicality: This invention focuses on protecting the energy consumption-time efficiency multi-objective task allocation algorithm based on reinforcement learning and the supporting hardware scheduling module. The core technology is innovative and independent, and can be widely applied to multi-scenario task scheduling of various unmanned swarms such as UAV swarms, unmanned vehicle swarms, and unmanned ship swarms (such as transportation inspection, emergency rescue coordination, environmental monitoring, etc.). It has broad application prospects and strong practicality and industrialization value. Attached Figure Description

[0019] The present application will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a module connection diagram in the embodiments of this application; Figure 2 This is a diagram of the reinforcement learning training module in the embodiments of this application; Figure 3 This is a block diagram of the task allocation algorithm in the embodiments of this application; Figure 4 This is a diagram of the hardware scheduling module in the embodiments of this application; Figure 5 This is a state-action mapping surface diagram in the embodiments of this application; Figure 6 This is a diagram illustrating the dynamic flow mechanism of hardware resources in this application embodiment. Detailed Implementation

[0020] To provide a clearer understanding of the technical features, objectives, and effects of this application, the specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0021] The embodiments of this application provide a method for optimizing task allocation and dynamic scheduling of hardware resources for unmanned clusters in multiple scenarios.

[0022] Please refer to Figure 1 , Figure 1 This is a module connection diagram of a task allocation and hardware resource dynamic scheduling optimization method for multi-scenario unmanned clusters in an embodiment of this application, including: The status monitoring module is used to collect dynamic operation data of the unmanned cluster in real time based on task parameters. The dynamic operation data includes the remaining hardware resources of each unmanned device, the current task load, real-time energy consumption, task execution time deviation, and the status of the unassigned task queue. The reinforcement learning training module is used to construct a reinforcement learning environment. It uses dynamic running data as the environment state space, the decision unit of the multi-objective task allocation algorithm as the agent, and the task allocation ratio and hardware resource allocation ratio of each unmanned device as the action space. It also constructs a weighted summation reward function to guide the agent to iterative training until a convergent optimal decision model is obtained. The multi-objective task allocation algorithm module is connected to the reinforcement learning training module. It is used to call the optimal decision model, combine task parameters to construct an optimization function with the dual objectives of minimizing energy consumption and minimizing timeliness deviation, and solve the task allocation scheme under the conditions of satisfying task allocation constraints, hardware resource constraints and timeliness constraints. The hardware scheduling module, in conjunction with the multi-objective task allocation algorithm module, is used to dynamically allocate hardware resources and obtain hardware resource allocation instructions based on the task allocation scheme and the hierarchical evaluation results of the hardware resource status of each unmanned device. The output execution module is used to convert task allocation schemes and hardware resource allocation instructions into execution instructions that can be recognized by unmanned equipment and then issue them for execution.

[0023] As one embodiment, the system includes a task input module, a reinforcement learning training module, a multi-objective task allocation algorithm module, a hardware scheduling module, a status monitoring module, and an output execution module. The multi-objective task allocation algorithm module, based on a decision model trained through reinforcement learning, achieves collaborative optimization of both energy consumption and timeliness, supporting real-time task allocation for large-scale unmanned swarms. The hardware scheduling module works in conjunction with the task allocation algorithm module to dynamically schedule and allocate hardware resources for the unmanned swarm. The scope of this invention includes the energy consumption-timeliness multi-objective task allocation algorithm based on reinforcement learning and its supporting hardware scheduling module, which can significantly improve the task execution efficiency of unmanned swarms, reduce overall energy consumption, and enhance the real-time performance and stability of large-scale swarm scheduling. It is applicable to various unmanned swarms (drones, unmanned vehicles, unmanned ships, etc.) and multi-scenario task scheduling scenarios.

[0024] The key task parameters include: task type, task priority, task execution time limit, task energy consumption threshold, task allocation ratio of unmanned equipment, and hardware resource allocation ratio of unmanned equipment. The status monitoring module collects dynamic operation data every 10 milliseconds. After collection, the data is standardized and fed back to the reinforcement learning training module, the multi-objective task allocation algorithm module, and the hardware scheduling module.

[0025] In one embodiment, the system receives task instructions from an external host computer or user, and decomposes these instructions using a parsing module to extract key task parameters, including but not limited to: task type (e.g., inspection, transportation, monitoring), task priority (high / medium / low), task execution time limit (i.e., preset time limit), task energy consumption threshold (i.e., the maximum allowed energy consumption during task execution), and hardware resource requirements (e.g., CPU computing power, memory capacity, communication bandwidth, battery capacity). After parsing, the standardized task parameters are synchronously transmitted to the reinforcement learning training module (for model fine-tuning) and the multi-objective task allocation algorithm module (for task allocation and solving), ensuring that subsequent modules obtain accurate and consistent task input.

[0026] In one embodiment, the status monitoring module is responsible for collecting real-time dynamic operational data of the unmanned swarm, providing data support for decision-making optimization of various modules in the system. The collected data includes: the hardware resource status of each unmanned device (CPU, memory, battery, bandwidth), task execution progress (percentage of completed tasks, remaining task duration), real-time energy consumption (energy consumption per unit time, cumulative energy consumption), task execution timeliness deviation (difference between actual execution time and preset time limit), and task queue status (number of unassigned tasks, number of tasks to be executed, and priority). The collection frequency can be set according to actual needs (e.g., once every 10ms). After standardization processing, the collected data is fed back to the reinforcement learning training module (for model fine-tuning), the multi-objective task allocation algorithm module (for task allocation adjustment), and the hardware scheduling module (for resource scheduling optimization), ensuring that each module can make optimal decisions based on real-time data.

[0027] Step S2 includes: The reward function expression is: ,in This is a reverse-quantification of energy consumption reward based on the overall task execution energy consumption. This is a reverse-quantification of time-efficiency reward item based on the deviation in task execution timeliness. , These are the adjustable weighting coefficients for energy consumption optimization goals and timeliness optimization goals, respectively; In the weighted summation reward function, the energy consumption reward item Designed using reverse quantization: setting a baseline energy consumption value. When the actual energy consumption Positive rewards are given when the value is below the benchmark. The lower the score, the greater the reward value; conversely, a negative penalty is imposed if the score exceeds the target. (Time-limited reward items) Designed using reverse quantization: setting a preset time limit When the actual execution time Positive rewards will be given within the preset time limit. The smaller the deviation, the greater the reward value; exceeding the time limit results in a negative penalty. , These are energy consumption reward coefficient and timeliness reward coefficient; weighting coefficient. and satisfy Users can flexibly adjust according to application scenarios that prioritize energy saving or time efficiency.

[0028] As one embodiment, the reinforcement learning environment is constructed as follows: the overall task execution system of the unmanned swarm is used as the reinforcement learning environment. The state space of the environment is defined as the dynamic operating state of the unmanned swarm, including the remaining hardware resources of each unmanned device (CPU, memory, battery, bandwidth), the current task load (the number of assigned but unfinished tasks and the resources required), real-time energy consumption (energy consumption per unit time), task execution timeliness deviation (the difference between the actual execution time and the preset time limit), and the queue status of unassigned tasks (number of tasks, priority, and required resources). The dynamism of the environment is reflected in the fact that the environment state space is updated in real time when unmanned devices execute tasks, the hardware state changes, or new tasks are added.

[0029] In one embodiment, the decision unit of the multi-objective task allocation algorithm module is used as a reinforcement learning agent. The core role of the agent is to output the optimal task allocation and resource scheduling strategy based on the environmental state. The action space is defined as all the decision actions that the agent can execute, specifically the task allocation ratio for each unmanned device (i.e., which tasks are assigned to which device) and the hardware resource allocation ratio (i.e., how much CPU, memory, and other resources are allocated to each device). The output of the actions must satisfy the hardware resource constraints and task execution constraints of the unmanned devices.

[0030] In one embodiment, the agent in the reinforcement learning environment employs an "exploration-exploitation" strategy to select actions (exploration means trying new decision-making strategies, and exploitation means adopting the current optimal decision-making strategy). After executing an action, the environment provides feedback on a new state and corresponding reward value. Based on the reward value, the agent updates its decision model parameters using a Q-learning algorithm or a deep reinforcement learning algorithm (DQN), gradually improving the rationality of its decisions. The iterative training process continues until the reward value converges to a preset threshold (i.e., the optimization effect of the decision model reaches the expected level), obtaining the optimal decision model, which is then output to the multi-objective task allocation algorithm module. Simultaneously, the unmanned swarm operation data fed back in real time by the state monitoring module is continuously input into the reinforcement learning environment. The agent fine-tunes the model parameters based on the new environmental data, achieving online adaptive updates of the model and ensuring that the decision model can adapt to the dynamic operational changes of the unmanned swarm (such as equipment failure, task addition, and sudden changes in energy consumption).

[0031] Step S4 includes: The optimal decision model used by the multi-objective task allocation algorithm module is trained based on the deep Q-network algorithm or the Q-learning algorithm, and the model parameters are updated online adaptively by receiving real-time running data from the state monitoring module.

[0032] The multi-objective task allocation algorithm module adopts a task grouping and parallel solution strategy, dividing the task queue into multiple sub-task groups and performing allocation and solution in parallel using multiple threads.

[0033] As one embodiment, the multi-objective task allocation algorithm module is one of the core inventive points of this invention. Based on the optimal decision model output by the reinforcement learning training module and combined with the task parameters parsed by the task input module, it achieves task allocation with dual objectives of energy consumption and time efficiency optimization, while supporting real-time scheduling of large-scale unmanned swarms. Its core working process is as follows: Figure 3 As shown.

[0034] As one example, the optimization function is constructed as follows: Combining task parameters and the state of the unmanned equipment, a dual-objective optimization function for energy consumption and timeliness is constructed, clearly defining the optimization objectives and constraints. The optimization objectives are "minimizing energy consumption" and "optimizing timeliness (i.e., minimizing timeliness deviation)". The constraints include task allocation constraints (each task is assigned to only one device), hardware resource constraints (the resources required for tasks assigned to a device do not exceed its available resources), and timeliness constraints (the total time for a device to execute tasks does not exceed its maximum schedulable time). The specific function expression and constraints are detailed below.

[0035] As one example, the solution is fast: the optimal decision model trained by reinforcement learning is invoked, and the decision rules of the model are used to reduce the number of iterations in traditional optimization algorithms, quickly obtaining a task allocation scheme that satisfies the dual-objective optimization and constraint conditions. For the real-time requirements of large-scale cluster scheduling, a "task grouping + parallel solution" strategy is adopted. The large-scale task queue is divided into multiple sub-task groups according to task type and priority. Combined with the device partitioning of the unmanned cluster (e.g., dividing devices into groups by region), the allocation solution of each sub-task group is executed in parallel by multiple threads, controlling the decision response time within a preset threshold (e.g., ≤100ms), adapting to the large-scale cluster scheduling requirements of ≥100 unmanned devices.

[0036] As one implementation example, dynamic adjustment involves receiving task execution progress, real-time energy consumption, timeliness deviation, and device status data from the status monitoring module, and dynamically fine-tuning the assigned task plan. For instance, when a device experiences an abnormally high energy consumption, its task allocation ratio is adjusted, transferring some tasks to idle devices with lower energy consumption; when a device experiences task execution delays that affect overall timeliness, its task priority is increased, and redundant hardware resources are scheduled to assist in execution, ensuring that both energy consumption and timeliness are always optimal.

[0037] The task grouping and parallel solution strategy controls the decision response time to within 100 milliseconds, making it suitable for large-scale cluster scheduling scenarios with 100 or more unmanned devices.

[0038] The hardware scheduling module includes: The resource monitoring unit receives dynamic operating data from the status monitoring module and categorizes the dynamic operating data of each unmanned device into three levels: normal, strained, and overloaded, based on resource occupancy rate, to obtain a graded evaluation result. The normal level is defined as a resource occupancy rate of less than or equal to 60%, the strained level is defined as a resource occupancy rate of greater than 60% and less than or equal to 80%, and the overload level is defined as a resource occupancy rate of greater than 80%. The resource allocation unit is used to dynamically allocate hardware resources according to the task allocation scheme and the hierarchical evaluation results, and obtain the hardware resource allocation status and scheduling adjustment results: allocate resources to normal level equipment according to the scheme, assist in scheduling redundant hardware resources of idle equipment in the same partition for tense level equipment, and immediately adjust the task allocation ratio for overload level equipment and transfer unexecuted tasks to idle equipment. The scheduling feedback unit is used to feed back the hardware resource allocation status and scheduling adjustment results to the status monitoring module and the multi-objective task allocation algorithm module in real time.

[0039] As one embodiment, the hardware scheduling module is another core protection point of the present invention. It works in deep conjunction with the multi-objective task allocation algorithm module to be responsible for the dynamic allocation and scheduling of unmanned cluster hardware resources, ensuring the efficient execution of the task allocation scheme and improving the utilization rate of hardware resources.

[0040] As one embodiment, the hardware scheduling module includes a resource monitoring unit, a resource allocation unit, and a scheduling feedback unit, with specific functions as follows: Figure 4 As shown: (1) Resource monitoring unit: Receives real-time hardware resource status data of each unmanned device from the status monitoring module, including CPU utilization, memory utilization, remaining battery power, communication bandwidth utilization, etc., and performs a graded assessment of the hardware resource status of each device, dividing it into three levels: "normal" (resource utilization ≤ 60%), "tight" (60% < resource utilization ≤ 80%), and "overload" (resource utilization > 80%). The graded assessment results are then transmitted to the resource allocation unit.

[0041] (2) Resource Allocation Unit: Based on the task allocation scheme output by the multi-objective task allocation algorithm module and combined with the hierarchical evaluation results of the resource monitoring unit, hardware resources are dynamically allocated. For devices with a hardware status of "normal", corresponding hardware resources are allocated according to the task allocation scheme; for devices with a status of "stress", redundant hardware resources (such as idle CPU computing power and remaining memory) of idle devices in the same partition are prioritized for assistance to alleviate their hardware load; for devices with a status of "overload", the task allocation ratio is immediately adjusted, some of their unexecuted tasks are transferred to idle devices, and new tasks are stopped from being allocated to them until the hardware status is restored to the "normal" or "stress" level. In addition, the resource allocation unit also supports dynamic scheduling of hardware resources. When a device completes its task and releases its hardware resources, redundant resources are promptly allocated to devices that need them to improve the overall hardware resource utilization rate.

[0042] (3) Scheduling Feedback Unit: The unit feeds back the allocation status of hardware resources and the scheduling adjustment results (such as resource allocation ratio and task transfer status) to the status monitoring module and the multi-objective task allocation algorithm module in real time. The status monitoring module updates the device hardware status based on the feedback data, and the multi-objective task allocation algorithm module fine-tunes the task allocation scheme based on the feedback data to achieve collaborative optimization of task allocation and hardware scheduling.

[0043] In one embodiment, the output execution module is responsible for converting the system's decision results into execution instructions that can be recognized by the unmanned equipment, controlling the unmanned swarm to complete the task. Specifically, it receives the task allocation scheme output by the multi-target task allocation algorithm module (e.g., a drone performing an area inspection task) and the hardware resource allocation instructions output by the hardware scheduling module (e.g., allocating 50% of CPU computing power for task execution). Through the instruction conversion unit, it converts these into dedicated execution instructions (e.g., flight instructions, driving instructions, task execution instructions) for the unmanned equipment (drones, unmanned vehicles, etc.), and sends them to each unmanned equipment through the communication module. At the same time, it receives execution feedback from the unmanned equipment in real time to confirm whether the task has started and is being executed normally. If an execution anomaly occurs (e.g., equipment not responding, task execution failing), the anomaly information is immediately fed back to the status monitoring module, triggering the system's dynamic adjustment mechanism (e.g., reallocating tasks, scheduling backup equipment).

[0044] A method for optimizing task allocation and dynamic scheduling of hardware resources in multi-scenario unmanned clusters, the method comprising: The dynamic operation data of the unmanned cluster is collected in real time based on task parameters. The dynamic operation data includes the remaining hardware resources of each unmanned device, the current task load, real-time energy consumption, task execution time deviation, and the status of the unassigned task queue. A reinforcement learning environment is constructed, with dynamic running data as the environment state space, decision-making units of multi-objective task allocation algorithms as agents, and task allocation ratios and hardware resource allocation ratios of each unmanned device as action space. A weighted summation reward function is constructed to guide the agent to iterative training until a convergent optimal decision model is obtained. The optimal decision model is invoked, and an optimization function with the dual objectives of minimizing energy consumption and minimizing timeliness deviation is constructed in combination with the task parameters. The task allocation scheme is solved under the conditions of satisfying the task allocation constraints, hardware resource constraints and timeliness constraints. Based on the task allocation plan and the hierarchical evaluation results of the hardware resource status of each unmanned device, hardware resources are dynamically allocated, and hardware resource allocation instructions are obtained. The task allocation scheme and hardware resource allocation instructions are converted into execution instructions that can be recognized by unmanned equipment and then issued for execution.

[0045] In one embodiment, this embodiment provides a reinforcement learning task allocation and hardware resource dynamic scheduling optimization system for multi-scenario unmanned swarms, which is applied to the environmental monitoring task of the unmanned swarm. The unmanned swarm size is 120 unmanned swarms, the task requirement is to conduct full-coverage environmental monitoring of a certain area, and the monitoring task is required to be completed within 2 hours (preset time limit), and the overall energy consumption does not exceed the preset threshold (average energy consumption of each unmanned swarm ≤ 500Wh).

[0046] 1. System Setup The system includes a task input module, a reinforcement learning training module, a multi-objective task allocation algorithm module, a hardware scheduling module, a status monitoring module, and an output execution module. These modules are interconnected via wireless communication. The specific setup is as follows: (1) Task input module: The host computer terminal is used to receive the environmental monitoring task instructions issued by the user and parse the task parameters: the task type is environmental monitoring, the task priority is high, the task execution time limit is 2 hours, the overall energy consumption threshold is 60000Wh (120 units × 500Wh), and the hardware resources required for each UAV monitoring task are: CPU computing power ≥ 20%, memory ≥ 1GB, communication bandwidth ≥ 10Mbps, and battery capacity ≥ 1000Wh.

[0047] (2) Reinforcement Learning Training Module: An industrial computer is used to build a reinforcement learning training environment. The environment status is defined as the CPU utilization rate, memory utilization rate, remaining battery power, real-time energy consumption, monitoring task execution progress, and queue status of unassigned monitoring tasks for 120 UAVs. The decision unit of the multi-objective task allocation algorithm is used as the intelligent agent, and the action space is the monitoring task allocation ratio and hardware resource allocation ratio of each UAV. The reward function is set. (Energy consumption weight) (Timeliness weight) Prioritize task timeliness; use the DQN deep reinforcement learning algorithm for iterative training for 1000 rounds until the reward value converges to a preset threshold (R≥0.8) to obtain the optimal decision model.

[0048] (3) Multi-objective task allocation algorithm module: integrated into the industrial computer, calling the optimal decision model obtained by reinforcement learning training, constructing a dual objective optimization function of energy consumption and time efficiency, with constraints that each monitoring task is allocated to only one UAV, the resources required for the task allocated to the UAV do not exceed its available resources, and the total execution time of each UAV does not exceed 2 hours; a task grouping (divided into 12 sub-task groups according to the monitoring area) and parallel solution strategy are adopted, and the task allocation solution is executed in multiple threads.

[0049] (4) Hardware scheduling module: integrated into the control center of the UAV cluster, including resource monitoring unit, resource allocation unit and scheduling feedback unit, to monitor the hardware resource status of 120 UAVs in real time and dynamically allocate resources such as CPU computing power, memory and communication bandwidth.

[0050] (5) Status monitoring module: Install status monitoring sensors on each UAV to collect the CPU usage, memory usage, remaining battery power, real-time energy consumption, and monitoring task execution progress of the UAV in real time. The collection frequency is once every 10ms, and the data is fed back to each module through wireless communication.

[0051] (6) Output execution module: Converts the task allocation scheme and hardware resource allocation instructions into flight instructions and monitoring instructions that can be recognized by the UAV, and sends them to each UAV through wireless communication to control the UAV to perform environmental monitoring tasks.

[0052] 2. System operation process (1) Task input and parsing: The host computer issues environmental monitoring task instructions, the task input module parses the task parameters, and transmits them synchronously to the reinforcement learning training module and the multi-objective task allocation algorithm module.

[0053] (2) Reinforcement learning model call: The multi-objective task allocation algorithm module calls the optimal decision model output by the reinforcement learning training module, and combines the task parameters and the initial state data of the UAV fed back by the state monitoring module to initialize the task allocation decision variables.

[0054] (3) Task allocation solution: Construct a dual objective optimization function of energy consumption and time efficiency, adopt task grouping and parallel solution strategy, quickly solve the monitoring task allocation scheme of 120 UAVs (each UAV is assigned a monitoring task in a corresponding area, and the task scope and execution order are clearly defined), and synchronize the scheme to the hardware scheduling module and the output execution module.

[0055] (4) Hardware resource scheduling: The hardware scheduling module receives the task allocation plan and dynamically allocates hardware resources based on the hardware status of the UAV fed back by the status monitoring module. For UAVs with a hardware status of "strained", the redundant CPU computing power and memory of idle UAVs in the same area are scheduled to assist; for UAVs with a hardware status of "overloaded", some unexecuted tasks are transferred to idle UAVs in a timely manner.

[0056] (5) Real-time monitoring and adjustment: The status monitoring module collects the UAV's operational data in real time and feeds it back to each module. The reinforcement learning training module fine-tunes the model parameters based on the real-time data; the multi-objective task allocation algorithm module fine-tunes the task allocation scheme based on the data; and the hardware scheduling module adjusts the hardware resource allocation strategy based on the data to ensure that the dual objectives of energy consumption and timeliness are met.

[0057] (6) Task Execution and Feedback: The output execution module sends instructions to each UAV, controls the UAV to execute the monitoring task, and receives execution feedback in real time to ensure the normal completion of the task. Table 1: Performance Comparison of Different Algorithms in Monitoring Tasks with 120 UAVs

[0058] As shown in Table 1, traditional algorithms and single-objective reinforcement learning often suffer from trade-offs. For example, while the standard QMIX algorithm has the shortest task completion time (1.78 hours), the lack of dynamic constraints on hardware resources leads to frequent scheduling of fully loaded devices, causing single-machine energy consumption to soar to 585.2 Wh, exceeding the safety threshold of 500 Wh. This invention introduces a weighted reward function design that balances energy consumption and time efficiency (…). While ensuring completion within 2 hours, the average energy consumption was reduced to 415.8Wh, effectively extending the overall battery life of the cluster.

[0059] Table 2: Comparison of Average Hardware Resource Utilization and Load Balancing

[0060] In traditional decoupled scheduling, the task allocation module is unaware of hardware status, leading to some devices experiencing CPU utilization exceeding 90% and overload, while other devices remain idle for extended periods. This invention's hardware scheduling module monitors the status in real time (classifying it into three levels: normal, strained, and overloaded), and triggers dynamic task transfer when "overloaded" occurs. As shown in Table 2, in a high-pressure scenario with 120 devices, this invention improved CPU utilization by over 130%, and the number of overload alarms dropped dramatically from 45 to 2, demonstrating the effectiveness of the software-hardware collaborative scheduling technology.

[0061] Table 3: Decision response time (ms) of different algorithms in large-scale unmanned clusters

[0062] This invention employs a "task grouping + multi-threaded parallel solution" strategy. As shown in Table 3, when the cluster size exceeds 100 devices, the response time of traditional algorithms increases exponentially; however, the system of this invention, even with 150 devices, still keeps the decision response time within the preset threshold of 100ms (only 92ms), solving the industry pain point of large-scale clusters being unable to be scheduled in real time.

[0063] This application also discloses a computer-readable storage medium storing multiple instructions that are adapted for a processor to load and execute the aforementioned method for task allocation and dynamic scheduling optimization of hardware resources for multi-scenario unmanned clusters.

[0064] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure.

[0065] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A task allocation and hardware resource dynamic scheduling optimization system for multi-scenario unmanned clusters, characterized in that the system... include: The status monitoring module is used to collect dynamic operation data of the unmanned cluster in real time based on task parameters. The dynamic operation data includes the remaining hardware resources of each unmanned device, the current task load, real-time energy consumption, task execution time deviation, and the status of the unassigned task queue. The reinforcement learning training module is used to construct a reinforcement learning environment. It uses dynamic running data as the environment state space, the decision unit of the multi-objective task allocation algorithm as the agent, and the task allocation ratio and hardware resource allocation ratio of each unmanned device as the action space. It also constructs a weighted summation reward function to guide the agent to iterative training until a convergent optimal decision model is obtained. The multi-objective task allocation algorithm module is connected to the reinforcement learning training module. It is used to call the optimal decision model, combine task parameters to construct an optimization function with the dual objectives of minimizing energy consumption and minimizing timeliness deviation, and solve the task allocation scheme under the conditions of satisfying task allocation constraints, hardware resource constraints and timeliness constraints. The hardware scheduling module, in conjunction with the multi-objective task allocation algorithm module, is used to dynamically allocate hardware resources and obtain hardware resource allocation instructions based on the task allocation scheme and the hierarchical evaluation results of the hardware resource status of each unmanned device. The output execution module is used to convert task allocation schemes and hardware resource allocation instructions into execution instructions that can be recognized by unmanned equipment and then issue them for execution.

2. The task allocation and hardware resource dynamic scheduling optimization system for multi-scenario unmanned clusters as described in claim 1, characterized in that, The key task parameters include: task type, task priority, task execution time limit, task energy consumption threshold, task allocation ratio of unmanned equipment, and hardware resource allocation ratio of unmanned equipment. The status monitoring module collects dynamic operation data every 10 milliseconds. After collection, the data is standardized and fed back to the reinforcement learning training module, the multi-objective task allocation algorithm module, and the hardware scheduling module.

3. The task allocation and hardware resource dynamic scheduling optimization system for multi-scenario unmanned clusters as described in claim 1, characterized in that, Step S2 includes: The reward function expression is: ,in This is a reverse-quantification of energy consumption reward based on the overall task execution energy consumption. This is a reverse-quantification of time-efficiency reward item based on the deviation in task execution timeliness. , These are the adjustable weighting coefficients for energy consumption optimization goals and timeliness optimization goals, respectively; In the weighted summation reward function, the energy consumption reward item Designed using reverse quantization: setting a baseline energy consumption value. When the actual energy consumption Positive rewards are given when the value is below the benchmark. The lower the score, the greater the reward value; conversely, a negative penalty is imposed if the score exceeds the target. (Time-limited reward items) Designed using reverse quantization: setting a preset time limit When the actual execution time Positive rewards will be given within the preset time limit. The smaller the deviation, the greater the reward value; exceeding the time limit results in a negative penalty. , These are energy consumption reward coefficient and timeliness reward coefficient; weighting coefficient. and satisfy Users can flexibly adjust according to application scenarios that prioritize energy saving or time efficiency.

4. The task allocation and hardware resource dynamic scheduling optimization system for multi-scenario unmanned clusters as described in claim 1, characterized in that, Step S4 includes: The optimal decision model used by the multi-objective task allocation algorithm module is trained based on the deep Q-network algorithm or the Q-learning algorithm, and the model parameters are updated online adaptively by receiving real-time running data from the state monitoring module.

5. The task allocation and hardware resource dynamic scheduling optimization system for multi-scenario unmanned clusters as described in claim 1, characterized in that, The multi-objective task allocation algorithm module adopts a task grouping and parallel solution strategy, dividing the task queue into multiple sub-task groups and performing allocation and solution in parallel using multiple threads.

6. The task allocation and hardware resource dynamic scheduling optimization system for multi-scenario unmanned clusters as described in claim 5, characterized in that, The task grouping and parallel solution strategy controls the decision response time to within 100 milliseconds, making it suitable for large-scale cluster scheduling scenarios with 100 or more unmanned devices.

7. The task allocation and hardware resource dynamic scheduling optimization system for multi-scenario unmanned clusters as described in claim 1, characterized in that, The hardware scheduling module includes: The resource monitoring unit receives dynamic operating data from the status monitoring module and categorizes the dynamic operating data of each unmanned device into three levels: normal, strained, and overloaded, based on resource occupancy rate, to obtain a graded evaluation result. The normal level is defined as a resource occupancy rate of less than or equal to 60%, the strained level is defined as a resource occupancy rate of greater than 60% and less than or equal to 80%, and the overload level is defined as a resource occupancy rate of greater than 80%. The resource allocation unit is used to dynamically allocate hardware resources according to the task allocation scheme and the hierarchical evaluation results, and obtain the hardware resource allocation status and scheduling adjustment results: allocate resources to normal level equipment according to the scheme, assist in scheduling redundant hardware resources of idle equipment in the same partition for tense level equipment, and immediately adjust the task allocation ratio for overload level equipment and transfer unexecuted tasks to idle equipment. The scheduling feedback unit is used to feed back the hardware resource allocation status and scheduling adjustment results to the status monitoring module and the multi-objective task allocation algorithm module in real time.

8. A method for optimizing task allocation and dynamic scheduling of hardware resources for multi-scenario unmanned clusters, implemented based on the system for optimizing task allocation and dynamic scheduling of hardware resources for multi-scenario unmanned clusters as described in any one of claims 1-7, characterized in that, The method includes: The dynamic operation data of the unmanned cluster is collected in real time based on task parameters. The dynamic operation data includes the remaining hardware resources of each unmanned device, the current task load, real-time energy consumption, task execution time deviation, and the status of the unassigned task queue. A reinforcement learning environment is constructed, with dynamic running data as the environment state space, decision-making units of multi-objective task allocation algorithms as agents, and task allocation ratios and hardware resource allocation ratios of each unmanned device as action space. A weighted summation reward function is constructed to guide the agent to iterative training until a convergent optimal decision model is obtained. The optimal decision model is invoked, and an optimization function with the dual objectives of minimizing energy consumption and minimizing timeliness deviation is constructed in combination with the task parameters. The task allocation scheme is solved under the conditions of satisfying the task allocation constraints, hardware resource constraints and timeliness constraints. Based on the task allocation plan and the hierarchical evaluation results of the hardware resource status of each unmanned device, hardware resources are dynamically allocated, and hardware resource allocation instructions are obtained. The task allocation scheme and hardware resource allocation instructions are converted into execution instructions that can be recognized by unmanned equipment and then issued for execution.

Images