A method, system, device and medium for scheduling a drone cluster

Abstract

The application discloses a kind of unmanned aerial vehicle cluster scheduling method, system, equipment and medium, belong to unmanned aerial vehicle control technical field.The method is by collecting the historical state data of unmanned aerial vehicle cluster, strategy network is trained using the first quantum optimization module with global search characteristics and the second quantum optimization module with local search characteristics;Real-time state data of each unmanned aerial vehicle is collected and input into strategy network, to obtain the predicted operation action of each unmanned aerial vehicle;Through Lyapunov function, the feasibility of each predicted operation action is determined, and the predicted operation action that does not satisfy the constraint is corrected to generate a set of operation scheduling instructions that satisfy the physical constraint;According to real-time state data and operation scheduling instruction set, generate a charging scheduling scheme;Finally, according to the operation scheduling instruction set and the charging scheduling scheme, the unmanned aerial vehicle cluster is cooperatively scheduled.The application can improve the reliability of deep reinforcement learning method in unmanned aerial vehicle cluster online task scheduling.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) control technology, and in particular to a method, system, device and medium for UAV swarm scheduling. Background Technology

[0002] With the rapid development of drone technology, drone swarm collaboration has become a key technology in complex scenarios such as emergency rescue and area patrol. Compared to single-drone operations, drone swarms have significant advantages in terms of mission coverage and fault tolerance. However, how to achieve efficient collaborative scheduling among multiple drones in dynamic environments has become a core challenge restricting its large-scale application.

[0003] To address the real-time nature of drone swarm task scheduling, existing technologies typically employ deep reinforcement learning (DLM) methods. These methods, through continuous interaction between the agent and the environment, autonomously learn and optimize scheduling strategies, and have been widely applied in the field of drone swarm task scheduling. However, the reliability of DLM methods in online drone swarm task scheduling remains significantly insufficient. On the one hand, the policy network requires extensive random exploration in the early stages of training to obtain effective policies, resulting in slow convergence and low training efficiency. On the other hand, drones are subject to strict physical constraints during task execution (such as flight speed limits, energy consumption limits, and collision avoidance requirements), but existing DLM methods lack real-time safety verification mechanisms when the policy network outputs actions. These output actions may violate physical constraints, leading to drone crashes, battery depletion, or mission failure. Therefore, the reliability of existing DLM methods in online drone swarm task scheduling is low, making it difficult to meet practical application requirements. Summary of the Invention

[0004] This application provides a method, system, device, and medium for scheduling unmanned aerial vehicle (UAV) swarms, which can solve the problem of low reliability in online task scheduling of UAV swarms using deep reinforcement learning methods in existing technologies.

[0005] Some embodiments of this application provide a method for scheduling unmanned aerial vehicle (UAV) swarms, including: Historical state data of each drone in the drone cluster is collected. Based on the historical state data, the optimal parameters of the pre-built policy network are searched alternately by a first quantum optimization module with global search characteristics and a second quantum optimization module with local search characteristics. The policy network is then initialized with the optimal parameters. Collect real-time status data of each of the aforementioned UAVs, input the real-time status data into the policy network, and obtain the predicted operation actions of each of the aforementioned UAVs; The feasibility of each predicted job action is determined by a preset Lyapunov function. If the determination result is that the physical constraints are not met, the predicted job action is modified to obtain a set of job scheduling instructions that meet the physical constraints. Based on the real-time status data and the job scheduling instruction set, a charging scheduling scheme for the UAV cluster is generated; The drone swarm is coordinated and scheduled according to the job scheduling instruction set and the charging scheduling scheme.

[0006] Compared to existing technologies, the above embodiments have the following advantages: This application pre-trains the policy network using a quantum optimization module, enabling the policy network to have an effective exploration direction in the early stages of training, avoiding blind exploration starting from random initialization; simultaneously, it uses a Lyapunov function to determine the feasibility of each predicted task action output by the policy network and corrects actions that do not meet the constraints, ensuring that the final output task scheduling instructions meet physical constraints such as flight speed, energy consumption, and collision avoidance. Based on this, a charging scheduling scheme is generated according to real-time state data and the task scheduling instruction set, enabling charging scheduling to coordinate with task execution. These methods improve the reliability of deep reinforcement learning methods in online task scheduling of UAV swarms from both training efficiency and execution security perspectives.

[0007] Further, the step of alternately searching for the optimal parameters of the pre-built policy network based on the historical state data using a first quantum optimization module with global search characteristics and a second quantum optimization module with local search characteristics, and initializing the policy network using the optimal parameters, includes: The parameters to be optimized in the policy network are encoded into multiple individuals, and all of these individuals constitute the current population; wherein each individual corresponds to a set of parameter codes; Based on the historical state data, the current population is iteratively updated alternately by the first quantum optimization module and the second quantum optimization module until a preset iteration termination condition is met, resulting in the final population. In each round of alternation, the first quantum optimization module performs a global search on the current population to obtain a first population, which is then passed to the second quantum optimization module. The second quantum optimization module performs a local search on the first population to obtain a second population, which is then passed back to the first quantum optimization module as the current population for the next round of alternation. The parameters corresponding to the globally optimal individuals in the final population are encoded and mapped to the initial parameters of the policy network, thereby completing the training of the policy network.

[0008] Compared to existing technologies, the above embodiments have the following advantages: By encoding the parameters to be optimized in the policy network as individuals in the population, and using the global search capability of the first quantum optimization module and the local search capability of the second quantum optimization module to iteratively update the population, a balance is achieved between global exploration and local fine-tuning search during the pre-training process. Ultimately, the parameters of the globally optimal individual are encoded and mapped to the initial parameters of the policy network. This pre-training method can provide better initial parameters for the policy network without increasing additional environmental interaction costs, reducing the overhead of random exploration in the early stages of training.

[0009] Further, the policy network includes a low-level feature extraction network, an action output branch, and a priority output branch; the step of inputting the real-time state data into the policy network to obtain the predicted operational actions of each UAV includes: The real-time status data of each of the aforementioned UAVs is input into the policy network; The underlying feature extraction network is used to extract features from the real-time state data to obtain shared features. The shared features are linearly mapped and nonlinearly activated through the action output branch to output the predicted operation actions of each UAV. The shared features are linearly mapped and value estimated through the priority output branch, and the task priority corresponding to each prediction operation is output.

[0010] Compared to existing technologies, the above embodiments have the following advantages: The policy network extracts shared features from real-time state data through a shared underlying feature extraction network. The action output branch and the priority output branch simultaneously output predicted job actions and task priorities based on these shared features. The two branches share the feature extraction network, ensuring consistency between the generation of predicted job actions and the inference of task priorities, while avoiding the increased system complexity that would be incurred by building and maintaining a separate computation module for priority inference.

[0011] Further, the real-time status includes: the remaining battery power of the drone and its distance from the charging station; the step of generating a charging scheduling scheme for the drone cluster based on the real-time status data and the job scheduling instruction set includes: Based on the remaining battery power and the distance, calculate the charging urgency index for each of the drones; Determine the task distribution location corresponding to each predicted job action in the job scheduling instruction set; A charging scheduling scheme for the drone cluster is generated based on the task distribution location, the task priority, and the charging urgency index.

[0012] Compared to existing technologies, the above embodiments have the following advantages: During the generation of the charging scheduling scheme, the charging urgency index of each UAV is calculated based on the remaining power and distance to the charging pile in real time. This index is then combined with the task priority given by the priority output branch of the policy network and the task distribution location corresponding to each predicted operation to jointly generate the charging scheduling scheme. This method ensures that the decision-making basis for charging scheduling simultaneously covers the UAV's own power status and the importance of the task to be executed, and that the selection of charging timing and route matches the current operation scheduling task.

[0013] Furthermore, generating a charging scheduling scheme for the drone swarm based on the task distribution location, the task priority, and the charging urgency index includes: The drones are sorted according to the charging urgency index and the task priority to generate a charging order; For each of the aforementioned drones, the path length cost is evaluated based on the path length between the drone's current location and the target charging station location. For each of the aforementioned drones, the risk cost of mission interruption is assessed based on the mission distribution location and the mission priority corresponding to the predicted operation action. For each of the aforementioned drones, the return path of the drone is searched and determined with the goal of minimizing the weighted sum of the path length cost and the mission interruption risk cost. A charging schedule for the drone cluster is generated based on the charging sequence and each of the return paths.

[0014] Compared to existing technologies, the above embodiments have the following advantages: In the path planning algorithm, the cost function includes both path length cost and task interruption risk cost. The task interruption risk cost is obtained based on the distance between the waypoint and the distribution location of each task, as well as the task priority of the corresponding predicted operation. This cost function design enables path planning to identify and avoid high-priority task areas while searching for shorter routes, reducing the risk of task interruption caused by the drone leaving the operation area to go for charging.

[0015] Furthermore, the predicted operation includes several operation components, each corresponding to a constraint condition; the feasibility of each predicted operation is determined using a preset Lyapunov function, and if the determination result is that the physical constraint is not met, the predicted operation is modified, including: The action components of each predicted operation are input into the Lyapunov function to calculate the penalty component based on the deviation of each action component from the corresponding constraint condition, and the stability evaluation value of each predicted operation is obtained by weighted summation based on each penalty component. If the stability assessment value exceeds the preset safety threshold, then identify the out-of-bounds action component in the corresponding predicted operation action that violates the corresponding constraint condition; Based on the constraints, each out-of-bounds action component in each predicted operation is corrected by a dimension-by-dimensional projection method.

[0016] Compared to existing technologies, the above embodiments have the following advantages: By calculating the penalty component for the deviation between each action component and its corresponding constraint condition using a Lyapunov function and then weighting and summing the results, a stability evaluation value is obtained. When the evaluation value exceeds a threshold, the specific out-of-bounds action component is identified, and each out-of-bounds action component is directly corrected to its corresponding constraint boundary using a dimension-by-dimensional projection method. This method can quickly locate the specific action dimension that causes the constraint to be unmet and directly correct it according to the constraint boundary, without introducing an additional optimization solution process, resulting in low computational overhead.

[0017] Furthermore, the coordinated scheduling of the drone swarm includes: Obtain the task execution feedback data of each UAV after the execution of the job scheduling instruction set; A reward function is constructed based on the deviation between the task execution feedback data and the preset optimization target; A preset policy optimization algorithm is used to perform gradient updates on the policy network based on the reward function.

[0018] Compared with existing technologies, the above embodiments have the following advantages: After executing the job scheduling instruction set, task execution feedback data of each UAV is obtained; a reward function is constructed based on the deviation between the feedback data and the preset optimization target; and a policy optimization algorithm is used to update the policy network using gradients. This feedback mechanism enables the policy network to continuously adjust according to the actual execution effect, achieving online learning and optimization.

[0019] Another embodiment of this application provides a drone swarm scheduling system, which is used to implement the steps of the drone swarm scheduling method of this application.

[0020] Another embodiment of this application also provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps of the UAV swarm scheduling method of this application.

[0021] Another embodiment of this application also provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform steps such as the UAV swarm scheduling method of this application. Attached Figure Description

[0022] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a drone swarm scheduling method provided in some embodiments of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0026] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0027] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0028] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0029] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0030] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0031] To address the real-time nature of drone swarm task scheduling, existing technologies typically employ deep reinforcement learning (DLM) methods. These methods, through continuous interaction between the agent and the environment, autonomously learn and optimize scheduling strategies, and have been widely applied in the field of drone swarm task scheduling. However, the reliability of DLM methods in online drone swarm task scheduling remains significantly insufficient. On the one hand, the policy network requires extensive random exploration in the early stages of training to obtain effective policies, resulting in slow convergence and low training efficiency. On the other hand, drones are subject to strict physical constraints during task execution (such as flight speed limits, energy consumption limits, and collision avoidance requirements), but existing DLM methods lack real-time safety verification mechanisms when the policy network outputs actions. These output actions may violate physical constraints, leading to drone crashes, battery depletion, or mission failure. Therefore, the reliability of existing DLM methods in online drone swarm task scheduling is low, making it difficult to meet practical application requirements.

[0032] Please refer to Figure 1 To address the issue of low reliability in online task scheduling of unmanned aerial vehicle (UAV) swarms using existing deep reinforcement learning methods, this application provides a UAV swarm scheduling method, comprising the following steps S101 to S105: S101: Collect historical state data of each drone in the drone cluster. Based on the historical state data, alternately search for the optimal parameters of the pre-built policy network using a first quantum optimization module with global search characteristics and a second quantum optimization module with local search characteristics, and initialize the policy network using the optimal parameters.

[0033] Preferably, in some embodiments of this application, any state data includes: UAV location status, remaining battery status, task node location status, task node rate requirement status, and channel status information; wherein, the UAV location status is represented by three-dimensional Cartesian coordinates, the remaining battery status is represented by the remaining battery percentage, the task node rate requirement status is represented by the minimum transmission rate threshold, and the channel status information includes signal-to-noise ratio, bandwidth availability, and bit error rate parameters.

[0034] Furthermore, in some embodiments of this application, the step of alternately searching for the optimal parameters of a pre-built policy network based on the historical state data using a first quantum optimization module with global search characteristics and a second quantum optimization module with local search characteristics, and initializing the policy network using the optimal parameters, includes: The parameters to be optimized in the policy network are encoded into multiple individuals, and all of these individuals constitute the current population; wherein each individual corresponds to a set of parameter codes; Based on the historical state data, the current population is iteratively updated alternately by the first quantum optimization module and the second quantum optimization module until a preset iteration termination condition is met, resulting in the final population. In each round of alternation, the first quantum optimization module performs a global search on the current population to obtain a first population, which is then passed to the second quantum optimization module. The second quantum optimization module performs a local search on the first population to obtain a second population, which is then passed back to the first quantum optimization module as the current population for the next round of alternation. The parameters corresponding to the globally optimal individuals in the final population are encoded and mapped to the initial parameters of the policy network, thereby completing the training of the policy network.

[0035] Preferably, in some embodiments of this application, the policy network is a policy network in a reinforcement learning method, and its corresponding action space is the predicted operation action of the UAV, including: the UAV's three-dimensional position scheduling action and transmission power allocation action.

[0036] Preferably, in some embodiments of this application, the first quantum optimization module is a quantum particle swarm optimization module, and the second quantum optimization module is an evolutionary quantum artificial bee colony optimization module; the hierarchical quantum particle swarm optimization module is guided by hierarchical inertial weights and the ratio of individual optimal to global optimal, and performs global search in combination with qubit encoding and quantum rotation gate update mechanism; the evolutionary quantum artificial bee colony optimization module introduces global optimal guidance and quantum state measurement value perturbation through observation bee position update strategy, and introduces fitness average value and sensitivity for local optimization through nectar source selection strategy; wherein, the hierarchical inertial weights decrease linearly with the number of iterations, and the sensitivity is determined by calculating the absolute value of the difference between nectar source fitness and population average fitness, which is used to measure the degree of deviation of nectar source from population average quality.

[0037] Preferably, in some embodiments of this application, before using the quantum particle swarm optimization module and the evolutionary quantum artificial bee colony optimization module for alternating iterative optimization, the encoded mapping relationship between each individual and the parameters of the policy network is first established. The specific process is as follows: First, the encoding mapping process from parameters to individuals is implemented: each individual is encoded using a quantum chromosome, which consists of several qubits. A single qubit is represented by a two-dimensional complex vector, and the sum of the squares of its probability amplitudes is 1, i.e. ,in and These represent the probability amplitudes of a qubit in the |0> and |1> states, respectively. Each qubit contains information about both the |0> and |1> states, thus a quantum chromosome can simultaneously represent a superposition of multiple candidate solutions. During population initialization, the parameters to be optimized in the policy network (including the weight matrices and bias vectors of each layer) are expanded into a one-dimensional parameter sequence in a preset order. Then, each parameter in this sequence is mapped to the probability amplitude of a qubit, forming the initial quantum chromosome. Each individual corresponds to a set of parameter codes, which is also a complete set of parameter values ​​for the corresponding policy network. During the alternating iterative optimization process, the first and second quantum optimization modules update the quantum chromosomes through a quantum rotation gate update mechanism. The quantum rotation gate acts on the probability amplitude of each qubit, controlling the rotation of the quantum state on the Bloch sphere by rotating the angle, thereby realizing probabilistic transition search in the solution space, rather than performing point-by-point traversal in the deterministic encoding space. The rotation angle is dynamically calculated based on the similarity between the individual's historical optimum and the global optimum. When the similarity is high, the rotation angle is small, performing a local fine-grained search; when the similarity is low, the rotation angle is large, performing a global breadth-of-field exploration.

[0038] After each iteration, the individual is remapped back to the parameters: measurement operations are performed on the updated quantum chromosome. The measurement uses Monte Carlo sampling, performing multiple independent measurements on each qubit according to its probability amplitude, randomly collapsing the quantum state into a binary encoded string. The number of measurements is set according to the decoding accuracy requirements. After obtaining the binary encoded string, it is segmented into equal-length segments according to the total number of parameters in the policy network, so that each parameter corresponds to a fixed-length binary substring. Then, each binary substring is converted into a real number within a preset numerical range through linear mapping, obtaining the initial values ​​of the weight matrix and bias vector for each layer. After decoding, a small random perturbation is added to each weight and bias value, with the perturbation amplitude less than one-hundredth of the decoded numerical range, to break the initialization symmetry between neurons in the same layer. The preset numerical range is pre-set according to the layer type and activation function characteristics of the policy network. For fully connected layers using the ReLU activation function, the preset numerical range of the weight matrix is ​​between -0.1 and 0.1, and the preset numerical range of the bias vector is between 0 and 0.01.

[0039] Through the above encoding mapping relationship, each update of the population during the iterative optimization process is essentially a search and optimization of the policy network parameter encoding.

[0040] Preferably, in some embodiments of this application, the process of alternating iterative optimization using a quantum particle swarm optimization module and an evolutionary quantum artificial bee colony optimization module is as follows: The alternating iterations employ a nested loop structure, rather than terminating after two modules are executed sequentially. During a complete pre-training process, the quantum particle swarm optimization module and the evolutionary quantum artificial bee colony optimization module are called alternately, forming a loop, until the overall convergence condition is met.

[0041] In each round of alternation, the quantum particle swarm optimization module is executed first. This module employs hierarchical inertia weights and the ratio of individual optimal to global optimal, combined with qubit encoding and a quantum rotation gate update mechanism for global search. The hierarchical inertia weights decrease linearly with the number of iterations, allowing the algorithm to have a large search step size in the early stages to cover a broad solution space, and gradually narrowing the search range in later stages to improve convergence accuracy. After running a preset number of generations, the quantum particle swarm optimization module pauses and transfers all individuals in the current population (including each individual's position, historical best position, and global optimal information) to the evolutionary quantum artificial bee colony optimization module, serving as its initial population. During this transfer, each individual in the quantum particle swarm optimization module is directly converted into a nectar source in the evolutionary quantum artificial bee colony optimization module; the individual's quantum chromosome becomes the encoding of the nectar source, and the individual's fitness value becomes the nectar quantity of the nectar source. Simultaneously, the currently recorded global optimal individual in the quantum particle swarm optimization module is individually marked as the optimal nectar source in the evolutionary quantum artificial bee colony optimization module to guide subsequent searches.

[0042] After receiving the aforementioned population, the Evolutionary Quantum Artificial Bee Colony Optimization Module performs a localized refined search based on it. The module introduces global optimal guidance and quantum state measurement perturbation through an observation bee position update strategy, and incorporates average fitness and sensitivity for local optimization through a nectar source selection strategy. Sensitivity is determined by calculating the absolute value of the difference between the nectar source's fitness and the population's average fitness; it measures the degree of deviation of the nectar source from the population's average quality. A greater deviation increases the probability of the nectar source being explored locally in subsequent searches. After running a preset number of generations, the Evolutionary Quantum Artificial Bee Colony Optimization Module pauses, converting all nectar sources in its population into individuals of the Quantum Particle Swarm Optimization Module, using the same conversion method as described above. Simultaneously, the optimal nectar source discovered by the Evolutionary Quantum Artificial Bee Colony Optimization Module is taken as the globally optimal individual of the Quantum Particle Swarm Optimization Module, and the historical optimal state corresponding to each nectar source is taken as the historical optimal state of the corresponding particle. A global replacement strategy is adopted, meaning the entire population of the Evolutionary Quantum Artificial Bee Colony Optimization Module completely covers the current population of the Quantum Particle Swarm Optimization Module.

[0043] The switching between the quantum particle swarm optimization module and the evolutionary quantum artificial bee colony optimization module is not a simple rotation with a fixed number of iterations, but rather an adaptive triggering mechanism based on the population convergence state. When the quantum particle swarm optimization module is running, it continuously monitors population diversity indicators. If the decrease in population diversity over several generations is less than a preset threshold, it indicates that the quantum particle swarm optimization module has entered a local convergence state, and the returns from continuing global search are decreasing. At this point, it triggers a switch to the evolutionary quantum artificial bee colony optimization module for local development. When the evolutionary quantum artificial bee colony optimization module is running, it continuously monitors the improvement rate of the optimal nectar source. If the fitness improvement of the optimal nectar source over several generations is less than a preset threshold, it indicates that the local search has approached saturation. At this point, it triggers a switch back to the quantum particle swarm optimization module, using a probabilistic transition through the quantum rotation gate to jump out of the current local optimum and enter a new global search region. Furthermore, a maximum number of iterations is set as a hard constraint. When the number of generations of a module reaches the preset upper limit, a forced switch is implemented regardless of the convergence state.

[0044] The above alternating iterations continue until the change in the global optimal fitness remains stable in multiple rounds of alternation, and the overall convergence condition is met before stopping, at which point the global optimal individual in the final population is output.

[0045] Preferably, in some embodiments of this application, the quantum rotation gate update process of the quantum particle swarm optimization module is as follows: the quantum rotation gate update dynamically calculates the rotation angle based on the similarity between the individual historical best and the global best, and controls the rotation of the quantum state on the Bloch sphere through this angle to realize the probabilistic transition search of the solution space.

[0046] Preferably, in some embodiments of this application, the similarity measurement methods between individual historical optimum and global optimum include the following two: Method 1 is Hamming distance measurement, which involves performing multiple Monte Carlo measurements on each quantum chromosome, randomly collapsing the probability amplitude of the qubit into a binary encoded string, calculating the Hamming distance between the two binary strings, and calculating the similarity based on the ratio of this distance to the chromosome length; Method 2 is Euclidean distance measurement, which involves extracting the probability amplitude value of each qubit, concatenating all probability amplitudes into a continuous vector, calculating the Euclidean distance between the two vectors, and calculating the similarity based on the ratio of this distance to the maximum possible distance.

[0047] Preferably, in some embodiments of this application, the rotation angle is determined by the following rule: the higher the similarity, the smaller the rotation angle, used for local fine-grained search; the lower the similarity, the larger the rotation angle, used for global breadth-of-field exploration. The specific value of the rotation angle is determined by the functional relationship between similarity and preset maximum and minimum rotation angles. The maximum rotation angle is preset to be within π / 4 radians, and the minimum rotation angle is preset to be a positive number greater than zero. This functional relationship can be any one of linear, quadratic, or exponential decay relationships: under a linear relationship, the rotation angle is proportional to the difference between one and the similarity; under a quadratic relationship, the rotation angle is proportional to the square of one minus the similarity; under an exponential decay relationship, the rotation angle decreases exponentially with increasing similarity. The sensitivity of the mapping can be adjusted by preset coefficients for all three relationships, and in practical applications, the choice can be made according to the task's preference for exploration and development. The rotation direction is determined based on the fitness comparison results. This design allows for large rotations when similarity is low to enhance global search capabilities, and small fine-grained searches when similarity is high to promote convergence.

[0048] By encoding the parameters to be optimized in the policy network as individuals in the population, and iteratively updating the population using the global search capability of the first quantum optimization module and the local search capability of the second quantum optimization module, a balance is achieved between global exploration and local refinement search during the pre-training process. Ultimately, the parameters of the globally optimal individual are encoded and mapped to the initial parameters of the policy network. This pre-training method can provide the policy network with better initial parameters without increasing the cost of additional environmental interaction, reducing the overhead of random exploration in the early stages of training.

[0049] S102: Collect real-time status data of each UAV, input the real-time status data into the policy network, and obtain the predicted operation actions of each UAV.

[0050] Furthermore, in some embodiments of this application, the policy network includes a low-level feature extraction network, an action output branch, and a priority output branch; the step of inputting the real-time state data into the policy network to obtain the predicted operational actions of each of the UAVs includes: The real-time status data of each of the aforementioned UAVs is input into the policy network; The underlying feature extraction network is used to extract features from the real-time state data to obtain shared features. The shared features are linearly mapped and nonlinearly activated through the action output branch to output the predicted operation actions of each UAV. The shared features are linearly mapped and value estimated through the priority output branch, and the task priority corresponding to each prediction operation is output.

[0051] Preferably, in some embodiments of this application, the policy network adopts a dual-head output structure, connecting an action output branch and a priority output branch respectively on the basis of a shared underlying feature extraction network. The action output branch is used to generate the predicted operation actions of each UAV, specifically the UAV's three-dimensional position scheduling command and transmission power allocation command; the priority output branch is used to generate the task priority corresponding to each predicted operation action. In actual operation, the real-time status data of each UAV is input into the policy network, and the input data is first processed by the underlying feature extraction network to obtain a shared feature representation. This shared feature is simultaneously fed to the action output branch and the priority output branch: the action output branch outputs the predicted operation actions of each UAV based on the shared feature, and the priority output branch outputs a Q-value estimate for each task node, which represents the expected cumulative reward for executing the scheduling action related to the task node in the current state. The original Q-values ​​of each task node generated by the priority output branch are normalized by Softmax to obtain the task priority in the range [0,1]. Because the priority output branch and the action output branch share the underlying feature extraction network and jointly participate in the gradient update of the Proximal Policy Optimization (PPO) algorithm, the task priority is dynamically adjusted synchronously with the optimization of the policy network, thus always reflecting the optimal task execution order in the current environment. This design allows priority inference and action decision-making to share the same set of feature representations, ensuring their consistency, while also eliminating the need to maintain a separate priority calculation module, reducing the overall system complexity.

[0052] The policy network extracts shared features from real-time state data through a shared underlying feature extraction network. The action output branch and the priority output branch simultaneously output predicted job actions and task priorities based on these shared features. The two branches share the feature extraction network, ensuring consistency between the generation of predicted job actions and the inference of task priorities, while avoiding the increased system complexity that would be incurred by building and maintaining a separate computation module for priority inference.

[0053] S103: The feasibility of each predicted operation is determined by a preset Lyapunov function. If the determination result is that the physical constraints are not met, the predicted operation is modified to obtain a set of operation scheduling instructions that meet the physical constraints.

[0054] Furthermore, in some embodiments of this application, the predicted task action includes several task components, each task component corresponding to a constraint condition; the feasibility determination of each predicted task action is performed using a preset Lyapunov function, and if the determination result is that the physical constraint is not met, the predicted task action is modified, including: The action components of each predicted operation are input into the Lyapunov function to calculate the penalty component based on the deviation of each action component from the corresponding constraint condition, and the stability evaluation value of each predicted operation is obtained by weighted summation based on each penalty component. If the stability assessment value exceeds the preset safety threshold, then identify the out-of-bounds action component in the corresponding predicted operation action that violates the corresponding constraint condition; Based on the constraints, each out-of-bounds action component in each predicted operation is corrected by a dimension-by-dimensional projection method.

[0055] Preferably, in some embodiments of this application, the predicted operation action includes several action components, including speed change, transmission power value, distance to nearby UAVs, and remaining battery power. These components are respectively constrained by a preset maximum speed change, a preset maximum transmission power value, a preset minimum safe distance, and a preset minimum battery power threshold.

[0056] Preferably, in some embodiments of this application, the Lyapunov function is a weighted combination, consisting of three components summed according to preset weights. The first component is the kinetic energy component, calculated based on the square of the changes in the UAV's mass and velocity, representing the energy cost required for the UAV to change its flight state. The second component is the power component, calculated based on the UAV's transmit power value, representing the impact of communication energy consumption on system stability. The third component is the constraint violation penalty component, calculated based on the deviation between the velocity change, transmit power value, distance from neighboring UAVs, and remaining battery power, and their respective thresholds; a penalty term is generated when the deviation is positive. The preset minimum safe distance is dynamically set according to the UAV model and flight speed, and the preset minimum battery power threshold is dynamically calculated based on the remaining mission time and the battery power required for recharging. Preferably, the preset weights of the kinetic energy component, power component, and constraint violation penalty component are adjusted according to the mission stage: the penalty weight is reduced in the early stage of the mission to encourage exploration, and the penalty weight is increased in the later stage of the mission to ensure safety.

[0057] Preferably, in some embodiments of this application, after inputting each motion component of the predicted operation into a Lyapunov function, a penalty component is calculated based on the deviation of each motion component from the corresponding constraint condition. The weighted sum of these penalty components yields the stability evaluation value of the predicted operation. If the stability evaluation value exceeds a preset safety threshold, the predicted operation is determined to not meet the constraints. In this case, each motion component in the predicted operation is traversed, and motion components exceeding the corresponding constraint boundaries are identified as out-of-bounds motion components. Out-of-bounds motion variables include at least one of velocity change exceeding the limit, transmission power exceeding the limit, collision avoidance distance exceeding the limit, or power consumption exceeding the limit.

[0058] Preferably, in some embodiments of this application, after identifying out-of-bounds action components, a step-by-step projection method is used for correction. Specifically, each out-of-bounds action variable is projected onto its corresponding constraint boundary. If the action variable value is greater than the upper constraint limit, it is assigned the upper constraint limit value; if the action variable value is less than the lower constraint limit, it is assigned the lower constraint limit value. When multiple action variables simultaneously exceed the limits, projection correction is performed sequentially in descending order of the degree of exceedance. After each correction, the Lyapunov function value is recalculated until the stability assessment value falls within the safety threshold or the preset maximum number of corrections is reached. The maximum number of corrections is typically three; exceeding this number forces the current correction result to be output and triggers a system alarm. The corrected action components are then recombined with the non-out-of-bounds action components to obtain the corrected predicted operation action that satisfies the physical constraints.

[0059] Preferably, in some embodiments of this application, the preset safety threshold is dynamically set. Its baseline value is a constant pre-calibrated based on the physical characteristics of the UAV swarm, which is associated with at least one of the following indicators: maximum permissible speed, maximum permissible transmission power, minimum safe distance, and minimum safe battery level. During mission execution, the safety threshold is dynamically adjusted according to the system state: when the average remaining battery level of the swarm is higher than the preset high water level, the safety threshold is lowered to allow a larger exploration range; when the average remaining battery level of the swarm is lower than the preset low water level, the safety threshold is raised to enforce more conservative actions. During emergency missions, the safety threshold can be temporarily relaxed, but the relaxation cannot exceed 1.2 to 1.5 times the baseline value, and the relaxed threshold must still meet collision avoidance and battery survival constraints. The preset high water level and preset low water level are pre-set based on the total mission duration and the distribution of charging stations.

[0060] By calculating the penalty component for the deviation between each action component and its corresponding constraint condition using Lyapunov functions and then weighting and summing them, a stability evaluation value is obtained. When the evaluation value exceeds a threshold, the specific out-of-bounds action component is identified, and each out-of-bounds action component is directly corrected to its corresponding constraint boundary using a dimension-by-dimensional projection method. This method can quickly locate the specific action dimension that causes the constraint to be unsatisfied and directly correct it according to the constraint boundary without introducing an additional optimization solution process, resulting in low computational cost.

[0061] S104: Generate a charging schedule for the UAV cluster based on the real-time status data and the job scheduling instruction set.

[0062] Furthermore, in some embodiments of this application, the real-time status includes: the remaining battery power of the drone and its distance from the charging station; generating a charging scheduling scheme for the drone cluster based on the real-time status data and the job scheduling instruction set includes: Based on the remaining battery power and the distance, calculate the charging urgency index for each of the drones; Determine the task distribution location corresponding to each predicted job action in the job scheduling instruction set; A charging scheduling scheme for the drone cluster is generated based on the task distribution location, the task priority, and the charging urgency index.

[0063] Preferably, in some embodiments of this application, the charging scheduling scheme is generated based on the remaining battery power, distance to the charging pile, and job scheduling instruction set in the real-time state. The real-time state includes the remaining battery percentage of each drone and the Euclidean distance between the drone and the charging pile. When generating the charging scheduling scheme, the charging urgency index of each drone is first calculated based on the remaining battery power and distance. This index is defined as a weighted sum of the reciprocal of the remaining battery percentage and the normalized distance: the reciprocal of the remaining battery percentage is multiplied by a first weighting coefficient, plus the Euclidean distance between the drone and the charging pile divided by a preset maximum charging distance, and then multiplied by a second weighting coefficient. The two are added together to obtain the original charging urgency value. The first weighting coefficient is preset to 0.6, and the second weighting coefficient is preset to 0.4. Drones with lower remaining battery power or farther away from the charging pile receive higher charging urgency values. The rule for determining the preset maximum charging distance is: the value manually set by the operator according to the task area range is preferred; if the operator does not set it, the maximum value of the Euclidean distance between any two charging piles in the task area is multiplied by 1.5. The original charging urgency values ​​are linearly normalized and mapped to the 0-1 range before being used for subsequent sorting and comparison.

[0064] During the generation of the charging scheduling scheme, the charging urgency index of each drone is calculated based on the remaining power and distance to the charging station in real time. This index is then combined with the task priority given by the priority output branch of the policy network and the task distribution location corresponding to each predicted operation to generate the charging scheduling scheme. This approach ensures that the decision-making basis for charging scheduling simultaneously includes the drone's own power status and the importance of the task to be performed, matching the charging timing and route selection with the current job scheduling task.

[0065] Furthermore, in some embodiments of this application, generating a charging scheduling scheme for the drone swarm based on the task distribution location, the task priority, and the charging urgency index includes: The drones are sorted according to the charging urgency index and the task priority to generate a charging order; For each of the aforementioned drones, the path length cost is evaluated based on the path length between the drone's current location and the target charging station location. For each of the aforementioned drones, the risk cost of mission interruption is assessed based on the mission distribution location and the mission priority corresponding to the predicted operation action. For each of the aforementioned drones, the return path of the drone is searched and determined with the goal of minimizing the weighted sum of the path length cost and the mission interruption risk cost. A charging schedule for the drone cluster is generated based on the charging sequence and each of the return paths.

[0066] Preferably, in some embodiments of this application, a charging scheduling scheme is generated based on three factors: task distribution location, task priority, and charging urgency index. This includes: sorting each drone according to the charging urgency index and task priority to generate a charging order, with drones having higher charging urgency and corresponding higher task priority returning to charging first; then, based on the current location of each drone, task distribution location, and charging order, using an improved A* algorithm to plan a return path for each drone to be charged. The cost function of the improved A* algorithm includes two items: path length cost and task interruption risk cost. The path length cost is the cumulative value of the Euclidean distance between adjacent path points, and the task interruption risk cost is obtained based on the distance between the path points on the return path and each task distribution location, as well as the task priority of the corresponding predicted task action; finally, a complete charging scheduling scheme is generated based on the charging order and each return path. Charging scheduling employs a hybrid mechanism combining event-triggered and periodic scheduling. Triggering conditions include immediate triggering when the remaining battery percentage of any drone falls below a preset low battery alarm threshold, and triggering when any drone's remaining battery is insufficient to return to the nearest charging station after completing its current task. When no event condition is triggered, the system proactively performs charging scheduling at fixed scheduling intervals to plan the order of drones that will need charging in advance.

[0067] In the path planning algorithm, the cost function includes both path length cost and task interruption risk cost. The task interruption risk cost is obtained based on the distance between the waypoints and the distribution locations of each task, as well as the task priority of the corresponding predicted operation. This cost function design enables path planning to identify and avoid high-priority task areas while searching for shorter routes, reducing the risk of task interruption caused by the drone leaving the operation area to go for charging.

[0068] S105: Coordinate the scheduling of the drone cluster according to the job scheduling instruction set and the charging scheduling scheme.

[0069] Furthermore, in some embodiments of this application, the coordinated scheduling of the drone swarm includes: Obtain the task execution feedback data of each UAV after the execution of the job scheduling instruction set; A reward function is constructed based on the deviation between the task execution feedback data and the preset optimization target; A preset policy optimization algorithm is used to perform gradient updates on the policy network based on the reward function.

[0070] Preferably, in some embodiments of this application, after the UAV swarm is coordinated and scheduled according to the job scheduling instruction set and the charging scheduling scheme, an online update process for the policy network is also included, specifically: First, task execution feedback data from each UAV is obtained after the actual execution of the job scheduling instruction set, including three indicators: task completion time, total cluster energy consumption, and load balancing. Then, a reward function is constructed based on the deviations of these three feedback data from the preset optimization target. Specifically, normalized deviation values ​​are calculated for task completion time, total cluster energy consumption, and load balancing. These normalized deviation values ​​are calculated using the relative error between the actual and target values, and then combined according to preset weights to form a comprehensive reward value. This comprehensive reward value serves as the basis for gradient updates in the Proximal Policy Optimization (PPO) algorithm. The preset weights are not fixed but dynamically adjusted according to the task type to adapt to the optimization preferences under different task scenarios.

[0071] The proximal policy optimization algorithm calculates the policy gradient based on the aforementioned reward function and updates the policy network accordingly. Since the priority output branch and the action output branch share the underlying feature extraction network and jointly participate in the gradient update of the proximal policy optimization algorithm, the task priority is dynamically adjusted synchronously with the optimization of the policy network. This allows the policy network to continuously optimize based on actual execution results, thereby continuously improving the quality of subsequent scheduling decisions.

[0072] After executing the task scheduling instruction set, task execution feedback data from each UAV is acquired. A reward function is constructed based on the deviation between the feedback data and the preset optimization target, and a policy optimization algorithm is used to update the policy network using gradients. This feedback mechanism enables the policy network to continuously adjust according to the actual execution results, achieving online learning and optimization.

[0073] In summary, the UAV swarm scheduling method provided in this application has the following advantages compared to existing technologies: This application pre-trains the policy network using a quantum optimization module, enabling the policy network to have an effective exploration direction from the initial training stage, avoiding blind exploration starting from random initialization. Simultaneously, it uses a Lyapunov function to determine the feasibility of each predicted task action output by the policy network and corrects actions that do not meet constraints, ensuring that the final output task scheduling instructions meet physical constraints such as flight speed, energy consumption, and collision avoidance. Based on this, a charging scheduling scheme is generated according to real-time status data and the task scheduling instruction set, enabling charging scheduling to coordinate with task execution. These methods improve the reliability of deep reinforcement learning methods in online task scheduling of UAV swarms from both training efficiency and execution security perspectives.

[0074] Based on the above method embodiments, one embodiment of this application provides a drone swarm scheduling system, which is used to implement the drone swarm scheduling method provided in any of the above method embodiments of this application.

[0075] It should be noted that the system embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0076] Based on the above embodiments of the UAV swarm scheduling method, another embodiment of this application provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the UAV swarm scheduling method of any embodiment of this application.

[0077] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete this application. The one or more module units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0078] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0079] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0080] Based on the above-described method embodiments, another embodiment of this application provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the UAV swarm scheduling method described in any of the above-described method embodiments of this application.

[0081] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above-described embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, an electrical signal, and a software distribution medium, etc.

Claims

1. A method for scheduling unmanned aerial vehicle (UAV) swarms, characterized in that, include: Historical state data of each drone in the drone cluster is collected. Based on the historical state data, the optimal parameters of the pre-built policy network are searched alternately by a first quantum optimization module with global search characteristics and a second quantum optimization module with local search characteristics. The policy network is then initialized with the optimal parameters. Collect real-time status data of each of the aforementioned UAVs, input the real-time status data into the policy network, and obtain the predicted operation actions of each of the aforementioned UAVs; The feasibility of each predicted job action is determined by a preset Lyapunov function. If the determination result is that the physical constraints are not met, the predicted job action is modified to obtain a set of job scheduling instructions that meet the physical constraints. Based on the real-time status data and the job scheduling instruction set, a charging scheduling scheme for the UAV cluster is generated; The drone swarm is coordinated and scheduled according to the job scheduling instruction set and the charging scheduling scheme.

2. The UAV swarm scheduling method as described in claim 1, characterized in that, The step of alternately searching for the optimal parameters of a pre-built policy network based on the historical state data using a first quantum optimization module with global search characteristics and a second quantum optimization module with local search characteristics, and initializing the policy network using the optimal parameters, includes: The parameters to be optimized in the policy network are encoded into multiple individuals, and all of these individuals constitute the current population; wherein each individual corresponds to a set of parameter codes; Based on the historical state data, the current population is iteratively updated alternately by the first quantum optimization module and the second quantum optimization module until a preset iteration termination condition is met, resulting in the final population. In each round of alternation, the first quantum optimization module performs a global search on the current population to obtain a first population, which is then passed to the second quantum optimization module. The second quantum optimization module performs a local search on the first population to obtain a second population, which is then passed back to the first quantum optimization module as the current population for the next round of alternation. The parameters corresponding to the globally optimal individuals in the final population are encoded and mapped to the initial parameters of the policy network, thereby completing the training of the policy network.

3. The method for scheduling unmanned aerial vehicle (UAV) swarms as described in claim 1, characterized in that, The policy network includes a low-level feature extraction network, an action output branch, and a priority output branch; The step of inputting the real-time status data into the policy network to obtain the predicted operational actions of each UAV includes: The real-time status data of each of the aforementioned UAVs is input into the policy network; The underlying feature extraction network is used to extract features from the real-time state data to obtain shared features. The shared features are linearly mapped and nonlinearly activated through the action output branch to output the predicted operation actions of each UAV. The shared features are linearly mapped and value estimated through the priority output branch, and the task priority corresponding to each prediction operation is output.

4. The UAV swarm scheduling method as described in claim 3, characterized in that, The real-time status includes: the remaining battery power of the drone and its distance from the charging station; the step of generating a charging scheduling scheme for the drone cluster based on the real-time status data and the job scheduling instruction set includes: Based on the remaining battery power and the distance, calculate the charging urgency index for each of the drones; Determine the task distribution location corresponding to each predicted job action in the job scheduling instruction set; A charging scheduling scheme for the drone cluster is generated based on the task distribution location, the task priority, and the charging urgency index.

5. The UAV swarm scheduling method as described in claim 4, characterized in that, The step of generating a charging scheduling scheme for the drone cluster based on the task distribution location, the task priority, and the charging urgency index includes: The drones are sorted according to the charging urgency index and the task priority to generate a charging order; For each of the aforementioned drones, the path length cost is evaluated based on the path length between the drone's current location and the target charging station location. For each of the aforementioned drones, the risk cost of mission interruption is assessed based on the mission distribution location and the mission priority corresponding to the predicted operation action. For each of the aforementioned drones, the return path of the drone is searched and determined with the goal of minimizing the weighted sum of the path length cost and the mission interruption risk cost. A charging schedule for the drone cluster is generated based on the charging sequence and each of the return paths.

6. The method for scheduling unmanned aerial vehicle (UAV) swarms as described in claim 1, characterized in that, The predicted task includes several task components, each corresponding to a constraint condition. The feasibility of each predicted task is determined using a preset Lyapunov function. If the determination result indicates that the physical constraints are not met, the predicted task is modified, including: The action components of each predicted operation are input into the Lyapunov function to calculate the penalty component based on the deviation of each action component from the corresponding constraint condition, and the stability evaluation value of each predicted operation is obtained by weighted summation based on each penalty component. If the stability assessment value exceeds the preset safety threshold, then identify the out-of-bounds action component in the corresponding predicted operation action that violates the corresponding constraint condition; Based on the constraints, each out-of-bounds action component in each predicted operation is corrected by a dimension-by-dimensional projection method.

7. The method for scheduling unmanned aerial vehicle (UAV) swarms as described in claim 1, characterized in that, The coordinated scheduling of the drone swarm includes: Obtain the task execution feedback data of each UAV after the execution of the job scheduling instruction set; A reward function is constructed based on the deviation between the task execution feedback data and the preset optimization target; A preset policy optimization algorithm is used to perform gradient updates on the policy network based on the reward function.

8. A drone swarm scheduling system, characterized in that, The UAV swarm scheduling system implements a UAV swarm scheduling method as described in any one of claims 1 to 7.

9. A terminal device, characterized in that, The device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement a drone swarm scheduling method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform a drone swarm scheduling method as described in any one of claims 1 to 7.

Images