Unmanned aerial vehicle cluster sequential decision method and system based on dynamic priority ranking

Abstract

The application provides a UAV cluster sequential decision method and system based on dynamic priority ranking, relates to the technical field of UAV cluster cooperative control, and comprises the following steps: obtaining observation data of each UAV in the cluster at a current time step, and preprocessing to obtain a joint observation sequence; inputting the joint observation sequence into a pre-constructed fixed priority scoring function, calculating priority scores of each UAV, arranging each UAV in descending order of priority scores, and obtaining a dynamic arrangement index; rearranging the joint observation sequence according to the dynamic arrangement index, encoding the rearranged observation sequence, and obtaining an observation representation sequence; and generating each UAV action at the current time step in sequence through causal mask autoregression based on the observation representation sequence. The application does not need to call a large language model at each environment time step, does not need to additionally train a ranking network, can dynamically adjust the action generation sequence, and improves the cooperative decision-making capability of the UAV cluster.

Description

Technical Field

[0001] This invention relates to the field of collaborative control technology for unmanned aerial vehicle (UAV) swarms, specifically to a method and system for UAV swarm sequence decision-making based on dynamic priority ranking. Background Technology

[0002] Unmanned aerial vehicle (UAV) swarms can be applied to tasks such as collaborative target tracking, area inspection, disaster search and rescue, and communication relay. In these tasks, multiple UAVs need to collaboratively generate actions based on onboard perception information, position and attitude information, communication link status, remaining energy, and task area constraints. Due to differences in the location, role, and environmental information of each UAV, the actions of different UAVs have varying impacts on the swarm's subsequent decisions. The order in which each UAV generates actions at the same decision-making moment may affect the collaborative decision-making effect of the UAV swarm. In related technologies, multi-agent reinforcement learning methods such as QMIX, VDN, and MAPPO, which involve centralized training and distributed execution, typically treat actions generated by multiple agents within the same environmental time step as parallel joint actions. Sequence modeling-based multi-agent action generation methods represent the joint action generation process as a sequence generation process, generating actions of each agent sequentially according to a pre-defined agent order. Some existing technologies also learn the action generation order of each agent through ranking networks, credit estimation networks, or leader-follower relationship models. Existing large language model-assisted reinforcement learning techniques utilize the semantic understanding capabilities of large language models to assist reinforcement learning systems in task modeling and policy training.

[0003] However, the above-mentioned technologies have the following shortcomings when applied to UAV swarm tasks: Parallel joint action generation methods do not explicitly describe the impact of preceding UAV actions on the generation process of subsequent UAV actions. Actions generated by UAVs that possess key information or undertake key tasks are difficult to be used as conditions for other UAVs to generate actions in the current time step in a timely manner; Sequential action generation methods usually adopt a fixed order, a random order, or an order with a low degree of correlation with the current state, making it difficult to dynamically adjust the action generation order based on the current perception information, position and attitude, communication status, and other information of the UAVs; Learnable ranking methods require setting up additional ranking networks, model parameters, and training objectives, which couples the ranking learning process with the multi-agent policy training process, and the ranking results are usually implicitly determined by the model parameters, lacking clear task semantic basis; If a large language model is called at each environmental time step to determine the UAV action generation order, it will increase model call latency, communication overhead, and computational resource consumption, making it difficult to adapt to the high-frequency and continuous environmental interaction requirements in UAV swarm reinforcement learning.

[0004] Therefore, existing technologies cannot dynamically determine the order of action generation based on the current task status of the drone swarm, and the sorting criteria are unclear, which limits the effectiveness of collaborative decision-making in drone swarms. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a method and system for drone swarm order decision-making based on dynamic priority ranking. This method dynamically adjusts the action generation order without requiring the invocation of a large language model at each environmental time step or the additional training of a ranking network, thereby improving the collaborative decision-making capability of the drone swarm.

[0006] According to some embodiments, the present invention adopts the following technical solution: The method for determining the order of drone swarms based on dynamic priority ranking includes: Obtain observation data from each UAV in the cluster at the current time step, and preprocess it to obtain a joint observation sequence; The joint observation sequence is input into a pre-constructed fixed priority scoring function to calculate the priority score of each UAV. The UAVs are then arranged in descending order of priority score to obtain a dynamic sorting index. The joint observation sequence is rearranged according to the dynamic permutation index, and the rearranged observation sequence is encoded to obtain the observation representation sequence; Based on the observation representation sequence, the actions of each UAV at the current time step are generated sequentially through causal mask autoregression; The pre-construction of the fixed priority scoring function is based on the task description, observation space description, and action space description of the UAV swarm. It generates priority sorting logic in natural language form, structured pseudocode combined with the observation space description, and an executable fixed priority scoring function in sequence through a large language model.

[0007] According to some embodiments, the present invention adopts the following technical solution: A drone swarm ordering decision system based on dynamic priority ranking includes: The acquisition module is configured to: acquire observation data from each UAV in the cluster at the current time step, and preprocess the data to obtain a joint observation sequence; The sorting module is configured to: input the joint observation sequence into a pre-built fixed priority scoring function, calculate the priority score of each UAV, sort the UAVs from high to low according to the priority score, and obtain a dynamic sorting index; The encoding module is configured to: rearrange the joint observation sequence according to the dynamic permutation index, and encode the rearranged observation sequence to obtain an observation representation sequence; The generation module is configured to: generate each UAV action at the current time step sequentially through causal mask autoregression based on the observation representation sequence; The pre-construction of the fixed priority scoring function is based on the task description, observation space description, and action space description of the UAV swarm. It generates priority sorting logic in natural language form, structured pseudocode combined with the observation space description, and an executable fixed priority scoring function in sequence through a large language model.

[0008] According to some embodiments, the present invention adopts the following technical solution: A computer program product includes a computer program that, when executed by a processor, implements the aforementioned UAV swarm order decision-making method based on dynamic priority sorting.

[0009] According to some embodiments, the present invention adopts the following technical solution: A non-transitory computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the aforementioned UAV swarm order decision-making method based on dynamic priority sorting.

[0010] According to some embodiments, the present invention adopts the following technical solution: An electronic device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the UAV swarm order decision method based on dynamic priority sorting.

[0011] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention pre-generates and verifies executable priority scoring functions. During UAV swarm training and inference, only the verified fixed scoring function is called to calculate the priority of each UAV, eliminating the need to call a large language model at each environmental time step. This helps reduce model call latency, communication overhead, and computational resource consumption. Furthermore, it eliminates the need to set up and train an independent ranking network; instead, the fixed priority scoring function directly determines the order of UAV actions, thus avoiding coupling between the ranking network training process and the UAV cooperative strategy training process. This reduces the need for additional model parameters, training objectives, and training data, thereby improving the stability of the training process.

[0012] The scoring function generation and verification process of this invention is decoupled from the UAV swarm reinforcement learning training process. While maintaining the original sequence encoding, autoregressive action generation and policy training framework, it mainly achieves dynamic sequential decision-making through priority calculation, observation sequence rearrangement and action backmapping, which is convenient for deployment in existing sequential multi-agent reinforcement learning systems.

[0013] This invention dynamically calculates priority scores based on the current airborne perception data, navigation attitude data, and task constraints of each UAV, and adjusts the action generation order of each UAV accordingly. Compared with a fixed or random order, this invention enables the action generation order to change with the current state and task progress of the UAV cluster, thereby improving the adaptability of the action generation order to actual task requirements. Attached Figure Description

[0014] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0015] Figure 1 This is a schematic diagram of the method flow provided in Example 1. Detailed Implementation

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0017] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0018] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0019] Example 1 One embodiment of the present invention provides a method for determining the order of unmanned aerial vehicle (UAV) swarms based on dynamic priority sorting, comprising: Step 1: Obtain the observation data of each UAV in the cluster at the current time step, and preprocess it to obtain a joint observation sequence; Step 2: Input the joint observation sequence into a pre-constructed fixed priority scoring function to calculate the priority score of each UAV, and arrange the UAVs in descending order of priority score to obtain a dynamic sorting index; Step 3: Rearrange the joint observation sequence according to the dynamic permutation index, and encode the rearranged observation sequence to obtain the observation representation sequence; Step 4: Based on the observation representation sequence, generate the UAV actions for the current time step sequentially through causal mask autoregression; The pre-construction of the fixed priority scoring function is based on the task description, observation space description, and action space description of the UAV swarm. It generates priority sorting logic in natural language form, structured pseudocode combined with the observation space description, and an executable fixed priority scoring function in sequence through a large language model.

[0020] This embodiment is applicable to cooperative multi-agent sequential decision-making scenarios, especially suitable for collaborative decision-making training when UAV swarms perform tasks such as cooperative target tracking, area inspection, disaster search and rescue, and communication relay. Addressing the problems of existing multi-agent reinforcement learning methods, such as difficulty in dynamically determining the action generation order based on the current task state, unclear sorting criteria, and difficulty in directly applying the sorting results to the UAV control process, the following technical issues are proposed to be resolved: (1) How to dynamically determine the decision priority of each UAV in the process of generating joint actions based on the airborne perception data, position and attitude data of each UAV at the current moment, and avoid using a fixed order or random order.

[0021] (2) How to transform the task description, observation space definition, action space definition and role responsibilities of the UAV cluster into an executable priority scoring function so that the sorting basis of each UAV has clear task semantics and can be interpreted and verified.

[0022] (3) How to avoid calling the large language model at each environmental time step of reinforcement learning training and inference in UAV clusters, while retaining its task semantic analysis capabilities, and reducing model call latency, communication overhead and computing resource consumption.

[0023] (4) How to reduce the risks of errors in the observation field correspondence, inconsistent input and output forms, code failure to execute, and sorting logic deviating from task requirements in the process of priority scoring function generation, and improve the executability and scenario adaptability of the scoring function.

[0024] (5) How to embed the dynamic arrangement result obtained based on the current UAV state into the multi-agent sequential action generation process so that the subsequent UAV can use the action information generated by the preceding UAV and generate its own action accordingly.

[0025] (6) How to achieve dynamic adjustment of the order of UAV action generation without additional training of an independent ranking network, and avoid the coupling between ranking model training and UAV collaborative strategy training.

[0026] (7) How to restore the UAV actions generated in dynamic order to the original UAV number, forming joint action parameters that can be received by the UAV simulation environment, mission payload control interface or flight control interface, so that the dynamic priority sorting results can be directly applied to the training and execution process of UAV clusters.

[0027] As one embodiment, to address the above-mentioned technical problems, such as Figure 1 As shown, the UAV swarm sequential decision-making method based on dynamic priority ranking of the present invention includes a fixed priority scoring function pre-construction stage and a sequential decision-making stage. In the fixed priority scoring function pre-construction stage, an executable scoring function for determining the decision priority of each UAV is generated based on the task description, observation space description, and action space description of the UAV swarm, and the scoring function is verified and corrected. The sequential decision-making stage is executed during both UAV swarm training and inference. In the sequential decision-making stage, the current airborne perception data, navigation attitude data, and task constraints of each UAV are input into the verified scoring function to obtain the priority score of each UAV. The action generation order of the UAVs is dynamically adjusted according to the priority scores, and the actions of each UAV are generated sequentially according to the adjusted order. After the actions are generated, each action is restored to its corresponding original UAV number to form a joint action that can be received by the UAV simulation environment or control interface.

[0028] Through the above processing, the large language model is only used for the generation and verification of the scoring function, and does not need to participate in the drone action ranking at each environmental time step; at the same time, it can dynamically adjust the action generation order according to the current input data of the drone swarm without the need to train an additional independent ranking network.

[0029] The specific implementation process is explained below: Before training and inference of the drone swarm, a fixed-priority scoring function is pre-built, which includes the following steps: Step S1: Enter the task description and space description.

[0030] The task description, observation space description, and action space description of the UAV swarm are obtained and organized into the input information required for generating the scoring function. In UAV swarm application scenarios, the observation space description explains the meaning, data dimensions, and value range of fields such as airborne perception data, navigation attitude data, and task constraints; the action space description explains the action parameters that the UAV simulation environment or control interface can receive. The input information in the scoring function generation stage can be represented as follows:

[0031] in, This describes the drone swarm mission. Indicates the description of the observation space. This represents the action space description.

[0032] Step S2: Generate priority sorting logic.

[0033] The input information The input analysis module determines the factors influencing the order of action generation based on the mission objectives of the UAV swarm, the meaning of the observed data, and the collaborative relationships between the UAVs, and forms a priority ranking logic in natural language form.

[0034] in, This indicates an analysis module based on a large language model. This indicates the generated priority sorting logic.

[0035] When the analysis module calls the large language model, it uses preset analysis prompts. The analysis prompts include at least a description of the UAV cluster task, a description of the observation space, a description of the action space, and priority ordering logic generation requirements. These prompts are used to constrain the large language model to analyze the factors affecting the generation order of each UAV action based on the input task information and observation fields.

[0036] As an optional implementation, the analysis prompts use the following template: Given a drone swarm mission description Observational space description Action space description This study analyzes the factors influencing the order in which UAV actions are generated in the current task. Using only the observation fields defined in the observation space description, it describes the direction of influence of each factor on the UAV decision priority, the triggering conditions, and the combined relationships between multiple factors. It outputs a natural language priority ranking logic that can be further converted into a deterministic scoring process; it does not directly generate UAV actions and does not introduce observation fields not defined in the input information.

[0037] Step S3: Construct a structured scoring process.

[0038] The priority sorting logic The input modeling module, combined with the observation space description, converts the priority ranking logic in natural language form into structured pseudocode, clarifying the method of reading observation data, scoring elements, and scoring calculation process:

[0039] in, This represents a modeling module based on a large language model. This represents the structured pseudocode used to generate the priority scoring function.

[0040] When the modeling module calls the large language model, it uses preset modeling prompts to convert the priority ranking logic in natural language form into a structured scoring process corresponding to the observation space.

[0041] As an optional implementation, the modeling prompts use the following template: "Based on priority sorting logic" and observation space description This involves converting the natural language-based sorting logic into structured pseudocode. For each scoring factor, it clarifies the corresponding observation fields, field reading methods, necessary data processing methods, conditional judgments, scoring calculation process, and the combination order of multiple scoring factors. Each drone calculates a priority score and forms a sequence of length [length missing]. Priority score sequences; fields not defined in the observation space description must not be used, and input and output formats must be clearly defined.

[0042] Step S4: Generate and validate the priority scoring function.

[0043] The structured pseudocode Input the programming module, which then generates an executable priority scoring function:

[0044] in, This represents the number of iterations in generating the scoring function. This represents a programming module based on a large language model. Indicates the first The secondary priority scoring function This represents the feedback information generated in the previous iteration; during the initial generation, Empty.

[0045] When the programming module calls the large language model, it uses preset programming prompts to generate an executable priority scoring function that conforms to the preset program interface based on the structured pseudocode.

[0046] As an optional implementation, the programming prompts use the following template: "Based on structured pseudocode" Generate an executable priority scoring function. The scoring function uses a joint observation sequence that conforms to the description of the observation space. As input, calculate a real-valued priority score for each drone and output a sequence of priority scores. Undefined fields in the observation space must not be accessed, large language models must not be called during the execution of the scoring function, and input dimensions, data types, and boundary conditions must be handled. Feedback information from the previous iteration should be provided. Then, the scoring function will be modified to address the issues pointed out, while keeping the input and output interfaces of the scoring function unchanged.

[0047] First, a code analysis tool is used to check the generated priority scoring function:

[0048] in, Indicates the first The results of the examination For priority scoring functions, This refers to a code inspection tool, which is implemented using a compiler, interpreter, or code execution program. It is used to compile and test the generated priority scoring function, check the program syntax, data types, input / output interfaces, and execution process, and output corresponding compilation information, exception information, or execution results.

[0049] If the priority scoring function fails the code detection, the verification module will then base its decision on the detection result. The problems with the scoring function are analyzed, and feedback information is generated:

[0050] in, For priority sorting logic, For structured pseudocode, For priority scoring functions, Indicates the first Feedback information generated during the verification process This represents the validation module based on a large language model.

[0051] When the verification module calls the large language model, it uses preset verification prompts to locate errors in the scoring function by combining priority sorting logic, structured pseudocode, and code detection results.

[0052] As an optional implementation, the verification prompt word adopts the following template: "Based on priority sorting logic" and structured pseudocode Check priority scoring function And combined with code detection results Determine if the scoring function has issues such as incorrect observation fields, inconsistent input / output formats, program errors, or scoring logic deviating from sorting requirements. Output the location of the problem, the problem type, and the corresponding modification requirements.

[0053] The programming module based on the large language model regenerates the scoring function according to the feedback information until the scoring function passes the check or reaches a preset number of iterations. The scoring function that passes the check is then designated as a fixed-priority scoring function. And it can be shared and used in subsequent drone swarm training and inference processes.

[0054] During the training and inference processes of the drone swarm, sequential decision-making based on dynamic priority ranking is performed, specifically including the following steps: Step S5: Obtain the current observation data of the drone cluster.

[0055] In the environment time step t during drone swarm training or inference, the current observation data of each drone is acquired. The observation data of the i-th drone at environment time step t can be represented as:

[0056] in, This represents airborne sensing data. Represents navigation attitude data, This represents task constraint data. After preprocessing the observed data, we obtain the first... Local observation of the environment at time step t using a drone The joint observation sequence of the environment at time step t is composed of local observations from all UAVs:

[0057] in, This indicates the number of drones.

[0058] Step S6: Calculate the drone priority and construct a dynamic sorting index.

[0059] Joint observation sequence Input fixed priority scoring function The priority score for each drone is obtained:

[0060] in, Indicates the first A drone in environmental time step Priority score.

[0061] The drones are arranged in descending order of priority score to obtain a dynamic sorting index:

[0062] in, , Indicates the first in a dynamic arrangement The original drone number corresponding to each position. When two or more drones have the same priority score, a stable sorting method is used to preserve their original number order.

[0063] Step S7: Rearrange the observation sequence according to the dynamic permutation index.

[0064] Based on dynamically arranged indexes For joint observation sequences The observation sequence is rearranged to obtain the rearranged sequence:

[0065] Feature embedding is performed on the rearranged observation sequence, and the corresponding sequence position encoding is superimposed to obtain the encoder input:

[0066] in, The observation embedding layer is used to map the local observations of different UAVs to a feature space of a unified dimension. As an optional implementation, the observation embedding layer can be implemented using a linear mapping layer or a multilayer perceptron. The position code represents the sequential position of the UAV in the current dynamic arrangement. As an optional implementation, the position code can use a learnable position vector and read the corresponding position vector according to the position of the UAV in the dynamic arrangement.

[0067] The input is fed into a sequence encoder to obtain a sequence of observation representations arranged in order of current priority:

[0068] in, Indicates sequence encoder parameters, Indicates the first in a dynamic arrangement The observation representation corresponding to each position.

[0069] Step S8: Generate drone actions autoregressively according to the dynamic arrangement order.

[0070] The autoregressive action generation module, according to the dynamically arranged index... The drone actions are generated sequentially in a predetermined order. In the dynamic arrangement... The drone actions corresponding to each location are:

[0071] in, The parameter is Action generation strategy Indicates the original number is The drone in the environment time step The generated action.

[0072] The autoregressive action generation module uses a causal mask to make the dynamic arrangement of the first... The drone corresponding to each location can utilize the previous You can access the motion information that a drone has already generated, but you cannot access the motion information of drones that have not yet been generated.

[0073] The action sequence generated according to the dynamic arrangement order can be represented as:

[0074] The UAV actions generated by the autoregressive dynamic arrangement index include heading angle, target speed, target altitude, acceleration, target waypoint, formation relative position, sensor pointing angle, or mission payload control identifier. The combined actions after back-mapping in step S9 form action parameters that can be received by the UAV simulation environment, mission payload control interface, or flight control interface, and are used to control each UAV to perform corresponding flight or mission actions.

[0075] Step S9: Map the action back to the original drone number.

[0076] After all drone actions are generated, they are arranged according to the dynamic index. The inverse mapping relationship will restore the actions generated according to the dynamic arrangement order to the original drone number:

[0077] in This indicates the joint actions arranged according to the original drone numbers:

[0078] Step S10: Perform the joint action and update the training network.

[0079] Joint action Execute the command in the drone swarm simulation environment or control interface to obtain shared rewards. Joint observation sequence with the next environmental time step :

[0080] in, This refers to the state transition process in an existing UAV swarm simulation environment or control interface, where the UAV swarm executes joint actions and returns environmental feedback within an environmental time step. Its input is the current joint action. The output is a shared reward. Joint observation sequence with the next environmental time step .

[0081] During the training phase, based on the current joint observation sequence Joint actions Shared rewards and the next joint observation sequence Training data is generated, and the trainable parameters of the sequence encoder and autoregressive action generation module are updated using the training data. A fixed-priority scoring function is used. It remains unchanged during the above update process and does not participate in the parameter update of the multi-agent policy network.

[0082] Step S11: Iterate through the training and perform inference.

[0083] Repeat steps S5 to S10 until the preset training termination condition is met, and obtain the trained UAV swarm sequential decision model.

[0084] During the inference phase, the fixed-priority scoring function will continue to be used. The system calculates the current priority of each drone and performs dynamic reordering, autoregressive action generation, and action backmapping in sequence, without having to call the large language model again or train the priority scoring function online.

[0085] This embodiment has the following advantages: (1) This invention pre-generates and verifies an executable priority scoring function through an analysis module, modeling module, programming module, and verification module based on a large language model. During the training and inference process of UAV clusters, only the verified fixed scoring function is called to calculate the priority of each UAV, without having to call the large language model at each environmental time step, which helps to reduce model call latency, communication overhead, and computational resource consumption.

[0086] (2) The present invention dynamically calculates priority scores based on the current airborne perception data, navigation attitude data, and task constraints of each UAV, and adjusts the action generation order of each UAV accordingly. Compared with a fixed order or a random order, the present invention enables the action generation order to change with the current state of the UAV cluster and the progress of the task, thereby improving the adaptability of the action generation order to the actual task requirements.

[0087] (3) This invention does not require setting up and training an independent ranking network. Instead, it uses a fixed priority scoring function to directly determine the order of UAV action generation, thereby avoiding the coupling between the ranking network training process and the UAV cooperative strategy training process, reducing the need for additional model parameters, training objectives and training data, and improving the stability of the training process.

[0088] (4) The present invention completes the ranking factor analysis, scoring process construction and executable scoring function generation in sequence through the analysis module, modeling module and programming module based on the large language model. The scoring function is checked and feedback is corrected through the verification module based on the large language model. This helps to reduce problems such as errors in the corresponding observation fields, inconsistent input and output forms, program failure to execute and scoring logic deviating from task requirements, and improves the executability and scenario adaptability of the priority scoring function.

[0089] (5) The present invention directly applies the dynamic priority sorting result to the sequential action generation process of the UAV cluster, so that the UAV in the later position can use the action information generated by the preceding UAV and generate its own action accordingly, thereby enhancing the sequential association between the actions of each UAV within the same environmental time step, which is conducive to improving the collaborative decision-making ability of the UAV cluster.

[0090] (6) The priority scoring function generated by this invention is derived from the UAV cluster task description, observation space description and action space description. The scoring results can correspond to clear task semantics and observation data, so that the UAV action generation order has an interpretable and verifiable basis for determination, and it is convenient to check and adjust the sorting results.

[0091] (7) After the action generation is completed, the present invention restores each UAV action to its original UAV number according to the inverse mapping relationship of the dynamic sorting index, forming a joint action that can be received by the UAV simulation environment or control interface. Therefore, the dynamic priority sorting does not change the original numbering relationship and action output interface of the UAVs, which facilitates the connection with existing UAV swarm training system and control system.

[0092] (8) The scoring function generation and verification process and the UAV swarm reinforcement learning training process of the present invention are decoupled from each other. While maintaining the original sequence encoding, autoregressive action generation and policy training framework, dynamic sequential decision-making is mainly achieved through priority calculation, observation sequence rearrangement and action backmapping, which is convenient for deployment in existing sequential multi-agent reinforcement learning systems.

[0093] (9) The present invention forms a complete processing flow consisting of the current input data of the UAV, priority scoring, dynamic reordering, sequential action generation, action back mapping and environmental feedback. It can be applied to UAV swarm tasks such as collaborative target tracking, area inspection, disaster search and communication relay, so that the dynamic priority sorting mechanism can directly act on the training and execution process of the UAV swarm.

[0094] Example 2 One embodiment of the present invention provides a drone swarm order decision-making system based on dynamic priority ranking, comprising: The acquisition module is configured to: acquire observation data from each UAV in the cluster at the current time step, and preprocess the data to obtain a joint observation sequence; The sorting module is configured to: input the joint observation sequence into a pre-built fixed priority scoring function, calculate the priority score of each UAV, sort the UAVs from high to low according to the priority score, and obtain a dynamic sorting index; The encoding module is configured to: rearrange the joint observation sequence according to the dynamic permutation index, and encode the rearranged observation sequence to obtain an observation representation sequence; The generation module is configured to: generate each UAV action at the current time step sequentially through causal mask autoregression based on the observation representation sequence; The pre-construction of the fixed priority scoring function is based on the task description, observation space description, and action space description of the UAV swarm. It generates priority sorting logic in natural language form, structured pseudocode combined with the observation space description, and an executable fixed priority scoring function in sequence through a large language model.

[0095] Example 3 One embodiment of the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned UAV swarm order decision-making method based on dynamic priority sorting.

[0096] Example 4 In one embodiment of the present invention, a non-transitory computer-readable storage medium is provided for storing computer instructions. When the computer instructions are executed by a processor, they implement the UAV swarm order decision method based on dynamic priority sorting.

[0097] Example 5 One embodiment of the present invention provides an electronic device, including: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the UAV swarm order decision method based on dynamic priority sorting.

[0098] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0099] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0100] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for determining the order of unmanned aerial vehicle (UAV) swarms based on dynamic priority ranking, characterized in that, include: Obtain observation data from each UAV in the cluster at the current time step, and preprocess it to obtain a joint observation sequence; The joint observation sequence is input into a pre-constructed fixed priority scoring function to calculate the priority score of each UAV. The UAVs are then arranged in descending order of priority score to obtain a dynamic sorting index. The joint observation sequence is rearranged according to the dynamic permutation index, and the rearranged observation sequence is encoded to obtain the observation representation sequence; Based on the observation representation sequence, the actions of each UAV at the current time step are generated sequentially through causal mask autoregression; The pre-construction of the fixed priority scoring function is based on the task description, observation space description, and action space description of the UAV swarm. It generates priority sorting logic in natural language form, structured pseudocode combined with the observation space description, and an executable fixed priority scoring function in sequence through a large language model.

2. The UAV swarm order decision-making method based on dynamic priority sorting as described in claim 1, characterized in that, The observation data includes airborne perception data, navigation attitude data, and mission constraint data.

3. The UAV swarm order decision-making method based on dynamic priority sorting as described in claim 1, characterized in that, The encoding includes: embedding features into the rearranged observation sequence and superimposing a learnable sequence position code, inputting a trainable sequence encoder to obtain the observation representation sequence.

4. The UAV swarm order decision-making method based on dynamic priority sorting as described in claim 1, characterized in that, The causal mask autoregression uses the following action generation strategy: a drone at any position in the dynamic arrangement index uses actions that have been generated before that position, but cannot access actions that have not yet been generated after that position.

5. The UAV swarm order decision-making method based on dynamic priority sorting as described in claim 1, characterized in that, Also includes: Based on the inverse mapping relationship of the dynamic arrangement index, the actions of each UAV generated in the rearranged order are restored to the original UAV number to form a joint action.

6. The UAV swarm order decision-making method based on dynamic priority sorting as described in claim 5, characterized in that, The joint action is executed, and the parameters of the sequence encoder and action generation strategy are updated based on the obtained reward, while the fixed priority scoring function remains unchanged.

7. A UAV swarm order decision-making system based on dynamic priority ranking, characterized in that, include: The acquisition module is configured to: acquire observation data from each UAV in the cluster at the current time step, and preprocess the data to obtain a joint observation sequence; The sorting module is configured to: input the joint observation sequence into a pre-built fixed priority scoring function, calculate the priority score of each UAV, sort the UAVs from high to low according to the priority score, and obtain a dynamic sorting index; The encoding module is configured to: rearrange the joint observation sequence according to the dynamic permutation index, and encode the rearranged observation sequence to obtain an observation representation sequence; The generation module is configured to: generate each UAV action at the current time step sequentially through causal mask autoregression based on the observation representation sequence; The pre-construction of the fixed priority scoring function is based on the task description, observation space description, and action space description of the UAV swarm. It generates priority sorting logic in natural language form, structured pseudocode combined with the observation space description, and an executable fixed priority scoring function in sequence through a large language model.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the UAV swarm order decision method based on dynamic priority sorting as described in any one of claims 1-6.

9. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the UAV swarm order decision method based on dynamic priority sorting as described in any one of claims 1-6.

10. An electronic device, characterized in that, include: The device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the UAV swarm order decision method based on dynamic priority sorting as described in any one of claims 1-6.

Images