A multi-task end-to-end UAV cruising and formation maintenance method

Abstract

This invention discloses a multi-task, end-to-end drone cruising and formation-maintaining method, comprising the following steps: acquiring drone and environmental information and inputting it into a dynamic formation control module, which controls the drone swarm to maintain a predetermined formation during flight to avoid obstacles; an adaptive strategy module calculates and outputs the drone flight strategy and path planning in real time based on dynamic environmental changes; each drone independently processes local environmental information to ensure global coordination of the drone swarm; and the drones perform cruising and formation-maintaining operations according to the resulting decision signals. By sensing environmental changes, adjusting the drone formation configuration in real time, and autonomously learning optimal strategies, this method ensures that drones can efficiently and reliably complete multiple tasks in complex scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicles (UAVs), and in particular relates to a multi-task end-to-end UAV cruising and formation maintenance method. Background Art

[0002] With the rapid development of drone technology, multi-UAV systems have found widespread application in numerous fields, such as disaster relief, environmental monitoring, military reconnaissance, and aerial patrol. However, as application scenarios become more complex, the challenges faced by drone swarms in performing multiple missions are becoming increasingly significant, particularly in terms of navigation methods and formation maintenance in complex environments. Addressing these challenges directly impacts the efficiency and safety of drone swarms in executing their missions.

[0003] Currently, traditional drone online decision-making strategies primarily focus on a single decision-making environment. However, the rise of multi-environment decision-making—that is, making continuous decisions across multiple environments within a single decision-making process—has presented new challenges and opportunities. Considering the potential cost-benefit differences in different environments, drones must accurately evaluate each decision to ensure the most cost-effective outcome.

[0004] In complex and ever-changing environments, drone formations must adjust their formation in real time based on the external environment and mission requirements. By dynamically adjusting the relative positions of drones, the formation can adapt to external interference and obstacles while maintaining the stability of the communication network. Dynamic formation control requires not only flexible drone maneuverability but also the ability for the system to quickly perceive and process environmental information to make optimal adjustments.

[0005] Traditional predefined strategies often fail to adapt to dynamically changing mission requirements and environmental complexity. However, through reinforcement learning, drones can continuously learn and optimize their decisions through interaction with their environment. In multi-task scenarios, RL can help drones autonomously learn how to efficiently plan flight paths and adjust formations in uncertain environments, even when facing conflicting missions and limited resources. This strategy empowers drone swarms with a higher level of intelligence, enabling them to cope with environmental uncertainties and achieve automated task allocation and dynamic adjustments.

[0006] Therefore, a novel, multi-task, end-to-end drone patrol and formation maintenance method is needed to address the multiple challenges in complex environments. This method ensures that drones can efficiently and reliably complete multiple tasks in complex scenarios by sensing environmental changes, adjusting the drone formation in real time, and autonomously learning optimal strategies. This innovative technology not only maintains communication connectivity and obstacle avoidance capabilities but also introduces an adaptive strategy that dynamically adjusts task priorities and optimizes system resource allocation, enhancing the collaborative capabilities of drone swarms in dynamic environments. Summary of the Invention

[0007] In view of this, the present invention aims to provide a multi-task end-to-end UAV cruising and formation maintenance method. The method integrates dynamic formation control and reinforcement learning adaptive strategies to provide a highly adaptive and robust solution suitable for complex, multi-task multi-UAV application scenarios, significantly improving the operational efficiency of UAV swarms and their ability to cope with complex environments. The method includes the following steps:

[0008] Step 1: Obtain drone and environment information and input it into the dynamic formation control module to control the drone swarm to maintain a predetermined formation during flight to avoid obstacles;

[0009] Step 2: Based on the changes in the dynamic environment, the adaptive strategy module calculates and outputs the UAV's flight strategy and path planning in real time;

[0010] Step 3: Each drone independently processes local environmental information to ensure global coordination of the drone cluster;

[0011] Step 4: The UAV performs cruise and formation maintenance according to the final decision signal generated.

[0012] Specifically, the dynamic formation control module updates the UAV motion according to the synthetic potential field. The total resultant force calculation formula of the synthetic potential field is:

[0013] F total (q)=F att (q)+∑F rep (q)+∑F drone (q)

[0014] Among them, F total (q) is the total force, which is used to guide the movement direction and speed of the UAV, ∑ represents the sum of all related objects, F att (q) represents the gravitational vector, pointing to the target position, and its calculation formula is: F att (q)=-k att (qq goal ), q represents the current position vector of the UAV, qgoal represents the target position vector, k att is the gravitational coefficient, which indicates the strength of gravity, F rep (q) represents the repulsive force vector, pointing in the direction away from the obstacle, ρ(q) represents the distance from the drone to the obstacle, ρ0 is the range of repulsion, the maximum distance of repulsion, k rep is the repulsion coefficient, which indicates the strength of the repulsive force, Represents the gradient of the distance function, the direction is from the obstacle to the drone, F drone (q) represents the repulsive force vector between UAVs, and its calculation formula is: d(q): The distance between the current drone and other drones, d0 represents the safe distance between drones, k drone is the repulsion coefficient between drones, Represents the gradient of the distance function, which is directed from other drones to the current drone.

[0015] Specifically, the adaptive strategy module includes the following steps:

[0016] Combine the features of multi-task learning to obtain the current state S t And process the input: S t =MTL(X t ,E t ,P t ) Among them, X t is the state of the drone itself, E t is environmental information, P t For task priority;

[0017] Based on the current state S t and priority information to adjust the weight of strategy selection: (Q(S t ,A)+λ·P t W) where λ is the adjustment coefficient, P t is the task priority, W is the weight matrix, A t represents the optimal action at the current moment, A represents the set of optional actions, and Q represents the action value function;

[0018] Use the Q-learning algorithm to update the action-value function Q in combination with future discounted rewards: Among them, α is the learning rate, γ is the discount factor, which is used to measure the importance of future rewards, and A ′ Indicates the possible actions at the next moment, Q(S t ,A t ) is the action value function, which means that in state S t Next, perform action At The expected total reward that can be obtained, R t Indicates immediate reward;

[0019] Using the policy network optimized by multi-task learning, the policy network parameters are updated by gradient descent to minimize the loss function of policy selection: Among them, θ is the policy network parameter, η is the learning rate, represents the gradient, is the loss function, and the calculation formula of the loss function is:

[0020]

[0021] in, Express expectations.

[0022] Preferably, the UAV independently processing local environment information in step 3 includes the following steps:

[0023] UAV i perceives its own and surrounding environment information through sensors and exchanges information with neighboring UAVs to form a local environment information set L i and the neighboring drone information set N i ;

[0024] UAV i is based on the local environment information set L i and the information set N of neighboring drones i , independently calculate the optimal decision D i , the calculation formula is: Among them, A i is the set of actions that can be performed by drone i, Q(L i ,A i ,N i ) is the action value function, which represents the local information L i and nearby drone information N i Next, perform action A i The expected total reward that can be obtained;

[0025] By communicating and exchanging information with neighboring drones, the decision of drone i is ensured. i The decision-making of its neighboring drones should be consistent with the global goal to avoid decision conflicts, D′ i =Cooperate(D i ,D neighbors ), where D neighbors is the decision set of neighboring drones, Cooperate is the collaborative function used to adjust decisions to avoid conflicts, and D′ i For local decision-making after coordination.

[0026] Preferably, the UAV independently processing local environment information in step 3 includes the following steps:

[0027] The perceived local environment information set L i and the information set N of neighboring drones i Perform fusion, generate local state representation, and extract key features V i , V i =f fusion (L i ,N i ) where f fusion Represents the feature fusion function, the key feature V extracted i Serves as input for subsequent decision-making;

[0028] According to the real-time environment and the importance of the task, the priority of the task is dynamically adjusted to generate the task weight W i to ensure that critical tasks are prioritized;

[0029] Use reinforcement learning algorithm to calculate the value Q(V) of each action in the current local state i ,A i ,W i ), Q(V i ,A i ,W i )=RL(V i ,A i ,W i ), where RL represents the reinforcement learning model, and the Q-value function generated by training is used to evaluate the value of each possible action;

[0030] According to the calculated Q value, select the action with the highest value as the current optimal decision D i , This step ensures that the drone can make the most profitable decision in the current state. i is the set of actions that can be performed by drone i, Q(V i ,A i ,W i ) is the action value function, expressed in the feature vector V i and task weight W i Next, perform action A i The expected total reward that can be obtained;

[0031] By exchanging information with neighboring drones, coordinating and adjusting decisions, conflicting decisions are avoided and the consistency of drone group behavior is ensured: D' i =Cooperate(D i ,D neighbors ), where D neighborsis the decision set of neighboring drones, Cooperate is the collaborative function, and D′ i In order to make local decisions after coordination, the decision is fine-tuned through the coordination mechanism to make the individual behaviors in the drone group compatible with each other.

[0032] Specifically, the collaborative function is used to achieve decision consistency adjustment, including the following steps:

[0033] UAV i collects updated decision information from neighboring UAVs to form a complete decision information matrix;

[0034] Analyze the decision information matrix and calculate the decision consistency error E between each pair of drones i , that is, the deviation between the path points of each drone at the same time step: E i (j,k)=‖P j ′ (t)-P ′ k (t)‖, where P j ′ (t) and P ′ k (t) represents the adjusted path points of UAVs j and k at time step t. If the error is greater than a preset tolerance threshold e tol , then it is considered that there is a consistency problem;

[0035] For each pair of drones j and k with consistency error, calculate the consistency adjustment vector ΔP ij , to minimize the deviation of the path, the consistency adjustment vector Among them, α is the adjustment coefficient, which controls the amplitude of the adjustment;

[0036] Apply the consistency adjustment vector to the corresponding drones and update their paths to reduce the global consistency error: in, represents the set of drones adjacent to drone i, and the final decision output is the adjusted path P i ″ .

[0037] The beneficial effects of the present invention are as follows: The method of the present invention combines multi-task learning and reinforcement learning, enabling drones to adaptively adjust their flight strategies according to environmental changes and prioritize key tasks. This dynamic task priority management ensures that different task objectives can collaborate efficiently, reduces task conflicts and maximizes the efficiency of task completion. At the same time, through distributed decision-making and collaborative adjustment mechanisms, the drone group can ensure the coordination and consistency of global behavior. Steps such as conflict detection and resolution, and consistency adjustment ensure that the actions between drones will not cause path conflicts or interference, thereby maintaining the integrity of the formation. In addition, each drone can make autonomous decisions based on local information and collaborate with other drones through real-time information sharing. This method has high robustness and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is the overall flow chart of the UAV cruising and formation maintenance method. DETAILED DESCRIPTION

[0039] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are merely some, rather than all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0040] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0041] This embodiment proposes a multi-task end-to-end UAV cruising and formation maintenance method, such as Figure 1 As shown, the following steps are included:

[0042] Step 1: Obtain drone and environment information and input it into the dynamic formation control module to control the drone swarm to maintain a predetermined formation during flight to avoid obstacles;

[0043] Step 2: Based on the changes in the dynamic environment, the adaptive strategy module calculates and outputs the UAV's flight strategy and path planning in real time;

[0044] Step 3: Each drone independently processes local environmental information to ensure global coordination of the drone cluster;

[0045] Step 4: The UAV performs cruise and formation maintenance according to the final decision signal generated.

[0046] In this embodiment, the main function of the dynamic formation control module is to dynamically adjust the relative positions of drones based on real-time environmental changes, ensuring that the entire drone group can maintain a predetermined formation during flight while avoiding collisions and loss of communication connections. This generally includes the following functions: formation structure maintenance: maintaining the stability of the drone formation in the face of environmental interference (such as wind, obstacles, and hostile interference); communication connectivity assurance: ensuring that the communication links between drones are not disrupted and maintaining the group's collaborative operation capabilities; obstacle avoidance: dynamically adjusting the formation configuration when obstacles appear in the flight path to safely pass or bypass them.

[0047] Specifically, the dynamic formation control module updates the UAV motion according to the synthetic potential field. The total resultant force calculation formula of the synthetic potential field is:

[0048] F total (q)=F att (q)+∑F rep (q)+∑F drone (q)

[0049] Among them, F total (q) is the total force, which is used to guide the movement direction and speed of the UAV, ∑ represents the sum of all related objects, F att (q) represents the gravitational vector, pointing to the target position, and its calculation formula is: F att (q)=-k att (qq goal ), q represents the current position vector of the UAV, q goal represents the target position vector, k att is the gravitational coefficient, which indicates the strength of gravity, F rep (q) represents the repulsive force vector, pointing in the direction away from the obstacle, ρ(q) represents the distance from the drone to the obstacle, ρ0 is the range of repulsion, the maximum distance of repulsion, k rep is the repulsion coefficient, which indicates the strength of the repulsive force, Represents the gradient of the distance function, the direction is from the obstacle to the drone, F drone (q) represents the repulsive force vector between UAVs, and its calculation formula is: The distance between the current drone and other drones, d0 represents the safe distance between drones, k drone is the repulsion coefficient between drones, Represents the gradient of the distance function, which is directed from other drones to the current drone.

[0050] Specifically, the adaptive strategy module includes the following steps:

[0051] Combine the features of multi-task learning to obtain the current state S t And process the input: S t =MTL(X t ,E t ,P t ) Among them, X t is the state of the drone itself, E t is environmental information, P t For task priority;

[0052] Based on the current state S t and priority information to adjust the weight of strategy selection: Among them, λ is the adjustment coefficient, P t is the task priority, W is the weight matrix, A t represents the optimal action at the current moment, A represents the set of optional actions, and Q represents the action value function;

[0053] Use the Q-learning algorithm to update the action-value function Q in combination with future discounted rewards: Among them, α is the learning rate, γ is the discount factor, which is used to measure the importance of future rewards, and A ′ Indicates the possible actions at the next moment, Q(S t ,A t ) is the action value function, which means that in state S t Next, perform action A t The expected total reward that can be obtained, R t Indicates immediate reward;

[0054] Using the policy network optimized by multi-task learning, the policy network parameters are updated by gradient descent to minimize the loss function of policy selection: Among them, θ is the policy network parameter, η is the learning rate, represents the gradient, is the loss function, and the calculation formula of the loss function is:

[0055]

[0056] in, Express expectations.

[0057] In this embodiment, the adaptive strategy module, within the multi-task end-to-end drone cruising and formation maintenance method, is primarily responsible for adjusting the drone's flight strategy and path planning in real time based on dynamic environmental changes. Its core purpose is to use reinforcement learning to enable the drone to adaptively optimize its cruising path and formation decisions in complex and changing environments, achieving optimal mission execution.

[0058] The Adaptive Strategy Module, through the collaborative work of real-time environmental perception, strategy selection, and strategy optimization, enables drones to autonomously learn and adjust their flight strategies in complex and changing environments. This module utilizes a deep reinforcement learning algorithm to continuously optimize drones' cruising paths and formation decisions, improving the efficiency and success rate of their missions. In a multi-tasking, end-to-end drone system, the Adaptive Strategy Module is key to achieving intelligent and adaptive capabilities in drone swarms.

[0059] Preferably, the UAV independently processing local environment information in step 3 includes the following steps:

[0060] UAV i perceives its own and surrounding environment information through sensors and exchanges information with neighboring UAVs to form a local environment information set L i and the neighboring drone information set N i ;

[0061] UAV i is based on the local environment information set L i and the information set N of neighboring drones i , independently calculate the optimal decision D i , the calculation formula is: Among them, A i is the set of actions that can be performed by drone i, Q(L i ,A i ,N i ) is the action value function, which represents the local information L i and nearby drone information N i Next, perform action A i The expected total reward that can be obtained;

[0062] By communicating and exchanging information with neighboring drones, the decision of drone i is ensured. i The decision-making of its neighboring drones should be consistent with the global goal to avoid decision conflicts, D′ i =Cooperate(D i ,D neighbors ), where D neighbors is the decision set of neighboring drones, Cooperate is the collaborative function used to adjust decisions to avoid conflicts, and D′ i For local decision-making after coordination.

[0063] The primary function of the distributed decision-making module in this embodiment is to ensure that each drone in the swarm can make independent decisions based on its own local information and that of its neighbors, without relying on a central control system. This distributed decision-making mechanism enhances the autonomy and robustness of the swarm, enabling the drone formation to maintain efficient mission execution in dynamic and complex environments.

[0064] Preferably, the UAV independently processing local environment information in step 3 includes the following steps:

[0065] The perceived local environment information set L i and the information set N of neighboring drones i Perform fusion, generate local state representation, and extract key features V i , V i =f fusion (L i ,N i ) where f fusion Represents the feature fusion function, the key feature V extracted i Serves as input for subsequent decision-making;

[0066] According to the real-time environment and the importance of the task, the priority of the task is dynamically adjusted to generate the task weight W i :W i =AdjustPriority(T i ,V i ) to ensure that key tasks are given priority; task priorities are adjusted by analyzing factors such as task time requirements and environmental threat levels;

[0067] Use reinforcement learning algorithm to calculate the value Q(V) of each action in the current local state i ,A i ,W i ): Q(V i ,A i ,W i )=RL(V i ,A i ,W i ), where RL represents the reinforcement learning model, and the Q-value function generated by training is used to evaluate the value of each possible action;

[0068] Based on the calculated Q value, select the action with the highest value as the current optimal decision: This step ensures that the drone can make the most profitable decision in the current state. i is the set of actions that can be performed by drone i, Q(V i ,A i ,W i ) is the action value function, expressed in the feature vector V i and task weight W i Next, perform action A i The expected total reward that can be obtained;

[0069] By exchanging information with neighboring drones, coordinating and adjusting decisions, conflicting decisions are avoided and the consistency of drone group behavior is ensured: D' i =Cooperate(D i ,D neighbors ), where D neighbors is the decision set of neighboring drones, Cooperate is the collaborative function, and D′ i In order to make local decisions after coordination, the decision is fine-tuned through the coordination mechanism to make the individual behaviors in the drone group compatible with each other.

[0070] The improved local decision-making module significantly enhances the decision-making efficiency and mission execution capabilities of drone swarms by enhancing adaptability to dynamic environments and coordinating with neighboring drones. This module's design not only ensures the autonomy of drones but also, through task priority management and decision-making coordination, ensures that the entire drone swarm can coordinate and efficiently execute missions in complex environments. This improvement enables drones to more flexibly and intelligently adjust their decisions when faced with multiple tasks and dynamic environments, ensuring the achievement of mission objectives.

[0071] Specifically, the collaborative function is used to achieve decision consistency adjustment, including the following steps:

[0072] UAV i collects updated decision information from neighboring UAVs to form a complete decision information matrix;

[0073] Analyze the decision information matrix and calculate the decision consistency error E between each pair of drones i , that is, the deviation between the path points of each drone at the same time step: E i (j,k)=‖P j ′ (t)-P ′ k (t)‖, where P j ′ (t) and P ′ k (t) represents the adjusted path points of UAVs j and k at time step t. If the error is greater than a preset tolerance threshold e tol , then it is considered that there is a consistency problem;

[0074] For each pair of drones j and k with consistency error, calculate the consistency adjustment vector ΔP ij , to minimize the deviation of the path, the consistency adjustment vector Among them, α is the adjustment coefficient, which controls the amplitude of the adjustment;

[0075] Apply the consistency adjustment vector to the corresponding drones and update their paths to reduce the global consistency error: in, represents the set of drones adjacent to drone i, and the final decision output is the adjusted path P i ″ .

[0076] The decision-making collaborative computing module ensures that drone swarms achieve global coordination based on independent decision-making through local decision-sharing, conflict detection and resolution, consistency adjustment, and feedback loops. Improvements to this module significantly improve the collaborative efficiency and mission success rate of multi-drone systems in complex environments.

[0077] This embodiment presents a novel multi-task, end-to-end drone cruising and formation maintenance method capable of addressing multiple challenges in complex environments. By sensing environmental changes, adjusting the drone formation in real time, and autonomously learning optimal strategies, this method ensures that drones can efficiently and reliably complete multiple tasks in complex scenarios. The innovation of this technology lies not only in its ability to maintain communication connectivity and obstacle avoidance, but also in the introduction of adaptive strategies that enable dynamic adjustment of task priorities and optimal allocation of system resources, enhancing the collaborative working capabilities of drone groups in dynamic environments.

[0078] As used herein, the word "preferred" is intended to serve as an example, instance, or illustration. Any aspect or design described herein as "preferred" is not necessarily to be construed as advantageous over other aspects or designs. Rather, the use of the word "preferred" is intended to present concepts in a concrete manner. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X employs A or B" is intended to mean any of the naturally inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then "X employs A or B" is satisfied in any of the foregoing examples.

[0079] Moreover, although the present disclosure has been shown and described with respect to one or implementation, those skilled in the art will think of equivalent variations and modifications based on reading and understanding of this specification and the accompanying drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the above-mentioned components (such as elements, etc.), the terms used to describe such components are intended to correspond to any component (unless otherwise indicated) that performs the specified function of the component (such as it is functionally equivalent), even if structurally different from the disclosed structure that performs the function in the exemplary implementation of the present disclosure shown herein. In addition, although the specific features of the present disclosure have been disclosed with respect to only one of several implementations, such features can be combined with one or other features of other implementations that can be desired and advantageous for a given or specific application. Moreover, insofar as the terms "including", "having", "containing" or their variations are used in specific embodiments or claims, such terms are intended to be included in a manner similar to the term "comprising".

[0080] The functional units in the embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or multiple or more units may be integrated into a single module. The aforementioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The aforementioned storage medium may be a read-only memory, a magnetic disk, or an optical disk, etc. The aforementioned devices or systems may execute the storage method in the corresponding method embodiment.

[0081] In summary, the above embodiment is one implementation method of the present invention, but the implementation method of the present invention is not limited to the described embodiment. Any other changes, modifications, substitutions, combinations, and simplifications that deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A multi-task end-to-end UAV cruising and formation maintenance method, characterized in that: The following steps are involved: Step 1: Obtain drone and environment information and input it into the dynamic formation control module to control the drone swarm to maintain a predetermined formation during flight to avoid obstacles; Step 2: Based on the changes in the dynamic environment, the adaptive strategy module calculates and outputs the UAV's flight strategy and path planning in real time; Step 3: Each drone independently processes local environmental information to ensure global coordination of the drone cluster; Step 4: The UAV performs cruise and formation maintenance according to the final decision signal generated; The dynamic formation control module updates the UAV motion according to the synthetic potential field. The total resultant force calculation formula of the synthetic potential field is: F total (q)=F att (q)+∑F rep (q)+∑F drone (q), Among them, F total (q) is the total force, which is used to guide the movement direction and speed of the UAV, ∑ represents the sum of all related objects, F att (q) represents the gravitational vector, pointing to the target position, and its calculation formula is: F att (q)=-k att (qq goal ), q represents the current position vector of the UAV, q goal represents the target position vector, k att is the gravitational coefficient, which indicates the strength of gravity, F rep (q) represents the repulsive force vector, pointing in the direction away from the obstacle, and its calculation formula is: Among them, ρ(q) represents the distance from the UAV to the obstacle, ρ0 is the range of repulsion, the maximum distance of repulsion, k rep is the repulsion coefficient, which indicates the strength of the repulsive force, Represents the gradient of the distance function, the direction is from the obstacle to the drone, F drone (q) represents the repulsive force vector between UAVs, and its calculation formula is: Among them, d(q) represents the distance between the current UAV and other UAVs, d0 represents the safe distance between UAVs, and k drone is the repulsion coefficient between drones, Represents the gradient of the distance function, which is directed from other drones to the current drone.

2. A multi-task end-to-end UAV cruising and formation maintenance method according to claim 1, characterized in that: The adaptive strategy module includes the following steps: Combine the features of multi-task learning to obtain the current state S t And process the input: S t =MTL(X t ,E t ,P t ) Among them, X t is the state of the drone itself, E t is environmental information, P t For task priority; Based on the current state S t and priority information to adjust the weight of strategy selection: Among them, λ is the adjustment coefficient, P t is the task priority, W is the weight matrix, A t represents the optimal action at the current moment, A represents the set of optional actions, and Q represents the action value function; Use the Q-learning algorithm to update the action-value function Q in combination with future discounted rewards: Q(S t , Among them, α is the learning rate, γ is the discount factor, which is used to measure the importance of future rewards, and A ′ Indicates the possible actions at the next moment, Q(S t ,A t ) is the action value function, which means that in state S t Next, perform action A t The expected total reward that can be obtained, R t Indicates immediate reward; Using the policy network optimized by multi-task learning, the policy network parameters are updated by gradient descent to minimize the loss function of policy selection: Among them, θ is the policy network parameter, η is the learning rate, represents the gradient, is the loss function, and the calculation formula of the loss function is: in, Express expectations.

3. A multi-task end-to-end UAV cruising and formation maintenance method according to claim 2, characterized in that: The UAV independently processes local environment information as described in step 3, including the following steps: UAV i perceives its own and surrounding environment information through sensors and exchanges information with neighboring UAVs to form a local environment information set L i and the neighboring drone information set N i ; UAV i is based on the local environment information set L i and the information set N of neighboring drones i , independently calculate the optimal decision D i , the calculation formula is: Among them, A i is the set of actions that can be performed by drone i, Q(L i ,A i ,N i ) is the action value function, which represents the local information L i and nearby drone information N i Next, perform action A i The expected total reward that can be obtained; By communicating and exchanging information with neighboring drones, the decision of drone i is ensured. i The decision-making of its neighboring drones should be consistent with the global goal to avoid decision conflicts, D′ i =Cooperate(D i ,D neighbors ), where D neighbors is the decision set of neighboring drones, Cooperate is the collaborative function used to adjust decisions to avoid conflicts, and D′ i For local decision-making after coordination.

4. The multi-task end-to-end UAV cruising and formation maintenance method according to claim 2, characterized in that: The UAV independently processes local environment information as described in step 3, including the following steps: The perceived local environment information set L i and the information set N of neighboring drones i Perform fusion, generate local state representation, and extract key features V i , V i =f fusion (L i ,N i ) where f fusion Represents the feature fusion function, the key feature V extracted i Serves as input for subsequent decision-making; According to the real-time environment and the importance of the task, the priority of the task is dynamically adjusted to generate the task weight W i to ensure that critical tasks are prioritized; Use reinforcement learning algorithm to calculate the value Q(V) of each action in the current local state i ,A i ,W i ), Q(V i ,A i ,W i )=RL(V i ,A i ,W i ), where RL represents the reinforcement learning model, and the Q-value function generated by training is used to evaluate the value of each possible action; According to the calculated Q value, select the action with the highest value as the current optimal decision D i , This step ensures that the drone can make the most profitable decision in the current state. i is the set of actions that can be performed by drone i, Q(V i ,A i ,W i ) is the action value function, expressed in the feature vector V i and task weight W i Next, perform action A i The expected total reward that can be obtained; By exchanging information with neighboring drones, coordinating and adjusting decisions, conflicting decisions are avoided and the consistency of drone group behavior is ensured: D' i =Cooperate(D i ,D neighbors ), where D neighbors is the decision set of neighboring drones, Cooperate is the collaborative function, and D′ i In order to make local decisions after coordination, the decision is fine-tuned through the coordination mechanism to make the individual behaviors in the drone group compatible with each other.

5. A multi-task end-to-end UAV cruising and formation maintenance method according to claim 3 or 4, characterized in that: The collaborative function is used to achieve decision consistency adjustment, including the following steps: UAV i collects updated decision information from neighboring UAVs to form a complete decision information matrix; Analyze the decision information matrix and calculate the decision consistency error E between each pair of drones i , that is, the deviation between the path points of each UAV at the same time step, and its calculation formula is: E i (j,k)=‖P j ′ (t)-P ′ k (t)‖, Among them, P j ′ (t) and P ′ k (t) represents the adjusted path points of UAVs j and k at time step t. If the error is greater than a preset tolerance threshold e tol , then it is considered that there is a consistency problem; For each pair of drones j and k with consistency error, calculate the consistency adjustment vector ΔP ij , in order to minimize the deviation of the path, the calculation formula of the consistency adjustment vector is: Among them, α is the adjustment coefficient, which controls the amplitude of the adjustment; Apply the consistency adjustment vector to the corresponding UAVs and update their paths to reduce the global consistency error. The update formula is: in, represents the set of drones adjacent to drone i, and the final decision output is the adjusted path P i ″ .

Images