Plant protection unmanned aerial vehicle cluster self-learning optimization formation control method

Abstract

The invention discloses a plant protection unmanned aerial vehicle cluster self-learning optimization formation control method, belongs to the technical field of self-learning optimization formation control methods, and improves the performance and adaptability of an unmanned aerial vehicle system. According to the patent technology, maximization of system performance can be achieved through ETCRF based on RL, the plant protection unmanned aerial vehicle can learn an optimal control strategy through interaction with the environment so as to reduce control cost and improve system efficiency, the system has higher adaptability, self-adaptive adjustment can be conducted according to environment changes and system requirements, and the system can be widely applied to the field of plant protection. Therefore, the plant protection unmanned aerial vehicle group system can better deal with dynamic environment and uncertain factors, so that the stability and robustness of the system are improved.

Description

Technical Field

[0001] The present invention relates to a self-learning optimization formation control method, in particular to a self-learning optimization formation control method for a plant protection UAV cluster, and belongs to the technical field of self-learning optimization formation control methods. Background Art

[0002] Due to the large area of ​​farmland and the wide range of plant protection operations, a single drone often cannot meet the plant protection needs of large-scale farmland. Therefore, multiple plant protection drones are needed to collaborate to complete plant protection operations.

[0003] In this case, how to achieve effective collaboration among multiple drones becomes a key issue, so it is necessary to study the collaborative control of plant protection drone swarms.

[0004] At the same time, multi-agent systems (MAS) have been widely used in fields such as intelligent transportation, industrial production, and the Internet of Things. The goal of multi-agent formation control is to achieve coordination and cooperation among multiple agents while maintaining a predetermined formation to complete specific tasks. At present, MAS has been used in agriculture, namely plant protection drone swarm technology, but the traditional plant protection drone swarm collaborative control method has the following disadvantages:

[0005] High resource consumption: The cooperative control technology of plant protection UAV swarms often requires high computing and communication resources to support information exchange and collaborative decision-making between UAVs. This increases the hardware cost of the system and may limit the scale and scalability of the system. When the plant protection UAV performs a mission, its measurement module, communication module, microprocessor, actuator module, etc. will consume a lot of energy. Traditional cooperative control methods may not be able to effectively reduce the energy consumption of these modules, thereby shortening the service life of the system.

[0006] Poor environmental adaptability. In practical applications, the plant protection drone swarm system may face communication limitations such as limited communication bandwidth, data packet loss, and communication delay. Traditional collaborative control methods may not be able to effectively cope with these limitations, resulting in system performance degradation or even failure. In addition, the plant protection drone swarm system is usually nonlinear and uncertain, which may cause the system to show poor robustness and adaptability when facing complex environments and emergencies.

[0007] Plant protection UAVs have poor fault tolerance when failures occur. Traditional plant protection UAV swarm collaborative control methods often focus on the overall collaborative performance of the system, while ignoring the real-time detection and diagnosis of internal faults of individual plant protection UAVs (such as actuator failures). When an actuator fails, traditional plant protection UAV swarm collaborative control methods usually rely on highly precise communication and synchronization mechanisms to ensure the consistency and coordination of the system, which leads to its lack of adaptability and learning ability, and may lack sufficient fault-tolerant control mechanisms to deal with actuator failures. It is impossible to quickly adjust the collaborative strategy to adapt to the new system state, resulting in system performance degradation or even collapse.

[0008] In view of the above defects, each drone in the plant protection drone swarm has independent perception and decision-making capabilities through communication and interaction with other drones. Therefore, formation control needs to consider the interaction, information exchange and cooperation strategy between drones. In addition, the randomness, nonlinearity and uncertainty problems in the plant protection drone swarm must be solved. These control methods only consider the stability of the plant protection drone swarm system, but do not further consider the control performance and efficiency of the plant protection drone swarm. In addition, the research content in recent years has only focused on consensus control issues, which leads to the following defects in the current plant protection drone swarm technology:

[0009] 1) Consumes a lot of communication and computing resources, because the existing plant protection drone swarm technology needs to update the controller at each sampling time, which greatly increases resource consumption.

[0010] 2) The computing and communication capabilities of drones are limited. In a swarm of plant protection drones, each drone transmits information through a communication network. Because it is usually equipped with a microprocessor, this will limit the communication capabilities of the drone, resulting in untimely information transmission.

[0011] Therefore, a self-learning optimization formation control method for plant protection UAV cluster is designed to solve the above problems. Summary of the invention

[0012] The main purpose of the present invention is to provide a self-learning optimization formation control method for a plant protection UAV cluster.

[0013] The purpose of the present invention can be achieved by adopting the following technical solutions:

[0014] The plant protection UAV cluster self-learning optimization formation control method includes the following steps:

[0015] Step 1: Based on a plant protection drone swarm with one leader and K followers, an undirected communication topology is used to describe the interconnection between agents;

[0016] Step 2: Establish a system model for following UAV i;

[0017] Step 3: Establish formation dynamic error calculation based on the system model of the following UAV i;

[0018] Step 4: Establish the discounted optimal value function of the following UAV i and satisfy the HJB equation to obtain the optimal formation control law;

[0019] Step 5: A reinforcement learning-based ETCRF control scheme is used to approximate the solution of the coupled HJB equations, ensuring that all uncertain followers accurately track the leader trajectory in the specified formation;

[0020] Step 6: Design an event-triggered robust formation controller;

[0021] Step 7: Get an approximation of the optimal value function for following drone i in the context of the neural network architecture.

[0022] Preferably, in step 1, describing the interconnection between agents using undirected communication topology specifically includes the following steps:

[0023] use in A set of nodes representing the agent itself;

[0024] is the set of edges, is the weighted adjacency matrix;

[0025] If and only if agent i and agent j are directly connected, then

[0026] And if Then α ij >0, if Then α ij = 0 and

[0027] make is the neighbor set of agent i, is the set of drone i and its neighbors,

[0028]

[0029] in A g The degree matrix of is the Laplace matrix;

[0030] Among them l ij =-α ij and

[0031] Preferably, the system model for following drone i in step 2 is as follows:

[0032]

[0033] in is the system status;

[0034] is the control input of the ith follower drone;

[0035] is the uncertainty of actuator fault matching;

[0036] and is a nonlinear system function.

[0037] Preferably, the system function and Including I f,i The compact set Ω with (0) = 0 is Lipschitz continuous, and the system (1) is stable on Ω;

[0038] Dynamic uncertainty satisfy in is a known function and The dynamics of the pilot drone can be expressed as:

[0039]

[0040] in is the system status;

[0041] is a differentiable function.

[0042] Preferably, the dynamic error calculation in step 3 includes the following:

[0043] (1) The nominal form of the formula is given by:

[0044]

[0045] The formation error of the i-th following drone is provided as:

[0046]

[0047] The vector constant n i represents the formation mode between UAV i and the pilot UAV, and c i ≥0 indicates pinning gain, so the dynamics of the error formation are calculated as:

[0048]

[0049] The new value function of the i-th following drone is expressed as:

[0050]

[0051] where γ is a positive constant, Utility Function Defined as

[0052]

[0053] where θ is a positive constant, and is a positive definite matrix.

[0054] Preferably, the Hami quantity of following drone i is expressed as:

[0055]

[0056] The discounted optimal value function of following drone i is:

[0057]

[0058] Satisfies the HJB equation:

[0059]

[0060] in is a set of acceptable controls, and the optimal formation control law is obtained:

[0061]

[0062] According to formula (8) and formula (9), the coupled HJB equation can be obtained as follows:

[0063]

[0064] Preferably, in step six, in the event triggering mechanism, the sampling state of the following drone i is expressed as:

[0065]

[0066] where Γ k represents the kth sampling moment. Therefore, the corresponding sampling error is:

[0067]

[0068] The event trigger error is expressed as:

[0069]

[0070] Based on formulas (9), (11) and (12), the ETCOF control law of the following UAV i is:

[0071]

[0072] And the system function is bounded, that is:

[0073]

[0074] in is a normal number.

[0075] Based on the ETCOF control law provided by formula (1), formula (2), formula (15) and formula (14) for the following UAV with dynamic uncertainty, if the trigger condition is met:

[0076]

[0077] Where λ is the design parameter, then the formation error of following UAV i is asymptotically stable.

[0078] Preferably, in step 7, in the context of a neural network architecture, the optimal value function for following drone i is approximately:

[0079]

[0080] in is the ideal weight vector, is the activation function, h c is the number of neurons in the hidden layer, is the approximation error. Further,

[0081]

[0082] The approximate value function is expressed as:

[0083]

[0084] in represents an estimate of the optimal weights, and we then obtain:

[0085]

[0086] According to formula (14) and formula (17), the approximate ETCOF control law is rewritten as:

[0087]

[0088] Combining formula (14) and formula (19), the approximate ETCOF control law is:

[0089]

[0090] According to formula (10) and formula (22), the approximate Hamiltonian is:

[0091]

[0092] By minimizing the objective function The weight adjustment law for judging the neural network is:

[0093]

[0094] where ξ c >0 and the learning rate is:

[0095]

[0096] Considering the formula (3) for the i-th nominal following drone, the weight update rule of the critic neural network is formula (24), then the weight estimation error of the critic neural network is It has uniformly eventual boundedness.

[0097] Beneficial technical effects of the present invention:

[0098] The plant protection UAV cluster self-learning optimization formation control method provided by the present invention improves the performance and adaptability of the UAV system. The patented technology can maximize the system performance through ETCRF based on RL. The plant protection UAV can learn the optimal control strategy through interaction with the environment to reduce the control cost and improve the system efficiency. The system has stronger adaptability and can be adaptively adjusted according to environmental changes and system requirements, which enables the plant protection UAV cluster system to better cope with dynamic environments and uncertain factors, thereby improving the stability and robustness of the system.

[0099] Improve agricultural production efficiency and stability. The plant protection drone swarm system can realize the automated operation of agricultural equipment through collaborative control, which can significantly improve agricultural production efficiency, reduce labor costs, and reduce the risk of human operation errors. The fault-tolerant formation control solution of the plant protection drone swarm system can ensure that the entire system can still operate normally by reallocating tasks and adjusting control strategies when some intelligent agents fail, thereby ensuring the stability and continuity of agricultural production.

[0100] Optimize resource allocation and reduce waste. The designed ETCRF controller is updated only at the triggering moment, so that the plant protection drone swarm system can calculate in real time and save communication resources. The system can automatically adjust the operation parameters according to the monitoring data, and can realize the precise management and dynamic scheduling of agricultural resources, thereby achieving the optimal allocation and efficient utilization of resources.

[0101] Responding to complex and changing environments. The plant protection drone swarm system can adapt to different complex environments by dynamically adjusting the formation and control strategy, and through collaborative operations and intelligent decision-making, it can reduce the impact of disasters on agricultural production and improve the disaster resistance of agricultural production. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] Figure 1 The present invention is a flowchart of a preferred embodiment of the plant protection UAV cluster self-learning optimization formation control method according to the present invention. DETAILED DESCRIPTION

[0103] In order to make the technical solution of the present invention more clear and specific to those skilled in the art, the present invention is further described in detail below in conjunction with embodiments and drawings, but the implementation manner of the present invention is not limited thereto.

[0104] like Figure 1 As shown, the plant protection UAV cluster self-learning optimization formation control method provided in this embodiment includes the following steps:

[0105] Step 1: Based on a plant protection drone swarm with one leader and K followers, an undirected communication topology is used to describe the interconnection between agents;

[0106] Step 2: Establish a system model for following UAV i;

[0107] Step 3: Establish formation dynamic error calculation based on the system model of the following UAV i;

[0108] Step 4: Establish the discounted optimal value function of the following UAV i and satisfy the HJB equation to obtain the optimal formation control law;

[0109] Step 5: A reinforcement learning-based ETCRF control scheme is used to approximate the solution of the coupled HJB equations, ensuring that all uncertain followers accurately track the leader trajectory in the specified formation;

[0110] Step 6: Design an event-triggered robust formation controller;

[0111] Step 7: Get an approximation of the optimal value function for following drone i in the context of the neural network architecture.

[0112] Improve the performance and adaptability of the drone system. This patented technology can maximize system performance through RL-based ETCRF. The plant protection drone can learn the optimal control strategy through interaction with the environment to reduce control costs and improve system efficiency. The system has stronger adaptability and can make adaptive adjustments according to environmental changes and system requirements, which enables the plant protection drone swarm system to better cope with dynamic environments and uncertain factors, thereby improving the stability and robustness of the system.

[0113] Improve agricultural production efficiency and stability. The plant protection drone swarm system can realize the automated operation of agricultural equipment through collaborative control, which can significantly improve agricultural production efficiency, reduce labor costs, and reduce the risk of human operation errors. The fault-tolerant formation control solution of the plant protection drone swarm system can ensure that the entire system can still operate normally by reallocating tasks and adjusting control strategies when some intelligent agents fail, thereby ensuring the stability and continuity of agricultural production.

[0114] Optimize resource allocation and reduce waste. The designed ETCRF controller is updated only at the triggering moment, so that the plant protection drone swarm system can calculate in real time and save communication resources. The system can automatically adjust the operation parameters according to the monitoring data, and can realize the precise management and dynamic scheduling of agricultural resources, thereby achieving the optimal allocation and efficient utilization of resources.

[0115] Responding to complex and changing environments. The plant protection drone swarm system can adapt to different complex environments by dynamically adjusting the formation and control strategy, and through collaborative operations and intelligent decision-making, it can reduce the impact of disasters on agricultural production and improve the disaster resistance of agricultural production.

[0116] In this embodiment, in step 1, describing the interconnection between agents using undirected communication topology specifically includes the following steps:

[0117] use in A set of nodes representing the agent itself;

[0118] is the set of edges, is the weighted adjacency matrix;

[0119] If and only if agent i and agent j are directly connected, then

[0120] And if Then α ij >0, if Then α ij = 0 and

[0121] make is the neighbor set of agent i, is the set of drone i and its neighbors,

[0122]

[0123] in A g The degree matrix of is the Laplace matrix;

[0124] in and

[0125] In this embodiment, the system model for following drone i in step 2 is as follows:

[0126]

[0127] in is the system status;

[0128] is the control input of the ith follower drone;

[0129] is the uncertainty of actuator fault matching;

[0130] and is a nonlinear system function.

[0131] In this embodiment, the system function and Including I f,i The compact set Ω with (0) = 0 is Lipschitz continuous, and the system (1) is stable on Ω;

[0132] Dynamic uncertainty satisfy in is a known function and The dynamics of the pilot drone can be expressed as:

[0133]

[0134] in is the system status;

[0135] is a differentiable function.

[0136] In this embodiment, the dynamic error calculation in step 3 includes the following:

[0137] (1) The nominal form of the formula is given by:

[0138]

[0139] The formation error of the i-th following drone is provided as:

[0140]

[0141] The vector constant represents the formation mode between UAV i and the pilot UAV, and c i≥0 indicates pinning gain, so the dynamics of the error formation are calculated as:

[0142]

[0143]

[0144] The new value function of the i-th following drone is expressed as:

[0145]

[0146] where γ is a positive constant, Utility Function Defined as

[0147]

[0148] where θ is a positive constant, and is a positive definite matrix.

[0149] In this embodiment, the Hami value of following drone i is expressed as:

[0150]

[0151] The discounted optimal value function of following drone i is:

[0152]

[0153] Satisfies the HJB equation:

[0154]

[0155] in is a set of acceptable controls, and the optimal formation control law is obtained:

[0156]

[0157] According to formula (8) and formula (9), the coupled HJB equation can be obtained as follows:

[0158]

[0159] In this embodiment, in step six, in the event triggering mechanism, the sampling state of the following drone i is expressed as:

[0160]

[0161] where Γ k represents the kth sampling moment. Therefore, the corresponding sampling error is:

[0162]

[0163] The event trigger error is expressed as:

[0164]

[0165] Based on formulas (9), (11) and (12), the ETCOF control law of the following UAV i is:

[0166]

[0167] And the system function is bounded, that is:

[0168]

[0169] in is a normal number.

[0170] Based on the ETCOF control law provided by formula (1), formula (2), formula (15) and formula (14) for the following UAV with dynamic uncertainty, if the trigger condition is met:

[0171]

[0172]

[0173] Where λ is the design parameter, then the formation error of following UAV i is asymptotically stable.

[0174] In this embodiment, in step 7, in the context of the neural network architecture, the optimal value function of following drone i is approximately:

[0175]

[0176] in is the ideal weight vector, is the activation function, h c is the number of neurons in the hidden layer, is the approximation error. Further,

[0177]

[0178] The approximate value function is expressed as:

[0179]

[0180] in represents an estimate of the optimal weights, and we then obtain:

[0181]

[0182] According to formula (14) and formula (17), the approximate ETCOF control law is rewritten as:

[0183]

[0184] Combining formula (14) and formula (19), the approximate ETCOF control law is:

[0185]

[0186] According to formula (10) and formula (22), the approximate Hamiltonian is:

[0187]

[0188] By minimizing the objective function The weight adjustment law for judging the neural network is:

[0189]

[0190] where ξ c >0 and the learning rate is:

[0191]

[0192] Considering the formula (3) for the i-th nominal following drone, the weight update rule of the critic neural network is formula (24), then the weight estimation error of the critic neural network is It has uniformly eventual boundedness.

[0193] The above are only further embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and concepts of the present invention within the scope disclosed by the present invention, which belong to the protection scope of the present invention.

Claims

1. The plant protection UAV cluster self-learning optimization formation control method is characterized by: The steps include: Step 1: Based on a plant protection drone swarm with one leader and K followers, an undirected communication topology is used to describe the interconnection between agents; Step 2: Establish a system model for following UAV i; Step 3: Establish formation dynamic error calculation based on the system model of the following UAV i; Step 4: Establish the discounted optimal value function of the following UAV i and satisfy the HJB equation to obtain the optimal formation control law; Step 5: A reinforcement learning-based ETCRF control scheme is used to approximate the solution of the coupled HJB equations, ensuring that all uncertain followers accurately track the leader trajectory in the specified formation; Step 6: Design an event-triggered robust formation controller; Step 7: Get an approximation of the optimal value function for following drone i in the context of the neural network architecture.

2. The plant protection UAV cluster self-learning optimization formation control method according to claim 1, characterized in that: In step 1, the undirected communication topology is used to describe the interconnection between agents, which specifically includes the following steps: use in A set of nodes representing the agent itself; is the set of edges, is the weighted adjacency matrix; If and only if agent i and agent j are directly connected, then And if Then α ij >0, if Then α ij =0 and make is the neighbor set of agent i, is the set of drone i and its neighbors, in A g The degree matrix of is the Laplace matrix; in and 3. The plant protection UAV cluster self-learning optimization formation control method according to claim 1, characterized in that: Step 2: The system model for following drone i is as follows: in is the system status; is the control input of the ith follower drone; is the uncertainty of actuator fault matching; and is a nonlinear system function.

4. The plant protection UAV cluster self-learning optimization formation control method according to claim 3 is characterized by: System functions and Including I f,i The compact set Ω with (0) = 0 is Lipschitz continuous, and the system (1) is stable on Ω; Dynamic uncertainty satisfy in is a known function and The dynamics of the pilot drone can be expressed as: in is the system status; is a differentiable function.

5. The plant protection UAV cluster self-learning optimization formation control method according to claim 4, characterized in that: The dynamic error calculation in step 3 includes the following: (1) The nominal form of the formula is given by: The formation error of the i-th following drone is provided as: The vector constant represents the formation mode between UAV i and the pilot UAV, and c i ≥0 indicates pinning gain, so the dynamics of the error formation are calculated as: The new value function of the i-th following drone is expressed as: where γ is a positive constant, Utility Function Defined as where θ is a positive constant, and is a positive definite matrix.

6. The plant protection UAV cluster self-learning optimization formation control method according to claim 5, characterized in that: The Hami quantity following UAV i is expressed as: The discounted optimal value function of following drone i is: Satisfies the HJB equation: in is a set of acceptable controls, and the optimal formation control law is obtained: According to formula (8) and formula (9), the coupled HJB equation can be obtained as follows:

7. The plant protection UAV cluster self-learning optimization formation control method according to claim 6, characterized in that: In step 6, in the event triggering mechanism, the sampling state of the following drone i is expressed as: where Γ k represents the kth sampling moment, and the corresponding sampling error is: The event trigger error is expressed as: Based on formulas (9), (11) and (12), the ETCOF control law of the following UAV i is: And the system function is bounded, that is: in is a positive constant; Based on the ETCOF control law provided by formula (1), formula (2), formula (15) and formula (14) for the following UAV with dynamic uncertainty, if the trigger condition is met: Where λ is the design parameter, then the formation error of following UAV i is asymptotically stable.

8. The plant protection UAV cluster self-learning optimization formation control method according to claim 7, characterized in that: In step 7, in the context of the neural network architecture, the optimal value function for following drone i is approximated as: in is the ideal weight vector, is the activation function, h c is the number of neurons in the hidden layer, To approximate the error, we further have: The approximate value function is expressed as: in represents an estimate of the optimal weights, and we then obtain: According to formula (14) and formula (17), the approximate ETCOF control law is rewritten as: Combining formula (14) and formula (19), the approximate ETCOF control law is: According to formula (10) and formula (22), the approximate Hamiltonian is: By minimizing the objective function The weight adjustment law for judging the neural network is: where ξ c >0 and the learning rate is: Considering the formula (3) for the i-th nominal following drone, the weight update rule of the critic neural network is formula (24), then the weight estimation error of the critic neural network is It has the property of being uniformly eventually bounded.

Images