A method and system for optimal control of UAV-UGV under preset time

Abstract

The application discloses a UAV-UGV optimal control method and system under preset time, relates to the technical field of tracking control, and comprises the following steps: acquiring UAV-UGV related data, inputting the UAV-UGV related data into a pre-established nonlinear dynamics model of the UAV and the UGV, and outputting to obtain a UAV-UGV error system, wherein the pre-established nonlinear dynamics model of the UAV and the UGV is constructed based on a communication topology structure of the UAV-UGV; calculating a heterogeneous formation tracking neighbor error of the UAV-UGV error system based on a reinforcement learning mode, obtaining an optimal control input sequence by solving an HJB equation, inputting the optimal control input sequence into a pre-established preset time reinforcement learning controller based on a neural network, and outputting to obtain a UAV and UGV optimal formation preset time.

Description

Technical Field

[0001] This invention relates to the field of tracking control technology, specifically to an optimal control method and system for UAV-UGV under a preset time. Background Technology

[0002] With the rapid development of unmanned aerial vehicles (UAVs) and unmanned vehicle (UGV) technologies, UAV-UGV cooperative formation control has gradually become an important research direction in fields such as intelligent transportation, logistics distribution, and border patrol. In these systems, UAVs and UGVs each possess different dynamic characteristics. UAVs have aerial maneuverability and can quickly reach target points, while UGVs are responsible for ground tasks such as load transportation and data collection. How to achieve efficient cooperation and formation tracking between UAVs and UGVs in heterogeneous environments has become an important research topic.

[0003] In UAV-UGV formation control, the system faces challenges such as heterogeneity, nonlinear dynamic models, and communication delays. Traditional control methods struggle to meet the requirements for efficient and precise control in addressing these issues. Summary of the Invention

[0004] To address the shortcomings mentioned in the background section, the present invention aims to provide an optimal control method and system for UAV-UGV under a preset time, which can cope with the different dynamic characteristics of UAV and UGV and ensure efficient and stable synchronous tracking control in complex environments.

[0005] Firstly, the objective of this invention can be achieved through the following technical solution: an optimal control method for UAV-UGV under a preset time, the method comprising the following steps:

[0006] Acquire UAV-UGV related data, input the UAV-UGV related data into a pre-established nonlinear dynamic model of UAV and UGV, and output the UAV-UGV error system. The pre-established nonlinear dynamic model of UAV and UGV is constructed based on the communication topology of UAV-UGV.

[0007] The neighbor error of the UAV-UGV error system is calculated by using reinforcement learning. The optimal control input sequence is obtained by solving the HJB equation. The optimal control input sequence is then input into a pre-established neural network-based preset time reinforcement learning controller, and the output is the preset time for the optimal formation of UAV and UGV.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the implementation method of the UAV-UGV-based communication topology is as follows:

[0009] The UAV-UGV system uses an undirected graph G to describe its communication network, which contains n UGVs and m UAVs. Nodes communicate with each other via non-negative elements ω. ij To communicate; defined as follows:

[0010] The adjacency matrix is ​​defined as follows:

[0011] Λ=[ω ij ]∈R (n+m)×(n+m)

[0012] ω represents the direct communication relationship between nodes in the system. If there is communication between node i and node j, then ω ij >0, otherwise ω ij =0;

[0013] The in-degree matrix is ​​defined as follows:

[0014]

[0015] D represents the sum of communication signals received by each node;

[0016] The Laplace matrix is ​​defined as follows:

[0017] L = D - Λ

[0018] The pin matrix is ​​defined as follows:

[0019] B = diag{b i}

[0020] Where b i Indicates the strength or weight of a communication link.

[0021] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the pre-established nonlinear dynamic model of the UAV and UGV includes the UGV dynamic model, the UAV dynamic model, the second-order model of the UAV-UGV heterogeneous formation, and the definition of a virtual leader;

[0022] The dynamic model of the UGV is described by the following equations:

[0023]

[0024] in:

[0025] x ugvi and y ugvi θ is the two-dimensional position of the i-th UGV; ugvi It is the azimuth angle of the i-th UGV. ugvi w is the velocity of the i-th UGV; ugvi It is the angular velocity of the i-th UGV; i∈Ω1,Ω1={1,2,…,n};

[0026] By introducing auxiliary variables, the control input model of UGV is obtained:

[0027] x ugvi =x ugvis +E ugvil cosθ ugvi

[0028] y ugvi =y ugvis +E ugvil sinθ ugvi (2)

[0029] Among them, E ugvil It starts from the beginning position of UGV (x ugvis ,y ugvis The distance to the current position;

[0030] The method for constructing the UAV dynamics model is as follows:

[0031] The dynamic model of a UAV takes into account the effects of gravity and thrust. The following is a model of a quadcopter UAV:

[0032]

[0033] Where, x uavi ,y uavi ,z uavi φ represents the position of the i-th UAV; uavi ,θ uavi ,ψ uavi These are the roll, pitch, and yaw angles of the i-th UAV, respectively; F i It controls the thrust; m i It is the mass of the UAV; τ xuavi ,τ yuavi ,τ zuavi It is the aerodynamic damping coefficient specific to the i-th UAV; i∈Ω2,Ω2={n+1,n+2,…,n+m};

[0034] The implementation method for constructing a second-order model of UAV-UGV heterogeneous formation is as follows:

[0035] The heterogeneous formation of UAVs and UGVs can be described by the following second-order dynamic model:

[0036] x i =σ xi

[0037]

[0038] Where, x i ,y i It refers to the location of the UAV or UGV; σxi ,σ yi It is an auxiliary variable;

[0039] For i∈Ω1,

[0040] x i =x ugvi ,y i =y ugvi ,u xi =u xugvi ,

[0041] For i∈Ω2,

[0042] x i =x uavi ,y i =y uavi ,u xi =u xuavi ,

[0043] The implementation method for a virtual leader is defined as follows:

[0044] In UAV-UGV formations, the dynamic model of the virtual leader is defined as follows:

[0045]

[0046] in:

[0047] p0 = [p 0x ,p 0y ] T This is the position of the virtual leader; q0 = [q 0x ,q 0y ] T It is the speed of the virtual leader; χ0 = [χ 0x ,χ 0y ] T It is a known control signal.

[0048] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the implementation method of the UAV-UGV error system is as follows:

[0049] The formation offset vector is constructed as follows:

[0050]

[0051] in, δ represents a known bounded input. i1 and δ i2 Representing state variables, defined as follows:

[0052]

[0053] Define the tracking errors for UAVs and UGVs, including position error and velocity error:

[0054] Position error:

[0055]

[0056] Speed ​​error:

[0057]

[0058] in, and p0 is the actual position of the i-th node; p0 is the position of the virtual leader; q0 is the speed of the virtual leader.

[0059] Based on the defined tracking error, construct a dynamic model of the state error:

[0060]

[0061] Combining the tracking error and the velocity error, we can construct the error state system equation:

[0062]

[0063] in,

[0064]

[0065] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the calculation of the heterogeneous formation tracking neighbor error of the UAV-UGV error system based on reinforcement learning includes defining the heterogeneous formation tracking neighbor error based on reinforcement learning, defining the optimal distributed performance index, and solving the HJB equation.

[0066] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the implementation method for defining the heterogeneous formation tracking neighbor error based on reinforcement learning is as follows:

[0067] The tracking neighbor error for the i-th UAV / UGV is defined as follows:

[0068] Location neighbor error:

[0069] Speed ​​Neighbor Error:

[0070] make have to:

[0071] Combined neighbor error:

[0072] make have to:

[0073] Dynamically combined neighbor error:

[0074] The optimal implementation method for distributed performance metrics is defined as follows:

[0075] Based on formula (14), the distributed performance index is defined as follows:

[0076]

[0077] in,

[0078]

[0079] Based on formula (15), the optimal distributed performance metric is defined as follows:

[0080]

[0081] The HJB equation is:

[0082] By taking the time derivative of (14) on both sides of equation (16), the HJB equation for the distribution is obtained as follows:

[0083]

[0084] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the process of obtaining the optimal control input sequence by solving the HJB equation is as follows:

[0085] Optimal control input sequence It has a unique correlation with the optimal distributed performance exponential function (16), therefore the optimal control input sequence It needs to be the unique governing solution of the distributed HJB equation (17). pass Let's find the solution, and the solution is as follows:

[0086]

[0087] That is, the optimal control input is:

[0088]

[0089] Optimal control input Transformed into:

[0090]

[0091] Where k1 and k2 are definite non-negative numbers, and ψ is an auxiliary function defined as follows:

[0092]

[0093] Time-varying function The definition is as follows:

[0094]

[0095] Unknown function It cannot be solved directly. Represented by NNs:

[0096]

[0097] in, This is the ideal weight for drones; W i It is a basis function vector; ξ i It is an approximation error.

[0098] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the neural network-based preset time reinforcement learning controller includes: a commentator network, an actor network, and an adaptive update rate based on a preset time;

[0099] The implementation process of a commenter network based on a preset time:

[0100] The commentator network design is as follows:

[0101]

[0102] in, It is the output of critics; This represents the critic's adaptive parameter matrix;

[0103] The actor network design is as follows:

[0104]

[0105] in, This represents the optimized control input; Represents the actor's adaptive parameter matrix;

[0106] The adaptive update rate design for the commenter network is as follows:

[0107]

[0108] Where, γ ci It is a constant obtained by the commentator;

[0109] The adaptive update rate design for the actor network is as follows:

[0110]

[0111] Where, γ ai It is a constant obtained by the actor.

[0112] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: for a UAV-UGV formation system, optimizing the tracking control input. Under the influence of this, the commentator performs calculations through the commentator-actor framework of reinforcement learning and updates according to the update rule (24); the actor adjusts according to the update rule (25), and its control input is given by formula (23), with parameters a, k1, k2, γ. ai ,γ ci The system will achieve the expected results if the following specific conditions are met:

[0113]

[0114] (k1-k2-1)ψ min -(k1+k2)a>0

[0115]

[0116] 2(a+ψ min +2)(k1-1)-(4-a-ψ min (k1+k2)(a+ψ) min )>0 (26)

[0117] Where, ψ min ψ is the minimum value; express The smallest eigenvalue;

[0118]

[0119] System error f xi and f vi Under optimized control (22), convergence is achieved, and each UAV and UGV can achieve synchronous tracking of position and velocity within a preset time T.

[0120] Secondly, in order to achieve the above objectives, the present invention discloses an optimal control system for UAV-UGV under a preset time, comprising:

[0121] The model generation module is used to acquire UAV-UGV related data, input the UAV-UGV related data into a pre-established nonlinear dynamic model of UAV and UGV, and output the UAV-UGV error system. The pre-established nonlinear dynamic model of UAV and UGV is constructed based on the communication topology of UAV-UGV.

[0122] The preset time module is used to calculate the neighbor error of the UAV-UGV error system in heterogeneous formation tracking based on reinforcement learning. It obtains the optimal control input sequence by solving the HJB equation, and inputs the optimal control input sequence into the pre-established neural network-based preset time reinforcement learning controller, and outputs the optimal formation preset time of UAV and UGV.

[0123] The beneficial effects of this invention are:

[0124] This invention defines the communication topology of a UAV-UGV system and constructs nonlinear dynamic models for both UAVs and UGVs. Combined with a second-order dynamic model of the heterogeneous formation, an error system suitable for heterogeneous environments is designed to describe the dynamic behavior and error feedback of each node in the formation. In terms of control design, a preset time function is innovatively introduced into the reinforcement learning algorithm, proposing an optimal UAV-UGV controller under a preset time. By constructing and solving the HJB equations, the optimal control input sequence is obtained, ensuring that the system can achieve synchronous tracking of the target within the preset time. This design not only improves the response speed of the control system but also ensures that the dynamic cooperation between UAVs and UGVs can be completed accurately and on time. To further optimize the controller's performance, the invention also introduces a neural network-based critic-actor framework. Through this framework, the system can adaptively adjust control parameters. The critic network is responsible for evaluating the effectiveness of the control strategy, and the actor network optimizes the control input based on the evaluation results. This dynamic adjustment mechanism ensures that the system can continuously optimize the control strategy and maintain the stability and accuracy of the formation when facing external disturbances or uncertainties. Attached Figure Description

[0125] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0126] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0127] Figure 2 This is a schematic diagram of the system structure of the present invention;

[0128] Figure 3 This is an experimental diagram of the spatial trajectory of the UAV-unmanned vehicle in this embodiment;

[0129] Figure 4 This is a trajectory error diagram of the UAV-unmanned vehicle in this embodiment on the x-axis and y-axis respectively. Detailed Implementation

[0130] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0131] Example 1:

[0132] The following is a description of the relevant terms used in the embodiments of this application:

[0133] Unmanned aerial vehicles (UAVs), also known as drones, are unmanned aircraft controlled by radio remote control equipment and their own programmed control devices. The term "UAV" is actually a general term for unmanned aerial vehicles, which, from a technical perspective, can be categorized as: unmanned fixed-wing aircraft, unmanned vertical takeoff and landing aircraft, unmanned airships, unmanned helicopters, unmanned multi-rotor aircraft, and unmanned paragliders, among others. Compared to manned aircraft, UAVs have advantages such as smaller size, lower cost, ease of use, lower requirements for the operational environment, and stronger battlefield survivability.

[0134] like Figure 1 As shown, an optimal control method for UAV-UGV under a preset time is characterized by the following steps:

[0135] S101: Obtain UAV-UGV related data, input the UAV-UGV related data into the pre-established nonlinear dynamic model of UAV and UGV, and output the UAV-UGV error system, wherein the pre-established nonlinear dynamic model of UAV and UGV is constructed based on the communication topology of UAV-UGV;

[0136] Define the communication topology of UAV-UGV

[0137] The UAV-UGV system uses an undirected graph G to describe its communication network, which contains n UGVs and m UAVs. Nodes communicate with each other via non-negative elements ω. ij To conduct communication. The specific definition is as follows:

[0138] The adjacency matrix is ​​defined as follows:

[0139] Λ=[ω ij ]∈R (n+m)×(n+m)

[0140] ω represents the direct communication relationship between nodes in the system. If there is communication between node i and node j, then ω ij >0, otherwise ω ij =0.

[0141] The in-degree matrix is ​​defined as follows:

[0142]

[0143] This represents the sum of communication signals received by each node.

[0144] The Laplace matrix is ​​defined as follows:

[0145] L = D - Λ

[0146] The pin matrix is ​​defined as follows:

[0147] B = diag{b i}

[0148] Where b i It represents the strength or weight of a communication link and is usually used to measure the ability of nodes to exchange information.

[0149] The pre-established nonlinear dynamic models of UAVs and UGVs include the UGV dynamic model, the UAV dynamic model, the second-order model of the UAV-UGV heterogeneous formation, and the definition of a virtual leader;

[0150] Constructing a UGV dynamic model

[0151] The dynamic model of the UGV can be described by the following equations:

[0152]

[0153] in:

[0154] x ugvi and y ugvi θ is the two-dimensional position of the i-th UGV; ugvi It is the azimuth angle of the i-th UGV. ugvi w is the velocity of the i-th UGV; ugvi It is the angular velocity of the i-th UGV; i∈Ω1,Ω1={1,2,…,n}.

[0155] Furthermore, by introducing auxiliary variables, the control input model of UGV can be obtained:

[0156] x ugvi =x ugvis +E ugvil cosθ ugvi

[0157] y ugvi =y ugvis +E ugvil sinθ ugvi (28)

[0158] Among them, Eugvil It starts from the beginning position of UGV (x ugvis ,y ugvis The distance to the current forward position.

[0159] III. Constructing a UAV Dynamics Model

[0160] The dynamics model of a UAV typically considers the effects of gravity and thrust. The following is a model of a quadcopter UAV:

[0161]

[0162] Where, x uavi ,y uavi ,z uavi φ represents the position of the i-th UAV; uavi ,θ uavi ,ψ uavi These are the roll, pitch, and yaw angles of the i-th UAV, respectively; F i It controls the thrust; m i It is the mass of the UAV; τ xuavi ,τ yuavi ,τ zuavi It is the aerodynamic damping coefficient specific to the i-th UAV; i∈Ω2,Ω2={n+1,n+2,…,n+m}.

[0163] IV. Constructing a Second-Order Model of UAV-UGV Heterogeneous Formation

[0164] The heterogeneous formation of UAVs and UGVs can be described by the following second-order dynamic model:

[0165] x i =σ xi

[0166]

[0167] Where, x i ,y i It refers to the location of the UAV or UGV; σ xi ,σ yi It is an auxiliary variable.

[0168] For i∈Ω1,

[0169] x i =x ugvi ,y i =y ugvi ,u xi =u xugvi ,a i =1,

[0170] For i∈Ω2,

[0171] x i =x uavi ,y i =y uavi ,u xi =u xuavi ,

[0172] V. Defining the Virtual Leader:

[0173] In a UAV-UGV formation, the dynamic model of the virtual leader can be defined as:

[0174]

[0175] in:

[0176] p0 = [p 0x ,p 0y ] T This is the position of the virtual leader; q0 = [q 0x ,q 0y ] T It is the speed of the virtual leader; χ0 = [χ 0x ,χ 0y ] T It is a known control signal.

[0177] Constructing a UAV-UGV error system

[0178] The formation offset vector is constructed as follows:

[0179]

[0180] in, δ represents a known bounded input. i1 and δ i2 This represents a state variable, specifically defined as follows:

[0181]

[0182] Define the tracking errors for UAVs and UGVs, including position error and velocity error:

[0183] Position error:

[0184]

[0185] Speed ​​error:

[0186]

[0187] in, and p0 is the actual position of the i-th node; p0 is the position of the virtual leader; q0 is the speed of the virtual leader.

[0188] Based on the defined tracking error, construct a dynamic model of the state error:

[0189]

[0190] Combining the tracking error and the velocity error, we can construct the error state system equation:

[0191]

[0192] in,

[0193]

[0194] S102: The neighbor error of the UAV-UGV error system is calculated based on the reinforcement learning method. The optimal control input sequence is obtained by solving the HJB equation. The optimal control input sequence is input into the pre-established neural network-based preset time reinforcement learning controller, and the output is the preset time of the optimal formation of UAV and UGV.

[0195] The calculation of neighbor tracking error in the UAV-UGV error system based on reinforcement learning includes defining the neighbor tracking error of the heterogeneous formation based on reinforcement learning, defining the optimal distributed performance index, and solving the HJB equation.

[0196] The implementation method for tracking neighbor errors in heterogeneous formations based on reinforcement learning is defined as follows:

[0197] The tracking neighbor error for the i-th UAV / UGV is defined as follows:

[0198] Location neighbor error:

[0199] Speed ​​Neighbor Error:

[0200] make have to:

[0201] Combined neighbor error:

[0202] make have to:

[0203] Dynamically combined neighbor error:

[0204] The optimal implementation method for distributed performance metrics is defined as follows:

[0205] Based on formula (14), the distributed performance index is defined as follows:

[0206]

[0207] in,

[0208]

[0209] Based on formula (15), the optimal distributed performance metric is defined as follows:

[0210]

[0211] The HJB equation is:

[0212] By taking the time derivative of (14) on both sides of equation (16), the HJB equation for the distribution is obtained as follows:

[0213]

[0214] The process of obtaining the optimal control input sequence by solving the HJB equation is as follows:

[0215] Optimal control input sequence It has a unique correlation with the optimal distributed performance exponential function (16), therefore the optimal control input sequence It needs to be the unique governing solution of the distributed HJB equation (17). pass Let's find the solution, and the solution is as follows:

[0216]

[0217] That is, the optimal control input is:

[0218]

[0219] Optimal control input Transformed into:

[0220]

[0221] Where k1 and k2 are definite non-negative numbers, and ψ is an auxiliary function defined as follows:

[0222]

[0223] Time-varying function The definition is as follows:

[0224]

[0225] Unknown function It cannot be solved directly. Represented by NNs:

[0226]

[0227] in, This is the ideal weight for drones; W i It is a basis function vector; ξ i It is an approximation error.

[0228] The neural network-based preset-time reinforcement learning controller includes: a preset-time commenter network, an actor network, and an adaptive update rate;

[0229] The implementation process of a commenter network based on a preset time:

[0230] The commentator network design is as follows:

[0231]

[0232] in, It is the output of critics; This represents the critic's adaptive parameter matrix;

[0233] The actor network design is as follows:

[0234]

[0235] in, This represents the optimized control input; Represents the actor's adaptive parameter matrix;

[0236] The adaptive update rate design for the commenter network is as follows:

[0237]

[0238] Where, γ ci It is a constant obtained by the commentator;

[0239] The adaptive update rate design for the actor network is as follows:

[0240]

[0241] Where, γ ai It is a constant obtained by the actor.

[0242] For UAV-UGV platooning systems, optimizing tracking control inputs is crucial. Under the influence of this, the commentator performs calculations through the commentator-actor framework of reinforcement learning and updates according to the update rule (24); the actor adjusts according to the update rule (25), and its control input is given by formula (23), with parameters a, k1, k2, γ. ai ,γ ciThe system will achieve the expected results if the following specific conditions are met:

[0243]

[0244] (k1-k2-1)ψ min -(k1+k2)a>0

[0245]

[0246] 2(a+ψ min +2)(k1-1)-(4-a-ψ min (k1+k2)(a+ψ) min )>0 (52)

[0247] Where, ψ min ψ is the minimum value; express The smallest eigenvalue;

[0248]

[0249] System error f xi and f vi Under optimized control (22), convergence is achieved, and each UAV and UGV can achieve synchronous tracking of position and velocity within a preset time T.

[0250] Specifically, such as Figure 3 As shown, the green and fluorescent green lines represent drones, the blue and red lines represent unmanned vehicles, and the purple line represents the overall trajectory.

[0251] like Figure 4 As shown in the figure, the green and fluorescent green lines represent drones, while the blue and red lines represent unmanned vehicles. The figure reveals that the trajectories of the drones and unmanned vehicles rapidly overlap, with the error approaching zero.

[0252] Example 2: Second aspect, such as Figure 2 As shown, in order to achieve the above objectives, this invention discloses an optimal control system for UAV-UGV under a preset time, comprising:

[0253] The model generation module 11 is used to acquire UAV-UGV related data, input the UAV-UGV related data into a pre-established nonlinear dynamic model of UAV and UGV, and output the UAV-UGV error system. The pre-established nonlinear dynamic model of UAV and UGV is constructed based on the communication topology of UAV-UGV.

[0254] The preset time module 12 is used to calculate the neighbor error of the UAV-UGV error system in heterogeneous formation tracking based on reinforcement learning. It obtains the optimal control input sequence by solving the HJB equation, inputs the optimal control input sequence into the pre-established preset time reinforcement learning controller based on neural network, and outputs the optimal formation preset time of UAV and UGV.

[0255] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.

[0256] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0257] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0258] The foregoing has shown and described the basic principles, main features, and advantages of this disclosure. Those skilled in the art should understand that this disclosure is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this disclosure. Various changes and modifications can be made to this disclosure without departing from its spirit and scope, and all such changes and modifications fall within the scope of this disclosure as claimed.

Claims

1. An optimal control method for UAV-UGV under a preset time, characterized in that, The method includes the following steps: Acquire UAV-UGV related data, input the UAV-UGV related data into a pre-established nonlinear dynamic model of UAV and UGV, and output the UAV-UGV error system. The pre-established nonlinear dynamic model of UAV and UGV is constructed based on the communication topology of UAV-UGV. The error of the UAV-UGV error system is calculated by heterogeneous formation tracking neighbor error based on reinforcement learning. The optimal control input sequence is obtained by solving the HJB equation. The optimal control input sequence is input into a pre-established neural network-based preset time reinforcement learning controller, and the output is the preset time of the optimal formation of UAV and UGV. The calculation of heterogeneous formation tracking neighbor error in the UAV-UGV error system based on reinforcement learning includes defining the heterogeneous formation tracking neighbor error based on reinforcement learning, defining the optimal distributed performance index, and solving the HJB equation. The implementation method for tracking neighbor errors in heterogeneous formations based on reinforcement learning is defined as follows: The tracking neighbor error for the i-th UAV / UGV is defined as follows: Location neighbor error: Speed ​​Neighbor Error: make have to: Combined neighbor error: make have to: Dynamically combined neighbor error: The optimal implementation method for distributed performance metrics is defined as follows: Based on formula (14), the distributed performance index is defined as follows: in, Based on formula (15), the optimal distributed performance metric is defined as follows: The HJB equation is: By taking the time derivative of equation (14) on both sides of equation (16), the HJB equation for the distribution is obtained as follows: The process of obtaining the optimal control input sequence by solving the HJB equation is as follows: Optimal control input sequence It has a unique association with the best distributed performance index (16), therefore the optimal control input sequence It needs to be the unique governing solution of the distributed HJB equation (17). pass Let's find the solution, and the solution is as follows: That is, the optimal control input is: Optimal control input Transformed into: Where k1 and k2 are definite non-negative numbers, and ψ is an auxiliary function defined as follows: Time-varying function The definition is as follows: Unknown function It cannot be solved directly. Represented by NNs: in, This is the ideal weight for drones; W i It is a basis function vector; ξ i It is an approximation error.

2. The optimal control method for UAV-UGV under a preset time according to claim 1, characterized in that, The implementation method of the communication topology based on UAV-UGV is as follows: The UAV-UGV system uses an undirected graph G to describe its communication network, which contains n UGVs and m UAVs. Nodes communicate with each other via non-negative elements ω. ij To communicate; defined as follows: The adjacency matrix is ​​defined as follows: L=[ω ij ]∈R (n+m)×(n+m) Λ represents the direct communication relationship between nodes in the system. If there is communication between node i and node j, then ω ij >0, otherwise ω ij =0; The in-degree matrix is ​​defined as follows: D represents the sum of communication signals received by each node; The Laplace matrix is ​​defined as follows: L=D-Λ The pin matrix is ​​defined as follows: B=diag{b i } Where b i Indicates the strength or weight of a communication link.

3. The optimal control method for UAV-UGV under a preset time according to claim 1, characterized in that, The pre-established nonlinear dynamic models of UAV and UGV include the UGV dynamic model, the UAV dynamic model, the second-order model of the UAV-UGV heterogeneous formation, and the definition of a virtual leader; The dynamic model of the UGV is described by the following equations: in: x ugvi and y ugvi θ is the two-dimensional position of the i-th UGV; ugvi It is the azimuth angle of the i-th UGV; v ugvi w is the velocity of the i-th UGV; ugvi It is the angular velocity of the i-th UGV; i∈Ω1,Ω1={1,2,…,n}; By introducing auxiliary variables, the control input model of UGV is obtained: x ugvi =x ugvis +E ugvil cosθ ugvi and ugvi / and ugvis +E ugvil sinθ ugvi (2) Among them, E ugvil It starts from the beginning position of UGV (x ugvis ,y ugvis The distance to the current position; The method for constructing the UAV dynamics model is as follows: The dynamic model of a UAV takes into account the effects of gravity and thrust. The following is a model of a quadcopter UAV: Where, x uavi ,y uavi ,z uavi φ represents the position of the i-th UAV; uavi ,θ uavi ,ψ uavi These are the roll, pitch, and yaw angles of the i-th UAV, respectively; F i It controls the thrust; m i It is the mass of the UAV; τ xuavi ,τ yuavi ,τ zuavi It is the aerodynamic damping coefficient specific to the i-th UAV; i∈Ω2,Ω2={n+1,n+2,…,n+m}; The implementation method for constructing a second-order model of UAV-UGV heterogeneous formation is as follows: The heterogeneous formation of UAVs and UGVs can be described by the following second-order dynamic model: x i =s xi Where, x i ,y i It refers to the location of the UAV or UGV; σ xi ,σ yi It is an auxiliary variable; For i∈Ω1, x i =x ugvi ,y i =y ugvi ,u xi =u xugvi ,a i =1, For i∈Ω2, x i =x uavi ,y i =y uavi ,u xi =u xuavi , The implementation method for a virtual leader is defined as follows: In UAV-UGV formations, the dynamic model of the virtual leader is defined as follows: in: p0 = [p 0x ,p 0y ] T This is the position of the virtual leader; q0 = [q 0x ,q 0y ] T It is the speed of the virtual leader; χ0 = [χ 0x ,χ 0y ] T It is a known control signal.

4. The optimal control method for UAV-UGV under a preset time according to claim 1, characterized in that, The implementation method of the UAV-UGV error system is as follows: The formation offset vector is constructed as follows: Where, θ i δ represents a known bounded input. i1 and δ i2 Representing state variables, defined as follows: Define the tracking errors for UAVs and UGVs, including position error and velocity error: Position error: Speed ​​error: in, and p0 is the actual position of the i-th node; p0 is the position of the virtual leader; q0 is the speed of the virtual leader. Based on the defined tracking error, construct a dynamic model of the state error: Combining the tracking error and the velocity error, we can construct the error state system equation: in, 5. The optimal control method for UAV-UGV under a preset time according to claim 1, characterized in that, The neural network-based preset time reinforcement learning controller includes: a commentator network, an actor network, and an adaptive update rate based on a preset time. The implementation process of a commenter network based on a preset time: The commentator network design is as follows: in, It is the output of critics; This represents the critic's adaptive parameter matrix; The actor network design is as follows: in, This represents the optimized control input; Represents the actor's adaptive parameter matrix; The adaptive update rate design for the commenter network is as follows: Where, γ ci It is a constant obtained by the commentator; The adaptive update rate design for the actor network is as follows: Where, γ ai It is a constant obtained by the actor.

6. The optimal control method for UAV-UGV under a preset time according to claim 5, characterized in that, For UAV-UGV platooning systems, optimizing tracking control inputs is crucial. Under the influence of this, the commentator performs calculations through the commentator-actor framework of reinforcement learning and updates according to formula (24); the actor is adjusted according to formula (25), and its control input is given by formula (23), with parameters a, k1, k2, γ. ai ,γ ci The system will achieve the expected results if the following specific conditions are met: (k1-k2-1)ψ min -(k1+k2)a>0 2(a+ψ min +2)(k1-1)-(4-a-ψ min )(k1+k2)(a+ψ min )>0 (26) Where, ψ min ψ is the minimum value; W i W i T The smallest eigenvalue; System error f xi and f vi Under optimized control, formula (22) converges, and each UAV and UGV can achieve synchronous tracking of position and velocity within a preset time T.

7. An optimal control system for a UAV-UGV under a preset time, employing the optimal control method for a UAV-UGV under a preset time as described in claim 1, characterized in that, include: The model generation module is used to acquire UAV-UGV related data, input the UAV-UGV related data into a pre-established nonlinear dynamic model of UAV and UGV, and output the UAV-UGV error system. The pre-established nonlinear dynamic model of UAV and UGV is constructed based on the communication topology of UAV-UGV. The preset time module is used to calculate the neighbor error of the UAV-UGV error system in heterogeneous formation tracking based on reinforcement learning. It obtains the optimal control input sequence by solving the HJB equation, and inputs the optimal control input sequence into the pre-established neural network-based preset time reinforcement learning controller, and outputs the optimal formation preset time of UAV and UGV.

Images