UAV-UGV optimal control method and system under preset time

Abstract

The invention discloses a UAV-UGV optimal control method and system under preset time, and relates to the technical field of tracking control, and the method comprises the following steps: obtaining UAV-UGV related data, inputting the UAV-UGV related data into a pre-established nonlinear kinetic model of a UAV and a UGV, and outputting to obtain a UAV-UGV error system, the pre-established nonlinear dynamic model of the UAV and the UGV is constructed on the basis of a communication topological structure of the UAV-UGV; and performing heterogeneous formation tracking neighbor error calculation on the UAV-UGV error system based on a reinforcement learning mode, solving an HJB equation to obtain an optimal control input sequence, inputting the optimal control input sequence into a pre-established preset time reinforcement learning controller based on a neural network, and outputting to obtain UAV and UGV optimal formation preset time.

Description

Technical Field

[0001] The present invention relates to the technical field of tracking control, and in particular to a UAV-UGV optimal control method and system under a preset time. Background Art

[0002] With the rapid development of unmanned aerial vehicle (UAV) and unmanned vehicle (UGV) technologies, UAV-UGV collaborative formation control has gradually become an important research direction in the fields of intelligent transportation, logistics distribution, border patrol, etc. In this type of system, UAVs and UGVs each have different dynamic characteristics. UAVs have air maneuverability and can quickly reach the target point; UGVs are responsible for ground tasks such as load transportation and data collection. How to achieve efficient collaboration and formation tracking of UAVs and UGVs in a heterogeneous environment has become an important research topic.

[0003] In UAV-UGV formation control, the system needs to face challenges such as heterogeneity, nonlinear dynamic models and communication delays. In view of these problems, traditional control methods are difficult to meet the requirements of efficient and accurate control. Summary of the invention

[0004] In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide a UAV-UGV optimal control method and system under a preset time, which can cope with the different dynamic characteristics of UAV and UGV and ensure efficient and stable synchronous tracking control in complex environments.

[0005] In a first aspect, the purpose of the present invention can be achieved by the following technical solution: a UAV-UGV optimal control method under a preset time, the method comprising the following steps:

[0006] Acquire UAV-UGV related data, input the UAV-UGV related data into a pre-established nonlinear dynamic model of the UAV and the UGV, and output a UAV-UGV error system, wherein the pre-established nonlinear dynamic model of the UAV and the UGV is constructed based on a communication topology structure of the UAV-UGV;

[0007] The heterogeneous formation tracking neighbor errors of the UAV-UGV error system are calculated based on reinforcement learning. The optimal control input sequence is obtained by solving the HJB equation. The optimal control input sequence is input into a pre-established neural network-based preset time reinforcement learning controller, and the optimal formation preset time of UAV and UGV is obtained as output.

[0008] In combination with the first aspect, in some implementations of the first aspect, the method further includes: a method for implementing the UAV-UGV-based communication topology structure is:

[0009] The UAV-UGV system uses an undirected graph G to describe its communication network, which contains n UGVs and m UAVs, and the nodes are connected by non-negative elements ω ij To communicate; defined as follows:

[0010] The adjacency matrix is ​​defined as follows:

[0011] Λ=[ω ij ]∈R (n+m)×(n+m)

[0012] Represents the direct communication relationship between nodes in the system. If there is communication between node i and node j, then ω ij >0, otherwise ω ij =0;

[0013] The in-degree matrix is ​​defined as follows:

[0014]

[0015] D represents the sum of the communication signals received by each node;

[0016] The Laplacian matrix is ​​defined as follows:

[0017] L=D-Λ

[0018] The pin matrix is ​​defined as follows:

[0019] B = diag{b i}

[0020] where b i Indicates the strength or weight of a communication link.

[0021] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: the pre-established nonlinear dynamic models of the UAV and UGV include a UGV dynamic model, a UAV dynamic model, a second-order model of the UAV-UGV heterogeneous formation, and defining a virtual leader;

[0022] The dynamic model of UGV is described by the following equation:

[0023]

[0024] in:

[0025] x ugvi and ugvi is the 2D position of the i-th UGV; θ ugvi is the azimuth of the i-th UGV. ugvi is the speed of the i-th UGV; w ugvi is the angular velocity of the i-th UGV; i∈Ω 1 ,Ω 1={1,2,…,n};

[0026] By introducing auxiliary variables, the control input model of UGV is obtained:

[0027] x ugvi =x ugvis +E ugvil cosθ ugvi

[0028] y ugvi =y ugvis +E ugvil sinθ ugvi (2)

[0029] Among them, E ugvil is the starting position of the UGV (x ugvis ,y ugvis ) to the current forward position;

[0030] The implementation method of building the UAV dynamics model is:

[0031] The dynamic model of UAV takes into account the effects of gravity and thrust. The following is a model of a quadrotor UAV:

[0032]

[0033] Among them, x uavi ,y uavi ,z uavi represents the position of the i-th UAV; φ uavi ,θ uavi ,ψ uavi are the roll, pitch and yaw angles of the i-th UAV; F i is the control thrust; m i is the mass of the UAV; τ xuavi ,τ yuavi ,τ zuavi is the aerodynamic damping coefficient specific to the i-th UAV; i∈Ω 2 ,Ω 2 ={n+1,n+2,…,n+m};

[0034] The implementation method of constructing the second-order model of UAV-UGV heterogeneous formation is:

[0035] The heterogeneous formation of UAVs and UGVs can be described by the following second-order dynamic model:

[0036] x i =σ xi

[0037]

[0038] Among them, x i ,y i is the position of the UAV or UGV; xi ,σ yi is an auxiliary variable;

[0039] For i∈Ω 1 ,

[0040] x i =x ugvi ,y i =y ugvi ,u xi =u xugvi ,

[0041] For i∈Ω 2 ,

[0042] x i =x uavi ,y i =y uavi ,u xi =u xuavi ,

[0043] The implementation method for defining a virtual leader is:

[0044] In the UAV-UGV formation, the dynamic model of the virtual leader is defined as:

[0045]

[0046] in:

[0047] p 0 =[p 0x ,p 0y ] T is the position of the virtual leader; q 0 =[q 0x ,q 0y ] T is the speed of the virtual leader; 0 =[χ 0x ,x 0y ] T is a known control signal.

[0048] In combination with the first aspect, in some implementations of the first aspect, the method further includes: a method for implementing the UAV-UGV error system is:

[0049] The formation offset vector is constructed as follows:

[0050]

[0051] in, represents a known bounded input, δ i1 and δ i2 Represents the state variable and is defined as follows:

[0052]

[0053] Define the tracking error of UAV and UGV, including position error and velocity error:

[0054] Position error:

[0055]

[0056] Speed ​​error:

[0057]

[0058] in, and is the actual position of the i-th node; p 0 is the position of the virtual leader; q 0 It is the speed of the virtual leader;

[0059] According to the defined tracking error, a dynamic model of state error is constructed:

[0060]

[0061] Combining the tracking error and velocity error, the error state system equation is formed:

[0062]

[0063] in,

[0064]

[0065] In combination with the first aspect, in certain implementations of the first aspect, the method also includes: the calculation of the heterogeneous formation tracking neighbor error of the UAV-UGV error system based on reinforcement learning includes defining the heterogeneous formation tracking neighbor error based on reinforcement learning, defining the optimal distributed performance indicator, and solving the HJB equation.

[0066] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the implementation method of defining the heterogeneous formation tracking neighbor error based on the reinforcement learning method is:

[0067] The tracking neighbor error for the i-th UAV / UGV is defined as follows:

[0068] Position neighbor error:

[0069] Speed ​​Neighbor Error:

[0070] make have to:

[0071] Combined neighbor error:

[0072] make have to:

[0073] Dynamically combined neighbor error:

[0074] The implementation method for defining the best distributed performance indicators is:

[0075] Based on formula (14), the distributed performance index is defined as follows:

[0076]

[0077] in,

[0078]

[0079] Based on formula (15), the optimal distributed performance index is defined as follows:

[0080]

[0081] The HJB equation is:

[0082] By taking the time derivative of (14) on both sides of equation (16), the HJB equation of the distribution is obtained as follows:

[0083]

[0084] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the process of obtaining the optimal control input sequence by solving the HJB equation is as follows:

[0085] Optimal control input sequence is uniquely associated with the best distributed performance index function (16), so the optimal control input sequence is required to be the unique control solution of the distributed HJB equation (17), pass To solve, the solution is as follows:

[0086]

[0087] That is, the optimal control input is:

[0088]

[0089] The optimal control input Transformed into:

[0090]

[0091] Among them, k 1 and k 2 is a definite non-negative number, and ψ is an auxiliary function defined as follows:

[0092]

[0093] Time-varying function is defined as follows:

[0094]

[0095] Unknown function Cannot be solved directly. Using NNs to represent:

[0096]

[0097] in, Is the ideal weight for drones; W i is the basis function vector; ξ i is the approximate error.

[0098] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: the neural network-based preset time reinforcement learning controller includes: a preset time-based critic network, an actor network, and an adaptive update rate;

[0099] The implementation process of the reviewer network based on preset time:

[0100] The critic network is designed as follows:

[0101]

[0102] in, It is the critic’s output; represents the critic's adaptive parameter matrix;

[0103] The actor network is designed as follows:

[0104]

[0105] in, represents the optimized control input; represents the actor adaptation parameter matrix;

[0106] The adaptive update rate of the critic network is designed as follows:

[0107]

[0108] Among them, γ ci is a constant obtained by the commentator;

[0109] The adaptive update rate of the actor network is designed as follows:

[0110]

[0111] Among them, γ ai is a constant obtained by the actor.

[0112] In combination with the first aspect, in some implementations of the first aspect, the method further includes: for the UAV-UGV formation system, when optimizing the tracking control input Under the action of , the critic is calculated through the critic-actor framework of reinforcement learning and updated according to the update rule (24); the actor is adjusted according to the update rule (25), and its control input is given by formula (23), and the parameters a, k 1 ,k 2 ,γ ai ,γ ci The system achieves the expected effect if the following specific conditions are met:

[0113]

[0114] (k 1 -k 2 -1)ψ min -(k 1 +k 2 )a>0

[0115]

[0116] 2(a+ψ min +2)(k 1 -1)-(4-a-ψ min )(k 1 +k 2 )(a+ψ min )>0 (26)

[0117] Among them, ψ min is the minimum value of ψ; express The minimum eigenvalue of ;

[0118]

[0119] System error f xi and f vi Under the optimal control (22), convergence is achieved and each UAV and UGV can achieve synchronous tracking of position and velocity within the preset time T.

[0120] In a second aspect, in order to achieve the above-mentioned object, the present invention discloses a UAV-UGV optimal control system under a preset time, comprising:

[0121] A model generation module, used for acquiring UAV-UGV related data, inputting the UAV-UGV related data into a pre-established nonlinear dynamic model of the UAV and UGV, and outputting a UAV-UGV error system, wherein the pre-established nonlinear dynamic model of the UAV and UGV is constructed based on a communication topology structure of the UAV-UGV;

[0122] The preset time module is used to calculate the heterogeneous formation tracking neighbor error of the UAV-UGV error system based on reinforcement learning. The optimal control input sequence is obtained by solving the HJB equation, and the optimal control input sequence is input into the pre-established neural network-based preset time reinforcement learning controller to output the optimal formation preset time of UAV and UGV.

[0123] Beneficial effects of the present invention:

[0124] The present invention defines the communication topology of the UAV-UGV system, and constructs the nonlinear dynamic models of UAV and UGV respectively. Combined with the second-order dynamic model of heterogeneous formations, an error system suitable for heterogeneous environments is designed to describe the dynamic behavior and error feedback of each node in the formation. In terms of control design, the preset time function is innovatively introduced into the reinforcement learning algorithm, and an optimal controller for UAV-UGV under preset time is proposed. By constructing and solving the HJB equation, the optimal control input sequence is obtained to ensure that the system can achieve the synchronous tracking target within the preset time. This design not only improves the response speed of the control system, but also ensures that the dynamic collaboration of UAV and UGV can be completed accurately and on time. In order to further optimize the performance of the controller, the invention also introduces a critic-actor framework based on neural networks. Through this framework, the system can adaptively adjust the control parameters, the critic network is responsible for evaluating the effect of the control strategy, and the actor network optimizes the control input according to the evaluation results. This dynamic adjustment mechanism ensures that the system can continuously optimize the control strategy and maintain the stability and accuracy of the formation in the face of external interference or uncertainty. BRIEF DESCRIPTION OF THE DRAWINGS

[0125] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0126] Figure 1 It is a schematic flow chart of the method of the present invention;

[0127] Figure 2 It is a schematic diagram of the system structure of the present invention;

[0128] Figure 3 is an experimental diagram of the UAV-UAV space trajectory of this embodiment;

[0129] Figure 4 1 and 2 are trajectory error diagrams of the UAV-UAV of this embodiment on the x-axis and y-axis respectively. DETAILED DESCRIPTION

[0130] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0131] Embodiment 1:

[0132] The following is an introduction to the relevant terms involved in the embodiments of the present application:

[0133] Unmanned aerial vehicles (UAVs) are unmanned aircraft that are controlled by radio remote control equipment and self-contained program control devices. UAVs are actually a general term for unmanned aerial vehicles. From a technical point of view, they can be divided into: unmanned fixed-wing aircraft, unmanned vertical take-off and landing aircraft, unmanned airships, unmanned helicopters, unmanned multi-rotor aircraft, unmanned paragliders, etc. Compared with manned aircraft, they have the advantages of small size, low cost, easy use, low requirements for combat environment, and strong battlefield survivability.

[0134] like Figure 1 As shown, a UAV-UGV optimal control method under a preset time is characterized in that the method includes the following steps:

[0135] S101: Acquire UAV-UGV related data, input the UAV-UGV related data into a pre-established nonlinear dynamic model of the UAV and UGV, and output a UAV-UGV error system, wherein the pre-established nonlinear dynamic model of the UAV and UGV is constructed based on a communication topology structure of the UAV-UGV;

[0136] Defining UAV-UGV communication topology

[0137] The UAV-UGV system uses an undirected graph G to describe its communication network, which contains n UGVs and m UAVs, and the nodes are connected by non-negative elements ω ij To communicate. The specific definitions are as follows:

[0138] The adjacency matrix is ​​defined as follows:

[0139] Λ=[ω ij ]∈R (n+m)×(n+m)

[0140] Represents the direct communication relationship between nodes in the system. If there is communication between node i and node j, then ω ij >0, otherwise ω ij =0.

[0141] The in-degree matrix is ​​defined as follows:

[0142]

[0143] Represents the sum of the communication signals received by each node.

[0144] The Laplacian matrix is ​​defined as follows:

[0145] L=D-Λ

[0146] The pin matrix is ​​defined as follows:

[0147] B = diag{b i}

[0148] where b i Indicates the strength or weight of a communication link, and is usually used to measure the ability of information exchange between nodes.

[0149] Pre-established nonlinear dynamic models of UAVs and UGVs include UGV dynamics model, UAV dynamics model, second-order model of UAV-UGV heterogeneous formation, and definition of virtual leaders;

[0150] Building a UGV dynamics model

[0151] The dynamic model of UGV can be described by the following equation:

[0152]

[0153] in:

[0154] x ugvi and ugvi is the 2D position of the i-th UGV; θ ugvi is the azimuth of the i-th UGV. ugvi is the speed of the i-th UGV; w ugviis the angular velocity of the i-th UGV; i∈Ω 1 ,Ω 1 ={1,2,…,n}.

[0155] Furthermore, by introducing auxiliary variables, the control input model of UGV can be obtained:

[0156] x ugvi =x ugvis +E ugvil cosθ ugvi

[0157] y ugvi =y ugvis +E ugvil sinθ ugvi (28)

[0158] Among them, E ugvil is the starting position of the UGV (x ugvis ,y ugvis ) to the current forward position.

[0159] 3. Constructing UAV dynamics model

[0160] The dynamic model of UAV usually takes into account the effects of gravity and thrust. The following is a model of a quadrotor UAV:

[0161]

[0162] Among them, x uavi ,y uavi ,z uavi represents the position of the i-th UAV; φ uavi ,θ uavi ,ψ uavi are the roll, pitch and yaw angles of the i-th UAV; F i is the control thrust; m i is the mass of the UAV; τ xuavi ,τ yuavi ,τ zuavi is the aerodynamic damping coefficient specific to the i-th UAV; i∈Ω 2 ,Ω 2 ={n+1,n+2,…,n+m}.

[0163] 4. Constructing a Second-Order Model of UAV-UGV Heterogeneous Formation

[0164] The heterogeneous formation of UAVs and UGVs can be described by the following second-order dynamic model:

[0165] x i =σ xi

[0166]

[0167] Among them, x i ,y i is the position of the UAV or UGV; xi ,σ yi is an auxiliary variable.

[0168] For i∈Ω 1 ,

[0169] x i =x ugvi ,y i =y ugvi ,u xi =u xugvi ,a i =1,

[0170] For i∈Ω 2 ,

[0171] x i =x uavi ,y i =y uavi ,u xi =u xuavi ,

[0172] 5. Defining Virtual Leaders:

[0173] In a UAV-UGV formation, the dynamic model of the virtual leader can be defined as:

[0174]

[0175] in:

[0176] p 0 =[p 0x ,p 0y ] T is the position of the virtual leader; q 0 =[q 0x ,q 0y ] T is the speed of the virtual leader; 0 =[χ 0x ,x 0y ] T is a known control signal.

[0177] Building a UAV-UGV Error System

[0178] The formation offset vector is constructed as follows:

[0179]

[0180] in, represents a known bounded input, δ i1 and δ i2 Represents the state variable, which is defined as follows:

[0181]

[0182] Define the tracking error of UAV and UGV, including position error and velocity error:

[0183] Position error:

[0184]

[0185] Speed ​​error:

[0186]

[0187] in, and is the actual position of the i-th node; p 0 is the position of the virtual leader; q 0 is the speed of the virtual leader.

[0188] According to the defined tracking error, a dynamic model of state error is constructed:

[0189]

[0190] Combining the tracking error and velocity error, the error state system equation is formed:

[0191]

[0192] in,

[0193]

[0194] S102: Calculate the heterogeneous formation tracking neighbor error of the UAV-UGV error system based on reinforcement learning, obtain the optimal control input sequence by solving the HJB equation, input the optimal control input sequence into a pre-established neural network-based preset time reinforcement learning controller, and output the optimal formation preset time of the UAV and UGV.

[0195] The calculation of heterogeneous formation tracking neighbor error for UAV-UGV error system based on reinforcement learning includes defining heterogeneous formation tracking neighbor error based on reinforcement learning, defining optimal distributed performance indicators and solving HJB equations.

[0196] The implementation method of heterogeneous formation tracking neighbor error based on reinforcement learning is defined as:

[0197] The tracking neighbor error for the i-th UAV / UGV is defined as follows:

[0198] Position neighbor error:

[0199] Speed ​​Neighbor Error:

[0200] make have to:

[0201] Combined neighbor error:

[0202] make have to:

[0203] Dynamically combined neighbor error:

[0204] The implementation method for defining the best distributed performance indicators is:

[0205] Based on formula (14), the distributed performance index is defined as follows:

[0206]

[0207] in,

[0208]

[0209] Based on formula (15), the optimal distributed performance index is defined as follows:

[0210]

[0211] The HJB equation is:

[0212] By taking the time derivative of (14) on both sides of equation (16), the HJB equation of the distribution is obtained as follows:

[0213]

[0214] The process of obtaining the optimal control input sequence by solving the HJB equation is as follows:

[0215] Optimal control input sequence is uniquely associated with the best distributed performance index function (16), so the optimal control input sequence is required to be the unique control solution of the distributed HJB equation (17), pass To solve, the solution is as follows:

[0216]

[0217] That is, the optimal control input is:

[0218]

[0219] The optimal control input Transformed into:

[0220]

[0221] Among them, k 1 and k 2 is a definite non-negative number, and ψ is an auxiliary function defined as follows:

[0222]

[0223] Time-varying function is defined as follows:

[0224]

[0225] Unknown function Cannot be solved directly. Using NNs to represent:

[0226]

[0227] in, Is the ideal weight for drones; W i is the basis function vector; ξ i is the approximate error.

[0228] The preset time reinforcement learning controller based on neural network includes: a critic network based on preset time, an actor network and an adaptive update rate;

[0229] The implementation process of the reviewer network based on preset time:

[0230] The critic network is designed as follows:

[0231]

[0232] in, It is the critic’s output; represents the critic's adaptive parameter matrix;

[0233] The actor network is designed as follows:

[0234]

[0235] in, represents the optimized control input; represents the actor adaptation parameter matrix;

[0236] The adaptive update rate of the critic network is designed as follows:

[0237]

[0238] Among them, γ ci is a constant obtained by the commentator;

[0239] The adaptive update rate of the actor network is designed as follows:

[0240]

[0241] Among them, γ ai is a constant obtained by the actor.

[0242] For UAV-UGV formation system, optimizing tracking control input Under the action of , the critic is calculated through the critic-actor framework of reinforcement learning and updated according to the update rule (24); the actor is adjusted according to the update rule (25), and its control input is given by formula (23), and the parameters a, k 1 ,k 2 ,γ ai ,γ ci The system achieves the expected effect if the following specific conditions are met:

[0243]

[0244] (k 1 -k 2 -1)ψ min -(k 1 +k 2 )a>0

[0245]

[0246] 2(a+ψ min +2)(k 1 -1)-(4-a-ψ min )(k 1 +k 2 )(a+ψ min )>0 (52)

[0247] Among them, ψ min is the minimum value of ψ; express The minimum eigenvalue of ;

[0248]

[0249] System error f xi and f viUnder the optimal control (22), convergence is achieved and each UAV and UGV can achieve synchronous tracking of position and velocity within the preset time T.

[0250] Specifically, Figure 3 As shown, the green line and fluorescent green line are drones, the blue line and red line are unmanned vehicles, and the purple line is the overall trajectory.

[0251] like Figure 4 As shown in the figure, the green and fluorescent green lines are UAVs, and the blue and red lines are unmanned vehicles. As can be seen from the figure, the trajectories of the UAV and the unmanned vehicle overlap quickly, and the error tends to 0.

[0252] Embodiment 2: In the second aspect, as Figure 2 As shown, in order to achieve the above-mentioned purpose, the present invention discloses a UAV-UGV optimal control system under a preset time, comprising:

[0253] A model generation module 11 is used to obtain UAV-UGV related data, input the UAV-UGV related data into a pre-established nonlinear dynamic model of the UAV and UGV, and output a UAV-UGV error system, wherein the pre-established nonlinear dynamic model of the UAV and UGV is constructed based on the communication topology of the UAV-UGV;

[0254] The preset time module 12 is used to calculate the heterogeneous formation tracking neighbor error of the UAV-UGV error system based on reinforcement learning, obtain the optimal control input sequence by solving the HJB equation, input the optimal control input sequence into a pre-established neural network-based preset time reinforcement learning controller, and output the optimal formation preset time of the UAV and UGV.

[0255] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.

[0256] It needs to be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to execute the above method. The storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0257] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0258] The above shows and describes the basic principles, main features and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, the present disclosure may have various changes and improvements, and these changes and improvements fall within the scope of the present disclosure to be protected.

Claims

1. A UAV-UGV optimal control method under a preset time, characterized in that: The method comprises the following steps: Acquire UAV-UGV related data, input the UAV-UGV related data into a pre-established nonlinear dynamic model of the UAV and the UGV, and output a UAV-UGV error system, wherein the pre-established nonlinear dynamic model of the UAV and the UGV is constructed based on a communication topology structure of the UAV-UGV; The heterogeneous formation tracking neighbor errors of the UAV-UGV error system are calculated based on reinforcement learning. The optimal control input sequence is obtained by solving the HJB equation. The optimal control input sequence is input into a pre-established neural network-based preset time reinforcement learning controller, and the optimal formation preset time of UAV and UGV is obtained as output.

2. The UAV-UGV optimal control method under a preset time according to claim 1, characterized in that: The implementation method of the UAV-UGV based communication topology structure is: The UAV-UGV system uses an undirected graph G to describe its communication network, which contains n UGVs and m UAVs, and the nodes are connected by non-negative elements ω ij To communicate; defined as follows: The adjacency matrix is ​​defined as follows: L=[ω ij ]∈R (n+m)×(n+m) Λ represents the direct communication relationship between nodes in the system. If there is communication between node i and node j, then ω ij >0, otherwise ω ij =0; The in-degree matrix is ​​defined as follows: D represents the sum of the communication signals received by each node; The Laplacian matrix is ​​defined as follows: L=D-Λ The pin matrix is ​​defined as follows: B=diag{b i } where b i Indicates the strength or weight of a communication link.

3. The UAV-UGV optimal control method under a preset time according to claim 1, characterized in that: The pre-established nonlinear dynamic models of UAV and UGV include a UGV dynamic model, a UAV dynamic model, a second-order model of a UAV-UGV heterogeneous formation, and a definition of a virtual leader; The dynamic model of UGV is described by the following equation: in: x ugvi and ugvi is the 2D position of the i-th UGV; θ ugvi is the azimuth of the i-th UGV; v ugvi is the speed of the i-th UGV; w ugvi is the angular velocity of the i-th UGV; i∈Ω1,Ω1={1,2,…,n}; By introducing auxiliary variables, the control input model of UGV is obtained: x ugvi =x ugvis +E ugvil cosθ ugvi and ugvi =and ugvis +E ugvil sinθ ugvi (2) Among them, E ugvil is the starting position of the UGV (x ugvis ,y ugvis ) to the current forward position; The implementation method of building the UAV dynamics model is: The dynamic model of UAV takes into account the effects of gravity and thrust. The following is a model of a quadrotor UAV: Among them, x uavi ,y uavi ,z uavi represents the position of the i-th UAV; φ uavi ,θ uavi ,ψ uavi are the roll, pitch and yaw angles of the i-th UAV; F i is the control thrust; m i is the mass of the UAV; τ xuavi ,τ yuavi ,τ zuavi is the aerodynamic damping coefficient specific to the i-th UAV; i∈Ω2,Ω2={n+1,n+2,…,n+m}; The implementation method of constructing the second-order model of UAV-UGV heterogeneous formation is: The heterogeneous formation of UAVs and UGVs can be described by the following second-order dynamic model: Among them, x i ,y i is the position of the UAV or UGV; xi ,σ yi is an auxiliary variable; For i∈Ω1, For i∈Ω2, The implementation method for defining a virtual leader is: In the UAV-UGV formation, the dynamic model of the virtual leader is defined as: in: p0=[p 0x ,p 0y ] T is the position of the virtual leader; q0 = [q 0x ,q 0y ] T is the speed of the virtual leader; χ0=[χ 0x ,x 0y ] T is a known control signal.

4. The UAV-UGV optimal control method under a preset time according to claim 1, characterized in that: The implementation method of the UAV-UGV error system is: The formation offset vector is constructed as follows: in, represents a known bounded input, δ i1 and δ i2 Represents the state variable and is defined as follows: Define the tracking error of UAV and UGV, including position error and velocity error: Position error: Speed ​​error: in, and is the actual position of the ith node; p0 is the position of the virtual leader; q0 is the speed of the virtual leader; According to the defined tracking error, a dynamic model of state error is constructed: Combining the tracking error and velocity error, the error state system equation is formed: in, 5. The UAV-UGV optimal control method under a preset time according to claim 1, characterized in that: The calculation of heterogeneous formation tracking neighbor errors for the UAV-UGV error system based on reinforcement learning includes defining heterogeneous formation tracking neighbor errors based on reinforcement learning, defining optimal distributed performance indicators, and solving HJB equations.

6. The UAV-UGV optimal control method under a preset time according to claim 5, characterized in that: The implementation method of defining the heterogeneous formation tracking neighbor error based on reinforcement learning is: The tracking neighbor error for the i-th UAV / UGV is defined as follows: Position neighbor error: Speed ​​Neighbor Error: make have to: Combined neighbor error: make have to: Dynamically combined neighbor error: The implementation method for defining the best distributed performance indicators is: Based on formula (14), the distributed performance index is defined as follows: in, Based on formula (15), the optimal distributed performance index is defined as follows: The HJB equation is: By taking the time derivative of (14) on both sides of equation (16), the HJB equation of the distribution is obtained as follows:

7. The UAV-UGV optimal control method under a preset time according to claim 1, characterized in that: The process of obtaining the optimal control input sequence by solving the HJB equation is as follows: Optimal control input sequence is uniquely associated with the best distributed performance index function (16), so the optimal control input sequence is required to be the unique control solution of the distributed HJB equation (17), pass To solve, the solution is as follows: That is, the optimal control input is: The optimal control input Transformed into: Among them, k1 and k2 are certain non-negative numbers, and ψ is an auxiliary function defined as follows: Time-varying function is defined as follows: Unknown function Cannot be solved directly. Using NNs to represent: in, Is the ideal weight for drones; W i is the basis function vector; ξ i is the approximate error.

8. The UAV-UGV optimal control method under a preset time according to claim 7, characterized in that: The preset time reinforcement learning controller based on neural network includes: a critic network based on preset time, an actor network and an adaptive update rate; The implementation process of the reviewer network based on preset time: The critic network is designed as follows: in, It is the critic’s output; represents the critic's adaptive parameter matrix; The actor network is designed as follows: in, represents the optimized control input; represents the actor adaptation parameter matrix; The adaptive update rate of the critic network is designed as follows: Among them, γ ci is a constant obtained by the commentator; The adaptive update rate of the actor network is designed as follows: Among them, γ ai is a constant obtained by the actor.

9. The UAV-UGV optimal control method under a preset time according to claim 8, characterized in that: For UAV-UGV formation system, optimizing tracking control input Under the action of , the critic is calculated through the critic-actor framework of reinforcement learning and updated according to the update rule (24); the actor is adjusted through the update rule (25), and its control input is given by formula (23), and the parameters a, k1, k2, γ ai ,γ ci The system achieves the expected effect if the following specific conditions are met: Among them, ψ min is the minimum value of ψ; express The minimum eigenvalue of ; System error f xi and f vi Under the optimal control (22), convergence is achieved and each UAV and UGV can achieve synchronous tracking of position and velocity within the preset time T.

10. A UAV-UGV optimal control system under a preset time, characterized in that: include: A model generation module, used for acquiring UAV-UGV related data, inputting the UAV-UGV related data into a pre-established nonlinear dynamic model of the UAV and UGV, and outputting a UAV-UGV error system, wherein the pre-established nonlinear dynamic model of the UAV and UGV is constructed based on a communication topology structure of the UAV-UGV; The preset time module is used to calculate the heterogeneous formation tracking neighbor error of the UAV-UGV error system based on reinforcement learning. The optimal control input sequence is obtained by solving the HJB equation, and the optimal control input sequence is input into the pre-established neural network-based preset time reinforcement learning controller to output the optimal formation preset time of UAV and UGV.

Images