Optimal consensus method and device for approximate dynamic programming of heterogeneous multi-UAV data collaboration

Abstract

The present invention provides an optimal consistent method and device for approximate dynamic programming of heterogeneous multi-UAV data collaboration, and relates to the field of multi-agent control technology. The method comprises: obtaining the state data of the heterogeneous multi-UAV system, constructing an identifier with a compensation signal, approximating the dynamic model information of the heterogeneous multi-UAV system, constructing a compensator with a dynamic signal, and obtaining a weight update rule of a neural network with a compensator identification; constructing a Hamiltonian function according to the weight update rule; constructing a cumulative square error with a collaborative signal according to the Hamiltonian function; using a gradient descent method to obtain a weight update rule of a cumulative execution-collaborative judgment neural network, and calculating the constant weights of two adjacent iterations; judging whether a preset threshold condition is met according to the error of the constant weights of two adjacent iterations; if met, stopping iteration, and outputting the optimal consistent solution of the heterogeneous multi-UAV system. The present invention can improve the scalability of the algorithm and reduce the complexity of the algorithm.

Description

Technical Field

[0001] The present invention relates to the field of multi-agent control technology, and in particular to an optimal consensus method and device for heterogeneous multi-UAV data collaborative approximate dynamic programming. Background Art

[0002] In multi-agent systems, cooperative control has a wide range of applications, including drone formations, autonomous vehicle formations, industrial robots, and ocean exploration. Heterogeneous multi-agent system consensus control is a research method that enables a group of agents with different characteristics or functions, including robots, drones, and sensors, to work in a dynamic environment to achieve a common goal. In the multi-drone consensus work, each drone is an agent, and multiple drones work together to complete tasks, such as search, patrol, and monitoring; the hardware and communication capabilities of different drones may be heterogeneous. By optimizing the consistency problem, it is possible to ensure that the information sharing and task allocation between drones are consistent, so that the cluster can coordinate and complete tasks, including efficient search and area coverage tasks. Adaptive dynamic programming is a method that combines dynamic programming and machine learning, mainly used to solve decision-making and control problems. Its core idea is to adapt to changes in a dynamic environment through learning and optimization, and find the optimal decision strategy. Therefore, adaptive dynamic programming is a key technology for learning the optimal consensus control solution of heterogeneous multi-drone systems, and has been widely used in the optimal consensus problem of multi-drone systems.

[0003] The methods currently used to deal with the optimal consensus problem of heterogeneous multi-UAV systems include: (1) a distributed optimization collaboration protocol for heterogeneous nonlinear systems on a directed graph with completely unknown models. The parameters of the judgment-execution neural network are adjusted through a dynamic trigger mechanism to determine the rules of non-periodic sampling and achieve the desired collaboration goal. However, continuous excitation conditions need to be met, and the increase in the number of neural networks increases the complexity of the algorithm calculation; (2) a new model-free event-triggered adaptive dynamic programming method is used to solve the optimal consensus problem of heterogeneous systems. However, this method is aimed at discrete linear dynamic models. Due to the continuity and nonlinearity of the system, the proposed model-free Q-learning algorithm is not applicable. The above two algorithms require global information and have poor scalability. Summary of the invention

[0004] In order to solve the technical problems of poor scalability, the need to meet continuous incentive conditions and high algorithm complexity in the heterogeneous multi-UAV system consistency method in the prior art, the embodiment of the present invention provides a heterogeneous multi-UAV data collaborative approximate dynamic programming optimal consistency method and device. The technical solution is as follows:

[0005] On the one hand, a method for optimal consistency of heterogeneous multi-UAV data collaboration approximate dynamic programming is provided, which is implemented by an optimal consistency device for heterogeneous multi-UAV data collaboration approximate dynamic programming, and the method includes:

[0006] S1. Obtain the status data of the heterogeneous multi-UAV system; construct the recognition neural network with compensator, the execution neural network and the collaborative evaluation neural network according to the approximate dynamic programming method;

[0007] S2. According to the state data of the heterogeneous multi-UAV system, an identifier with a compensation signal is constructed; according to the identifier with the compensation signal, the dynamic model information of the heterogeneous multi-UAV system is approximated to obtain the state data of the identifier with the compensation signal; according to the state data of the identifier, a compensator with a dynamic signal is constructed to obtain the weight update rule of the recognition neural network with the compensator;

[0008] S3, constructing a Hamiltonian function according to the weight update rule;

[0009] S4. Constructing a cumulative square error with a cooperative signal according to the Hamiltonian function;

[0010] S5. According to the cumulative square error, a gradient descent method is used to obtain a weight update rule of the collaborative evaluation neural network and a weight update rule of the execution neural network;

[0011] S6. Calculate the error of constant weights between two adjacent iterations according to the weight update rule of the collaborative judgment neural network and the weight update rule of the execution neural network; determine whether the error meets a preset threshold condition; if so, stop iteration and output the optimal consistent solution of the heterogeneous multi-UAV system to keep the state of the heterogeneous multi-UAV system consistent.

[0012] On the other hand, a device for optimal consistency of heterogeneous multi-UAV data collaboration approximate dynamic programming is provided, and the device is applied to an optimal consistency method for heterogeneous multi-UAV data collaboration approximate dynamic programming, and the device includes:

[0013] The acquisition and construction unit is used to obtain the status data of the heterogeneous multi-UAV system; according to the approximate dynamic programming method, the recognition neural network with compensator, the execution neural network and the collaborative evaluation neural network are constructed;

[0014] A construction and acquisition unit is used to construct an identifier with a compensation signal according to the state data of the heterogeneous multi-UAV system; approximate the dynamic model information of the heterogeneous multi-UAV system according to the identifier with the compensation signal to obtain the state data of the identifier with the compensation signal; construct a compensator with a dynamic signal according to the state data of the identifier, and obtain the weight update rule of the recognition neural network with the compensator;

[0015] A first construction unit, configured to construct a Hamiltonian function according to the weight updating rule;

[0016] A second construction unit, configured to construct a cumulative square error with a cooperative signal according to the Hamiltonian function;

[0017] An acquisition and calculation unit, used to obtain a weight update rule of the collaborative evaluation neural network and a weight update rule of the execution neural network according to the cumulative square error by using a gradient descent method;

[0018] The output unit is used to calculate the error of the constant weights of two adjacent iterations according to the collaborative judgment neural network weight update rule and the execution neural network weight update rule; determine whether the error meets a preset threshold condition; if so, stop iteration and output the optimal consistent solution of the heterogeneous multi-unmanned aerial vehicle system to keep the state of the heterogeneous multi-unmanned aerial vehicle system consistent.

[0019] On the other hand, a device for optimal consistency of heterogeneous multi-UAV data collaborative approximate dynamic programming is provided, and the device for optimal consistency of heterogeneous multi-UAV data collaborative approximate dynamic programming includes: a processor; a memory, and the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, any one of the above-mentioned optimal consistency methods for heterogeneous multi-UAV data collaborative approximate dynamic programming is implemented.

[0020] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned optimal consistency methods for heterogeneous multi-UAV data collaborative approximate dynamic programming.

[0021] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0022] The embodiment of the present invention first obtains the state data of the heterogeneous multi-UAV system; constructs a recognition neural network with a compensator, an execution neural network and a collaborative judgment neural network according to an approximate dynamic programming method; secondly, constructs an identifier with a compensation signal according to the state data of the heterogeneous multi-UAV system; approximates the dynamic model information of the heterogeneous multi-UAV system according to the identifier with the compensation signal, and obtains the state data of the identifier with the compensation signal; constructs a compensator with a dynamic signal according to the state data of the identifier, and obtains the weight update rule of the recognition neural network with the compensator; constructs a Hamiltonian function according to the weight update rule; constructs a cumulative square error with a collaborative signal according to the Hamiltonian function; according to the cumulative square error, adopts the gradient descent method to obtain the weight update rule of the collaborative judgment neural network and the weight update rule of the execution neural network; finally, according to the weight update rule of the collaborative judgment neural network and the weight update rule of the execution neural network, calculates the error of the constant weights of two adjacent iterations; judges whether the error meets the preset threshold condition; if so, stops the iteration, outputs the optimal consistent solution of the heterogeneous multi-UAV system, and makes the state of the heterogeneous multi-UAV system consistent.

[0023] The embodiment of the present invention does not require any global information, increases the scalability of the algorithm, and is applicable to situations where the dynamics of each drone system are different. The cumulative square error with a collaborative signal is used to obtain a weight update rule for the constant of the execution-collaborative judgment neural network, wherein the weight update rule for the constant of the execution-collaborative judgment neural network uses the information of the drone system and the information of its neighboring drone systems, and obtains the optimal consistent solution for the heterogeneous multi-drone system by adaptively adjusting the network weights. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 This is a diagram of the implementation process of an optimal consensus method for approximate dynamic programming of heterogeneous multi-UAV data collaboration provided by an embodiment of the present invention;

[0026] Figure 2 It is a flow chart of an optimal consensus method for approximate dynamic programming of heterogeneous multi-UAV data collaboration provided by an embodiment of the present invention;

[0027] Figure 3 It is a block diagram of an optimal consistent device for heterogeneous multi-UAV data collaboration approximate dynamic programming provided by an embodiment of the present invention;

[0028] Figure 4 It is a structural schematic diagram of an optimal consistent device for heterogeneous multi-UAV data collaboration approximate dynamic programming provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0030] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0031] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0032] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0033] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0034] The embodiment of the present invention provides a method for optimal consistency of heterogeneous multi-UAV data collaboration approximate dynamic programming, which can be implemented by a heterogeneous multi-UAV data collaboration approximate dynamic programming optimal consistency device, and the heterogeneous multi-UAV data collaboration approximate dynamic programming optimal consistency device can be a terminal or a server. Figure 1The figure shows the implementation process of the optimal consensus method for heterogeneous multi-UAV data collaboration approximate dynamic programming; in a feasible implementation method, a white noise signal is obtained; based on the white noise signal, the state data of the heterogeneous multi-UAV system is obtained, and the parameters of the recognition-execution-collaboration judgment neural network with a compensator are initialized; based on the recognition neural network and the compensator, the weight update rule of the recognition neural network with a compensator is obtained; by executing the optimal control of the neural network and the performance index function of the collaborative judgment neural network, a Hamiltonian residual signal is constructed; based on the Hamiltonian residual signal, the cumulative square error with the collaborative signal is calculated; based on the cumulative square error with the collaborative signal, the constant weight update rule of the recognition-execution-collaboration judgment neural network with a compensator is calculated; based on the constant weight update rule of the recognition-execution-collaboration judgment neural network with a compensator, it is calculated whether the constant weight error of two adjacent iterations meets the preset threshold, and if so, the optimal consistent solution of the heterogeneous multi-UAV system is output. If not, the steps of obtaining the weight update rule of the recognition neural network with compensator are re-executed until the preset threshold condition is met, and the optimal consistent solution of the heterogeneous multi-UAV system is output; wherein, the constant weight of the collaborative judgment neural network is input into the communication network, and the communication network outputs the constant weight of the collaborative judgment neural network of the neighboring UAV system, which is used to calculate the constant weight update rule of the recognition-execution-collaborative judgment neural network with compensator.

[0035] like Figure 2 The flowchart of the optimal consensus method for heterogeneous multi-UAV data collaboration approximate dynamic programming is shown. The processing flow of the method may include the following steps:

[0036] S1. Obtain the status data of the heterogeneous multi-UAV system; construct a recognition neural network with compensator, an execution neural network, and a collaborative evaluation neural network based on the approximate dynamic programming method.

[0037] In a feasible implementation, the number of iterations is initialized to zero; the weight constants and activation function of the recognition neural network with compensator are initialized, the weight constants and activation function of the execution neural network are initialized, and the weight constants and activation function of the collaborative judgment neural network are initialized.

[0038] Optionally, the specific implementation process of S1 for obtaining the status data of the heterogeneous multi-UAV system may include S11-S12:

[0039] S11, obtaining white noise signal data;

[0040] S12, inputting the white noise signal data into the dynamic function of the heterogeneous multi-UAV system to obtain the state data of the heterogeneous multi-UAV system; wherein the state data of the heterogeneous multi-UAV system is expressed by the following formula (1):

[0041] (1)

[0042] in, In a multi-UAV system, Status data of each drone; Indicates Control input data of each drone; Indicates System dynamics functions of a UAV system; Indicates The input dynamic function of the UAV system; Indicates The derivative of the state of the UAV system.

[0043] S2. According to the state data of the heterogeneous multi-UAV system, an identifier with a compensation signal is constructed; according to the identifier with the compensation signal, the dynamic model information of the heterogeneous multi-UAV system is approximated to obtain the state data of the identifier with the compensation signal; according to the state data of the identifier, a compensator with a dynamic signal is constructed to obtain the weight update rule of the recognition neural network with the compensator.

[0044] Among them, the heterogeneous multi-UAV system is a continuous-time nonlinear system; the heterogeneous means that it can be responsible for searching, patrolling and monitoring different UAVs; and the dynamic information of the UAVs is different.

[0045] Optionally, the specific implementation process of S2 may include S21-S23:

[0046] S21. According to the state data of the heterogeneous multi-UAV system, a recognition neural network is used to approximate the dynamic model information of the heterogeneous multi-UAV system, and the state data of the identifier with the compensation signal is obtained; wherein the process of approximating the dynamic model information of the heterogeneous multi-UAV system is expressed by the following formula (2) and formula (3):

[0047] (2)

[0048] (3)

[0049] in, Indicates The estimated value of the system dynamic function of the UAV system; Indicates An estimate of the input dynamic function of the UAV system; Represents the first constant weight of the recognition neural network; Represents the second constant weight of the recognition neural network; Represents the first activation function of the recognition neural network; Represents the second activation function of the recognition neural network;

[0050] S22. Construct a compensator with a dynamic signal according to the state data of the identifier with the compensation signal and the error of the state data of the heterogeneous multi-UAV system; wherein the process of constructing the compensator with a dynamic signal is represented by the following formula (4) and formula (5):

[0051] (4)

[0052] (5)

[0053] in, represents a positive definite compensation matrix; represents a positive compensation dynamic parameter; A status signal indicating the recognizer; represents a positive constant parameter; Indicates the status of the drone; Indicates the control of the drone; Represents the constant weights of the recognition neural network; represents the activation function of the recognizer; Indicates compensation signal The derivative of represents the derivative of the recognizer state; represents an exponential function with base e;

[0054] Among them, the status data of the heterogeneous multi-UAV system includes position data and speed data.

[0055] S23. Using Lyapunov's stability theorem, the weight update rule of the neural network with compensator recognition is obtained; wherein the weight update rule of the neural network with compensator recognition is expressed by the following formula (6):

[0056] (6)

[0057] in, Indicates a tuning parameter greater than 0; Represents the derivative of the constant weights of the recognition neural network with compensator.

[0058] Among them, Lyapunov's stability theorem is a conventional technical means, and the specific implementation process is not further elaborated in the present invention.

[0059] S3. Construct the Hamiltonian function according to the weight update rule.

[0060] Optionally, the specific implementation process of S3 may include S31-S34:

[0061] S31. According to the weight update rule, the collaborative evaluation neural network is used to estimate the performance index function, which is expressed by the following formula (7):

[0062] (7)

[0063] in, represents the constant weight of the collaborative judgment neural network; represents the estimated value of the performance indicator function; represents the activation function of the collaborative judge neural network;

[0064] S32, using the optimality principle, obtain the execution neural network approximation optimal control; wherein, the execution neural network approximation optimal control is expressed by the following formula (8):

[0065] (8)

[0066] in, represents the estimated value of the optimal control; represents a positive definite symmetric matrix; Represents the second activation function of the recognition neural network; Represents the second constant weight of the recognition neural network; represents the constant weights for executing the neural network; Represents the partial derivative of the collaborative critic neural network activation function.

[0067] In a feasible implementation, the optimality principle is a method used in dynamic programming and reinforcement learning to find optimal strategies and decisions. It solves optimization problems by recursively optimizing sub-problems. It is a conventional technical means, and the specific implementation process is not further elaborated in the present invention.

[0068] S33, constructing a Hamiltonian function according to the performance indicator function and executing a neural network to approximate optimal control;

[0069] S34. According to the Hamiltonian function, the process of obtaining the residual signal is expressed by the following formula (9):

[0070] (9)

[0071] in, represents the Hamiltonian function, represents a positive definite matrix; represents the Hamiltonian residual.

[0072] S4. Construct the cumulative square error with cooperative signal according to the Hamiltonian function.

[0073] Among them, the cumulative square error with cooperative signals includes: the constant weights of the cooperative evaluation neural network of the heterogeneous multi-UAV system and the constant weights of its neighboring UAVs.

[0074] Optionally, the cumulative square error of S4 with the cooperative signal is expressed by the following formula (10):

[0075] (10)

[0076] in, represents the discount factor signal; represents the cumulative square error signal; Indicates greater than A positive constant of express The Hamiltonian residual at time t; Represents the connection relationship between drones i and j; Indicates the number of drones; express Neighbors collaborate to judge the weights of the neural network; Represents a system combination signal; Indicated in System combined signal value at time; Represents the time derivative of the activation function of the collaborative critic neural network.

[0077] in, It can be expressed by the following formula (11):

[0078] (11)

[0079] S5. According to the cumulative square error, the gradient descent method is used to obtain the weight update rule of the collaborative judgment neural network and the weight update rule of the execution neural network.

[0080] Optionally, the specific implementation process of S5 may include S51-S52:

[0081] S51. Using the gradient descent method, the weight update rule of the cumulative collaborative judgment neural network is obtained, which is expressed by the following formula (12):

[0082] (12)

[0083] in, Represents time-varying functions of state and input; represents the derivative of the constant weights of the collaborative judge neural network; Represents the first adjustment parameter, which is a positive number;

[0084] S52, using the gradient descent method, obtain the update rule for executing the constant weights of the neural network, which is expressed by the following formula (13):

[0085] (13)

[0086] in, represents a positive definite regulation matrix; Represents the second adjustment parameter, which is a positive number.

[0087] Among them, the gradient descent method is a commonly used optimization algorithm for solving unconstrained nonlinear optimization problems. The basic idea is to start from an initial point and iteratively update the parameters along the direction of the fastest gradient descent of the objective function until a local minimum is found; among them, the gradient descent method is a conventional technical means, and the specific implementation process of the gradient descent method is not further elaborated in this invention.

[0088] S6. Calculate the error of constant weights between two adjacent iterations according to the collaborative judgment neural network weight update rules and the execution neural network weight update rules; determine whether the error meets the preset threshold condition; if so, stop iteration and output the optimal consistent solution for the heterogeneous multi-UAV system to keep the state of the heterogeneous multi-UAV system consistent.

[0089] If not satisfied, the number of iterations is increased by 1, and steps S2-S5 are repeated.

[0090] In a feasible implementation, the accumulated signals of the heterogeneous multi-UAV system are set to satisfy the following formula (14):

[0091] (14)

[0092] According to formula (14), it can be concluded that the algorithm proposed in the present invention does not need to meet the continuity condition.

[0093] Optionally, the preset threshold condition of S6 is expressed by the following formula (15):

[0094] (15)

[0095] in, represents a preset first threshold; Indicates the preset second threshold. Indicates the number of iterations The constant weight of the collaborative judgment neural network when ; Indicates the number of iterations The constant weights of the neural network when executing ; Indicates the number of iterations The constant weight of the collaborative judgment neural network when ; Indicates the number of iterations The constant weights of the neural network when executing .

[0096] Among them, in the multi-UAV consistency work, multiple UAVs need to work together to complete tasks, including search, patrol and monitoring tasks; the hardware and communication capabilities of different UAVs may be heterogeneous. By solving the optimal consistency problem of the multi-UAV system, it can ensure that the information sharing and task allocation between UAVs are consistent, so that the UAV cluster can complete tasks in a coordinated manner, including efficient search and area coverage tasks.

[0097] Among them, the optimal consistent solution of the heterogeneous multi-UAV system can keep the state of the multi-UAV system consistent. By obtaining the optimal consistent solution of the heterogeneous multi-UAV system, the heterogeneous multi-UAVs can maintain consistent speed and position when flying in formation, so that multiple UAVs can coordinate to complete efficient search, patrol monitoring and area coverage tasks.

[0098] The embodiment of the present invention first obtains the state data of the heterogeneous multi-UAV system; constructs a recognition neural network with a compensator, an execution neural network and a collaborative judgment neural network according to an approximate dynamic programming method; secondly, constructs an identifier with a compensation signal according to the state data of the heterogeneous multi-UAV system; approximates the dynamic model information of the heterogeneous multi-UAV system according to the identifier with the compensation signal, and obtains the state data of the identifier with the compensation signal; constructs a compensator with a dynamic signal according to the state data of the identifier, and obtains the weight update rule of the recognition neural network with the compensator; constructs a Hamiltonian function according to the weight update rule; constructs a cumulative square error with a collaborative signal according to the Hamiltonian function; according to the cumulative square error, adopts the gradient descent method to obtain the weight update rule of the collaborative judgment neural network and the weight update rule of the execution neural network; finally, according to the weight update rule of the collaborative judgment neural network and the weight update rule of the execution neural network, calculates the error of the constant weights of two adjacent iterations; judges whether the error meets the preset threshold condition; if so, stops the iteration, outputs the optimal consistent solution of the heterogeneous multi-UAV system, and makes the state of the heterogeneous multi-UAV system consistent.

[0099] The embodiment of the present invention does not require any global information, increases the scalability of the algorithm, and is applicable to situations where the dynamics of each drone system are different. The weight update rule of the execution-collaboration judgment neural network constant is obtained by using the cumulative square error with a collaborative signal. The weight update rule of the execution-collaboration judgment neural network constant uses the information of the drone system and the information of its neighboring drone systems. The optimal consistent solution for the heterogeneous multi-drone system is obtained by adaptively adjusting the network weights.

[0100] Figure 3The invention is a block diagram of an optimal consistent device for approximate dynamic programming of heterogeneous multi-UAV data collaboration according to an exemplary embodiment. The device is used for the optimal consistent method for approximate dynamic programming of heterogeneous multi-UAV data collaboration. Figure 3 The device includes an acquisition and construction unit 310, a construction and acquisition unit 320, a first construction unit 330, a second construction unit 340, an acquisition and calculation unit 350 and an output unit 360. Among them:

[0101] The acquisition and construction unit 310 is used to acquire the status data of the heterogeneous multi-UAV system; construct a recognition neural network with a compensator, an execution neural network, and a collaborative evaluation neural network according to an approximate dynamic programming method;

[0102] The construction and acquisition unit 320 is used to construct an identifier with a compensation signal according to the state data of the heterogeneous multi-UAV system; approximate the dynamic model information of the heterogeneous multi-UAV system according to the identifier with the compensation signal to obtain the state data of the identifier with the compensation signal; construct a compensator with a dynamic signal according to the state data of the identifier, and obtain the weight update rule of the recognition neural network with the compensator;

[0103] A first construction unit 330, configured to construct a Hamiltonian function according to the weight update rule;

[0104] A second construction unit 340 is used to construct a cumulative square error with a cooperative signal according to the Hamiltonian function;

[0105] An acquisition and calculation unit 350 is used to obtain a weight update rule of the collaborative evaluation neural network and a weight update rule of the execution neural network according to the cumulative square error by using a gradient descent method;

[0106] The output unit 360 is used to calculate the error of the constant weights of two adjacent iterations according to the collaborative judgment neural network weight update rule and the execution neural network weight update rule; determine whether the error meets a preset threshold condition; if so, stop iteration and output the optimal consistent solution of the heterogeneous multi-UAV system to keep the state of the heterogeneous multi-UAV system consistent.

[0107] Optionally, the acquisition and construction unit 310 is used to:

[0108] Get white noise signal data;

[0109] The white noise signal data is input into the dynamic function of the heterogeneous multi-UAV system to obtain the state data of the heterogeneous multi-UAV system. The state data of the heterogeneous multi-UAV system is expressed by the following formula (1):

[0110] (1)

[0111] in, In a multi-UAV system, Status data of each drone; Indicates Control input data of each drone; Indicates System dynamics functions of a UAV system; Indicates The input dynamic function of the UAV system; Indicates The derivative of the state of the UAV system.

[0112] Optionally, the constructing and obtaining unit 320 is used to:

[0113] According to the state data of the heterogeneous multi-UAV system, a recognition neural network is used to approximate the dynamic model information of the heterogeneous multi-UAV system, and the state data of the identifier with a compensation signal is obtained; wherein the process of approximating the dynamic model information of the heterogeneous multi-UAV system is expressed by the following formula (2) and formula (3):

[0114] (2)

[0115] (3)

[0116] in, Indicates The estimated value of the system dynamic function of the UAV system; Indicates An estimate of the input dynamic function of the UAV system; Represents the first constant weight of the recognition neural network; Represents the second constant weight of the recognition neural network; Represents the first activation function of the recognition neural network; Represents the second activation function of the recognition neural network;

[0117] According to the state data of the identifier with compensation signal and the error of the state data of the heterogeneous multi-UAV system, a compensator with dynamic signal is constructed; wherein the process of constructing the compensator with dynamic signal is expressed by the following formula (4) and formula (5):

[0118] (4)

[0119] (5)

[0120] in, represents a positive definite compensation matrix; represents a positive compensation dynamic parameter; A status signal indicating the recognizer; represents a positive constant parameter; Indicates the status of the drone; Indicates the control of the drone; Represents the constant weights of the recognition neural network; represents the activation function of the recognizer; Indicates compensation signal The derivative of represents the derivative of the recognizer state; represents an exponential function with base e;

[0121] Using Lyapunov's stability theorem, the weight update rule of the recognition neural network with compensator is obtained; wherein, the weight update rule of the recognition neural network with compensator is expressed by the following formula (6):

[0122] (6)

[0123] in, Indicates a tuning parameter greater than 0; Represents the derivative of the constant weights of the recognition neural network with compensator.

[0124] Optionally, the first constructing unit 330 is used to:

[0125] According to the weight update rule, a collaborative evaluation neural network is used to estimate the performance index function, which is expressed by the following formula (7):

[0126] (7)

[0127] in, represents the constant weight of the collaborative judgment neural network; represents the estimated value of the performance indicator function; represents the activation function of the collaborative judge neural network;

[0128] The optimality principle is adopted to obtain the execution neural network approximation optimal control; wherein, the execution neural network approximation optimal control is expressed by the following formula (8):

[0129] (8)

[0130] in, represents the estimated value of the optimal control; represents a positive definite symmetric matrix; Represents the second activation function of the recognition neural network; Represents the second constant weight of the recognition neural network; represents the constant weights for executing the neural network; Represents the partial derivative of the collaborative critic neural network activation function.

[0131] Construct Hamiltonian function based on performance index function and executing neural network to approximate optimal control;

[0132] According to the Hamiltonian function, the process of obtaining the residual signal is expressed by the following formula (9):

[0133] (9)

[0134] in, represents the Hamiltonian function, represents a positive definite matrix; represents the Hamiltonian residual.

[0135] Optionally, the cumulative square error with the cooperative signal is expressed by the following formula (10):

[0136] (10)

[0137] in, represents the discount factor signal; represents the cumulative square error signal; Indicates greater than A positive constant of express The Hamiltonian residual at time t; Represents the connection relationship between drones i and j; Indicates the number of drones; express Neighbors collaborate to judge the weights of the neural network; Represents a system combination signal; Indicated in System combined signal value at time; Represents the time derivative of the activation function of the collaborative critic neural network.

[0138] Optionally, the acquisition and calculation unit 350 is used to:

[0139] The gradient descent method is used to obtain the weight update rule of the cumulative collaborative judgment neural network, which is expressed by the following formula (11):

[0140] (11)

[0141] in, Represents time-varying functions of state and input; represents the derivative of the constant weights of the collaborative judge neural network; Represents the first adjustment parameter, which is a positive number;

[0142] The gradient descent method is used to obtain the update rule for the constant weights of the neural network, which is expressed by the following formula (12):

[0143] (12)

[0144] in, represents a positive definite regulation matrix; Represents the second adjustment parameter, which is a positive number.

[0145] Optionally, the preset threshold condition is expressed by the following formula (13):

[0146] (13)

[0147] in, represents a preset first threshold; Indicates the preset second threshold. Indicates the number of iterations The constant weight of the collaborative judgment neural network when ; Indicates the number of iterations The constant weights of the neural network when executing ; Indicates the number of iterations The constant weight of the collaborative judgment neural network when ; Indicates the number of iterations The constant weights of the neural network when executing .

[0148] The embodiment of the present invention first obtains the state data of the heterogeneous multi-UAV system; constructs a recognition neural network with a compensator, an execution neural network and a collaborative judgment neural network according to an approximate dynamic programming method; secondly, constructs an identifier with a compensation signal according to the state data of the heterogeneous multi-UAV system; approximates the dynamic model information of the heterogeneous multi-UAV system according to the identifier with the compensation signal, and obtains the state data of the identifier with the compensation signal; constructs a compensator with a dynamic signal according to the state data of the identifier, and obtains the weight update rule of the recognition neural network with the compensator; constructs a Hamiltonian function according to the weight update rule; constructs a cumulative square error with a collaborative signal according to the Hamiltonian function; according to the cumulative square error, adopts the gradient descent method to obtain the weight update rule of the collaborative judgment neural network and the weight update rule of the execution neural network; finally, according to the weight update rule of the collaborative judgment neural network and the weight update rule of the execution neural network, calculates the error of the constant weights of two adjacent iterations; judges whether the error meets the preset threshold condition; if so, stops the iteration, outputs the optimal consistent solution of the heterogeneous multi-UAV system, and makes the state of the heterogeneous multi-UAV system consistent.

[0149] The embodiment of the present invention does not require any global information, increases the scalability of the algorithm, and is applicable to situations where the dynamics of each drone system are different. The weight update rule of the execution-collaboration judgment neural network constant is obtained by using the cumulative square error with a collaborative signal. The weight update rule of the execution-collaboration judgment neural network constant uses the information of the drone system and the information of its neighboring drone systems. The optimal consistent solution for the heterogeneous multi-drone system is obtained by adaptively adjusting the network weights.

[0150] Figure 4 is a schematic diagram of a structure of an optimal consistent device for heterogeneous multi-UAV data collaboration approximate dynamic programming provided by an embodiment of the present invention, such as Figure 4 As shown, the optimal consistent device for heterogeneous multi-UAV data collaboration approximate dynamic planning may include the above Figure 3 The heterogeneous multi-UAV data collaborative approximate dynamic programming optimal consistent device shown. Optionally, the heterogeneous multi-UAV data collaborative approximate dynamic programming optimal consistent device 410 may include a first processor 2001.

[0151] Optionally, the heterogeneous multi-UAV data collaborative approximate dynamic programming optimal consensus device 410 may also include a memory 2002 and a transceiver 2003.

[0152] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0153] Combine the following Figure 4The components of the optimal consensus device 410 for heterogeneous multi-UAV data collaboration approximate dynamic programming are introduced in detail:

[0154] The first processor 2001 is the control center of the heterogeneous multi-UAV data collaborative approximate dynamic programming optimal consensus device 410, which can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0155] Optionally, the first processor 2001 can perform various functions of the heterogeneous multi-UAV data collaborative approximate dynamic programming optimal consistency device 410 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0156] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 4 CPU0 and CPU1 are shown in FIG.

[0157] In a specific implementation, as an embodiment, the heterogeneous multi-UAV data collaborative approximate dynamic programming optimal consensus device 410 may also include multiple processors, such as Figure 4 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0158] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0159] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be accessed through the interface circuit ( Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0160] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0161] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 4 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0162] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or can exist independently, and can be used to coordinate the approximate dynamic programming of the interface circuit of the optimal consistent device 410 through the heterogeneous multi-UAV data. Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0163] It should be noted that Figure 4 The structure of the heterogeneous multi-UAV data collaborative approximate dynamic programming optimal consensus device 410 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0164] In addition, the technical effects of the heterogeneous multi-UAV data collaborative approximate dynamic programming optimal consistency device 410 can refer to the technical effects of the heterogeneous multi-UAV data collaborative approximate dynamic programming optimal consistency method described in the above method embodiment, and will not be repeated here.

[0165] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0166] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0167] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0168] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0169] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0170] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0171] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0172] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0173] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0174] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0175] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0176] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0177] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An optimal consensus method for approximate dynamic programming of heterogeneous multi-UAV data collaboration, characterized by: The method comprises: S1. Obtain the status data of the heterogeneous multi-UAV system; construct the recognition neural network with compensator, the execution neural network and the collaborative evaluation neural network according to the approximate dynamic programming method; S2. According to the state data of the heterogeneous multi-UAV system, an identifier with a compensation signal is constructed; according to the identifier with the compensation signal, the dynamic model information of the heterogeneous multi-UAV system is approximated to obtain the state data of the identifier with the compensation signal; according to the state data of the identifier, a compensator with a dynamic signal is constructed to obtain the weight update rule of the recognition neural network with the compensator; According to the error between the state data of the identifier with the compensation signal and the state data of the heterogeneous multi-UAV system, a compensator with a dynamic signal is constructed; wherein the process of constructing the compensator with a dynamic signal is expressed by the following formula (1) and formula (2): （1） （2） in, represents a positive definite compensation matrix; represents a positive compensation dynamic parameter; A signal indicating the state of the recognizer; represents a positive constant parameter; Indicates the status of the drone; Indicates the control of the drone; Represents the constant weight of the recognition neural network; represents the activation function of the recognizer; Indicates compensation signal The derivative of represents the derivative of the recognizer state; represents an exponential function with base e; S3, constructing a Hamiltonian function according to the weight update rule; S4. Constructing a cumulative square error with a cooperative signal according to the Hamiltonian function; The cumulative square error of S4 with the cooperative signal is expressed by the following formula (3): （3） in, represents the discount factor signal; represents the cumulative square error signal; Indicates greater than A positive constant of express The Hamiltonian residual at time t; Represents the connection relationship between drones i and j; Indicates the number of drones; express Neighbors collaborate to judge the weights of the neural network; Represents a system combination signal; Indicated in System combined signal value at time; represents the derivative of the activation function of the collaborative judge neural network with respect to time; S5. According to the cumulative square error, a gradient descent method is used to obtain a weight update rule of the collaborative evaluation neural network and a weight update rule of the execution neural network; Among them, the gradient descent method is used to obtain the weight update rule of the collaborative judgment neural network, which is expressed by the following formula (4): （4） in, Represents time-varying functions of state and input; represents the derivative of the constant weights of the collaborative judge neural network; Represents the first adjustment parameter, which is a positive number; S6. Calculate the error of constant weights between two adjacent iterations according to the collaborative judgment neural network weight update rule and the execution neural network weight update rule; determine whether the error meets a preset threshold condition; if so, stop iteration and output the optimal consistent solution of the heterogeneous multi-UAV system to keep the state of the heterogeneous multi-UAV system consistent.

2. The optimal consensus method for heterogeneous multi-UAV data collaboration approximate dynamic programming according to claim 1 is characterized in that: The S1 acquires the status data of the heterogeneous multi-UAV system, including: S11, obtaining white noise signal data; S12, inputting the white noise signal data into the dynamic function of the heterogeneous multi-UAV system to obtain the state data of the heterogeneous multi-UAV system; wherein the state data of the heterogeneous multi-UAV system is expressed by the following formula (5): （5） in, In a multi-UAV system, Status data of each drone; Indicates Control input data of each drone; Indicates System dynamics functions of a UAV system; Indicates The input dynamic function of the UAV system; Indicates The derivative of the state of the UAV system.

3. The optimal consensus method for heterogeneous multi-UAV data collaboration approximate dynamic programming according to claim 2 is characterized in that: The S2 constructs an identifier with a compensation signal according to the state data of the heterogeneous multi-UAV system; approximates the dynamic model information of the heterogeneous multi-UAV system according to the identifier with the compensation signal to obtain the state data of the identifier with the compensation signal; According to the state data of the identifier, a compensator with a dynamic signal is constructed to obtain a weight update rule of the recognition neural network with the compensator, including: S21. According to the state data of the heterogeneous multi-UAV system, a recognition neural network is used to approximate the dynamic model information of the heterogeneous multi-UAV system to obtain the state data of the identifier with the compensation signal; wherein the process of approximating the dynamic model information of the heterogeneous multi-UAV system is represented by the following formula (6) and formula (7): （6） （7） in, Indicates The estimated value of the system dynamic function of the UAV system; Indicates An estimate of the input dynamic function of the UAV system; Represents the first constant weight of the recognition neural network; Represents the second constant weight of the recognition neural network; Represents the first activation function of the recognition neural network; Represents the second activation function of the recognition neural network; S22. Using Lyapunov's stability theorem, the weight update rule of the neural network with compensator identification is obtained; wherein the weight update rule of the neural network with compensator identification is expressed by the following formula (8): （8） in, Indicates a tuning parameter greater than 0; Represents the derivative of the constant weights of the recognition neural network with compensator.

4. The optimal consensus method for heterogeneous multi-UAV data collaboration approximate dynamic programming according to claim 1 is characterized in that: The step S3 constructs a Hamiltonian function according to the weight update rule, including: S31. According to the weight update rule, a collaborative evaluation neural network is used to estimate the performance index function, which is expressed by the following formula (9): （9） in, represents the constant weight of the collaborative judgment neural network; represents the estimated value of the performance indicator function; represents the activation function of the collaborative judge neural network; S32, using the optimality principle, obtain the execution neural network approximation optimal control, which is expressed by the following formula (10): （10） in, represents the estimated value of the optimal control; represents a positive definite symmetric matrix; Represents the second activation function of the recognition neural network; Represents the second constant weight of the recognition neural network; represents the constant weights for executing the neural network; represents the partial derivative of the activation function of the collaborative judge neural network; S33, constructing a Hamiltonian function according to the performance indicator function and executing a neural network to approximate optimal control; S34. According to the Hamiltonian function, the process of obtaining the residual signal is expressed by the following formula (11): （11） in, represents the Hamiltonian function, represents a positive definite matrix; represents the Hamiltonian residual.

5. The optimal consensus method for heterogeneous multi-UAV data collaboration approximate dynamic programming according to claim 1 is characterized in that: The step S5 adopts the gradient descent method to obtain the weight update rule of the collaborative evaluation neural network and the weight update rule of the execution neural network, including: The gradient descent method is used to obtain the update rule for the constant weights of the neural network, which is expressed by the following formula (12): （12） in, represents a positive definite regulation matrix; Represents the second adjustment parameter, which is a positive number.

6. The optimal consensus method for heterogeneous multi-UAV data collaboration approximate dynamic programming according to claim 1 is characterized in that: The preset threshold condition of S6 is expressed by the following formula (13): （13） in, represents a preset first threshold; represents a preset second threshold; Indicates the number of iterations The constant weight of the collaborative judgment neural network when ; Indicates the number of iterations The constant weights of the neural network when executing ; Indicates the number of iterations The constant weight of the collaborative judgment neural network when ; Indicates the number of iterations The constant weights of the neural network when executing .

7. A device for optimal consistency of heterogeneous multi-UAV data collaboration approximate dynamic programming, the device for optimal consistency of heterogeneous multi-UAV data collaboration approximate dynamic programming is used to implement the optimal consistency method of heterogeneous multi-UAV data collaboration approximate dynamic programming as claimed in any one of claims 1 to 6, characterized in that: The device comprises: The acquisition and construction unit is used to obtain the status data of the heterogeneous multi-UAV system; according to the approximate dynamic programming method, the recognition neural network with compensator, the execution neural network and the collaborative evaluation neural network are constructed; A construction and acquisition unit is used to construct an identifier with a compensation signal according to the state data of the heterogeneous multi-UAV system; approximate the dynamic model information of the heterogeneous multi-UAV system according to the identifier with the compensation signal to obtain the state data of the identifier with the compensation signal; construct a compensator with a dynamic signal according to the state data of the identifier, and obtain the weight update rule of the recognition neural network with the compensator; According to the error between the state data of the identifier with the compensation signal and the state data of the heterogeneous multi-UAV system, a compensator with a dynamic signal is constructed; wherein the process of constructing the compensator with a dynamic signal is expressed by the following formula (1) and formula (2): （1） （2） in, represents a positive definite compensation matrix; represents a positive compensation dynamic parameter; A signal indicating the state of the recognizer; represents a positive constant parameter; Indicates the status of the drone; Indicates the control of the drone; Represents the constant weight of the recognition neural network; represents the activation function of the recognizer; Indicates compensation signal The derivative of represents the derivative of the recognizer state; represents an exponential function with base e; A first construction unit, configured to construct a Hamiltonian function according to the weight updating rule; A second construction unit, configured to construct a cumulative square error with a cooperative signal according to the Hamiltonian function; The cumulative square error with cooperative signal is expressed by the following formula (3): （3） in, represents the discount factor signal; represents the cumulative square error signal; Indicates greater than A positive constant of express The Hamiltonian residual at time t; Represents the connection relationship between drones i and j; Indicates the number of drones; express Neighbors collaborate to judge the weights of the neural network; Represents a system combination signal; Indicated in System combined signal value at time; represents the derivative of the activation function of the collaborative judge neural network with respect to time; An acquisition and calculation unit, used to obtain a weight update rule of the collaborative evaluation neural network and a weight update rule of the execution neural network according to the cumulative square error by using a gradient descent method; Among them, the gradient descent method is used to obtain the weight update rule of the collaborative judgment neural network, which is expressed by the following formula (4): （4） in, Represents time-varying functions of state and input; represents the derivative of the constant weights of the collaborative judge neural network; Represents the first adjustment parameter, which is a positive number; The output unit is used to calculate the error of the constant weights of two adjacent iterations according to the collaborative judgment neural network weight update rule and the execution neural network weight update rule; determine whether the error meets a preset threshold condition; if so, stop iteration and output the optimal consistent solution of the heterogeneous multi-unmanned aerial vehicle system to keep the state of the heterogeneous multi-unmanned aerial vehicle system consistent.

8. An optimal consensus method and device for heterogeneous multi-UAV data collaboration approximate dynamic programming, characterized in that: The optimal consensus device for heterogeneous multi-UAV data collaborative approximate dynamic programming includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 6.

Images