A neural network controller design method for coordinating UAV swarm formations

Abstract

The present application belongs to the field of wireless communication technology and unmanned aerial vehicle control technology, and specifically relates to a neural network controller design method for coordinating drone cluster formations. By establishing a single drone dynamics model and a system state space equation, and using the drone's own sensors to obtain the state vector at the corresponding moment, the network weight coefficient is reversely updated according to the loss function, thereby inducing the gradient descent of the control input, continuously updating the control input, and obtaining the optimal control strategy at different moments through multiple strategy iterations. It does not rely on the construction of traditional loss functions and does not require a large amount of prior model parameter data, thereby greatly improving the real-time performance and stability of the drone formation.

Description

Technical Field

[0001] The present application belongs to the field of wireless communication technology and unmanned aerial vehicle control technology, and specifically relates to a neural network controller design method for coordinating unmanned aerial vehicle cluster formations. Background Art

[0002] The statements in this section merely provide background technical information related to this application and do not necessarily constitute prior art.

[0003] With the recent development of artificial intelligence (AI), drones can now use wireless communication to collect information from other drone sensors, facilitating formation control. Multiple drones flying in formation, collaborating in reconnaissance and combat, can significantly increase the success rate of combat missions. UAV formation flight can improve the efficiency of completing single missions in military reconnaissance, target strikes, communications relay, electronic countermeasures, battlefield assessment, and harassment. Multiple drones can simultaneously attack a single target from different positions and angles, enhancing strategic strike capabilities. They can also simultaneously strike multiple targets, creating visual and auditory disruption and shortening combat duration. During reconnaissance, drones in formation can capture images from multiple angles and in all directions. Linear quadratic regulators, optimal control, model predictive control, robust control, and adaptive dynamic programming have been designed to control UAV formations. However, all of these control designs require the knowledge of a large number of parameters.

[0004] In recent years, there has been a strong interest in the design of artificial neural networks using physical model formulations based on Lagrangian and Hamiltonian functions as priors for various tasks.

[0005] After searching the existing literature, it was found that in the patent document with publication number CN115328203A, entitled: A data-driven large-scale UAV cluster formation simulation acceleration method and system, the simulation speed of the dynamic process of large-scale UAV cluster formation behavior is significantly improved by establishing a low-dimensional feature subspace of the original large-scale UAV cluster formation state, and the composite agent model composed of a linear initial model and a deep neural network can accelerate the simulation speed of large-scale UAV cluster formation behavior while maintaining high simulation accuracy; however, this method requires prior knowledge of the UAV formation's driving trajectory, and cannot make the UAVs gradually form a formation from a disordered state, and cannot be applied to the field of UAV formation control.

[0006] The method disclosed in the application document, publication number CN113220021, entitled "A Collaborative Adaptive Tracking Control Method for Flight Formations Based on a Virtual Leader," incorporates a coordination strategy for a virtual pilot UAV. The relative positions of each UAV and the pilot are set as feedback information, and backstepping is used to design control inputs to achieve coordinated trajectory tracking according to the flight formation. A neural network is used to estimate dynamic uncertainty, online data is collected to construct prediction error estimation capabilities, and tracking error is used to design an adaptive update law for the neural network weights. However, this method only uses positional expectations as feedback, which cannot achieve the formation requirement of uniform flight speed for the UAVs.

[0007] In addition, the method disclosed in the patent document with publication number CN115877871A and titled: A non-zero-sum game drone formation control method based on reinforcement learning proposes a non-zero-sum game drone formation control method based on reinforcement learning, which fully utilizes the adaptive ability of reinforcement learning, intelligently obtains some unknown quantities of the drone formation system and uses state iteration to learn the optimal controller; however, this method has a complex algorithm, is difficult to solve, and takes a long time, which greatly reduces the possibility of real-time control. Summary of the Invention

[0008] Based on this, in response to the above technical problems, this application provides a neural network controller design method for coordinating drone cluster formations.

[0009] The technical solution adopted by this application to solve the problems existing in the prior art is:

[0010] This application proposes a design method for a neural network controller for coordinating drone swarm formations, which is implemented through the following steps:

[0011] S1: For the drone fleet, establish a drone dynamics model based on linear time-invariant expressions and write the state space equations of the entire system;

[0012] S2: Each UAV obtains its own position and velocity information from the sensor and obtains the state vector of the system at a certain moment;

[0013] S3: Use the neural network to update the network weight coefficients in reverse according to the minimum root mean square error loss function, induce the control input gradient descent according to the gradient descent of the weight coefficients, and calculate the updated control input;

[0014] S4: If the number of neural network iterations reaches the set value, or the drone's flight state converges to the desired state, then proceed to the next step S5, otherwise proceed to the previous step S3;

[0015] S5: Input the optimal control strategy into the drone queue model, and the drone queue updates its own state and enters step S2.

[0016] Preferably, the UAV dynamics model in step 1 is a longitudinal dynamics model of a single UAV based on a linear time-invariant expression established for a UAV cluster formation flying in the air, and the expression is:

[0017] ①

[0018] in, 、 are the system state transfer matrix and the system control quantity matrix respectively, is a matrix about the current state of the drone, It is the matrix of the control input of the pilot drone. is the system status information, For the control input of the drone, is the number of following drones in the cluster.

[0019] Preferably, the variables in formula ① are expanded and specifically written as:

[0020] ,

[0021]

[0022] ; Where: subscript 0 represents the pilot drone, subscript 1 to Representatives follow the drone, The position and velocity of the UAV are expressed as and , No. The drone is next to the drone in front of it. The distance between the racks is expressed as ; Represents the sensitivity of the following drone to the difference between the actual distance between itself and the adjacent drone and the expected distance. Represents the sensitivity of the following drone to the difference between its own and the neighboring drone’s actual and expected flight speeds; It represents the expected flight time interval in seconds, indicating that the expected flight distance between two adjacent UAVs at different speeds is different; is the time step;

[0023] The state space equation of the system is obtained by integrating the states of the drone fleet at each moment.

[0024] Preferably, the 、 They are respectively the system state transfer matrix and the system control quantity matrix expanded by iterative calculation according to the existing technology.

[0025] Preferably, the Equal to the acceleration of the pilot drone.

[0026] Preferably, in step 2, each drone obtains its own position and speed information from its own sensor, and the frequency of information acquisition is determined by the sampling time interval. Decision, obtain information and transmit it to the pilot drone through inter-drone communication to obtain the state vector of the system at a certain moment .

[0027] Preferably, the root mean square error loss function expression in step 3 is:

[0028] , ②

[0029] Where, Indicates the control termination time The actual state of the system, Indicates the system at the control termination time expected state;

[0030] According to the principle of taking the minimum value of the above-mentioned root mean square error loss function, the network weight coefficient is updated in reverse , due to the weight coefficient Gradient descent will induce the control input Gradient descent, so that the control input can be continuously calculated and updated , thereby updating the neural network;

[0031] Preferably, the gradient expression adopts the existing formula, which is:

[0032] , ③

[0033] Where, Represents the learning rate, subscript Represents the order of the current gradient descent, represents the gradient, Indicates the weight Down gradient;

[0034] The iterative expression of the control input is:

[0035] , ④

[0036] Where, for The Jacobian matrix of , The square matrix is The linear transformation coefficients on .

[0037] Preferably, the neural network input node is , The number of drones in the cluster = .

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] 1. The present invention uses a neural network ordinary differential equation solver to solve the optimal control strategy, continuously backpropagates the root mean square error between the actual state and the desired state to update the network parameters, and uses the principle of gradient descent of the weight coefficient to induce the gradient descent of the control input. Multiple strategy iterations enable the drone cluster to obtain the optimal control input.

[0040] 2. This invention combines deep learning with physical modeling, does not rely on the construction of traditional loss functions, and does not require a large number of model parameters as priors. It can solve complex high-dimensional control problems while ensuring the solution rate, greatly improving the real-time performance and stability of the formation. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings in the specification, which constitute a part of this application, are used to provide further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute improper limitations on this application.

[0042] Figure 1 This is the architecture diagram of the neural network controller;

[0043] Figure 2 Schematic diagram of UAV formation control based on neural network control strategy;

[0044] Figure 3 This is a control flow diagram of a neural network-based drone cluster control method of the present application;

[0045] Figure 4 This is a schematic diagram of the drone experimental location for a neural network-based drone cluster control method of this application. DETAILED DESCRIPTION

[0046] The present application will be further described below with reference to the accompanying drawings and embodiments.

[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0048] In the present disclosure, terms such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "side", "bottom", etc. indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. They are merely relational words determined for the convenience of describing the structural relationships of the various parts or elements of the present disclosure, and do not specifically refer to any part or element in the present disclosure, and should not be understood as limitations on the present disclosure.

[0049] Figure 1 and Figure 2 This paper describes the theoretical basis of a neural network controller design method for coordinating UAV swarm formations in this application. Figure 1 This is a diagram of the neural network controller architecture. In this application, neural network system training is achieved through the root mean square error backpropagation algorithm. This algorithm calculates the error between the network output and the target value and propagates the error from the output layer back to the input layer to gradually adjust the parameters and weights in the network, so that the network can better approach the target value.

[0050] In the RMS error back propagation neural network, the control input matrix Determine which nodes are controlled, the root mean square error function is expressed in The deviation between the target value state and the expected state at the moment, the control input calculated by the artificial neural network Substituting this into the system state equation allows the neural network to update its parameters based on the gradient descent of the loss function. A neural network has three layers: input layer, hidden layer, and output layer. In the hidden layer, each neuron has a full-flow activation function that converts the node input into its corresponding output signal.

[0051] like Figure 2 The figure shows a schematic diagram of UAV formation control based on a neural network control strategy. First, relevant data in the environment corresponding to the UAV cluster are obtained, and the relevant data in the environment corresponding to the UAV cluster, including the initial state of each UAV such as the position and speed of the UAV, are collected and stored in the database using UAV sensors. Then, the target value state of the UAV cluster, including the expected position and speed, is input into the initial network, that is, the neural network before backpropagation through the loss function. Then, the loss is calculated based on the difference between the output and the target through the loss function, and the loss is backpropagated to each node in the network to update the weight of the network to form an updated network, that is, the neural network after backpropagation through the loss function. Then, the strategy is executed through the updated network, and the pilot UAV continuously adjusts the control input, that is, the acceleration, through the neural network, so that the UAV cluster can achieve the desired state within the specified time period on the basis of receiving the real-time status information of the following UAVs and responding to the acceleration control of the pilot vehicle. The entire drone cluster can achieve the desired speed and desired distance flight.

[0052] Among them, the loss function is based on the state difference. When it is minimum, reversely solve the corresponding , 、 The time steps Status information and control input of the drone, is the weight of the neural network, Indicates the obtained The optimal value of The status information of the first drone includes the speed of the following drones , its own drone (i.e. The first drone) and the follower drone (i.e. The relative speed between the two drones , the distance between the two drones .Bundle , , As input, it outputs continuous action.

[0053] in is the time step, is the number of follower drones in the formation, Through these interactions, the optimal strategy for UAV following control is derived.

[0054] According to the above theoretical basis, if Figure 3 As shown, the present invention discloses a neural network controller design method for coordinating UAV cluster formations. The method is to solve ordinary differential equations through a neural network with root mean square error back propagation, and includes the following steps:

[0055] S1: For the drone fleet, establish a drone dynamics model based on linear time-invariant expressions and write the state space equations of the entire system;

[0056] S2: Each UAV obtains its own position and velocity information from the sensor and obtains the state vector of the system at a certain moment;

[0057] S3: Use the neural network to update the network weight coefficients in reverse according to the minimum root mean square error loss function, induce the control input gradient descent according to the gradient descent of the weight coefficients, and calculate the updated control input;

[0058] S4: If the number of neural network iterations reaches the set value, or the drone's flight state converges to the desired state, then proceed to the next step S5, otherwise proceed to the previous step S3;

[0059] S5: Input the optimal control strategy into the drone queue model, and the drone queue updates its own state and enters step S2.

[0060] In some embodiments, the UAV dynamics model in step 1 is a single UAV longitudinal dynamics model based on a linear time-invariant expression established for a UAV swarm formation flying in the air, and the expression is:

[0061] ①

[0062] in, 、 are the system state transfer matrix and the system control quantity matrix respectively, is a matrix about the current state of the drone, It is the matrix of the control input of the pilot drone. is the system status information, For the control input of the drone, is the number of following drones in the cluster.

[0063] In step S3, the neural network input node is , The number of drones in the cluster = .

[0064] Expand the variables in formula ① and write it as follows:

[0065] ,

[0066]

[0067] ; Where: subscript 0 represents the pilot drone, subscript 1 to Representatives follow the drone, The position and velocity of the UAV are expressed as and , No. The drone is next to the drone in front of it. The distance between the racks is expressed as ; Represents the sensitivity of the following drone to the difference between the actual distance between itself and the adjacent drone and the expected distance. Represents the sensitivity of the following drone to the difference between its own and the neighboring drone’s actual and expected flight speeds; It represents the expected flight time interval in seconds, indicating that the expected flight distance between two adjacent UAVs at different speeds is different; is the time step; then the state space equation of the system is obtained based on the integration of the state of the UAV fleet at each moment. 、 are the system state transfer matrix and system control quantity matrix expanded by iterative calculation respectively.

[0068] In some embodiments, the is the acceleration of the pilot drone.

[0069] In step S2, each drone obtains its own position and speed information from its own sensor. The frequency of information acquisition is determined by the sampling time interval. Decision, obtain information and transmit it to the pilot drone through inter-drone communication to obtain the state vector of the system at a certain moment .

[0070] The root mean square error loss function expression in step S3 is:

[0071] , ②

[0072] Where, Indicates the control termination time The actual state of the system, Indicates the system at the control termination time expected state;

[0073] According to the principle of taking the minimum value of the above-mentioned root mean square error loss function, the network weight coefficient is updated in reverse , due to the weight coefficient Gradient descent will induce the control input Gradient descent, so that the control input can be continuously calculated and updated , thereby updating the neural network;

[0074] In some embodiments, the gradient expression is:

[0075] , ③

[0076] Where, Represents the learning rate, subscript Represents the order of the current gradient descent, represents the gradient, Indicates the weight Down gradient;

[0077] The iterative expression of the control input is:

[0078] , ④

[0079] Where, for The Jacobian matrix of , The square matrix is The linear transformation coefficients on .

[0080] like Figure 4The figure below shows an example of a drone formation application scenario. The leading drone controls its own acceleration by transmitting and receiving information, thereby guiding the following drones to form a formation.

[0081] The above are merely preferred embodiments of the present application and are not intended to limit the present application. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

[0082] Although the above describes the specific implementation methods of the present application in conjunction with the accompanying drawings, it does not limit the scope of protection of the present application. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present application, various modifications or variations that can be made by technical personnel in this field without creative work are still within the scope of protection of the present application.

Claims

1. A design method for a neural network controller for coordinating a swarm of drones, characterized by: This is achieved by following these steps: S1: For the drone fleet, establish a drone dynamics model based on linear time-invariant expressions and write the state space equations of the entire system; S2: Each UAV obtains its own position and velocity information from the sensor and obtains the state vector of the system at a certain moment; S3: Use the neural network to update the network weight coefficients in reverse according to the minimum root mean square error loss function, induce the control input gradient descent according to the gradient descent of the weight coefficients, and calculate the updated control input; S4: If the number of neural network iterations reaches the set value, or the drone's flight state converges to the desired state, then proceed to the next step S5, otherwise proceed to the previous step S3; S5: Input the optimal control strategy into the UAV fleet model, and the UAV fleet updates its own state and enters step S2; The root mean square error loss function expression in step S3 is: ， ② Where, Indicates the control termination time The actual state of the system, Indicates the system at the control termination time expected state; According to the principle of taking the minimum value of the above-mentioned root mean square error loss function, the network weight coefficient is updated in reverse , weight coefficient Gradient descent will induce the control input Gradient descent, and then continuously calculate and update the control input , thereby updating the neural network.

2. The method for designing a neural network controller for coordinating a swarm of drones according to claim 1, characterized in that: The UAV dynamics model in step 1 is a single UAV longitudinal dynamics model based on a linear time-invariant expression established for a UAV cluster formation flying in the air. The expression is: ① in, 、 are the system state transfer matrix and the system control quantity matrix respectively, is a matrix about the current state of the drone, It is the matrix of the control input of the pilot drone. is the system status information, For the control input of the drone, is the number of following drones in the cluster.

3. The method for designing a neural network controller for coordinating a swarm of drones according to claim 2, characterized in that: Expand the variables in formula ① and write it as follows: ， Where: subscript 0 represents the pilot drone, subscript 1 to Representatives follow the drone, The position and velocity of the UAV are expressed as and , No. The drone is next to the drone in front of it. The distance between the racks is expressed as ; Represents the sensitivity of the following drone to the difference between the actual distance between itself and the adjacent drone and the expected distance. Represents the sensitivity of the following drone to the difference between its own and the neighboring drone’s actual and expected flight speeds; It represents the expected flight time interval in seconds, indicating that the expected flight distance between two adjacent UAVs at different speeds is different; is the time step; The state space equation of the system is obtained by integrating the states of the drone fleet at each moment.

4. The method for designing a neural network controller for coordinating a swarm of drones according to claim 2, characterized in that: described 、 are the system state transfer matrix and system control quantity matrix expanded by iterative calculation respectively.

5. The method for designing a neural network controller for coordinating a swarm of drones according to claim 2, characterized in that: described is the acceleration of the pilot drone.

6. The method for designing a neural network controller for coordinating a swarm of drones according to claim 1, characterized in that: In step S2, each drone obtains its own position and speed information from its own sensor. The frequency of information acquisition is determined by the sampling time interval. Decision, obtain information and transmit it to the pilot drone through inter-drone communication to obtain the state vector of the system at a certain moment .

7. The method for designing a neural network controller for coordinating a swarm of drones according to claim 1, characterized in that: The gradient expression is: ， ③ Where, Represents the learning rate, subscript Represents the order of the current gradient descent, represents the gradient, Indicates the weight Down gradient; The iterative expression of the control input is: ， ④ Where, for The Jacobian matrix of can be expressed as , The square matrix is The linear transformation coefficients on .

8. The method for designing a neural network controller for coordinating a swarm of drones according to claim 1, characterized in that: The neural network input node is , The number of drones in the cluster = .

Images