Distributed formation method of unmanned aerial vehicle cluster based on reinforcement learning

Abstract

The invention discloses a distributed formation method of an unmanned aerial vehicle cluster based on reinforcement learning. The distributed formation method comprises the steps that step (1), a formation target state function and a simulation model of environmental uncertainty factors are obtained, and an unmanned aerial vehicle formation simulation model is established; step (2), under the interference of the environmental uncertainty factors, based on the unmanned aerial vehicle formation simulation model established in the step (1), a Q learning method is adopted to train the unmanned aerial vehicle cluster to update a flight strategy table; step (3), the value of the completion degree of the formation target state is calculated according to the obtained formation target state function, the obtained value of the completion degree of the formation target state is compared with a preset value of the formation target state, whether the formation target state is reached or not is judged according to the comparison results, if the formation target state is reached, a step (4) is performed, and if not, the step (2) is entered; and step (4), the updated flight strategy table is saved. According to the distributed formation method of the unmanned aerial vehicle cluster based on reinforcement learning, flight strategy parameters with adaptability are provided for the cluster, and the stability and robustness of the unmanned aerial vehicle cluster formation are guaranteed.

Description

technical field [0001] The invention belongs to the field of automatic control of unmanned aerial vehicles, and in particular relates to a distributed formation method of unmanned aerial vehicle clusters based on reinforcement learning, which is used for adaptive formation control of unmanned aerial vehicle clusters under different formation targets. Background technique [0002] With the development of UAV technology, the application of UAV in military and civilian fields has been greatly expanded. In the military field, UAVs are considered to be able to replace manned aircraft to perform tasks such as "boring, harsh, dangerous, and deep" due to their low casualties, low life-cycle cost, and strong continuous combat capability; in the civilian field , UAVs have demonstrated their advantages in geological exploration, earthquake relief, emergency communications, and freight transportation. However, due to the limited communication distance, computing power and battery energ...

AI Technical Summary

Problems solved by technology

[0003] At present, the formation mode can be divided into structured formation mode and unstructured formation mode: the structured formation mode relies on the physical leader or the virtual leader to form a preset structured formation, however, the formation and maintenance of the formation depend on the leader The leader's control of the global information puts forward higher requirements on the performance of the communication network in the cluster, and due to the central control method, the failure of the leader will have a serious impact on the cluster formation; compared with the structured formation mode, the very The structured formation mode adopts a non-centered control method, which has great advantages in formation stability and network availability. However, as the formation scale increases, the control ability of the unstructured formation mode to the cluster behavior is much lower than that of the structured formation. Mode, unable to effectively unify cluster behavior

[0004] In open airspace, due to the influence of unknown factors such as airflow, temperature, and terrain, the formation of UAV swarms is extremely vulnerable to damage

The existing model-based formation methods are affected by modeling capabilities and are only applicable to some airspaces, which are not sufficiently robust and universal

Images