Multi-unmanned aerial vehicle local route planning method and device based on improved Q-Learning

A multi-UAV, route planning technology, applied in neural learning methods, based on specific mathematical models, navigation calculation tools, etc., can solve the problems of long planning time, low algorithm convergence speed, long path search time, etc., to achieve rapid planning Effect

Pending Publication Date: 2021-04-27
NANJING UNIV OF AERONAUTICS & ASTRONAUTICS +2
View PDF0 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Ant colony algorithm has the advantages of strong robustness and good information feedback ability, but the convergence speed of the algorithm is low and it is easy to fall into local optimum
The genetic algorithm is robust because it does not depend on the model, but for complex battlefield environments, the convergence speed of the algorithm is slow, resulting in a long path search time
The A* algorithm has the advantages of simple algorithm and easy for engineers to take the lead, but its calculation amount is large and the planning time is long

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-unmanned aerial vehicle local route planning method and device based on improved Q-Learning
  • Multi-unmanned aerial vehicle local route planning method and device based on improved Q-Learning
  • Multi-unmanned aerial vehicle local route planning method and device based on improved Q-Learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0065] A kind of multi-UAV local route planning method based on improved Q-Learning of the present invention, see figure 1 shown, including

[0066] Step S1, obtaining the location of the newly added sudden threat source, determining the newly added sudden threat area in the flight environment, and then determining the starting point and target point of the local route replanning that can avoid the newly added sudden threat area.

[0067] The specific implementation process is:

[0068] According to the nature of multi-UAV collaborative local route planning tasks, when multi-UAVs fly to a certain waypoint along the global route, they use the sensor to detect the surrounding environment to collect information such as the position coordinates and radius of the sudden threat source. The local flight environment of the aircraft is updated.

[0069] The process of updating the UAV flight environment is to read the new mission environment. In the two-dimensional environment space...

Embodiment 2

[0128] The embodiment of the present invention is simulation analysis:

[0129] Figure 4 It is a comparison result diagram of the convergence experiment of the Q-Learning algorithm and the Q-Learning algorithm based on the CMAC neural network in the same simulation environment in the specific embodiment of the present invention. The horizontal axis in the figure is the number of iterations, that is, the number of algorithm learning times, and the vertical axis is Cumulative remuneration value. Depend on Figure 4 It can be seen that both algorithms converge, but the convergence speeds of the algorithms are different. Based on the Q-Learning algorithm, when the number of learning times is 120, the cumulative reward reaches the maximum value and no longer increases. At this time, the cumulative reward value is about 62; when the Q-Learning algorithm based on the CMAC neural network is 50 times, the cumulative reward The reward has reached the maximum value, and the cumulativ...

Embodiment 3

[0146] Correspondingly, an improved Q-Learning-based multi-UAV local route planning device of the present invention includes:

[0147] According to the location of the newly added sudden threat source, determine the newly added sudden threat area in the flight environment, and then determine the starting point and target point of the local route replanning that can avoid the newly added sudden threat area;

[0148] According to the starting point, target point and threat area of ​​multi-UAV local route re-planning, the multi-UAV local route planning system model in the process of multi-UAV local re-planning movement is determined;

[0149] Using the improved Q-Learning algorithm of the generalization ability of the cerebellar neural network to solve the above multi-UAV local route planning system model, and plan the optimal local route of multi-UAV;

[0150] According to the length of the re-planned local route section of the multi-UAV, adjust the speed of the multi-UAV when f...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an improved QLearning-based multi-unmanned aerial vehicle local airline planning method and an improved QLearning-based multi-unmanned aerial vehicle local airline planning device. The method specifically comprises the following steps: updating the flight environment of the unmanned aerial vehicle by utilizing sudden threat source information detected by a sensor; designing an action space and an action selection strategy, constructing a reward function, and constructing a multi-unmanned aerial vehicle local route planning system model; a cerebellar neural network algorithm is adopted as a state generalization method to be combined with a QLearning algorithm, a model is solved, an action strategy enabling an accumulated return value to be maximum is quickly found out, and therefore an optimal local airway under the multi-unmanned-aerial-vehicle uncertain environment is quickly planned. According to the invention, path search of multi-unmanned aerial vehicle local airline planning in an unknown environment can be quickly realized.

Description

technical field [0001] The present invention relates to the technical field of UAV route planning, in particular to a multi-UAV local route planning method based on improved Q-Learning, and also relates to a multi-UAV local route planning device based on improved Q-Learning. Background technique [0002] With the development of aviation technology, the use of multi-UAVs to conduct coordinated operations in complex and changeable environments has been widely used. Carrying out the research on UAV route planning method, while reducing the burden and inconvenience of manual route planning, can make full use of known terrain, threat and other information to complete the global route that meets its own constraints and mission requirements, in order to realize UAV low-altitude Penetration and covert flight provide technical guarantee. However, the complex and changeable mission environment makes the global route unable to fully guarantee the flight safety of the UAV during the mi...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06Q10/04G06N7/00G06N3/08G01C21/20
CPCG06Q10/047G06N3/08G01C21/20G06N7/01
Inventor 刘蓉袁佳乐张衡羊书杰
Owner NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap