Unmanned aerial vehicle formation collision avoidance method based on deep reinforcement learning
A reinforcement learning, unmanned aerial vehicle technology, applied in mechanical equipment, combustion engines, internal combustion piston engines, etc., to achieve the effect of improving mobility
Pending Publication Date: 2022-02-08
北航(四川)西部国际创新港科技有限公司
View PDF0 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0005] Aiming at the problem of multi-UAV formation collision avoidance, the present invention proposes a UAV formation collision avoidance method based on deep reinforcement learning, and coordinates and controls the UAV formation as a whole to achieve the purpose of avoiding collision and successfully completing tasks
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0051] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and examples.
[0052] A collision avoidance method for UAV formations based on deep reinforcement learning, the specific steps are as follows:
[0053] Step 1: First select the deep reinforcement learning model as the main framework, and then set the initial parameters according to the industry's mature experiments. It is clear that the training goal is to output the strategy that enables the UAV to autonomously avoid collisions, and on this basis, set different constraints This enables UAVs to maintain formation to a certain extent. The environment involved in the invention includes leaders, followers, and obstacles, which will be divided by superscripts F, L, and O for ease of distinction.
[0054] The state space of the UAV at time t can be expressed as s t , the...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention provides an unmanned aerial vehicle formation collision avoidance method based on deep reinforcement learning, and the method comprises the steps of determining that a training target is to output a strategy enabling unmanned aerial vehicles to achieve the autonomous collision avoidance flight, and enabling the unmanned aerial vehicles to keep formation through setting different constraint conditions; training the unmanned aerial vehicles in a simulation environment, setting different reward values by selecting different behaviors to generate a strategy based on a collision avoidance behavior, and recording various state information and collision avoidance strategies of the unmanned aerial vehicles; processing external environment information by adopting an LSTM (Long Short Term Memory) mode in a recurrent neural network, and training on the basis of an initial strategy in combination with the state information of the unmanned aerial vehicles; adding different constraint conditions on the basis of collision avoidance, so that the unmanned aerial vehicles keep a certain formation to fly on the basis of avoiding inter-queue collision, and continuous operation optimization is performed through the model. According to the invention, effective unification of unmanned aerial vehicle collision avoidance and formation is realized, resources can be effectively integrated, and the optimal collision avoidance behavior can be obtained by adjusting the behavior of an individual in real time.
Description
technical field [0001] The invention relates to the field of deep reinforcement learning and the technical field of unmanned aerial vehicles, in particular to a method for avoiding collisions of unmanned aerial vehicle formations based on deep reinforcement learning. Background technique [0002] In recent years, multi-agents have attracted more and more researches due to their great potential in different fields. The fields involved include coordinated exploration of monitoring and rescue, coordinated control of satellite clusters, and formation control of drones. The basic concept of a multi-agent system is to use individuals to cooperate to solve complex tasks that cannot be accomplished by a single agent even with expensive equipment. Formation control is a basic problem in multi-agent systems, and its goal is to achieve and maintain a certain formation shape, so that multi-agent systems can jointly complete specific tasks. Formation keeping is an important issue in fo...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G05D1/10
CPCG05D1/104Y02T10/40
Inventor 张学军王思峰唐立
Owner 北航(四川)西部国际创新港科技有限公司