Unmanned aerial vehicle cluster meeting method based on deep reinforcement learning

Through deep reinforcement learning technology, the use of neural network systems to adjust the flight trajectory and speed of the drone cluster in real time, solving the problems of slow convergence speed and poor shape consistency of traditional drone cluster algorithms, achieving more efficient cluster shape maintenance and The task is completed.

CN118915818APending Publication Date: 2024-11-08JIANGSU DIGITAL EAGLE TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410959085.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The traditional drone cluster algorithm has a slow convergence speed and a long task completion time. It is impossible to adjust the flight trajectory in time to deal with high-altitude obstacles, resulting in a large gap between the cluster shape and the expected effect.

Method used

Using a deep reinforcement learning method, the connection between the drone and the control system is established through the neural network system, predict and adjust the flight trajectory in real time, maintain the consistency of the cluster shape, and use Bluetooth connection and neural network for information transmission and processing, and adjust in real time. The drone's flight speed and trajectory are used to avoid obstacles.

Benefits of technology

It improves the shape consistency and mission efficiency of the drone cluster during flight, ensures that the flight route can be adjusted in time when encountering high-altitude obstacles, maintains the stability of the cluster shape, and narrows the gap with the expected effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118915818A_ABST
    Figure CN118915818A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of machine learning and route planning, and discloses an unmanned aerial vehicle cluster meeting method based on deep reinforcement learning, which comprises the following steps: establishing a neural network system, and establishing a relation between an unmanned aerial vehicle and the neural network system. When the information is dissimilar, the control system controls the whole unmanned aerial vehicle of the unmanned aerial vehicle cluster to decelerate, the flight speed and the flight path are revised again through the network system processing unit, the unmanned aerial vehicle continues to fly according to the revised result, and the unmanned aerial vehicle does not fly when flying. And at the moment, the unmanned aerial vehicle flight information is continuously matched with the unmanned aerial vehicle flight information of the previous moment through a network system within a certain time period, so that the difference between the unmanned aerial vehicle cluster and the expected unmanned aerial vehicle cluster is reduced to a certain extent when the unmanned aerial vehicle cluster flies. The device is practical and suitable for being widely popularized and used.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning and route planning, and specifically relates to a drone cluster rendezvous method based on deep reinforcement learning. Background Art

[0002] In order to achieve precise strikes on a target area by drone swarms or complete reconnaissance and search missions in a certain area and improve the success rate of completing the mission, multiple drones are often required to fly to a target area from different directions to complete the rendezvous mission. Traditional solutions to drone swarm rendezvous missions mostly use basic consistency algorithms and consistency algorithms based on past state differences. These algorithm systems have slow convergence speeds, take a long time to complete missions, and have low effectiveness and feasibility.

[0003] However, when establishing a cluster, most of the existing drones use a characteristic operation trajectory, which ensures that the distances between two drones are similar during flight. However, when flying, the existing drones may have a flight trajectory at one moment and a trajectory at the next moment, and when the change is large, it is often impossible to make timely changes, resulting in this state for a period of time. Therefore, it is easy for the shape of the drone cluster to be quite different from the expected effect.

[0004] In view of this, the present invention is proposed. Summary of the invention

[0005] In order to solve the above technical problems, the basic concept of the technical solution adopted by the present invention is: A drone cluster rendezvous method based on deep reinforcement learning, including the following methods: S1. Establish a neural network system and connect the drone with the neural network system; S2. The flight trajectory of the drone is estimated through the neural network system, and the drone is allowed to fly according to the preset flight trajectory, and multiple drones are placed in a drone cluster state, and the shape of the drone cluster state at this time is retained and displayed; S3. The drone may encounter high-altitude obstacles during flight; S4, based on the above-mentioned UAV encountering a high-altitude obstacle, the flight speed of the UAV is controlled by the control system, and the flight route of the UAV is re-preset and planned, so that the re-preset and planned flight trajectory still conforms to the aircraft initialization cluster form; S5. The reset flight trajectory is displayed on the neural network system, and a simulation match is performed based on the shape of the drone cluster in the neural network at the last moment and the shape of the drone cluster in the preset drone cluster flight trajectory. S6. Based on the above, when the drone cluster avoids high-altitude obstacles, it is still in the original drone cluster shape.

[0006] As a preferred embodiment of the present invention, a method for establishing a connection between the specific network nervous system and the drone based on the above S1 is: the user logs in to the network nervous system and enables the network nervous system to be turned on. At this time, the user establishes a wireless connection through Bluetooth between the drones to be used, and the information of the drones can be reflected on the network nervous system.

[0007] As a preferred embodiment of the present invention, the specific method steps based on the estimated flight trajectory in S2 above and allowing the drone cluster to be displayed are: the user edits on the network neural system according to the requirements that the drone needs to achieve, so that multiple drones can establish a drone cluster. When the drones establish a cluster, the established cluster information images can be transmitted to the neural network system for response. At this time, the user plans the optimal driving trajectory of the drone in the cluster state on the neural network system based on the starting position and the end position of the drone.

[0008] As a preferred embodiment of the present invention, a specific method for re-presetting and planning the flight route of the drone based on the above-mentioned S4 when the drone encounters a high-altitude obstacle during driving is as follows: when the drone cluster is flying, the area occupied by the control is large, and there is a certain distance between the two drones. When one of the drones encounters a high-altitude obstacle while driving in front, the drone will be able to transmit information to the neural network system through its own camera function. When the neural network system receives this signal, it will be able to perform emergency processing on it through the processor. During the processing, the flight speed and flight trajectory of the drone will be able to be readjusted, and the drone will be able to maintain the initialized cluster drone form after processing.

[0009] As a preferred embodiment of the present invention, the specific method for simulating and matching the drone cluster in the above S5 is as follows: when the processor presets a route, preprocessing will be performed at this time. During the preprocessing, new drone cluster trajectory information will be formed, and the information can be reflected on the network system, and the information is extracted by the processing unit and matched with the information of the initial position of the aircraft cluster to determine whether there is similarity: when similar, the drone can continue to fly, and when not similar, it is determined that the drone flight trajectory and speed need to be modified again, and the above steps are repeated until the aircraft receives a similar command to continue flying, and the drone cluster flies according to the revised command.

[0010] As a preferred embodiment of the present invention, the specific flight method after the UAV avoids high-altitude obstacles in the above S6 is as follows: after the UAV avoids the high-altitude obstacles, it is still in a cluster state and is flying according to the re-planned route. Therefore, the flight speed of the UAV is restored to the flight speed before avoiding the obstacle through the processing unit in the network system.

[0011] As a preferred embodiment of the present invention, based on the above-mentioned specific simulation matching steps: when a drone establishes a drone cluster, the drone cluster information will be transmitted to the network system at this time, and the network system can reflect in real time the drone cluster information at a certain time end when the drone cluster is flying and the flight time of the drone at the previous moment, and match it on the network system to determine whether the information is similar; when the information is similar, the drone can maintain normal flight, and when the information is dissimilar, the control system controls the entire drone cluster to slow down, and revises the flight speed and flight trajectory through the network system processing unit, and allows the drone to continue flying according to the revised results. When the drone is flying, it continues to match the drone flight information of the previous moment on the network system within a certain period of time.

[0012] Compared with the prior art, the present invention has the following beneficial effects: According to the present invention, when establishing a drone cluster, the drone will transmit the information of the drone cluster to the network system, and the network system can reflect the drone cluster information at a certain time when the drone cluster is flying in real time and match the flight time of the drone at the previous moment on the network system to determine whether the information is similar; when the information is similar, the drone can maintain normal flight, and when the information is not similar, the control system controls the entire drone cluster to decelerate, and revises the flight speed and flight trajectory through the network system processing unit, and allows the drone to continue flying according to the revised results, and when the drone is flying, it continues to match the drone flight information at the previous moment on the network system within a certain period of time, so that the gap between the drone cluster and the expectation when flying is reduced to a certain extent.

[0013] The specific implementation modes of the present invention are further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In the attached picture: Figure 1 It is a schematic diagram of the overall process of the present invention; Figure 2 It is a schematic diagram of the matching process of the present invention; Figure 3 It is a schematic diagram of the flight trajectory of the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The following embodiments are used to illustrate the present invention.

[0016] like Figures 1 to 3 As shown, a drone cluster rendezvous method based on deep reinforcement learning includes the following methods: S1. Establish a neural network system and connect the drone with the neural network system; S2. The flight trajectory of the drone is estimated through the neural network system, and the drone is allowed to fly according to the preset flight trajectory, and multiple drones are placed in a drone cluster state, and the shape of the drone cluster state at this time is retained and displayed; S3. The drone may encounter high-altitude obstacles during flight; S4, based on the above-mentioned UAV encountering a high-altitude obstacle, the flight speed of the UAV is controlled by the control system, and the flight route of the UAV is re-preset and planned, so that the re-preset and planned flight trajectory still conforms to the aircraft initialization cluster form; S5. The reset flight trajectory is displayed on the neural network system, and a simulation match is performed based on the shape of the drone cluster in the neural network at the last moment and the shape of the drone cluster in the preset drone cluster flight trajectory. S6. Based on the above, when the drone cluster avoids high-altitude obstacles, it is still in the original drone cluster shape.

[0017] In a specific implementation, the method for establishing a connection between the specific network nervous system and the drone in S1 is: the user logs in to the network nervous system and enables the network nervous system to be turned on. At this time, the user establishes a wireless connection between the drones to be used through Bluetooth, and the information of the drones can be reflected on the network nervous system; Establish (x, y, z coordinates on the system; for example, a 10*10*10 three-dimensional space is divided into 10*10*10=1000 small cubes, each cube represents a grid, and the center of the cube is (xyz). If the obstacle is within this grid, the grid is 1, and the blank is 0).

[0018] Furthermore, the specific method steps based on the estimated flight trajectory in the above S2 and the display of the drone cluster are: the user edits the network neural system according to the requirements that the drone needs to achieve, so that multiple drones can establish a drone cluster. When the drones establish a cluster, the established cluster information images can be transmitted to the neural network system for response. At this time, the user plans the optimal driving trajectory of the drone in the cluster state on the neural network system based on the starting position and the end position of the drone.

[0019] Furthermore, based on the above S4, the specific method for re-presetting and planning the flight route of the drone when the drone encounters a high-altitude obstacle during driving is as follows: when the drone cluster is flying, the area occupied by the control is large, and there is a certain distance between the two drones. When one of the drones encounters a high-altitude obstacle while driving in front, the drone will be able to transmit information to the neural network system through its own camera function. When the neural network system receives this signal, it will be able to perform emergency processing on it through the processor. During the processing, the flight speed and flight trajectory of the drone will be able to be readjusted, and the drone will be able to maintain the initialized cluster drone form after processing.

[0020] Furthermore, the specific method for simulating and matching the drone cluster in the above S5 is as follows: when the processor presets a route, preprocessing will be performed at this time. During the preprocessing, new drone cluster trajectory information will be formed, and the information can be reflected on the network system, and the information is extracted by the processing unit and matched with the information of the initial position of the aircraft cluster to determine whether there is similarity: when similar, the drone can continue to fly, and when not similar, it is determined that the drone flight trajectory and speed need to be modified again, and the above steps are repeated until the aircraft receives a similar command to continue flying, and the drone cluster flies according to the revised command.

[0021] Furthermore, based on the specific flight method of the drone after avoiding high-altitude obstacles in the above S6, the drone is still in a cluster state after avoiding the high-altitude obstacles, and is also flying according to the re-planned route. Therefore, the flight speed of the drone is restored to the flight speed before avoiding the obstacle through the processing unit in the network system.

[0022] Furthermore, based on the above specific simulation matching steps: when the drone establishes a drone cluster, the drone cluster information will be transmitted to the network system at this time, and the network system can reflect the drone cluster information at a certain time end when the drone cluster is flying in real time and match the flight time of the drone at the previous moment on the network system to determine whether the information is similar; when the information is similar, the drone can maintain normal flight, and when the information is not similar, the control system controls the entire drone cluster to slow down, and revises the flight speed and flight trajectory through the network system processing unit, and allows the drone to continue flying according to the revised results. When the drone is flying, it continues to match the drone flight information at the previous moment on the network system within a certain period of time.

[0023] The implementation principle of a drone cluster rendezvous method based on deep reinforcement learning in this embodiment is as follows: when a drone establishes a drone cluster, the drone cluster information is transmitted to the network system at this time, and the network system can reflect in real time the drone cluster information at a certain time end when the drone cluster is flying and match the flight time of the drone at the previous moment on the network system to determine whether the information is similar; when the information is similar, the drone can maintain normal flight, and when the information is dissimilar, the control system controls the entire drone cluster to slow down, and revises the flight speed and flight trajectory through the network system processing unit, and allows the drone to continue flying according to the revised results. When the drone is flying, it continues to match the drone flight information of the previous moment on the network system within a certain period of time, where Flight 1, Flight 2 and Flight 3 represent the drone itself; V represents speed.

Claims

1. A drone cluster rendezvous method based on deep reinforcement learning, characterized in that: Includes the following methods: S1. Establish a neural network system and connect the drone with the neural network system; S2. The flight trajectory of the drone is estimated through the neural network system, and the drone is allowed to fly according to the preset flight trajectory, and multiple drones are placed in a drone cluster state, and the shape of the drone cluster state at this time is retained and displayed; S3. The drone may encounter high-altitude obstacles during flight; S4, based on the above-mentioned UAV encountering a high-altitude obstacle, the flight speed of the UAV is controlled by the control system, and the flight route of the UAV is re-preset and planned, so that the re-preset and planned flight trajectory still conforms to the aircraft initialization cluster form; S5. The reset flight trajectory is displayed on the neural network system, and a simulation match is performed based on the shape of the drone cluster in the neural network at the last moment and the shape of the drone cluster in the preset drone cluster flight trajectory. S6. Based on the above, when the drone cluster avoids high-altitude obstacles, it is still in the original drone cluster shape.

2. The method for gathering drone clusters based on deep reinforcement learning according to claim 1, characterized in that: The method for establishing a connection between the specific network nervous system and the drone based on the above S1 is: the user logs in to the network nervous system and enables the network nervous system to be turned on. At this time, the user establishes a wireless connection between the drones to be used through Bluetooth, and the information of the drone can be reflected on the network nervous system.

3. The method for gathering drone clusters based on deep reinforcement learning according to claim 1, characterized in that: The specific method and steps for displaying the estimated flight trajectory based on the above S2 and the drone cluster are as follows: the user edits the network neural system according to the requirements that the drone needs to achieve, so that multiple drones can establish a drone cluster. When the drones establish a cluster, the established cluster information images can be transmitted to the neural network system for response. At this time, the user plans the optimal driving trajectory of the drone in the cluster state on the neural network system based on the starting position and the end position of the drone.

4. The method for gathering drone clusters based on deep reinforcement learning according to claim 1, characterized in that: The specific method for re-presetting and planning the flight route of the drone based on the above S4 when the drone encounters a high-altitude obstacle during driving is: when the drone cluster is flying, the area occupied by the control is large, and there is a certain distance between the two drones. When one of the drones encounters a high-altitude obstacle while driving in front, the drone will be able to transmit information to the neural network system through its own camera function. When the neural network system receives this signal, it will be able to perform emergency processing on it through the processor. During the processing, the flight speed and flight trajectory of the drone will be able to be readjusted, and the drone will be able to maintain the initialized cluster drone form after processing.

5. The method for gathering drone clusters based on deep reinforcement learning according to claim 1, characterized in that: The specific method for simulating and matching the drone cluster based on the above S5 is: when the processor presets a route, preprocessing will be performed at this time. During the preprocessing, new drone cluster trajectory information will be formed, and the information can be reflected on the network system, and the information is extracted by the processing unit and matched with the information of the initial position of the aircraft cluster to determine whether there is similarity: when similar, the drone can continue to fly, and when not similar, it is determined that the drone flight trajectory and speed need to be modified again, and the above steps are repeated until the aircraft receives a similar command to continue flying, and the drone cluster flies according to the revised command.

6. The method for gathering drone clusters based on deep reinforcement learning according to claim 1, characterized in that: Based on the specific flight method of the drone after avoiding high-altitude obstacles in the above S6, the drone is still in a cluster state after avoiding the high-altitude obstacles, and is also flying according to the re-planned route. Therefore, the flight speed of the drone is restored to the flight speed before avoiding the obstacle through the processing unit in the network system.

7. The method for gathering drone clusters based on deep reinforcement learning according to claim 1, characterized in that: Based on the above specific simulation matching steps: when the drone establishes a drone cluster, the drone cluster information will be transmitted to the network system at this time, and the network system can reflect the drone cluster information at a certain time when the drone cluster is flying in real time and match the flight time of the drone at the previous moment on the network system to determine whether the information is similar; when the information is similar, the drone can maintain normal flight, and when the information is not similar, the control system controls the entire drone cluster to slow down, and revises the flight speed and flight trajectory through the network system processing unit, and allows the drone to continue flying according to the revised results. When the drone is flying, it continues to match the drone flight information at the previous moment on the network system within a certain period of time.

Citation Information

Cited By

  • Multi-unmanned aerial vehicle cooperative navigation method based on deep reinforcement learning

    CN120800386A