Unmanned aerial vehicle cluster route planning method and system based on behavior cloning and medium
By adopting a behavioral cloning-based route planning method in the drone cluster, using neural network imitation learning to improve path planning efficiency, the problem of inefficiency of traditional algorithms in complex scenarios is solved, and efficient and accurate route planning is achieved.
Patent Information
- Application Number
- CN202411468308.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-21
- Publication Date
- 2025-02-14
AI Technical Summary
In the case of large-scale drone clusters and complex scenarios, traditional A* algorithms and heuristic algorithm benchmark improvement algorithms have the problem of low path planning efficiency.
Using a drone cluster route planning method based on behavior cloning, through the combination of state space module, teaching module, behavior cloning module and action space module, the route planning algorithm is used to generate path planning experience data for neural network imitation and learning, thereby improving Efficiency of route planning.
It significantly improves the efficiency of drone cluster route planning, making it comparable to traditional route planning algorithms, and at the same time improves the accuracy of route planning.
Smart Images
Figure CN119437231A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of drone technology, and in particular to a drone cluster route planning method, system and medium based on behavior cloning. Background Art
[0002] The traditional improved algorithm based on the A* algorithm and heuristic algorithm traverses all reachable nodes in the scene to find the shortest path from the starting point to the target point. However, it has the problem of low path planning efficiency when the drone cluster is large and the scene is complex. Summary of the invention
[0003] The embodiments of the present application provide a method, system and medium for UAV cluster route planning based on behavior cloning, which uses a route planning algorithm as an "expert" to generate path planning experience data for neural network imitation learning. As the actual data collected is improved, a neural network with route planning accuracy comparable to that of a traditional route planning algorithm can be obtained, thereby significantly improving route planning efficiency.
[0004] The present application embodiment provides a UAV cluster route planning system based on behavior cloning, including:
[0005] A state space module, used to obtain key factor data of collaborative task planning and perform vector processing on the key factor data;
[0006] A teaching module is used to solve the route planning scheme based on the route planning algorithm under the current spatial constraints by using the vectors obtained by processing;
[0007] A behavior cloning module is used to learn the route planning solution solved by the teaching module based on the neural network model, and interact with the cluster UAV test environment to generate state data, so as to add labels to the generated state data to train the neural network model;
[0008] The action space module converts the final planning scheme determined by the teaching module and the behavior cloning module into a standard vector, and decodes it into a task allocation instruction and inputs it into the cluster UAV test environment, so that the cluster UAV executes the route planning instruction according to the joint action instruction set with a time relationship under the given constraints or task rules.
[0009] Optionally, the state space module is specifically used to construct three 0-1 matrices, which are respectively used as a feasible path matrix, a drone position matrix and a target position matrix, and the sizes of the feasible path matrix, the drone position matrix and the target position matrix are adapted to the scene size.
[0010] Optionally, the teaching module is specifically used to record the solved route planning solution in a fixed format and perform standardization processing to train the neural network model after the standardization processing.
[0011] Optionally, the behavior cloning module is specifically used to utilize the neural network model to interact with the cluster drone test environment and record the collected status data, and to add labels to the collected status data to train the neural network model, wherein the data with correct labels within a cycle is updated to the teaching data set of the behavior cloning module to incrementally update and train the neural network model.
[0012] Optionally, the action space module is specifically used to generate new state data after executing the route planning instructions under given constraints or task rules.
[0013] Optionally, the teaching module and the behavior cloning module are further used for:
[0014] When the swarm of drones does not reach the preset scale, the route planning algorithm is directly used to solve the route planning scheme;
[0015] When the drone swarm reaches a certain size, the route planning solution is obtained based on imitation learning of the neural network model.
[0016] The present application embodiment also proposes a UAV cluster route planning method based on behavior cloning, including:
[0017] Acquire key factor data of collaborative task planning, and perform vectorization processing on the key factor data;
[0018] Using the processed vectors, the route planning scheme is solved based on the route planning algorithm under the current spatial constraints;
[0019] Learning the route planning solution based on the neural network model and interacting with the cluster UAV test environment to generate state data, so as to add labels to the generated state data to train the neural network model;
[0020] The final planning scheme determined by the solution or the neural network model is converted into a standard vector and decoded into a task assignment instruction and input into the cluster UAV test environment, so that the cluster UAV executes the route planning instruction according to the joint action instruction set with a timing relationship under the given constraints or task rules.
[0021] An embodiment of the present application also proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the aforementioned method for UAV cluster route planning based on behavior cloning are implemented.
[0022] The embodiment of the present application uses a route planning algorithm as an "expert" to generate path planning experience data for the neural network to imitate and learn. As the actual data collected is improved, a neural network with route planning accuracy comparable to that of a traditional route planning algorithm can be obtained, thereby significantly improving route planning efficiency.
[0023] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0025] Figure 1 This is a schematic diagram of the architecture of the UAV cluster route planning system according to an embodiment of the present application;
[0026] Figure 2 A schematic diagram of the state space module construction of the UAV cluster route planning system according to an embodiment of the present application;
[0027] Figure 3 This is a schematic diagram of the standardization process of the teaching module of the UAV cluster route planning system according to an embodiment of the present application;
[0028] Figure 4 This is a schematic diagram of the route generation process based on behavior cloning of the drone cluster route planning system in an embodiment of the present application. DETAILED DESCRIPTION
[0029] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0030] The present application embodiment provides a UAV swarm route planning system based on behavior cloning, which mainly includes:
[0031] 1. State space module construction
[0032] The state space module collects the key factors that affect route planning, vectorizes and standardizes the data, and then inputs it into the teaching module.
[0033] 2. Teaching module construction and sampling
[0034] The route planning results are generated using the traditional traversal search algorithm, and the expert experience samples are vectorized and standardized.
[0035] 2. Construction and training of behavior cloning module
[0036] The behavior cloning module uses a supervised learning algorithm to generate teaching label samples of <key influencing factors of route planning, route planning scheme> in the learning and teaching module.
[0037] 4. Action space module construction
[0038] The action space module generates standardized task instructions based on the output results and inputs them into the simulation environment.
[0039] Specifically, Figure 1 As shown, the UAV cluster route planning system based on behavior cloning in the embodiment of the present application includes:
[0040] The state space module is used to obtain key factor data of collaborative task planning and vectorize the key factor data. For example, in some examples, key factors affecting task performance can be collected, and the data can be vectorized and standardized before being input into the teaching module.
[0041] The teaching module is used to solve the route planning scheme based on the route planning algorithm under the current spatial constraints by using the vectors obtained by processing. In some embodiments, the route planning scheme P based on expert experience solved by the traditional route planning algorithm can be used.
[0042] The behavior cloning module is used to learn the route planning solution solved by the teaching module based on the neural network model, and interact with the cluster drone test environment to generate state data, so as to add labels to the generated state data to train the neural network model.
[0043] The action space module converts the final planning scheme determined by the teaching module and the behavior cloning module into a standard vector, and decodes it into a task allocation instruction and inputs it into the cluster UAV test environment, so that the cluster UAV executes the route planning instruction according to the joint action instruction set with a time relationship under the given constraints or task rules.
[0044] In some embodiments, Figure 2 As shown, the state space module is specifically used to construct three 0-1 matrices, which are respectively used as feasible route matrices A=(aij ) m×n 、UAV position matrix B=(b ij ) m×n and target position matrix C = (c ij ) m×n The sizes of the feasible path matrix, the UAV position matrix and the target position matrix are adapted to the scene size, and 1 and 0 in the matrix represent True and False, respectively.
[0045] In some embodiments, the teaching module is specifically used to record the solved route planning solution in a fixed format and perform standardization processing to train the neural network model after the standardization processing. Figure 3 As shown, the format of the route planning solution P based on expert experience solved by the route planning algorithm is as follows:
[0046] {agent1:[position start ,position target ,route1],
[0047] agent2:[position start ,position target ,route2], ...
[0049] agentN:[position start ,position target ,route N ]}
[0050] The expert experience data is stored in JSON format to improve the data reading efficiency. In addition, the expert experience data is standardized to facilitate the training of neural networks.
[0051] In some embodiments, the behavior cloning module is specifically used to utilize the neural network model to interact with the cluster drone test environment and record the collected status data, and to add labels to the collected status data to train the neural network model, wherein the data with correct labels within a cycle is updated to the teaching data set of the behavior cloning module to incrementally update and train the neural network model.
[0052] In order to make the training data distribution of the neural network consistent with the current strategy π of the behavior cloning module θThe collected data is as consistent as possible, and the neural network in the behavior cloning module is used to interact with the cluster drone experimental environment, and the periodically collected mission status data is stored in the data set. The traditional route planning algorithm is used to label the data in the newly collected status data set, and the data with the correct label is updated to the teaching data set of the behavior cloning module within the periodic time, and the neural network strategy is incrementally updated and trained.
[0053] In some embodiments, the action space module is specifically used to generate new state data after executing the route planning instructions under given constraints or task rules.
[0054] In some embodiments, the teaching module and the behavior cloning module are further used to:
[0055] When the swarm of drones does not reach the preset scale, the route planning algorithm is directly used to solve the route planning scheme;
[0056] When the drone swarm reaches a certain size, the route planning solution is obtained based on imitation learning of the neural network model.
[0057] As a supplementary explanation of the system of the embodiment of the present application, the embodiment of the present application also proposes a regional search model solution process based on the behavior cloning method, such as Figure 4 shown.
[0058] Figure 4 In the figure, the process of the classic route planning algorithm is (1)→(2), that is, the switch of the simulation system is placed in position I, and the classic planning algorithm in the teaching module uses the classic planning algorithm according to the task state O of the experimental environment to feedback the optimal planning instruction. The process of the planning algorithm based on imitation learning is (9)→(10), that is, the switch of the experimental environment is reset to position III, the neural network in the behavior cloning module fixes the weights of the neural network of the fitted task planning function, takes the task state of the experimental environment as the input of the neural network, and takes the output of the neural network as the planning instruction.
[0059] The actual workflow is: when the swarm of drones has not reached a certain scale, the classic planning algorithm process (1)→(2) is started; when the drone swarm reaches a certain scale, the planning algorithm process based on imitation learning is started (9)→(10). The specific steps of imitation learning are as follows:
[0060] Step 1: Set the simulation system switch to position III and let the neural network π θ Interact with the experimental environment, execute steps (9)→(10), periodically collect key influencing factors that affect task effectiveness, and construct a data set of key influencing factors of task effectiveness.
[0061] Step 2: Set the experimental environment switch to position II and use the classic planning algorithm to calculate the task state data set D π In the paper, the fitness function of cluster drones under different key influencing factors of mission effectiveness is given, and the corresponding mission planning scheme is given as the data label to construct the teaching data set D.
[0062] Step 3: Merge the teaching data set and use D as the sample data set to perform neural network supervised learning, that is, execute steps (6)→(7)→(8) until the accuracy reaches the threshold and then go to step 1.
[0063] The embodiment of the present application uses a route planning algorithm as an "expert" to generate path planning experience data for the neural network to imitate and learn. As the actual data collected is improved, a neural network with route planning accuracy comparable to that of a traditional route planning algorithm can be obtained, thereby significantly improving route planning efficiency.
[0064] The present application embodiment also proposes a UAV cluster route planning method based on behavior cloning, including:
[0065] Acquire key factor data of collaborative task planning, and perform vectorization processing on the key factor data;
[0066] Using the processed vectors, the route planning scheme is solved based on the route planning algorithm under the current spatial constraints;
[0067] Learning the route planning solution based on the neural network model and interacting with the cluster UAV test environment to generate state data, so as to add labels to the generated state data to train the neural network model;
[0068] The final planning scheme determined by the solution or the neural network model is converted into a standard vector and decoded into a task assignment instruction and input into the cluster UAV test environment, so that the cluster UAV executes the route planning instruction according to the joint action instruction set with a timing relationship under the given constraints or task rules.
[0069] An embodiment of the present application also proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the aforementioned method for UAV cluster route planning based on behavior cloning are implemented.
[0070] It should be noted that in the various embodiments of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.
[0071] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0072] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0073] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. A UAV swarm route planning system based on behavior cloning, characterized in that: include: A state space module, used to obtain key factor data of collaborative task planning and perform vector processing on the key factor data; A teaching module is used to solve the route planning scheme based on the route planning algorithm under the current spatial constraints by using the vectors obtained by processing; A behavior cloning module is used to learn the route planning solution solved by the teaching module based on the neural network model, and interact with the cluster UAV test environment to generate state data, so as to add labels to the generated state data to train the neural network model; The action space module converts the final planning scheme determined by the teaching module and the behavior cloning module into a standard vector, and decodes it into a task allocation instruction and inputs it into the cluster UAV test environment, so that the cluster UAV executes the route planning instruction according to the joint action instruction set with a time relationship under the given constraints or task rules.
2. The UAV swarm route planning system based on behavior cloning as claimed in claim 1, characterized in that: The state space module is specifically used to construct three 0-1 matrices, which are respectively used as a feasible path matrix, a drone position matrix and a target position matrix. The sizes of the feasible path matrix, the drone position matrix and the target position matrix are adapted to the scene size.
3. The UAV swarm route planning system based on behavior cloning as claimed in claim 1, characterized in that: The teaching module is specifically used to record the solved route planning scheme in a fixed format and perform standardization processing to train the neural network model after the standardization processing.
4. The UAV swarm route planning system based on behavior cloning as claimed in claim 3, characterized in that: The behavior cloning module is specifically used to interact with the cluster drone test environment using the neural network model and record the collected status data, and to add labels to the collected status data to train the neural network model, wherein the data with correct labels within a cycle is updated to the teaching data set of the behavior cloning module to incrementally update and train the neural network model.
5. The UAV swarm route planning system based on behavior cloning as claimed in claim 1, characterized in that: The action space module is specifically used to generate new state data after executing the route planning instructions under given constraints or task rules.
6. The UAV swarm route planning system based on behavior cloning as claimed in claim 1, characterized in that: The teaching module and the behavior cloning module are also used for: When the swarm of drones does not reach the preset scale, the route planning algorithm is directly used to solve the route planning scheme; When the drone swarm reaches a certain size, the route planning solution is obtained based on imitation learning of the neural network model.
7. A UAV cluster route planning method based on behavior cloning, characterized in that: include: Acquire key factor data of collaborative task planning, and perform vectorization processing on the key factor data; Using the processed vectors, the route planning scheme is solved based on the route planning algorithm under the current spatial constraints; Learning the route planning solution based on the neural network model and interacting with the cluster UAV test environment to generate state data, so as to add labels to the generated state data to train the neural network model; The final planning scheme determined by the solution or the neural network model is converted into a standard vector and decoded into a task assignment instruction and input into the cluster UAV test environment, so that the cluster UAV executes the route planning instruction according to the joint action instruction set with a timing relationship under the given constraints or task rules.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the UAV cluster route planning method based on behavior cloning as described in claim 7 are implemented.