Unmanned aerial vehicle group cooperative surveying and mapping path planning method and system adopting reinforcement learning and application

Abstract

The invention relates to an unmanned aerial vehicle group cooperative surveying and mapping path planning method and system adopting reinforcement learning and application, and relates to the technical field of unmanned aerial vehicle surveying and mapping, and the method comprises the steps: randomly deploying a plurality of unmanned aerial vehicle group cooperative surveying and mapping paths covering a target area based on the constraints of unmanned aerial vehicle navigation boundaries and the like; constructing a fitness function by taking the routing inspection path length and the like as fitness factors, and evaluating a fitness value; when convergence is not satisfied, clustering the paths based on the fitness value, and constructing a multi-cluster unmanned aerial vehicle group cooperative surveying and mapping path; sorting the paths according to the fitness from good to bad, generating first-level to N-level paths, and constructing a reward tree; and updating the paths from the second level to the Nth level based on the reward tree, and circulating until a target surveying and mapping path meeting the convergence fitness is generated and output. According to the method, the problems of high trial and error cost and difficulty in convergence caused by a large number of clusters and difficulty in searching reward points in traditional unmanned aerial vehicle group cooperative surveying and mapping path planning are solved, and an effective solution is provided for intelligent path planning of unmanned aerial vehicle group cooperative surveying and mapping.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) mapping, and in particular to a method, system and application for collaborative mapping path planning of UAV swarms using reinforcement learning. Background Technology

[0002] With the development of UAV swarm collaborative mapping technology, efficient and accurate path planning has become an important foundation for improving mapping efficiency and reliability. Currently, traditional UAV swarm collaborative mapping path planning suffers from high trial-and-error costs and convergence difficulties due to the large number of UAVs involved and the difficulty in effectively determining reward points. Furthermore, it struggles to meet the real-time and efficiency requirements of path planning in complex scenarios.

[0003] Existing path planning methods suffer from significant discrepancies between planning results and actual collaborative needs due to the large number of drone swarms and the difficulty in finding reward points. This increases computational resource consumption and path optimization time costs, and also fails to meet the requirements for efficient and stable path planning in the intelligent transformation of drone swarm collaborative mapping. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method, system, and application for UAV swarm collaborative mapping path planning using reinforcement learning, which improves the efficiency and convergence stability of UAV swarm collaborative mapping path planning and reduces trial-and-error costs.

[0005] This invention is implemented as follows: a method for collaborative mapping path planning using UAV swarms with reinforcement learning, comprising:

[0006] Based on the drone navigation boundary, drone scheduling quantity constraints, and drone interval constraints, several drone swarms are randomly deployed to cover the target area for collaborative mapping paths.

[0007] Using inspection path length, inspection energy consumption, and safety level as fitness factors, a fitness function is constructed to evaluate several fitness values ​​of collaborative mapping paths of several UAV swarms.

[0008] When the fitness values ​​do not meet the convergence fitness, the collaborative mapping paths of the UAV swarms are clustered based on the fitness values ​​to construct a multi-cluster collaborative mapping path of UAV swarms, wherein the multi-cluster collaborative mapping path of UAV swarms includes multiple centroid UAV swarm collaborative mapping paths.

[0009] Based on the multiple centroid UAV swarm collaborative mapping paths, the collaborative mapping paths of the multiple UAV swarms are sorted according to fitness from best to worst, generating first-level UAV swarm collaborative mapping paths up to N-level UAV swarm collaborative mapping paths, and constructing a reward tree.

[0010] Based on the reward tree, the collaborative mapping path of the second-level UAV swarm up to the collaborative mapping path of the N-level UAV swarm is updated to obtain the updated collaborative mapping path of the UAV swarm. The loop is executed until a target mapping path that satisfies the convergence fitness is generated and output. Here, the collaborative mapping path of the k-level UAV swarm is the reward path of the collaborative mapping path of the z-level UAV swarm, k < z, k and z are integers, k∈[1,N-1], z∈[2,N].

[0011] Furthermore, based on UAV flight boundaries, UAV scheduling quantity constraints, and UAV interval constraints, several UAV swarm collaborative mapping paths covering the target area are randomly deployed, including:

[0012] Based on the UAV flight boundary, the UAV scheduling quantity constraint, and the UAV interval constraint, several initial UAV swarms are randomly deployed to collaboratively map the path.

[0013] From the plurality of initial UAV swarm collaborative mapping paths, a first initial UAV swarm collaborative mapping path is extracted, wherein the first initial UAV swarm collaborative mapping path includes the first UAV mapping path up to the Qth UAV mapping path.

[0014] From the first UAV mapping path, the first moment's flight altitude is extracted, and combined with the first UAV model, the UAV coverage calibration table is input to calibrate the first moment's mapping coverage and add it to the first UAV mapping coverage. The UAV coverage calibration table is used to store the UAV model, UAV altitude and UAV mapping coverage with a one-to-one correspondence with failure duration constraints. The UAV mapping coverage represents the mapping boundary coordinates relative to the UAV's flight position as the origin.

[0015] Until the mapping path of the Qth UAV is processed, the mapping coverage area of ​​the Qth UAV is determined;

[0016] The union of the mapping coverage of the first UAV up to the mapping coverage of the Qth UAV is used to obtain the first initial UAV swarm collaborative mapping path coverage.

[0017] If the coverage area of ​​the first initial UAV swarm collaborative mapping path covers the target area, the first initial UAV swarm collaborative mapping path is added to the plurality of UAV swarm collaborative mapping paths; otherwise, it is deleted.

[0018] Furthermore, from the mapping path of the first UAV, the flight altitude at the first moment is extracted, and combined with the UAV model, the UAV coverage calibration table is input to calibrate the mapping coverage at the first moment, including:

[0019] The first UAV model is input into the UAV coverage calibration table for sorting to obtain a first-level matching binary array, wherein each of the first-level matching binary arrays includes a flight altitude record value and a coverage calibration value.

[0020] Based on the flight altitude at the first moment, the first-level matching binary array is traversed and compared with the flight altitude record value to extract the set of coverage calibration values ​​where the flight altitude deviation is less than or equal to the flight altitude deviation threshold.

[0021] When the set of coverage calibration values ​​is not empty, extract the minimum value and set it as the survey coverage at the first time.

[0022] When the set of coverage calibration values ​​is empty, the coverage calibration model bound to the first UAV model is retrieved from the coverage calibration model library, the flight altitude at the first moment is processed, the mapping coverage at the first moment is generated, and the UAV coverage calibration table is updated using the flight altitude at the first moment, the first UAV model and the mapping coverage at the first moment.

[0023] The coverage calibration model is generated by machine learning training using multiple sets of data. Each set of data includes the flight altitude record value of a preset model of UAV and the coordinate label of the mapping boundary with the UAV's flight position as the origin.

[0024] Furthermore, a fitness function is constructed using inspection path length, inspection energy consumption, and safety level as fitness factors, including:

[0025] Collect drone inspection test data that meets the preset drone model and preset inspection path coordinate sequence, calculate the proportion of drone out-of-control data in the drone inspection test data, and use 1 to subtract the proportion of drone out-of-control data to set the safety level identifier.

[0026] Using the security level identifier as supervision and the preset inspection path coordinate sequence as input, a preset UAV model security level evaluator is constructed and added to the security level evaluation model library;

[0027] Based on the aforementioned safety level assessment model library, and combining path length weight, energy consumption weight, and safety level weight, the fitness function is constructed as follows:

[0028] Specifically, when the safety level is less than or equal to the safety level threshold, the fitness function is equal to the worst value; otherwise, the fitness function is equal to the weighted comprehensive value of the inspection path length, inspection energy consumption, and safety level, which combines the path length weight, energy consumption weight, and safety level weight.

[0029] Furthermore, the multi-cluster UAV swarm collaborative mapping path includes multiple centroid UAV swarm collaborative mapping paths, including:

[0030] Extract the collaborative mapping path of the first cluster of UAVs from the collaborative mapping path of the multi-cluster UAV swarm;

[0031] Extract the first UAV swarm collaborative mapping path from the first cluster of UAV swarm collaborative mapping paths, calculate the similarity between it and the collaborative mapping paths of UAV swarms within the cluster, and calculate the mean value. Set the mean value as the centroid parameter of the first path and add it to the path centroid parameter set.

[0032] Based on the set of path centroid parameters, the maximum value is extracted and set as the first centroid UAV swarm collaborative mapping path, and added to the multiple centroid UAV swarm collaborative mapping paths.

[0033] Furthermore, based on the multiple centroid UAV swarm collaborative mapping paths, the collaborative mapping paths of the multi-cluster UAV swarms are sorted according to fitness from best to worst, generating first-level UAV swarm collaborative mapping paths up to N-level UAV swarm collaborative mapping paths, and constructing a reward tree, including:

[0034] Based on the multiple centroid UAV swarm collaborative mapping paths, the collaborative mapping paths of the multiple UAV swarms are sorted according to fitness from best to worst to obtain the first-level initial UAV swarm collaborative mapping path up to the N-level initial UAV swarm collaborative mapping path.

[0035] The first-level initial UAV swarm collaborative mapping path is sorted by similar path random optimal fitness to obtain the first-level UAV swarm collaborative mapping path, wherein similar path refers to the path set with path similarity less than or equal to the path similarity threshold;

[0036] The Nth-level UAV swarm collaborative mapping path is obtained by performing similar path random optimal fitness sorting on the initial UAV swarm collaborative mapping path.

[0037] Furthermore, based on the reward tree, the collaborative mapping paths of the second-level UAV swarm up to the N-level UAV swarm are updated to obtain the updated UAV swarm collaborative mapping paths, including:

[0038] Step 1: From the second-level UAV swarm collaborative mapping path to the N-level UAV swarm collaborative mapping path, randomly extract the selected UAV swarm collaborative mapping path of the i-level UAV swarm collaborative mapping path, i∈[2,N], where i is an integer;

[0039] Step 2: Extract the ij-level UAV swarm collaborative mapping path from the second-level UAV swarm collaborative mapping path to the N-level UAV swarm collaborative mapping path. The ij-level UAV swarm collaborative mapping path has the ij-centroid UAV swarm collaborative mapping path, where j is an integer and the initial value of j is equal to 1.

[0040] Step 3: When the path similarity between the ij centroid UAV swarm collaborative mapping path and the selected UAV swarm collaborative mapping path is greater than or equal to the similarity threshold, and ij is greater than 1, increment j by one, and return to step 2 to execute the loop.

[0041] Step 4: When the path similarity between the ij-level centroid UAV swarm collaborative mapping path and the selected UAV swarm collaborative mapping path is less than the similarity threshold, the ij-level UAV swarm collaborative mapping path is used as the reward path to guide the update of the selected UAV swarm collaborative mapping path.

[0042] Step 5: When the path similarity between the ij centroid UAV swarm collaborative mapping path and the selected UAV swarm collaborative mapping path is greater than or equal to the similarity threshold, and ij equals 1, return to Step 1 to execute the loop;

[0043] When the collaborative mapping path of the entire UAV swarm is traversed, the updated collaborative mapping path of the UAV swarm is output.

[0044] A UAV swarm collaborative mapping path planning system employing reinforcement learning, the system being used to execute the aforementioned UAV swarm collaborative mapping path planning method employing reinforcement learning, the system comprising:

[0045] The path random deployment module is used to randomly deploy several UAV swarms to collaboratively map the target area based on UAV flight boundaries, UAV scheduling quantity constraints, and UAV interval constraints.

[0046] The fitness function construction module is used to construct a fitness function with inspection path length, inspection energy consumption and safety level as fitness factors, and evaluate several fitness values ​​of collaborative mapping paths of several UAV swarms.

[0047] The path clustering construction module is used to cluster the several UAV swarm collaborative mapping paths based on the several fitness values ​​when the several fitness values ​​do not meet the convergence fitness, and construct multi-cluster UAV swarm collaborative mapping paths, wherein the multi-cluster UAV swarm collaborative mapping paths include multiple centroid UAV swarm collaborative mapping paths.

[0048] The reward tree construction module is used to sort the collaborative mapping paths of multiple centroid UAV swarms according to their fitness from best to worst based on the collaborative mapping paths of the multiple centroid UAV swarms, generate first-level UAV swarm collaborative mapping paths up to N-level UAV swarm collaborative mapping paths, and construct a reward tree.

[0049] The path update iteration module is used to update the collaborative mapping path of the second-level UAV swarm up to the N-level UAV swarm based on the reward tree, obtain the updated UAV swarm collaborative mapping path, and execute the loop until a target mapping path that meets the convergence fitness is generated and output. Here, the k-level UAV swarm collaborative mapping path is the reward path of the z-level UAV swarm collaborative mapping path, k < z, k and z are integers, k∈[1,N-1], z∈[2,N].

[0050] A computer-readable storage medium, characterized in that the computer-readable storage medium includes a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to perform the above-described method.

[0051] A computer device includes a memory, a processor, and a program stored in the memory and executable thereon, which, when executed by the processor, performs the steps of the method described above.

[0052] One or more technical solutions provided in this invention have at least the following technical effects or advantages:

[0053] This invention proposes a method and system for collaborative mapping path planning using UAV swarms, employing reinforcement learning. Through multi-cluster path clustering and reward tree hierarchical optimization, it achieves efficient path planning in complex scenarios. First, initial paths are deployed based on constraints such as UAV flight boundaries. A fitness function that integrates path length, energy consumption, and safety level is used for evaluation. Paths that do not meet convergence requirements are clustered, and centroid paths are extracted. Then, multi-level paths are generated by sorting the centroid paths by fitness, constructing a reward tree, and eliminating similar redundant paths. Based on the reward tree, secondary path updates are guided by centroid path similarity matching until a converged target path is output. This method effectively solves the convergence problem caused by the large number of path solution groups in traditional methods, significantly reduces trial-and-error costs, and significantly improves convergence efficiency, providing an efficient and reliable intelligent solution for collaborative mapping using UAV swarms.

[0054] This invention achieves precise path planning for UAV swarms based on reinforcement learning by integrating multi-source information such as path clustering, centroid features, and reward tree construction. It overcomes the limitations of traditional methods that struggle to find reward points due to the large size of UAV swarms, significantly improving the efficiency and reliability of path planning in complex environments and providing a scientific path for the intelligent development of collaborative UAV swarm operations. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart illustrating the UAV swarm collaborative mapping path planning method using reinforcement learning, as provided in an embodiment of the present invention.

[0057] Figure 2 This is a schematic diagram of the structure of a UAV swarm collaborative mapping path planning system using reinforcement learning, provided in an embodiment of the present invention.

[0058] In the diagram: 01 Path random deployment module, 02 Fitness function construction module, 03 Path clustering construction module, 04 Reward tree construction module, 05 Path update and iteration module. Detailed Implementation

[0059] This invention provides a method and system for collaborative mapping path planning of UAV swarms using reinforcement learning, which solves the technical problems in the prior art, such as the large number of UAV swarms and the difficulty in determining reward points, resulting in high trial and error costs and difficulty in path planning convergence.

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0062] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0063] Example 1, as shown in the appendix Figure 1 As shown, this invention provides a method for collaborative mapping path planning of UAV swarms using reinforcement learning, the method comprising the following steps:

[0064] S100: Based on UAV navigation boundaries, UAV scheduling quantity constraints, and UAV interval constraints, randomly deploy several UAV swarms to collaboratively map the target area;

[0065] In this embodiment of the invention, in the scenario of collaborative mapping path planning by a swarm of unmanned aerial vehicles (UAVs), in order to ensure that the path covers the target area and meets the operational constraints of the UAVs, it is necessary to achieve effective deployment of the initial path through multi-dimensional constraints.

[0066] Specifically, based on the UAV flight boundaries, scheduling quantity constraints, and interval constraints, several initial UAV swarm collaborative mapping paths are randomly generated within the target area.

[0067] Furthermore, a first initial UAV swarm collaborative mapping path is extracted from several initial paths. This path contains a complete sequence from the first UAV mapping path to the Qth UAV mapping path.

[0068] For the first UAV mapping path, its flight altitude at the first moment is extracted, and the UAV coverage calibration table is consulted in conjunction with the UAV model. After calibrating the mapping coverage at the first moment, it is added to the mapping coverage of the first UAV.

[0069] Further, following this process, the mapping path of the Qth UAV is processed to complete the calibration of the mapping coverage area of ​​the Qth UAV.

[0070] Subsequently, the union of the mapping coverage of the first to Qth UAVs is taken to obtain the first initial UAV swarm collaborative mapping path coverage.

[0071] If the coverage area completely covers the target area, the initial path is retained; otherwise, it is deleted, ensuring that the final retained path meets the requirement of full coverage of the target area.

[0072] By employing random deployment and coverage verification logic under multiple constraints, an initial path set that conforms to actual operational scenarios is provided for subsequent path optimization, laying a foundation for the efficient execution of collaborative mapping by UAV swarms.

[0073] Step S100 in the method provided in this embodiment of the invention includes:

[0074] Based on the UAV flight boundary, the UAV scheduling quantity constraint, and the UAV interval constraint, several initial UAV swarms are randomly deployed to collaboratively map the path.

[0075] From the plurality of initial UAV swarm collaborative mapping paths, a first initial UAV swarm collaborative mapping path is extracted, wherein the first initial UAV swarm collaborative mapping path includes the first UAV mapping path up to the Qth UAV mapping path.

[0076] From the first UAV mapping path, the first moment's flight altitude is extracted, and combined with the first UAV model, the UAV coverage calibration table is input to calibrate the first moment's mapping coverage and add it to the first UAV mapping coverage. The UAV coverage calibration table is used to store the UAV model, UAV altitude and UAV mapping coverage with a one-to-one correspondence with failure duration constraints. The UAV mapping coverage represents the mapping boundary coordinates relative to the UAV's flight position as the origin.

[0077] Until the mapping path of the Qth UAV is processed, the mapping coverage area of ​​the Qth UAV is determined;

[0078] The union of the mapping coverage of the first UAV up to the mapping coverage of the Qth UAV is used to obtain the first initial UAV swarm collaborative mapping path coverage.

[0079] If the coverage area of ​​the first initial UAV swarm collaborative mapping path covers the target area, the first initial UAV swarm collaborative mapping path is added to the plurality of UAV swarm collaborative mapping paths; otherwise, it is deleted.

[0080] In this embodiment of the invention, in the scenario of collaborative mapping by a UAV swarm, in order to ensure that the initial path meets the coverage requirements and complies with the operational constraints, it is necessary to verify the effectiveness of the path through multi-dimensional constraints and dynamic calibration mechanisms.

[0081] Specifically, based on the drone flight boundaries (such as no-fly zones and signal coverage areas), the upper limit of the number of drones to be dispatched (such as the upper limit of the total number of drones to be deployed), and the safety interval threshold (such as the safety distance threshold between drones), several initial drone swarm collaborative mapping paths are randomly generated within the target area.

[0082] Subsequently, for each initial path, the navigation parameters of each UAV mapping path are analyzed sequentially. Taking the first initial UAV swarm collaborative mapping path as an example, it contains a complete sequence from the mapping path of the first UAV to the mapping path of the Qth UAV.

[0083] Specifically, for the mapping path of the first UAV, its flight altitude at the first moment is extracted, and the specific model of the UAV is obtained simultaneously to provide key parameters for subsequent coverage calibration.

[0084] Furthermore, the first UAV model is input into the UAV coverage calibration table for sorting. This calibration table stores the correspondence between UAV models, altitudes, and mapping coverage areas with failure duration constraints.

[0085] The method provided in this embodiment of the invention includes the following steps: "Extracting the flight altitude at the first moment from the mapping path of the first UAV, and inputting the UAV coverage calibration table in combination with the UAV model to calibrate the mapping coverage at the first moment."

[0086] The first UAV model is input into the UAV coverage calibration table for sorting to obtain a first-level matching binary array, wherein each of the first-level matching binary arrays includes a flight altitude record value and a coverage calibration value.

[0087] Based on the flight altitude at the first moment, the first-level matching binary array is traversed and compared with the flight altitude record value to extract the set of coverage calibration values ​​where the flight altitude deviation is less than or equal to the flight altitude deviation threshold.

[0088] When the set of coverage calibration values ​​is not empty, extract the minimum value and set it as the survey coverage at the first time.

[0089] When the set of coverage calibration values ​​is empty, the coverage calibration model bound to the first UAV model is retrieved from the coverage calibration model library, the flight altitude at the first moment is processed, the mapping coverage at the first moment is generated, and the UAV coverage calibration table is updated using the flight altitude at the first moment, the first UAV model and the mapping coverage at the first moment.

[0090] The coverage calibration model is generated by machine learning training using multiple sets of data. Each set of data includes the flight altitude record value of a preset model of UAV and the coordinate label of the mapping boundary with the UAV's flight position as the origin.

[0091] In this embodiment of the invention, when determining the coverage area of ​​a UAV mapping, it is necessary to first conduct an integrated analysis of the UAV model and flight altitude to accurately determine the coverage area.

[0092] Specifically, the first UAV model is input into the UAV coverage calibration table (which stores the correspondence between model, altitude and coverage with failure duration constraints), and a first-level matching binary array is obtained through model sorting.

[0093] For example, a certain model of UAV "A1" may have a first-level matching binary array containing recorded values ​​such as [(50m, 100m×100m), (100m, 200m×200m)], which correspond to the coverage range calibration values ​​at different flight altitudes.

[0094] Subsequently, based on the flight altitude at the first moment (e.g., 75m), the array is traversed and compared with the flight altitude record value. The deviation threshold is set to ±10m. Then, the deviation between 75m and 50m is 25m (exceeding the threshold), and the deviation between 75m and 100m is 25m (exceeding the threshold). At this time, the set of coverage calibration values ​​is empty.

[0095] For example, if the initial flight altitude is 80m, and the deviation from 100m is 20m (assuming the threshold is 20m), then the set of coverage range calibration values ​​that meet the conditions includes 200m×200m, and the minimum value is taken as the coverage range.

[0096] When the set is empty, the model bound to "A1" in the coverage calibration model library is retrieved. This model generates the coverage range (e.g., 150m×150m) by inputting the flight altitude (e.g., 75m) and using the trained neural network mapping relationship. At the same time, (75m, A1, 150m×150m) is updated to the calibration table, forming a dynamic learning mechanism.

[0097] The model training data includes multiple sets of navigation altitude and mapping boundary coordinate labels, such as (60m, 120m×120m). The parameters are optimized through supervised learning to ensure the accuracy of the generated coverage.

[0098] This process employs a dual mechanism of prioritizing calibration table lookup and dynamic model generation. It leverages historical data to improve efficiency while adapting to new scenarios through machine learning. This ensures that drones of different models and altitudes can accurately calibrate their coverage areas in real-time mapping, providing reliable data support for subsequent path coverage verification.

[0099] After the coverage range of a single drone is determined, the coverage range of multiple drones needs to be further integrated and verified to ensure that the initial path effectively covers the target area.

[0100] Specifically, starting from the first UAV mapping path, the process proceeds sequentially up to the Qth UAV mapping path. For each UAV, the steps of "model input - altitude comparison - coverage calibration" are repeated, and the mapping coverage at each moment is added to the first UAV mapping coverage set in sequence.

[0101] For example, assume that Q is 3, meaning it includes 3 drones. The first drone, model A1, calibrates a coverage area of ​​150m × 150m at an altitude of 75m, the second drone, model A2, calibrates a coverage area of ​​200m × 200m at an altitude of 100m, and the third drone, model A3, calibrates a coverage area of ​​100m × 100m at an altitude of 50m. In this case, the set of coverage areas mapped by the first drone is [150m × 150m, 200m × 200m, 100m × 100m].

[0102] Furthermore, the union of all coverage areas within this set (the first UAV mapping coverage area set) is taken to obtain the first initial UAV swarm collaborative mapping path coverage area.

[0103] During the union operation, the mapping boundary coordinates with the flight positions of each UAV as the origin are uniformly transformed to the target area coordinate system, merging overlapping areas to form a complete coverage area outline. For example, the union of the coverage areas of three UAVs A1, A2, and A3 may form an irregular polygon with a coverage area of ​​350m × 250m.

[0104] Finally, the coverage area of ​​the union is compared with the boundary coordinates of the target area. If the coverage area completely includes all coordinate points of the target area, the first initial UAV swarm collaborative mapping path is determined to meet the coverage requirements and is added to the set of valid paths; if there are uncovered parts of the target area (such as a corner coordinate not falling into the union area), a deletion operation is performed to ensure that the retained paths can completely cover the target area.

[0105] This process, through the steps of "single-machine construction - multi-machine integration - coverage verification", combines a dual calibration mechanism of calibration table and dynamic model, which not only ensures the accuracy of the coverage of a single UAV, but also realizes the integrity verification of the collaborative coverage of UAV swarm through union operation, effectively solving the problem of incomplete or repeated coverage in traditional path planning.

[0106] S200: Using inspection path length, inspection energy consumption, and safety level as fitness factors, a fitness function is constructed to evaluate several fitness values ​​of collaborative mapping paths of several UAV swarms.

[0107] In this embodiment of the invention, in the scenario of collaborative mapping and path planning by a swarm of unmanned aerial vehicles (UAVs), in order to accurately evaluate and optimize the path, it is necessary to construct a fitness function that integrates multiple key indicators.

[0108] The function uses inspection path length, inspection energy consumption, and safety level as core evaluation factors, and achieves scientific judgment on path planning through quantitative calculation.

[0109] First, collect drone inspection test data that meets the preset drone model and preset inspection path coordinate sequence, and calculate the proportion of drone out-of-control data. The safety level is then determined by "1 - proportion of out-of-control data".

[0110] Furthermore, using the safety level identifier as the supervision signal and the preset inspection path coordinate sequence as input, a safety level evaluator for a specific UAV model is constructed and stored in the safety level evaluation model library.

[0111] Based on the safety level assessment model library, a fitness function is constructed by combining path length weight, energy consumption weight, and safety level weight.

[0112] Among them, the inspection path length and energy consumption data can be read in real time according to the drone model and flight parameters, while the safety level is dynamically calculated by the evaluator based on the real-time flight path.

[0113] Specifically, when the safety level is less than or equal to a preset threshold, the fitness function is directly assigned the worst value, achieving a hard screening of "direct elimination"; if the safety level meets the standard, the fitness value is obtained through weighted comprehensive calculation to quantify the overall quality of the path.

[0114] By constructing a fitness function, we can ensure the safety baseline of path planning while comprehensively optimizing efficiency and energy consumption indicators, providing a scientific evaluation basis for subsequent path clustering, ranking, and iterative updates of UAV swarms.

[0115] Step S200 in the method provided in this embodiment of the invention includes:

[0116] Collect drone inspection test data that meets the preset drone model and preset inspection path coordinate sequence, calculate the proportion of drone out-of-control data in the drone inspection test data, and use 1 to subtract the proportion of drone out-of-control data to set the safety level identifier.

[0117] Using the security level identifier as supervision and the preset inspection path coordinate sequence as input, a preset UAV model security level evaluator is constructed and added to the security level evaluation model library;

[0118] Based on the aforementioned safety level assessment model library, and combining path length weight, energy consumption weight, and safety level weight, the fitness function is constructed as follows:

[0119] Specifically, when the safety level is less than or equal to the safety level threshold, the fitness function is equal to the worst value; otherwise, the fitness function is equal to the weighted comprehensive value of the inspection path length, inspection energy consumption, and safety level, which combines the path length weight, energy consumption weight, and safety level weight.

[0120] In this embodiment of the invention, in the scenario of collaborative mapping path planning by UAV swarms, in order to scientifically evaluate the merits of the path and guide the optimization direction, it is necessary to construct a fitness function that comprehensively considers multiple dimensions of indicators.

[0121] Specifically, by integrating three core factors—inspection path length, inspection energy consumption, and safety level—an evaluation system capable of quantifying path quality is formed.

[0122] First, collect drone inspection test data that meets the preset drone model and preset inspection path coordinate sequence, and calculate the proportion of drone out-of-control data (i.e., number of drone out-of-control times / number of drone inspections). Subtract this proportion from 1 to obtain the safety level identifier, i.e., "safety level identifier = 1 - proportion of drone out-of-control data".

[0123] For example, if a certain type of drone experiences 5 out-of-control incidents in 100 inspection tests, and the out-of-control data accounts for 5% (5 / 100×100%), then the safety level is rated as 0.95 (1-5%). This rating reflects the ability of path planning to ensure the safe operation of the drone.

[0124] Furthermore, using the safety level identifier as the monitoring signal and the preset inspection path coordinate sequence as the input, a safety level evaluator for a preset UAV model is constructed and added to the safety level evaluation model library.

[0125] Specifically, a security level evaluator is constructed using machine learning algorithms (such as convolutional neural networks CNN). A preset inspection path coordinate sequence is used as input features, and the security level identifier is used as the output label. Through training, the model learns the mapping relationship between path space features and security risks.

[0126] Specifically, the collected preset inspection path coordinate sequences and corresponding safety level identifiers are first divided into training and validation sets in an 8:2 ratio. For example, 800 sets are extracted from 1000 sets of historical data as the training set and 200 sets as the validation set to ensure that the data distribution reflects the path safety characteristics under different scenarios.

[0127] Furthermore, the CNN model is trained using the training set. A network architecture consisting of 3 convolutional layers, 2 pooling layers, and 2 fully connected layers is constructed.

[0128] The convolutional layer uses a 3×3 convolutional kernel to extract local spatial features of the path coordinate sequence (such as inflection point coordinates and height change points), the pooling layer reduces the feature dimension and retains key information, and the fully connected layer maps the abstract features to a safety level prediction value between 0 and 1.

[0129] During training, the path coordinate sequence is arranged into a two-dimensional matrix in chronological order and input into the model. The weights are optimized through backpropagation algorithm to minimize the mean squared error (MSE) between the predicted safety level and the actual safety level.

[0130] Meanwhile, the initial learning rate was set to 0.001, the batch size to 32, and the number of iterations to 100. After the 10th training round, the MSE on the validation set was 0.08; by the 30th round, the MSE had dropped to 0.03, indicating that the model's ability to extract path space features gradually improved with training iterations.

[0131] An early stopping mechanism is implemented during training, meaning training stops when the MSE on the validation set fluctuates less than 0.005 for eight consecutive rounds to avoid overfitting. For example, if the validation set MSE stabilizes at 0.025 and does not decrease further for eight rounds after training to the 45th round, the early stopping mechanism is triggered, indicating that the model has converged.

[0132] Finally, the trained security level evaluator can directly receive the real-time inspection path coordinate sequence and output the corresponding security level prediction value.

[0133] For example, if a coordinate sequence containing frequent altitude changes is input, the model outputs a safety level of 0.85 (higher than the safety threshold of 0.7), indicating that the safety risk of the path is low; if the input contains the coordinates of a path with continuous low-altitude flight, the model may output 0.65, triggering a safety warning.

[0134] After obtaining the trained safety level evaluator, the real-time inspection path coordinate sequence can be input into the evaluator to obtain the safety level prediction value, and then added to the safety level evaluation model library.

[0135] Furthermore, based on a safety level assessment model library, and combining path length weight, energy consumption weight, and safety level weight, a fitness function is constructed. This function quantifies path quality using multi-dimensional indicators, enabling a scientific evaluation of UAV swarm collaborative mapping paths.

[0136] Among them, the inspection path length and inspection energy consumption can be directly obtained according to the UAV model and flight parameters, while the safety level is dynamically evaluated by the evaluator on the real-time flight path. The evaluator is based on the model in the safety level evaluation model library. After inputting the real-time path coordinate sequence, the evaluator outputs the safety level prediction value between 0 and 1, reflecting the safety risk level of the path.

[0137] Specifically, when evaluating the safety level, if the safety level is less than or equal to the safety level threshold, the fitness function is directly equal to the worst value regardless of the performance of other indicators, thus achieving a hard screening of "unsafe direct elimination" and ensuring that high-risk paths are eliminated first.

[0138] Conversely, if the safety level is greater than the safety level threshold, the fitness function is equal to the weighted composite value of the inspection path length, inspection energy consumption, and safety level, which combines the path length weight, energy consumption weight, and safety level weight. That is, fitness function = path length × weight + inspection energy consumption × weight + safety level × weight.

[0139] The weights for each indicator are determined based on the characteristics of the UAV model, mission requirements, and historical data statistics to balance the importance of different indicators in path evaluation. The path length weight reflects the importance attached to mapping efficiency, the energy consumption weight reflects the consideration of the UAV's endurance, and the safety level weight highlights the requirements for flight safety.

[0140] For example, in the scenario of UAV mapping path planning, considering the impact of path length, energy consumption, and safety level on the task, the weights of path length, energy consumption, and safety level are set to 0.3, 0.3, and 0.4. If the inspection length of a certain path is 0.8, the energy consumption is 0.7, and the safety level is 0.9, then the fitness value is 0.8×0.3+0.7×0.3+0.9×0.4=0.81. The higher the fitness value, the better the overall performance of the path, and vice versa.

[0141] S300: When the fitness values ​​do not meet the convergence fitness, the collaborative mapping paths of the UAV swarms are clustered based on the fitness values ​​to construct a multi-cluster collaborative mapping path of UAV swarms, wherein the multi-cluster collaborative mapping path of UAV swarms includes multiple centroid UAV swarm collaborative mapping paths.

[0142] In this embodiment of the invention, in the scenario of collaborative mapping path planning for UAV swarms, when the fitness value does not reach the convergence condition, it is necessary to group the UAV mapping paths through clustering in order to construct a more representative multi-cluster path set.

[0143] Specifically, the first cluster of UAV swarm collaborative mapping paths is extracted from several UAV swarm collaborative mapping paths. This cluster is divided based on the similarity of fitness values, and paths with similar fitness values ​​are grouped together using a clustering algorithm.

[0144] Furthermore, the first UAV swarm collaborative mapping path is extracted from the first cluster of UAV swarm collaborative mapping paths, and similarity calculation is performed with other paths in the cluster to measure the shape similarity of the path coordinate sequence. Then, the mean value is calculated as the centroid parameter of the first path and added to the path centroid parameter set.

[0145] Furthermore, based on the set of path centroid parameters, the maximum value is extracted, and the corresponding path is set as the first centroid UAV swarm collaborative mapping path, which is then added to the set of multiple centroid UAV swarm collaborative mapping paths. This centroid path represents the typical characteristics of its cluster and can effectively characterize the common features of paths within the cluster.

[0146] Finally, repeat the above steps to process all clusters, and finally construct a multi-cluster path set containing multiple centroid UAV swarm collaborative mapping paths.

[0147] This clustering method divides the vast path solution space into several representative clusters, each dominated by centroid paths, effectively reducing the problem's complexity. Extracting centroid paths provides a clear direction for subsequent path updates.

[0148] Step S300 in the method provided in this embodiment of the invention includes:

[0149] Extract the collaborative mapping path of the first cluster of UAVs from the collaborative mapping path of the multi-cluster UAV swarm;

[0150] Extract the first UAV swarm collaborative mapping path from the first cluster of UAV swarm collaborative mapping paths, calculate the similarity between it and the collaborative mapping paths of UAV swarms within the cluster, and calculate the mean value. Set the mean value as the centroid parameter of the first path and add it to the path centroid parameter set.

[0151] Based on the set of path centroid parameters, the maximum value is extracted and set as the first centroid UAV swarm collaborative mapping path, and added to the multiple centroid UAV swarm collaborative mapping paths.

[0152] In this embodiment of the invention, in the scenario of UAV swarm collaborative mapping path planning, in order to effectively cluster the paths and extract representative centroid paths, it is necessary to extract the paths of each cluster sequentially from the multi-cluster UAV swarm collaborative mapping paths, and determine the centroid paths by calculating the path similarity within the cluster.

[0153] Specifically, the first cluster of UAV swarm collaborative mapping paths is extracted from the existing multi-cluster UAV swarm collaborative mapping paths. This cluster is a set of paths with similar characteristics, partitioned using a clustering algorithm (such as K-means).

[0154] The clustering process calculates the difference in fitness values ​​among all paths and groups paths with similar fitness values ​​into the same cluster. For example, paths with a fitness value difference of less than 0.1 are grouped into one cluster, thus forming the initial multi-cluster structure.

[0155] Furthermore, the first UAV swarm collaborative mapping path (such as path N) is extracted from the first cluster of UAV swarm collaborative mapping paths, and its similarity is calculated with each other UAV swarm collaborative mapping path in the cluster.

[0156] The similarity calculation uses Euclidean distance to measure the similarity of paths in terms of coordinate sequence, shape, etc.

[0157] Specifically, for the coordinate sequences of two paths, the shorter path is completed to the same length as the longer path (e.g., unified to n coordinate points) by interpolation. Then, the average value of the sum of the Euclidean distances of the corresponding coordinate points is calculated and used as the first path centroid parameter of the cluster.

[0158] For example, if there are 3 paths N, M and O in the first cluster, path N is extracted as the benchmark, the Euclidean distance similarity between N and M is calculated to be 0.7, and the similarity between N and O is 0.8. The mean is 0.75 ((0.7+0.8) / 2), which is set as the centroid parameter of the first path and added to the set of path centroid parameters.

[0159] Furthermore, based on the set of path centroid parameters, the maximum value (e.g., 0.8) is extracted, and the corresponding path (e.g., path O) becomes the first centroid UAV swarm collaborative mapping path for the cluster. This path is then added to the collaborative mapping paths of multiple centroid UAV swarms. This centroid path can best represent the common characteristics of paths within the cluster, such as path shape, coverage, or energy efficiency.

[0160] Finally, the above operations (cluster extraction - similarity calculation - centroid parameter calculation - centroid path determination) are repeated for each cluster in the multi-cluster UAV swarm collaborative mapping path, ultimately forming a set containing multiple centroid UAV swarm collaborative mapping paths. These centroid paths can effectively characterize the typical features of each cluster, compressing the originally complex path solution space into several representative centroid paths, significantly reducing the problem complexity.

[0161] S400: Based on the multiple centroid UAV swarm collaborative mapping paths, sort the multi-cluster UAV swarm collaborative mapping paths according to fitness from best to worst, generate first-level UAV swarm collaborative mapping paths up to N-level UAV swarm collaborative mapping paths, and construct a reward tree;

[0162] In this embodiment of the invention, in the scenario of collaborative mapping path planning by UAV swarms, in order to optimize the mapping path and improve the optimization efficiency, it is necessary to sort the multi-cluster paths based on the fitness value of the centroid path, and at the same time remove similar paths to generate multi-level paths and form a reward tree structure.

[0163] Specifically, firstly, based on the collaborative mapping paths of multiple centroid UAV swarms, the collaborative mapping paths of the multi-cluster UAV swarms are sorted in order of fitness value from best to worst, thereby obtaining the first-level initial UAV swarm collaborative mapping path up to the N-level initial UAV swarm collaborative mapping path.

[0164] Furthermore, the initial UAV swarm collaborative mapping paths are randomly selected based on optimal fitness for similar paths. This involves grouping paths with similarity less than or equal to a path similarity threshold into the same set of similar paths, removing all similar paths in this set except for the path with the best fitness, and retaining only the path with the highest fitness in each set as the initial UAV swarm collaborative mapping path.

[0165] Similarly, following the logic of "sorting - removing similar paths - retaining the best", the initial UAV swarm collaborative mapping paths from level two to level N are processed sequentially. At each level, the path with the best fitness is selected from the set of similar paths, and finally the level N UAV swarm collaborative mapping paths are obtained, thus completing the construction of the reward tree.

[0166] In the reward tree structure, a hierarchical reward guidance relationship is formed between paths at each level. Among them, the k-level UAV swarm collaborative mapping path serves as the reward path for the z-level path (k < z). Since similar paths have been eliminated in the early stage, each level of path has uniqueness and representativeness, which can accurately guide the direction of subsequent path updates.

[0167] By constructing a reward tree based on centroid path sorting and removing similar paths, a large number of paths can be transformed into a simple hierarchical structure, effectively avoiding the waste of computing resources caused by redundancy of similar paths.

[0168] Step S400 in the method provided in this embodiment of the invention includes:

[0169] Based on the multiple centroid UAV swarm collaborative mapping paths, the collaborative mapping paths of the multiple UAV swarms are sorted according to fitness from best to worst to obtain the first-level initial UAV swarm collaborative mapping path up to the N-level initial UAV swarm collaborative mapping path.

[0170] The first-level initial UAV swarm collaborative mapping path is sorted by similar path random optimal fitness to obtain the first-level UAV swarm collaborative mapping path, wherein similar path refers to the path set with path similarity less than or equal to the path similarity threshold;

[0171] The Nth-level UAV swarm collaborative mapping path is obtained by performing similar path random optimal fitness sorting on the initial UAV swarm collaborative mapping path.

[0172] In this embodiment of the invention, in the scenario of collaborative mapping path planning by UAV swarms, it is necessary to sort multiple cluster paths based on the centroid path fitness value, eliminate similar redundant paths, and generate a multi-level path construction reward tree to optimize the path planning system.

[0173] Specifically, firstly, based on multiple centroid UAV swarm collaborative mapping paths, the paths are sorted according to their fitness values ​​from best to worst. This sorting clearly distinguishes the quality of the paths, thus obtaining initial first-level UAV swarm collaborative mapping paths up to N-level initial UAV swarm collaborative mapping paths.

[0174] Furthermore, similar path random optimal fitness selection is performed on the initial UAV swarm collaborative mapping path to obtain the initial UAV swarm collaborative mapping path.

[0175] In practice, the path similarity is first calculated using the Euclidean distance algorithm. Paths with a path similarity less than or equal to the path similarity threshold (e.g., 0.7) are divided into the same set of similar paths. Then, the path with the best fitness is randomly selected from each set, and the remaining similar paths are removed to reduce the number of redundant paths and improve optimization efficiency.

[0176] For example, if there are 5 paths in the initial collaborative mapping path of a UAV swarm, with coordinate sequences P1, P2, P3, P4, and P5, the similarity between each path is calculated using the Euclidean distance algorithm. It is found that the similarity between P1 and P2, and P3, is 0.6 and 0.7 respectively, both ≤ the path similarity threshold of 0.7, while the similarity between P4 and P5 is 0.5. Therefore, P1, P2, and P3 are grouped into one set of similar paths, and P4 and P5 are grouped into another set of similar paths.

[0177] Assuming that the fitness value of P1 is 0.85, P2 is 0.78, and P3 is 0.82, randomly select P1 (0.85) with the best fitness from this set as the first-level UAV swarm collaborative mapping path, and remove P2 and P3; similarly, select P5 (e.g., 0.9) with higher fitness from P4 and P5, and remove P4.

[0178] Furthermore, following the steps of "sorting-similarity calculation-optimal sorting" described above, the initial UAV swarm collaborative mapping paths from level two to level N-1 are processed sequentially, and finally, the initial UAV swarm collaborative mapping paths at level N are randomly sorted by optimal fitness based on similarity.

[0179] Through a step-by-step sorting operation from level one to level N, a reward tree structure containing N levels of UAV swarm collaborative mapping paths is finally constructed. Each level of path is the optimal representative after removing similar redundancies from the corresponding initial path set, and is sorted from best to worst in terms of fitness, providing a hierarchical optimization foundation for subsequent path updates based on the reward tree.

[0180] S500: Based on the reward tree, update the collaborative mapping path of the second-level UAV swarm up to the collaborative mapping path of the N-level UAV swarm, obtain the updated collaborative mapping path of the UAV swarm, and execute the loop until a target mapping path that satisfies the convergence fitness is generated and output. Here, the collaborative mapping path of the k-level UAV swarm is the reward path of the collaborative mapping path of the z-level UAV swarm, k < z, k and z are integers, k∈[1, N-1], z∈[2, N].

[0181] In this embodiment of the invention, in the scenario of collaborative mapping path planning by a UAV swarm, in order to achieve iterative optimization of the path and converge to the target solution, it is necessary to perform hierarchical updates of the secondary paths based on the reward tree structure.

[0182] Specifically, firstly, from the collaborative mapping paths of UAV swarms from level 2 to level N, the selected collaborative mapping paths of UAV swarms at level i (i∈[2,N]) are randomly extracted.

[0183] Subsequently, ij-level paths (j initially valued at 1) are extracted from the same path range. These paths must include ij-centroid UAV swarm collaborative mapping paths and serve as potential reward path candidates.

[0184] Specifically, when the similarity between the centroid path ij and the selected path is greater than or equal to the threshold, and ij is greater than 1, the search continues to the next higher level by increasing the value of j, until an ij-level path with a similarity less than the threshold is found.

[0185] At this point, the path at level ij is used as the reward path to guide the selected path in updating its coordinate sequence, energy consumption parameters, and other features, optimizing it towards the high fitness direction of the reward path. If the similarity is always greater than or equal to the threshold and ij drops to level 1, a new selected path is chosen to avoid getting trapped in local optima.

[0186] Furthermore, by iterating through all paths from level two to level N, the fitness value is re-evaluated after each update until a target mapping path that meets the convergence condition is generated.

[0187] In this process, the k-level path serves as the reward path for the z-level path, forming a "hierarchical guidance-feedback optimization" logic. For example, the first-level path provides optimization direction for the second-level path, and the second-level path updates by referring to the centroid features of the first-level path, and so on, ultimately achieving efficient convergence of the path solution group.

[0188] By using a reward tree-based path update method, guided by centroid path similarity matching and hierarchical rewards, the massive path solution space is transformed into an ordered iterative optimization process, which significantly reduces trial-and-error costs and improves the convergence efficiency and target path quality of UAV swarm collaborative mapping path planning.

[0189] Step S500 in the method provided in this embodiment of the invention includes:

[0190] Step 1: From the second-level UAV swarm collaborative mapping path to the N-level UAV swarm collaborative mapping path, randomly extract the selected UAV swarm collaborative mapping path of the i-level UAV swarm collaborative mapping path, i∈[2,N], where i is an integer;

[0191] Step 2: Extract the ij-level UAV swarm collaborative mapping path from the second-level UAV swarm collaborative mapping path to the N-level UAV swarm collaborative mapping path. The ij-level UAV swarm collaborative mapping path has the ij-centroid UAV swarm collaborative mapping path, where j is an integer and the initial value of j is equal to 1.

[0192] Step 3: When the path similarity between the ij centroid UAV swarm collaborative mapping path and the selected UAV swarm collaborative mapping path is greater than or equal to the similarity threshold, and ij is greater than 1, increment j by one, and return to step 2 to execute the loop.

[0193] Step 4: When the path similarity between the ij-level centroid UAV swarm collaborative mapping path and the selected UAV swarm collaborative mapping path is less than the similarity threshold, the ij-level UAV swarm collaborative mapping path is used as the reward path to guide the update of the selected UAV swarm collaborative mapping path.

[0194] Step 5: When the path similarity between the ij centroid UAV swarm collaborative mapping path and the selected UAV swarm collaborative mapping path is greater than or equal to the similarity threshold, and ij equals 1, return to Step 1 to execute the loop.

[0195] When the collaborative mapping path of the entire UAV swarm is traversed, the updated collaborative mapping path of the UAV swarm is output.

[0196] In this embodiment of the invention, in the scenario of collaborative mapping path planning for UAV swarms, in order to solve the problem of convergence difficulties caused by the large number of UAV mapping path clusters in the existing technical solutions, it is necessary to perform hierarchical updates of secondary paths based on the reward tree structure.

[0197] This process transforms the complex path optimization problem into an ordered hierarchical iterative process by dynamically matching the similarity between the reward path and the path to be updated, effectively reducing trial and error costs and improving convergence efficiency.

[0198] Specifically, firstly, select i-level paths (such as the third-level path when i=3) are randomly extracted from the second-level to N-level paths. Then, extract ij-level paths (second-level paths when j=1) from the same level range. These paths must contain the corresponding centroid path as reward candidates.

[0199] If the similarity between the secondary centroid path and the selected path ≥ threshold (e.g., 0.6), and i - j = 2 > 1, then increase the value of j to 2, and continue to extract the i - j = 1 level path (primary path) for comparison; if the similarity between the primary centroid path and the selected path is still ≥ threshold, it indicates that the current reward path is too similar to the path to be updated, which may lead to the optimization falling into a local optimum, and it is necessary to return to step one to re - select a new selected path.

[0200] When the similarity between the i - j centroid path of a certain level and the selected path < threshold (e.g., the similarity between the i - j = 2 level centroid path and the selected path is 0.5), use this i - j level path as the reward path to guide the selected path to update features such as the coordinate sequence and energy consumption parameters, and optimize it towards the high - fitness direction of the reward path.

[0201] Exemplarily, if the selected fourth - level path P4 is selected, extract the third - level path P3 (including the third - level centroid path CP3), calculate the similarity between CP3 and P4 as 0.7 ≥ threshold, and i - j = 3 > 1. After adding 1 to j, extract the second - level path P2 (including the second - level centroid path CP2), calculate the similarity between CP2 and P4 as 0.4 < threshold. At this time, use P2 as the reward path, and by adjusting parameters such as the inflection point coordinates and flight altitude of P4, make it approach the efficient path features of P2.

[0202] After each update, it is necessary to re - evaluate the fitness value of the selected path based on the inspection path length, inspection energy consumption, and safety level. If the convergence condition is not met (e.g., the fitness value does not reach the preset convergence threshold), then continue to loop through other paths. When all paths from the second level to the Nth level are updated, output the target path that meets the convergent fitness.

[0203] In this process, the k - level path serves as the reward path for the z - level path (k < z), forming a multi - level optimization feedback mechanism (e.g., the primary path provides a basic optimization direction for the secondary path, the secondary path refers to the features of the primary path and combines its own centroid path for iteration, and so on). Eventually, the target path reduces the inspection energy consumption, increases the safety level, and at the same time improves the convergence efficiency, meeting the industry optimization standard compared with traditional reinforcement learning methods.

[0204] Through the above - mentioned specific implementation manners, the embodiments of the present invention achieve the following technical effects:

[0205] This invention proposes a UAV swarm collaborative mapping path planning method using reinforcement learning. Through multi-cluster path clustering and reward tree hierarchical optimization, it achieves efficient path planning in complex scenarios. First, based on UAV flight boundaries, scheduling quantity, and interval constraints, initial paths are randomly deployed. A fitness function that integrates path length, energy consumption, and safety level is used for evaluation. Paths that do not meet the convergence criteria are clustered, and centroid paths are extracted. Simultaneously, the fitness sequence of centroid paths is analyzed, and multi-level paths are generated by sorting them from best to worst. Similar and redundant paths are removed, and a reward tree structure is constructed. Finally, based on the reward tree, secondary path updates are guided by centroid path similarity matching. When the similarity between a certain level's centroid path and the path to be updated is below a threshold, that level's path is used as the reward path to trigger an update until a target path that meets the conditions is output.

[0206] The method provided by this invention solves the convergence problem caused by the large path cluster in traditional methods. Through the steps of "constraint deployment - fitness evaluation - clustering to improve centroid - reward tree construction - hierarchical update", it effectively reduces the trial and error cost and enhances the convergence efficiency, improves the scientificity and reliability of path planning in complex environments, and provides an innovative solution for intelligent collaborative mapping of UAV swarms.

[0207] Example 2, as shown in the appendix Figure 2 As shown, based on the inventive concept of the UAV swarm collaborative mapping path planning method using reinforcement learning provided in Embodiment 1, this invention also provides a UAV swarm collaborative mapping path planning system using reinforcement learning, specifically including:

[0208] The path random deployment module 01 is used to randomly deploy several UAV swarms to collaboratively map the target area based on UAV navigation boundaries, UAV scheduling quantity constraints, and UAV interval constraints.

[0209] The fitness function construction module 02 is used to construct a fitness function with inspection path length, inspection energy consumption and safety level as fitness factors, and evaluate several fitness values ​​of the collaborative mapping path of several UAV swarms.

[0210] The path clustering construction module 03 is used to cluster the several UAV swarm collaborative mapping paths based on the several fitness values ​​when the several fitness values ​​do not meet the convergence fitness, and construct multi-cluster UAV swarm collaborative mapping paths, wherein the multi-cluster UAV swarm collaborative mapping paths include multiple centroid UAV swarm collaborative mapping paths.

[0211] The reward tree construction module 04 is used to sort the collaborative mapping paths of multiple centroid UAV swarms according to fitness from best to worst based on the collaborative mapping paths of the multiple centroid UAV swarms, generate first-level UAV swarm collaborative mapping paths up to N-level UAV swarm collaborative mapping paths, and construct a reward tree.

[0212] The path update iteration module 05 is used to update the collaborative mapping path of the second-level UAV swarm up to the N-level UAV swarm based on the reward tree, obtain the updated UAV swarm collaborative mapping path, and execute the loop until a target mapping path that meets the convergence fitness is generated and output. Here, the k-level UAV swarm collaborative mapping path is the reward path of the z-level UAV swarm collaborative mapping path, k < z, k and z are integers, k∈[1, N-1], z∈[2, N].

[0213] In one embodiment, the path random deployment module 01 is further configured to:

[0214] Based on the UAV flight boundary, the UAV scheduling quantity constraint, and the UAV interval constraint, several initial UAV swarms are randomly deployed to collaboratively map the path.

[0215] From the plurality of initial UAV swarm collaborative mapping paths, a first initial UAV swarm collaborative mapping path is extracted, wherein the first initial UAV swarm collaborative mapping path includes the first UAV mapping path up to the Qth UAV mapping path.

[0216] From the first UAV mapping path, the first moment's flight altitude is extracted, and combined with the first UAV model, the UAV coverage calibration table is input to calibrate the first moment's mapping coverage and add it to the first UAV mapping coverage. The UAV coverage calibration table is used to store the UAV model, UAV altitude and UAV mapping coverage with a one-to-one correspondence with failure duration constraints. The UAV mapping coverage represents the mapping boundary coordinates relative to the UAV's flight position as the origin.

[0217] Until the mapping path of the Qth UAV is processed, the mapping coverage area of ​​the Qth UAV is determined;

[0218] The union of the mapping coverage of the first UAV up to the mapping coverage of the Qth UAV is used to obtain the first initial UAV swarm collaborative mapping path coverage.

[0219] If the coverage area of ​​the first initial UAV swarm collaborative mapping path covers the target area, the first initial UAV swarm collaborative mapping path is added to the plurality of UAV swarm collaborative mapping paths; otherwise, it is deleted.

[0220] In one embodiment, fitness function construction module 02 is further configured to:

[0221] Collect drone inspection test data that meets the preset drone model and preset inspection path coordinate sequence, calculate the proportion of drone out-of-control data in the drone inspection test data, and use 1 to subtract the proportion of drone out-of-control data to set the safety level identifier.

[0222] Using the security level identifier as supervision and the preset inspection path coordinate sequence as input, a preset UAV model security level evaluator is constructed and added to the security level evaluation model library;

[0223] Based on the aforementioned safety level assessment model library, and combining path length weight, energy consumption weight, and safety level weight, the fitness function is constructed as follows:

[0224] Specifically, when the safety level is less than or equal to the safety level threshold, the fitness function is equal to the worst value; otherwise, the fitness function is equal to the weighted comprehensive value of the inspection path length, inspection energy consumption, and safety level, which combines the path length weight, energy consumption weight, and safety level weight.

[0225] In one embodiment, the path clustering building module 03 is also used for:

[0226] Extract the collaborative mapping path of the first cluster of UAVs from the collaborative mapping path of the multi-cluster UAV swarm;

[0227] Extract the first UAV swarm collaborative mapping path from the first cluster of UAV swarm collaborative mapping paths, calculate the similarity between it and the collaborative mapping paths of UAV swarms within the cluster, and calculate the mean value. Set the mean value as the centroid parameter of the first path and add it to the path centroid parameter set.

[0228] Based on the set of path centroid parameters, the maximum value is extracted and set as the first centroid UAV swarm collaborative mapping path, and added to the multiple centroid UAV swarm collaborative mapping paths.

[0229] In one embodiment, the reward tree building module 04 is also used for:

[0230] Based on the multiple centroid UAV swarm collaborative mapping paths, the collaborative mapping paths of the multiple UAV swarms are sorted according to fitness from best to worst to obtain the first-level initial UAV swarm collaborative mapping path up to the N-level initial UAV swarm collaborative mapping path.

[0231] The first-level initial UAV swarm collaborative mapping path is sorted by similar path random optimal fitness to obtain the first-level UAV swarm collaborative mapping path, wherein similar path refers to the path set with path similarity less than or equal to the path similarity threshold;

[0232] The Nth-level UAV swarm collaborative mapping path is obtained by performing similar path random optimal fitness sorting on the initial UAV swarm collaborative mapping path.

[0233] In one embodiment, the path update iteration module 05 is also used for:

[0234] Step 1: From the second-level UAV swarm collaborative mapping path to the N-level UAV swarm collaborative mapping path, randomly extract the selected UAV swarm collaborative mapping path of the i-level UAV swarm collaborative mapping path, i∈[2,N], where i is an integer;

[0235] Step 2: Extract the ij-level UAV swarm collaborative mapping path from the second-level UAV swarm collaborative mapping path to the N-level UAV swarm collaborative mapping path. The ij-level UAV swarm collaborative mapping path has the ij-centroid UAV swarm collaborative mapping path, where j is an integer and the initial value of j is equal to 1.

[0236] Step 3: When the path similarity between the ij centroid UAV swarm collaborative mapping path and the selected UAV swarm collaborative mapping path is greater than or equal to the similarity threshold, and ij is greater than 1, increment j by one, and return to step 2 to execute the loop.

[0237] Step 4: When the path similarity between the ij-level centroid UAV swarm collaborative mapping path and the selected UAV swarm collaborative mapping path is less than the similarity threshold, the ij-level UAV swarm collaborative mapping path is used as the reward path to guide the update of the selected UAV swarm collaborative mapping path.

[0238] Step 5: When the path similarity between the ij centroid UAV swarm collaborative mapping path and the selected UAV swarm collaborative mapping path is greater than or equal to the similarity threshold, and ij equals 1, return to Step 1 to execute the loop.

[0239] When the collaborative mapping path of the entire UAV swarm is traversed, the updated collaborative mapping path of the UAV swarm is output.

[0240] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented, in whole or in part, as a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape) or an optical medium.

[0241] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0242] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0243] This specification and accompanying drawings are merely illustrative examples of the invention and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its scope. Therefore, if such modifications and variations fall within the scope of the invention and its equivalents, the invention is intended to include these modifications and variations.

Claims

1. A method for collaborative mapping path planning using UAV swarms with reinforcement learning, characterized in that, include: Based on the drone navigation boundary, drone scheduling quantity constraints, and drone interval constraints, several drone swarms are randomly deployed to cover the target area for collaborative mapping paths. Using inspection path length, inspection energy consumption, and safety level as fitness factors, a fitness function is constructed to evaluate several fitness values ​​of collaborative mapping paths of several UAV swarms. When the fitness values ​​do not meet the convergence fitness, the collaborative mapping paths of the UAV swarms are clustered based on the fitness values ​​to construct a multi-cluster collaborative mapping path of UAV swarms, wherein the multi-cluster collaborative mapping path of UAV swarms includes multiple centroid UAV swarm collaborative mapping paths. Based on the multiple centroid UAV swarm collaborative mapping paths, the collaborative mapping paths of the multiple UAV swarms are sorted according to fitness from best to worst, generating first-level UAV swarm collaborative mapping paths up to N-level UAV swarm collaborative mapping paths, and constructing a reward tree. Based on the reward tree, the collaborative mapping path of the second-level UAV swarm up to the collaborative mapping path of the N-level UAV swarm is updated to obtain the updated collaborative mapping path of the UAV swarm. The loop is executed until a target mapping path that satisfies the convergence fitness is generated and output. Here, the collaborative mapping path of the k-level UAV swarm is the reward path of the collaborative mapping path of the z-level UAV swarm, k < z, k and z are integers, k∈[1,N-1], z∈[2,N].

2. The UAV swarm collaborative mapping path planning method using reinforcement learning as described in claim 1, characterized in that, Based on UAV flight boundaries, UAV scheduling quantity constraints, and UAV interval constraints, several UAV swarm collaborative mapping paths are randomly deployed to cover the target area, including: Based on the UAV flight boundary, the UAV scheduling quantity constraint, and the UAV interval constraint, several initial UAV swarms are randomly deployed to collaboratively map the path. From the plurality of initial UAV swarm collaborative mapping paths, a first initial UAV swarm collaborative mapping path is extracted, wherein the first initial UAV swarm collaborative mapping path includes the first UAV mapping path up to the Qth UAV mapping path. From the first UAV mapping path, the first moment's flight altitude is extracted, and combined with the first UAV model, the UAV coverage calibration table is input to calibrate the first moment's mapping coverage and add it to the first UAV mapping coverage. The UAV coverage calibration table is used to store the UAV model, UAV altitude and UAV mapping coverage with a one-to-one correspondence with failure duration constraints. The UAV mapping coverage represents the mapping boundary coordinates relative to the UAV's flight position as the origin. Until the mapping path of the Qth UAV is processed, the mapping coverage area of ​​the Qth UAV is determined; The union of the mapping coverage of the first UAV up to the mapping coverage of the Qth UAV is used to obtain the first initial UAV swarm collaborative mapping path coverage. If the coverage area of ​​the first initial UAV swarm collaborative mapping path covers the target area, the first initial UAV swarm collaborative mapping path is added to the plurality of UAV swarm collaborative mapping paths; otherwise, it is deleted.

3. The UAV swarm collaborative mapping path planning method using reinforcement learning as described in claim 2, characterized in that, From the mapping path of the first UAV, the flight altitude at the first moment is extracted. Combined with the UAV model, the UAV coverage calibration table is used to calibrate the mapping coverage area at the first moment, including: The first UAV model is input into the UAV coverage calibration table for sorting to obtain a first-level matching binary array, wherein each of the first-level matching binary arrays includes a flight altitude record value and a coverage calibration value. Based on the flight altitude at the first moment, the first-level matching binary array is traversed and compared with the flight altitude record value to extract the set of coverage calibration values ​​where the flight altitude deviation is less than or equal to the flight altitude deviation threshold. When the set of coverage calibration values ​​is not empty, extract the minimum value and set it as the survey coverage at the first time. When the set of coverage calibration values ​​is empty, the coverage calibration model bound to the first UAV model is retrieved from the coverage calibration model library, the flight altitude at the first moment is processed, the mapping coverage at the first moment is generated, and the UAV coverage calibration table is updated using the flight altitude at the first moment, the first UAV model and the mapping coverage at the first moment. The coverage calibration model is generated by machine learning training using multiple sets of data. Each set of data includes the flight altitude record value of a preset model of UAV and the coordinate label of the mapping boundary with the UAV's flight position as the origin.

4. The UAV swarm collaborative mapping path planning method using reinforcement learning as described in claim 1, characterized in that, A fitness function is constructed using inspection path length, inspection energy consumption, and safety level as fitness factors, including: Collect drone inspection test data that meets the preset drone model and preset inspection path coordinate sequence, calculate the proportion of drone out-of-control data in the drone inspection test data, and use 1 to subtract the proportion of drone out-of-control data to set the safety level identifier. Using the security level identifier as supervision and the preset inspection path coordinate sequence as input, a preset UAV model security level evaluator is constructed and added to the security level evaluation model library; Based on the aforementioned safety level assessment model library, and combining path length weight, energy consumption weight, and safety level weight, the fitness function is constructed as follows: Specifically, when the safety level is less than or equal to the safety level threshold, the fitness function is equal to the worst value; otherwise, the fitness function is equal to the weighted comprehensive value of the inspection path length, inspection energy consumption, and safety level, which combines the path length weight, energy consumption weight, and safety level weight.

5. The UAV swarm collaborative mapping path planning method using reinforcement learning according to claim 1, characterized in that, The multi-cluster UAV swarm collaborative mapping path includes multiple centroid UAV swarm collaborative mapping paths, including: Extract the collaborative mapping path of the first cluster of UAVs from the collaborative mapping path of the multi-cluster UAV swarm; Extract the first UAV swarm collaborative mapping path from the first cluster of UAV swarm collaborative mapping paths, calculate the similarity between it and the collaborative mapping paths of UAV swarms within the cluster, and calculate the mean value. Set the mean value as the first path centroid parameter and add it to the path centroid parameter set. Based on the set of path centroid parameters, the maximum value is extracted and set as the first centroid UAV swarm collaborative mapping path, and added to the multiple centroid UAV swarm collaborative mapping paths.

6. The UAV swarm collaborative mapping path planning method using reinforcement learning according to claim 1, characterized in that, Based on the multiple centroid UAV swarm collaborative mapping paths, the collaborative mapping paths of the multi-cluster UAV swarms are sorted according to fitness from best to worst, generating first-level UAV swarm collaborative mapping paths up to N-level UAV swarm collaborative mapping paths, and constructing a reward tree, including: Based on the multiple centroid UAV swarm collaborative mapping paths, the collaborative mapping paths of the multiple UAV swarms are sorted according to fitness from best to worst to obtain the first-level initial UAV swarm collaborative mapping path up to the N-level initial UAV swarm collaborative mapping path. The first-level initial UAV swarm collaborative mapping path is sorted by similar path random optimal fitness to obtain the first-level UAV swarm collaborative mapping path, wherein similar path refers to the path set with path similarity less than or equal to the path similarity threshold; The Nth-level UAV swarm collaborative mapping path is obtained by performing similar path random optimal fitness sorting on the initial UAV swarm collaborative mapping path.

7. The method for collaborative mapping path planning using UAV swarms with reinforcement learning as described in claim 1, characterized in that, Based on the reward tree, the collaborative mapping paths of the second-level UAV swarm up to the N-level UAV swarm are updated to obtain the updated UAV swarm collaborative mapping paths, including: Step 1: From the second-level UAV swarm collaborative mapping path to the N-level UAV swarm collaborative mapping path, randomly extract the selected UAV swarm collaborative mapping path of the i-level UAV swarm collaborative mapping path, i∈[2,N], where i is an integer; Step 2: Extract the ij-level UAV swarm collaborative mapping path from the second-level UAV swarm collaborative mapping path to the N-level UAV swarm collaborative mapping path. The ij-level UAV swarm collaborative mapping path has the ij-centroid UAV swarm collaborative mapping path, where j is an integer and the initial value of j is equal to 1. Step 3: When the path similarity between the ij centroid UAV swarm collaborative mapping path and the selected UAV swarm collaborative mapping path is greater than or equal to the similarity threshold, and ij is greater than 1, increment j by one, and return to step 2 to execute the loop. Step 4: When the path similarity between the ij-level centroid UAV swarm collaborative mapping path and the selected UAV swarm collaborative mapping path is less than the similarity threshold, the ij-level UAV swarm collaborative mapping path is used as the reward path to guide the update of the selected UAV swarm collaborative mapping path. Step 5: When the path similarity between the ij centroid UAV swarm collaborative mapping path and the selected UAV swarm collaborative mapping path is greater than or equal to the similarity threshold, and ij equals 1, return to Step 1 to execute the loop; When the collaborative mapping path of the entire UAV swarm is traversed, the updated collaborative mapping path of the UAV swarm is output.

8. A collaborative mapping path planning system for unmanned aerial vehicle (UAV) swarms employing reinforcement learning, characterized in that, The system is used to execute the UAV swarm collaborative mapping path planning method using reinforcement learning as described in any one of claims 1-7, and the system comprises: The path random deployment module is used to randomly deploy several UAV swarms to collaboratively map the target area based on UAV flight boundaries, UAV scheduling quantity constraints, and UAV interval constraints. The fitness function construction module is used to construct a fitness function with inspection path length, inspection energy consumption and safety level as fitness factors, and evaluate several fitness values ​​of collaborative mapping paths of several UAV swarms. The path clustering construction module is used to cluster the several UAV swarm collaborative mapping paths based on the several fitness values ​​when the several fitness values ​​do not meet the convergence fitness, and construct multi-cluster UAV swarm collaborative mapping paths, wherein the multi-cluster UAV swarm collaborative mapping paths include multiple centroid UAV swarm collaborative mapping paths. The reward tree construction module is used to sort the collaborative mapping paths of multiple centroid UAV swarms according to their fitness from best to worst based on the collaborative mapping paths of the multiple centroid UAV swarms, generate first-level UAV swarm collaborative mapping paths up to N-level UAV swarm collaborative mapping paths, and construct a reward tree. The path update iteration module is used to update the collaborative mapping path of the second-level UAV swarm up to the N-level UAV swarm based on the reward tree, obtain the updated UAV swarm collaborative mapping path, and execute the loop until a target mapping path that meets the convergence fitness is generated and output. Here, the k-level UAV swarm collaborative mapping path is the reward path of the z-level UAV swarm collaborative mapping path, k < z, k and z are integers, k∈[1,N-1], z∈[2,N].

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the method as described in any one of claims 1-7.

10. A computer device, characterized in that, The computer device includes a memory, a processor, and a program stored in and executable on the memory, the program being executed by the processor to implement the steps of the method as described in any one of claims 1-7.

Images