A Monte Carlo Tree Search-Based Method for UAV 3D Coverage Path Planning

Abstract

This invention discloses a UAV 3D coverage path planning method based on Monte Carlo tree search. The method uniformly samples the search space and angles according to positional and angular resolutions, eliminating viewpoints located inside the model. Viewpoints that are too close or too far from the model are eliminated based on safe distance and sensor detection distance. The visible point cloud sequence of the viewpoint is calculated based on the angle between the viewpoint camera's line of sight and the point cloud point normal vector, and unqualified viewpoints are then eliminated accordingly. A sub-viewpoint sequence is generated for each viewpoint based on distance relationships. Then, for the current viewpoint, expansion, simulation, backtracking, and selection are performed according to the Monte Carlo tree search concept until the coverage meets the requirements. This invention ensures the detection quality of UAVs when detecting large 3D structures, while reducing redundancy in the full-coverage path, improving the detection efficiency and energy utilization of the UAV detection process. Furthermore, its viewpoint generation process ensures good algorithm flexibility and robustness.

Description

Technical Field

[0001] This invention belongs to the field of UAV path planning technology, specifically relating to a UAV three-dimensional coverage path planning method based on Monte Carlo tree search. Background Technology

[0002] Unmanned Aerial Vehicles (UAVs) are aircraft that fly remotely or autonomously via computer programs, possessing a variety of functions and applications. With the continuous development of UAV technology, the use of UAVs for three-dimensional full-coverage inspection of large and important facilities such as bridges and oil storage tanks has been widely applied in fields such as urban planning, environmental monitoring, and safety assessment.

[0003] When using drones for full-coverage 3D inspection of large and complex structures, it is necessary to ensure that the drone's sensors fully cover the surface of the 3D object to prevent missed detections. Simultaneously, a safe distance must be maintained between the drone and the 3D structure to guarantee the safety of both the drone and the object being inspected. However, using a handheld remote controller to operate the drone makes it impossible to assess whether the sensors achieve full coverage during flight, and maintaining a constant distance between the drone and the object increases inspection time and risk. Therefore, safe, efficient, and accurate full-coverage path planning for large 3D structures is a significant challenge and has important research implications for the practical application of drones in this field.

[0004] Research on full-coverage path planning for large 3D structures using UAVs is still insufficient. To achieve surface detection of large 3D structures, it is generally necessary to generate viewpoints that cover the model based on the available structural model, and then select these viewpoints to generate a full-coverage path. Viewpoint selection methods based on the Traveling Salesman Problem (TSP) are very common in full-coverage path planning algorithms. Besides TSP, search algorithms are also frequently used in the viewpoint selection process for full-coverage path planning. However, these full-coverage path planning algorithms have many shortcomings. For example, TSP-based 3D full-coverage path planning algorithms, due to limitations in their viewpoint generation methods, usually require models with specific accuracy. Low-accuracy models can lead to missed detections, while high-accuracy models can result in redundant full-coverage paths, leading to low algorithm robustness. Search-based full-coverage path planning algorithms often ignore the angle between the camera's line of sight and the surface to be detected when generating viewpoints, leading to distorted information and difficulty in completing the detection task. Furthermore, existing commonly used search-based full-coverage path planning algorithms all rely on greedy algorithms to expand paths, making it difficult to utilize the global information of the model, which also results in path redundancy. Therefore, solving problems such as low robustness, path redundancy, and poor detection quality has become a research hotspot in the field of full-coverage path planning algorithms.

[0005] Monte Carlo tree search (MCS) is a powerful search and decision-making algorithm that efficiently explores possible solutions in the search space through stochastic simulation and backpropagation, and optimizes and makes decisions based on the simulation results. It has already been applied in the field of UAV path planning. For example, the UAV path planning method based on Monte Carlo tree search proposed by Sheng Kexin et al. demonstrates the algorithm's advantages in path planning. However, the problem of full-coverage path planning for large 3D structures requires consideration of more constraints and variables, while ensuring path integrity and detection quality. Existing path planning methods based on this algorithm can only handle a very limited number of path points, and their feasibility in 3D space has not been verified, thus failing to meet the requirements of full-coverage path planning for large 3D structures. Summary of the Invention

[0006] To address the shortcomings of existing full-coverage path planning algorithms for large 3D structure detection using unmanned aerial vehicles (UAVs), this invention provides a UAV 3D coverage path planning method based on Monte Carlo tree search. This method not only ensures the detection quality of each viewpoint during the detection process but also selects viewpoints based on global information from a prior model, avoiding redundancy in the full-coverage path and improving the efficiency of the detection process and the energy utilization of the UAV. Furthermore, this method can perform full-coverage path planning tasks for various large 3D structures, exhibiting strong robustness and high flexibility.

[0007] To achieve the above objectives, the technical solution adopted by this invention is: a method for UAV 3D coverage path planning based on Monte Carlo tree search, comprising the following steps:

[0008] The search space and angle are uniformly sampled according to the position resolution and angle resolution to generate an initial viewpoint;

[0009] Viewpoints located inside the model in the initial viewpoints are removed. Then, viewpoints that are too close to or too far from the model are removed based on the safety distance and sensor detection distance. Finally, the visible point cloud sequence of the viewpoint is calculated based on the angle between the viewpoint camera's line of sight and the point cloud point normal vector. Unqualified viewpoints in the visible point cloud sequence of the viewpoint are removed, resulting in a set of viewpoints evenly distributed around the object to be detected.

[0010] For each viewpoint in a set of viewpoints around the object to be detected, a sub-viewpoint sequence is generated based on the distance relationship. Then, based on the idea of ​​Monte Carlo tree search, the process of expansion, simulation, backtracking and viewpoint selection is performed until the coverage meets the requirements, and a full coverage path for 3D detection is obtained.

[0011] The process of uniformly sampling the search space and angle according to positional and angular resolutions to generate an initial viewpoint includes the following steps:

[0012] First, the sampling space is uniformly divided into multiple grids according to a set resolution, resulting in multiple non-overlapping sub-regions. The vertices of each sub-region represent the position of the drone. At the vertices of each sub-region, sampling is performed according to a set angular resolution to obtain a more detailed viewpoint. The sampling formula is as follows:

[0013] ViewPoint ijkn =(x i ,y j ,z k ,ψ n )

[0014]

[0015] ViewPoint ijkn For the generated viewpoint; x i ,y j ,z k ,ψ n The coordinates and rotation angles of this viewpoint; x0, y0, z0 are the starting points of the entire sampling space; R position R represents the positional resolution. angle To obtain an angular resolution, a series of initial viewpoints are obtained.

[0016] Eliminating viewpoints located inside the model involves the following steps:

[0017] The viewpoint is compared with the boundary of the object to be detected. Viewpoints within the boundary are filtered out, and viewpoints outside the boundary are retained, as shown in the following formula:

[0018]

[0019] Where, f(VP) i ) represents the viewpoint VP i The output of the boundary filtering function, if viewpoint VP i If not within the boundary, then f(VP) i A value of 1 indicates that the viewpoint meets the requirements; if the viewpoint VP... i Within the boundary, then f(VP) i A value of 0 indicates that the viewpoint needs to be filtered out.

[0020] The specific criteria for eliminating viewpoints based on safe distance and sensor detection distance are as follows:

[0021] The remaining viewpoints after removing internal viewpoints of the model are further filtered using the safe distance of the drone and the detection range of the camera. Viewpoints that are too close to the object to be detected or are outside the camera's detection range are removed, and only viewpoints that meet the requirements are retained as the final feasible viewpoints. Specifically:

[0022] d min ≤d i ≤d max

[0023] Where: d min This represents the minimum safe distance between the drone and the object to be detected. The value of the minimum safe distance is determined based on the drone's motion characteristics; d max This represents the maximum distance between the drone and the object to be detected; the maximum distance value is based on the sensor's detection range. i For viewpoint VP i The actual distance of the distance model; a set of viewpoints uniformly distributed around the object to be detected is obtained, the distance between the viewpoints and the object to be detected meeting both the safety requirements of the UAV and the detection requirements of the sensor.

[0024] The process of calculating the visible point cloud sequence of a viewpoint based on the angle between the viewpoint camera's line of sight and the point cloud's point normal vector, and then removing unqualified viewpoints from the viewpoint's visible point cloud sequence, includes the following steps:

[0025] Each viewpoint is scored based on the area of ​​the three-dimensional structure effectively detected by the viewpoint, and viewpoints with scores below a set value are eliminated.

[0026] Point cloud files are used as the model files for the 3D structure to be detected. The number of intersections between the rays formed by points in the view frustum pointing to the camera's location and the entire model is calculated to determine whether a point is occluded. Specifically:

[0027]

[0028] Among them, P p P represents the current position. cam V is the location of the camera; V is the ray formed from the current point in the frustum to the location of the camera; intersection() is a function to calculate the number of intersections between the ray and the model; intersectionNum is the number of intersections between the ray and the model; if the number of intersectionNum is not zero, the current point is occluded and cannot be directly detected by the camera, so the current point is removed. After completing the occlusion removal operation for every point in the frustum, the remaining points in the frustum can be directly detected by the camera.

[0029] For each viewpoint, among the remaining point cloud data points P within the view frustum, find the K nearest neighbor points around point P. The resulting set of neighbor points for point P is represented as:

[0030] N(P) = {P1, P2, ..., P} n}

[0031] Where N(P) is the set of points in the neighborhood of point P;

[0032] Suppose there exists a plane such that the sum of the squares of the perpendicular distances from all points in the neighborhood set N(P) of a cloud point P to the plane is minimized. Taking the normal vector of the plane as the normal vector of point P, the fitted plane for all points in N(P) is represented by the following equation:

[0033] z = a*x + b*y + c

[0034] Solving the following equation yields the specific values ​​of the fitted plane:

[0035]

[0036] Where (a,b,c) * The parameters representing the fitted plane; z i x i y i Let V be the coordinates of a point in N(P). Using the viewpoint coordinates as the observation point coordinates, we obtain the normal vector of the fitted plane, and the normal vector of point P is V. P Represented as:

[0037]

[0038] The specific way of expression is determined by the coordinates of the observation point;

[0039] Next, the angle θ between the normal vector of each point and the line of sight of the corresponding camera is calculated, and the angle θ is compared with a pre-set threshold θ0. If θ is less than θ0, the point will be added to the visible point cloud sequence of the current viewpoint. After traversing the remaining point cloud data in the view frustum corresponding to the current viewpoint after occlusion removal, the current viewpoint is scored according to the visible point cloud sequence, as follows:

[0040]

[0041] Where total_points is the total number of point clouds within the current view frustum; visible_points is the number of visible point cloud sequences at the current viewpoint, and the ratio of the number of visible point cloud sequences to the total number of point clouds is used as the score value.

[0042] The angle θ between the normal vector of each point and the camera's line of sight corresponding to the viewpoint is calculated as follows:

[0043] The normal vector of the fitted plane and the Z-axis of the camera coordinate system are both expressed in the world coordinate system, as follows:

[0044]

[0045] in, Z is the rotation transformation matrix between the body coordinate system and the sensor coordinate system. cam This represents the direction of the Z-axis in the camera coordinate system, with specific values ​​of [0 0 1]. T V cam-world The Z-axis of the camera coordinate system is the vector in the world coordinate system;

[0046] Next, the angle θ between the normal vector of each point and the Z-axis in the camera coordinate system is calculated using the following formula:

[0047]

[0048] The sub-viewpoint sequence is generated for each viewpoint based on distance relationships as follows:

[0049] Connect the initial viewpoints and determine the distance threshold d. th For each viewpoint VP i In other words, if there exists a viewpoint VP j Make the distance between the two less than the distance threshold d th Then the viewpoint VP j As a viewpoint VP i One of the subviewpoint sequences, specifically represented as:

[0050]

[0051] Where dis() is the distance function; Child(VP) i ) is VP i The sub-viewpoint sequence, viewpoint VP i Distance threshold d th All other viewpoints within will be added to viewpoint VP. i In the sub-viewpoint sequence, the distance threshold d th Adjustments can be made based on specific application scenarios;

[0052] When extending a path, a viewpoint is selected from the sequence of sub-viewpoints of the current viewpoint as the path extension until the target point is reached or the maximum number of iterations is reached.

[0053] Performing expansion, simulation, backtracking, and selection on the current viewpoint based on the idea of ​​Monte Carlo tree search includes the following steps:

[0054] a) Expansion

[0055] Let Child(VP) current ) represents the current viewpoint VP current The subview sequence, the expansion process is from Child(VP) current In step ), select nodes that have not yet been scored to prepare for the next step of simulation scoring, represented as:

[0056]

[0057] Among them, VP expand For extended viewpoints; isScore() represents viewpoint VP. i The scoring status during the expansion process of the current viewpoint; if viewpoint VP i If a viewpoint has not been scored during the expansion process of the current viewpoint, it is true; otherwise, it is false. The viewpoint sequence is generated by a distance threshold. A viewpoint may be in the sub-viewpoint sequence of multiple viewpoints. Regardless of whether the sub-viewpoint has been expanded or scored during the expansion process of other viewpoints, it needs to be expanded during the expansion process of the current viewpoint.

[0058] b) Simulation

[0059] During the viewpoint selection process, after the corresponding sub-viewpoints are selected through expansion, the sub-viewpoints are simulated. In each step of the simulation, a greedy algorithm is used to select the sub-viewpoint with the largest increase in coverage to continue the simulation.

[0060] The simulation process begins by setting the initial viewpoint to the expanded viewpoint obtained in the previous step:

[0061] VP simulate =VP expand

[0062] Then, from the sub-viewpoints of the simulated viewpoint, select the sub-viewpoint with the largest increase in coverage as the next simulated viewpoint:

[0063]

[0064] CoverIns() is a function for calculating coverage growth. It simulates the current sub-viewpoint a set number of times before proceeding to the next step of expansion or selection.

[0065] c) Backtracking

[0066] After completing the simulation of the sub-viewpoint, the simulation results are backtracked up the simulation path to the root node. Each node has two parameters: the number of visits (visCount) and the reward value (winCount). During the backtracking process, the number of visits to each node is incremented by 1. If the simulation result is a victory, the reward value of the node is also incremented by 1; otherwise, the reward value remains unchanged.

[0067] Viewpoint visits are equated to node visits. During backtracking, the viewpoint visit count for each viewpoint on the path is incremented by 1. The viewpoint reward value is equated to the node reward value. During backtracking, the viewpoint reward value is increased based on the increase in coverage of the current viewpoint relative to the initial simulated viewpoint. At the end of the simulation, the total path coverage corresponding to the next viewpoint is greater than the total path coverage of the corresponding viewpoint at the start of the simulation. The increase in total path coverage during the simulation represents the reward value for simulating child nodes.

[0068] A greedy algorithm is used to perform full-coverage path planning for different models, and the average coverage rate of each viewpoint is calculated. The average coverage rate is then used as the standard coverage rate.

[0069] d) Selection

[0070] After all viewpoints have been expanded and simulated, the visit count (visCount) and reward value (winCount) of each viewpoint are updated. Sub-viewpoints are selected based on their specific visit count (visCount) and reward value (winCount) to form a full-coverage path. An improved UCT value is used to calculate the selection probability of each sub-viewpoint, and the sub-viewpoint with the higher UCT value is selected as the current viewpoint to expand the path until the path coverage reaches the algorithm's convergence condition.

[0071]

[0072] The viewpoint reward value is calculated as follows:

[0073]

[0074] Among them, result CovCoverage is the viewpoint reward value for traversing the path upwards from the last viewpoint after the simulation ends. final The total path coverage of the last viewpoint after the simulation ends; Coverage start The total path coverage is the initial viewpoint at the start of the simulation; n is the number of simulations.

[0075] In the improved UCT value calculation, the value of the exploration constant c is modified, and the coverage growth factor is added. The improved UCT value calculation formula is as follows:

[0076]

[0077] Where c1 is the exploration constant, c2 is the sub-viewpoint coverage coefficient, and Coverage i This represents the increase in coverage of the current child node compared to its parent node.

[0078] Compared with the prior art, the present invention has at least the following beneficial effects:

[0079] Based on the method described in this invention, the method for generating initial viewpoints can effectively ensure the uniformity and completeness of viewpoints, thereby improving the accuracy and efficiency of path planning. Furthermore, the shape and detail dimensions of the 3D structural model have no impact on the initial viewpoint generation process. This characteristic also makes the method of this invention more robust and adaptable to more diverse models. Based on the method described in this invention, after viewpoint filtering, the viewpoint selected from the visible point cloud sequence is calculated based on the angle between the viewpoint camera's line of sight and the point cloud's point normal vector. This not only ensures the safety of the UAV during detection but also improves the detection effect in the full-coverage path planning algorithm. It avoids the problem of excessively large angles between the camera's line of sight and the surface of the object to be detected, as well as camera distortion caused by excessively large angles. The process of filtering viewpoints is also the process of narrowing the search space for viewpoint selection, improving the speed of the viewpoint selection process in the full-coverage path planning algorithm and reducing the algorithm's convergence time.

[0080] Furthermore, the prior global information of the 3D model is considered during the viewpoint search and selection process, and this prior global information is backtracked to each viewpoint of the simulation path using a reward value based on standard coverage. In this way, global information can be used to select the optimal sub-viewpoint, thereby expanding the full coverage path. The method described in this invention can avoid the full coverage path from falling into local optima and effectively avoid redundancy in the full coverage path, thereby improving the efficiency of UAVs in detection tasks targeting large 3D structures.

[0081] Furthermore, through the improved UCT value calculation method, the value of the latter part of the UCT value calculation formula increases with fewer explorations. Therefore, points that are explored less frequently will have more exploration opportunities. The larger the proportion of the latter part of the formula, the more inclined the viewpoint selection will be to choose viewpoints with fewer visits, and the path will be more inclined to explore unknown areas. Conversely, the smaller the proportion of the latter part of the formula, the more inclined the viewpoint selection will be to choose viewpoints with more visits, and the path will be more inclined to explore the current area. Therefore, the value of the exploration constant c can be modified to adjust the search tendency of the entire algorithm. Adding sub-viewpoint information, such as coverage growth, to the formula helps the full-coverage path planning algorithm select the best sub-viewpoint. The improved UCT value calculation method takes into account both the simulation results and the quality of the current child node, which helps the algorithm select the optimal sub-viewpoint. It can evaluate each viewpoint and select the sub-viewpoint with the highest UCT value to expand the path. Combining the idea of ​​Monte Carlo tree search and prior global information, it makes viewpoint selection more accurate and comprehensive. Attached Figure Description

[0082] Figure 1 This is a flowchart of an implementable method of the present invention;

[0083] Figure 2 This is a schematic diagram illustrating the initial viewpoint generation of the present invention;

[0084] Figure 3 This is a schematic diagram illustrating the occlusion removal from the viewpoint of this invention.

[0085] Figure 4 This is a schematic diagram of point normal vector filtering in this invention;

[0086] Figure 5 This is a schematic diagram illustrating the generation of sub-viewpoint sequences in this invention;

[0087] Figure 6 This is a schematic diagram of the Monte Carlo tree search process of the present invention;

[0088] Figure 7 The results of the implementation on the fighter jet model; Detailed Implementation

[0089] The technical solutions in the embodiments of the present invention will be further described in detail below:

[0090] The process of the method proposed in this invention is as follows: Figure 1 As shown, the details are as follows:

[0091] Step 1: Initial viewpoint generation.

[0092] The process includes the following steps: uniformly sampling the search space and angles according to position resolution and angle resolution to generate an initial viewpoint.

[0093] First, the sampling space is uniformly divided into grids at a certain resolution, resulting in non-overlapping sub-regions. The vertices of each sub-region represent the drone's position. Then, at each vertex of the sub-region, sampling is performed at a certain angular resolution to obtain more detailed viewpoints. Figure 2 As shown. The sampling formula is as follows:

[0094] ViewPoint ijkn =(x i ,y j ,z k ,ψ n )

[0095]

[0096] ViewPoint ijkn For the generated viewpoint; x i ,y j ,z k ,ψ n The coordinates and rotation angles of this viewpoint; x0, y0, z0 are the starting points of the entire sampling space; R position R represents the positional resolution. angle This provides the angular resolution. This yields a series of initial viewpoints for subsequent full-coverage path planning.

[0097] Step 2: Viewpoint filtering.

[0098] First, viewpoints located inside the fighter jet model are eliminated; then, viewpoints that are too close or too far from the model are eliminated based on safe distance and sensor detection distance; finally, the visible point cloud sequence of the viewpoint is calculated based on the angle between the viewpoint camera's line of sight and the point cloud point normal vector, and unqualified viewpoints are eliminated accordingly. Specifically, the following steps are included:

[0099] 1) Eliminate viewpoints located inside the model

[0100] Before performing normal-vector-based filtering, viewpoints located inside the model must first be eliminated, as these viewpoints are ineffective for the detection task. The viewpoints need to be compared with the boundaries of the object to be detected to determine if they are within the boundaries. Viewpoints within the boundaries will be filtered out; only viewpoints outside the boundaries can meet the requirements of the full-coverage path planning algorithm. See the following formula for details:

[0101]

[0102] Where, f(VP) i ) represents the viewpoint VP i The output of the boundary filtering function. If the viewpoint VPi If not within the boundary, then f(VP) i A value of 1 indicates that the viewpoint meets the requirements of the full coverage path planning algorithm; if the viewpoint VP... i Within the boundary, then f(VP) i A value of 0 indicates that the viewpoint needs to be filtered out.

[0103] 2) Eliminate viewpoints based on safe distance and sensor detection distance.

[0104] Next, the remaining viewpoints need to be filtered a second time using the drone's safe distance and the camera's detection range. Viewpoints that are too close to the object to be detected or are outside the camera's detection range are eliminated, and only those that meet the requirements are retained as the final feasible viewpoints. This process can be specifically described as follows:

[0105] d min ≤d i ≤d max (0.1)

[0106] Where: d min This is the minimum safe distance between the drone and the object to be detected; the choice of this value is based on the drone's motion characteristics. max This represents the maximum distance between the drone and the object to be detected; the choice of this value is based on the sensor's detection range. i For viewpoint VP i The actual distance from the fighter jet model.

[0107] After this viewpoint removal is completed, a set of viewpoints will be evenly distributed around the object to be detected. The distance between the viewpoints and the object to be detected will meet both the safety requirements of the UAV and the detection requirements of the sensor.

[0108] 3) Viewpoint filtering based on the normal vectors of point cloud points

[0109] Cameras are the most commonly used sensors for large-scale 3D structure inspection, and the angle between the camera's line of sight and the surface of the object being inspected directly affects the inspection quality. This invention constrains the viewpoints, eliminating viewpoints with excessively large angles between the camera's line of sight and the surface of the object being inspected. In the viewpoint filtering process based on normal vectors, each viewpoint is scored based on the area of ​​the 3D structure effectively detected by that viewpoint. Viewpoints with lower scores are eliminated, thereby obtaining a set of viewpoints that can meet the inspection task and ensure inspection quality.

[0110] A view frustum is a physical model representing the camera's detection range. Objects within the view frustum can be captured by the camera. However, not all surfaces of the object being detected that are obscured within the view frustum can be detected. This invention extracts the object surfaces that can be detected by the camera and uses a point cloud file as the model file for the fighter jet. Each point constituting the model can become a research object. By calculating the number of intersections between the rays formed by the rays from a point in the view frustum to the camera's location and the entire fighter jet model, it can be determined whether a point is obscured. Figure 3 As shown. Specifically, it can be represented as:

[0111]

[0112] Among them, P p P represents the current position. cam V represents the camera's location; V is the ray formed from the current point in the view frustum to the camera's location; intersection() is a function that calculates the number of intersections between the ray and the fighter model; intersectionNum is the number of intersections between the ray and the fighter model. Therefore, if the number of intersectionNum is not zero, it means that the current point is occluded and cannot be directly detected by the camera, and the current point will be removed. After removing the occlusion of each point in the view frustum, the remaining points in the view frustum can be directly detected by the camera.

[0113] Then, the remaining point cloud data within the view frustum corresponding to each viewpoint after occlusion culling is processed. Specifically, for a point P in the view frustum, its K nearest neighbors are found. As a possible example, the nearest neighbors are 8 or 16, and the set of neighbors is represented as follows:

[0114] N(P) = {P1, P2, ..., P} n}

[0115] Here, N(P) is the set of points in the neighborhood of point P. These point cloud data points will be constructed into a point cloud search tree (such as a KD-Tree or Octree) to facilitate the calculation of the normal vector. Assuming the existence of a plane such that the sum of the squares of the perpendicular distances from all points in N(P) to this plane is minimized, the normal vector of this plane is taken as the normal vector of point P. The fitted plane for all points in N(P) is represented by the following equation:

[0116] z = a*x + b*y + c

[0117] Solving the following equation will yield the specific values ​​of the fitted plane:

[0118]

[0119] Where (a,b,c) *The parameters representing the fitted plane; z i x i y i Let V be the coordinates of a point in N(P). Meanwhile, to ensure the range of the angle between the normal vector and the camera's line of sight, the viewpoint coordinates are used as the observation point coordinates. The resulting normal vector of the fitted plane is the normal vector V of point P. P ,like Figure 4 As shown. It can be represented as:

[0120]

[0121] The specific way it is expressed is determined by the coordinates of the observation point.

[0122] After calculating the normal vector of the midpoint of the view frustum using the method described above, the next step is to calculate the angle between the normal vector of each point and the camera's line of sight corresponding to that point. This is used to determine whether the viewpoint can complete the detection task of the object surface represented by that point. Specifically:

[0123] First, the normal vector of this plane and the Z-axis of the camera coordinate system are both expressed in the world coordinate system, as follows:

[0124]

[0125] in, Z is the rotation transformation matrix between the body coordinate system and the sensor coordinate system, which is the same as defined above; cam This represents the direction of the Z-axis in the camera coordinate system, with specific values ​​of [0 0 1]. T V cam-world The Z-axis of the camera coordinate system is the vector in the world coordinate system;

[0126] Next, the angle θ between the normal vector of each point and the Z-axis in the camera coordinate system is calculated using the following formula:

[0127]

[0128] The included angle θ is compared with a pre-set threshold θ0. If θ is less than θ0, the point will be added to the visible point cloud sequence of the current viewpoint; otherwise, the point will be ignored. After traversing all the remaining point cloud data within the view frustum corresponding to the current viewpoint after occlusion removal, the current viewpoint is scored based on its visible point cloud sequence, as follows:

[0129]

[0130] Here, `total_points` represents the total number of point clouds within the current view frustum, and `visible_points` represents the number of visible point cloud sequences at the current viewpoint. The scoring formula uses the ratio of the number of visible point cloud sequences to the total number of point clouds as the score. A higher score is awarded if the current viewpoint can see more point clouds. Conversely, a low score indicates that the area of ​​the object surface that this viewpoint can detect is too small, and this viewpoint will be filtered out.

[0131] Step 3: Monte Carlo Tree Search for Viewpoint Selection. First, a sequence of sub-viewpoints is generated for each viewpoint based on distance relationships. Then, for the current viewpoint, expansion, simulation, backtracking, and selection are performed based on the idea of ​​Monte Carlo tree search until the coverage requirement is met. During this process, the backtracking and selection methods are improved based on the characteristics and requirements of full coverage path planning.

[0132] Specifically, the following steps are included:

[0133] 1) Generate a sub-viewpoint sequence for each viewpoint based on distance relationships.

[0134] To achieve full-coverage path planning, the initial viewpoints need to be connected, generating a sub-viewpoint sequence for each viewpoint to facilitate full-coverage path planning. Therefore, a distance threshold d is first determined. th For each viewpoint VP i In other words, if there exists a viewpoint VP j Make the distance between the two less than the distance threshold d th So, viewpoint VP j Will become the viewpoint VP i A member of the subview sequence, such as Figure 5 As shown. This process can be represented as:

[0135]

[0136] Where dis() is the distance function; Child(VP) i ) is VP i The sub-viewpoint sequence. Therefore, viewpoint VP i Distance threshold d th All other viewpoints within will be added to viewpoint VP. i In the sub-viewpoint sequence. Distance threshold d th This can be adjusted according to specific application scenarios. When extending the path, a viewpoint will be selected from the sub-viewpoint sequence of the current viewpoint as the path extension until the target point is reached or the maximum number of iterations is reached.

[0137] 2) Based on the Monte Carlo tree search concept, perform expansion, simulation, backtracking, and selection on the current viewpoint.

[0138] After generating a sub-viewpoint sequence for each viewpoint, viewpoints within the sub-viewpoint sequence need to be selected progressively to form a full-coverage path. To achieve this, this invention employs the concept of Monte Carlo Tree Search (MCTS) for depth-oriented searching, enabling the viewpoint selection process to better utilize prior global information from the fighter model, avoiding redundancy in the full-coverage path, and improving the energy utilization efficiency of the UAV. The following will describe the search and selection process according to the four steps of MCTS: selection, expansion, simulation, and backtracking. Figure 6 As shown.

[0139] a) Expansion

[0140] This section first introduces an extension method for the viewpoint selection process in the full-coverage path planning algorithm based on Monte Carlo tree search. Assume Child(VP) current ) represents the current viewpoint VP current The subview sequence, the expansion process is from Child(VP) current Selecting nodes that have not yet been scored to prepare for the next step of simulation scoring can be represented as follows:

[0141]

[0142] Among them, VP expand For extended viewpoints; isScore() represents viewpoint VP. i In the scoring status during the expansion process of the current viewpoint, if viewpoint VP i The condition is true if the sub-viewpoint has not yet been scored during the expansion process of the current viewpoint, and false otherwise. In this paper, the sub-viewpoint sequence is generated using a distance threshold, so a viewpoint may be in the sub-viewpoint sequence of multiple viewpoints. Therefore, regardless of whether a sub-viewpoint has been expanded or scored during the expansion process of other viewpoints, it needs to be expanded during the expansion process of the current viewpoint.

[0143] b) Simulation

[0144] During viewpoint selection, once a corresponding sub-viewpoint is selected through expansion, simulation of this sub-viewpoint begins. Simulation is the process of evaluating the quality of the sub-viewpoint and contributing to the full coverage path planning. In this invention, to ensure the quality of the full coverage path, instead of using the random selection of child nodes commonly used in Monte Carlo tree search for each simulation step, a greedy algorithm is used to select the sub-viewpoint with the largest increase in coverage for each simulation step.

[0145] The simulation process begins by setting the initial viewpoint of the simulation to the viewpoint obtained in the previous step, which is called the simulation viewpoint:

[0146] VP simulate =VP expand

[0147] Then, from the sub-viewpoints of the simulated viewpoint, select the sub-viewpoint with the largest increase in coverage as the next simulated viewpoint:

[0148]

[0149] CoverIns() is the function for calculating coverage growth. This sub-viewpoint selection method determines the termination condition of the simulation: according to the definition of Monte Carlo tree search, the current sub-viewpoint needs to be simulated until a clear result is obtained. However, when performing 3D full-coverage path planning, each simulation requires calculating the coverage growth of all child nodes in the current child node sequence. Therefore, the full-coverage path planning algorithm of this invention is designed to terminate the simulation after a certain number of simulations, and then proceed to the next step of expansion or selection.

[0150] c) Backtracking

[0151] After simulating the sub-viewpoints, a backtracking process is required, tracing the simulation results back up the simulation path to the root node. In the commonly used Monte Carlo tree search algorithm, each node has two parameters: the number of visits (visCount) and the reward value (winCount). Regardless of the simulation result, the number of visits for each node is incremented by 1 during the backtracking process. If the simulation result is a victory, the node's reward value is also incremented by 1; otherwise, the reward value remains unchanged. The backtracking process can be represented as:

[0152]

[0153] in, The simulation result for the expanded child node is true if the simulation result is a victory, otherwise it is false.

[0154] In this invention, the termination condition for sub-viewpoint simulation is not the end of the search, and the backtracking method is also improved accordingly. Specifically, the number of viewpoint visits is equivalent to the number of node visits, and during backtracking, the viewpoint visit count for each viewpoint on the path needs to be incremented by 1. The viewpoint reward value is equivalent to the node reward value, and during backtracking, the viewpoint reward value needs to be increased based on the increase in coverage of the current viewpoint relative to the initial simulated viewpoint. At the end of the simulation, the total path coverage corresponding to the last viewpoint must be greater than the total path coverage of the corresponding viewpoint at the start of the simulation. The increase in total path coverage during the simulation actually represents the reward value for simulating sub-nodes. However, since the number of simulations at the end of the simulation should change depending on different scenarios, it is unreasonable to use only the increase in total path coverage during the simulation to measure the reward value of simulated sub-nodes, reducing the flexibility and robustness of the algorithm.

[0155] To achieve a reasonable setting of viewpoint reward values ​​and improve the robustness of the algorithm, a standard coverage is proposed. standard The concept of ) is used. In order to obtain the standard coverage, a greedy algorithm is used to perform full coverage path planning on different models, and the average coverage of each viewpoint is calculated. This average coverage is then used as the standard coverage.

[0156] The path coverage will increase with each additional viewpoint, and whether the increase is significant or insignificant is measured by the standard coverage. This invention proposes the following method for calculating viewpoint reward value:

[0157]

[0158] Among them, result Cov Coverage is the viewpoint reward value for traversing the path upwards from the last viewpoint after the simulation ends. final The total path coverage of the last viewpoint after the simulation ends; Coverage start Let n be the total path coverage of the initial viewpoint at the start of the simulation; n is the number of simulations. In the full-coverage path planning algorithm based on Monte Carlo tree search of this invention, the backtracking process can be represented as:

[0159]

[0160] The above formula shows that when the average path coverage increase for each viewpoint during simulation exceeds the standard coverage of that viewpoint, the reward value obtained by each viewpoint during the final backtracking will be greater than 1. Conversely, if the average path coverage increase for each viewpoint during simulation is less than the standard coverage of that viewpoint, the reward value obtained by each viewpoint during the final backtracking will be less than 1. Therefore, using the standard coverage to calculate the viewpoint reward value can more objectively evaluate the quality of the simulated path, and the magnitude of the viewpoint reward value does not change with the number of simulations, thereby improving the robustness and flexibility of the 3D full-coverage path planning algorithm.

[0161] d) Selection

[0162] In the current child node sequence, after all viewpoints have been expanded and simulated, the visit count (visCount) and reward value (winCount) of each viewpoint are updated. Next, child viewpoints are selected to form a full-coverage path based on their specific visit count and reward value. This invention is based on the commonly used Monte Carlo tree search method, using the UCT value to calculate the selection probability of each child viewpoint. The calculation formula is as follows:

[0163]

[0164] Where c is the exploration constant; simCount is the total number of simulations so far. This represents the impact of the increase in viewpoint reward value, or path coverage, on viewpoint selection. The formula represents the impact of the number of explorations on viewpoint selection. In the latter half of the formula, the fewer the number of explorations, the larger the value. Therefore, points that have been explored less will have more exploration opportunities. A larger proportion of the latter half of the formula indicates a greater preference for viewpoints with fewer visits during viewpoint selection, and a path that tends to explore unknown areas. Conversely, a smaller proportion of the latter half of the formula indicates a greater preference for viewpoints with more visits during viewpoint selection, and a path that tends to explore the current area. This invention adjusts the search tendency of the entire algorithm by modifying the value of the exploration constant c, selecting sub-viewpoints with higher UCT values ​​as the current viewpoint to expand the path until the path coverage reaches the algorithm's convergence condition.

[0165]

[0166] In practical applications, the above-mentioned UCT value calculation method still has shortcomings. If only the simulation results are used when selecting, the information of the sub-viewpoint will not be fully utilized, and the detection quality of the selected sub-viewpoint on the object surface will be poor, thus affecting the quality of the entire full coverage path.

[0167] This invention adds sub-viewpoint information, such as coverage growth, to the formula to help the full-coverage path planning algorithm select the optimal sub-viewpoint. The improved UCT value calculation method takes into account both simulation results and the quality of the current child node, which helps to select the optimal sub-viewpoint and avoid redundancy in the 3D full-coverage path.

[0168] The improved formula for calculating the UCT value is as follows:

[0169]

[0170] Where c1 is the exploration constant; c2 is the sub-viewpoint coverage coefficient; Coverage i This represents the increase in coverage of the current child node compared to its parent node. The improved UCT value calculation method takes into account both the simulation results and the quality of the current child node, helping the algorithm select the optimal sub-viewpoint and avoid redundancy in the 3D full coverage path.

[0171] Based on the present invention, an implementable method is used for the final planning results of a fighter jet model, as follows: Figure 7 As shown.

[0172] Finally, it should be noted that the above description is only for illustrating specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art should understand that any modifications or variations made based on the technical solutions and inventive concepts of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for UAV 3D coverage path planning based on Monte Carlo tree search, characterized in that, Includes the following steps: The search space and angle are uniformly sampled according to the position resolution and angle resolution to generate an initial viewpoint; The process involves removing viewpoints located inside the model from the initial viewpoints, then removing viewpoints that are too close or too far from the model based on the safety distance and sensor detection distance. Finally, the visible point cloud sequence of the viewpoint is calculated based on the angle between the viewpoint camera's line of sight and the point cloud's point normal vector, and unqualified viewpoints are removed from the visible point cloud sequence, resulting in a set of viewpoints evenly distributed around the object to be detected. The steps involved in calculating the visible point cloud sequence of the viewpoint based on the angle between the viewpoint camera's line of sight and the point cloud's point normal vector, and then removing unqualified viewpoints from the visible point cloud sequence, are as follows: Each viewpoint is scored based on the area of ​​the three-dimensional structure effectively detected by the viewpoint, and viewpoints with scores below a set value are eliminated. Point cloud files are used as the model files for the 3D structure to be detected. The number of intersections between the rays formed by points in the view frustum pointing to the camera's location and the entire model is calculated to determine whether a point is occluded. Specifically: in, This represents the current position. The location of the camera; The ray formed from the current point in the view frustum to the location of the camera; A function to calculate the number of intersections between a ray and the model; This represents the number of intersections between the ray and the model. If the number of points is not zero, the current point is occluded and cannot be directly detected by the camera. The current point is removed. After the occlusion removal operation is completed for every point in the frustum, the remaining points in the frustum can be directly detected by the camera. For each viewpoint, the cloud points of the remaining point cloud data within the view frustum. Find cloud points Surrounding The nearest neighbor points will be used to obtain cloud points. The set of neighborhood points of is represented as: in, For point The set of points within the neighborhood; Suppose there exists a plane, cloud points neighborhood point set The sum of the squares of the perpendicular distances from all points in the plane to the given plane is minimized, and the normal vector of the plane is taken as the point. The normal vector is expressed by the following equation. Fitting plane for all points: Solving the following equation yields the specific values ​​of the fitted plane: in, The parameters representing the fitted plane; , , for The coordinates of the midpoint are used, and the viewpoint coordinates are used as the observation point coordinates to obtain the normal vector of the fitted plane. normal vector Represented as: The specific way of expression is determined by the coordinates of the observation point; Next, the angle between the normal vector of each point and the camera's line of sight corresponding to the viewpoint is calculated. and the included angle With a preset threshold Compare, if Less than If the point is found to be occluded, it will be added to the visible point cloud sequence of the current viewpoint. After traversing all the remaining point cloud data within the view frustum corresponding to the current viewpoint and removing any occlusions, the current viewpoint is scored based on the visible point cloud sequence, as shown in the following formula: in, It represents the total number of point clouds within the current view frustum; The score is calculated as the ratio of the number of visible point cloud sequences to the total number of point clouds at the current viewpoint. For each viewpoint in a set of viewpoints around the object to be detected, a sub-viewpoint sequence is generated based on the distance relationship. Then, based on the idea of ​​Monte Carlo tree search, the process of expansion, simulation, backtracking and viewpoint selection is performed until the coverage meets the requirements, and a full coverage path for 3D detection is obtained.

2. The UAV 3D coverage path planning method based on Monte Carlo tree search according to claim 1, characterized in that, The process of uniformly sampling the search space and angle according to positional and angular resolutions to generate an initial viewpoint includes the following steps: First, the sampling space is uniformly divided into multiple grids according to a set resolution, resulting in multiple non-overlapping sub-regions. The vertices of each sub-region represent the position of the drone. At the vertices of each sub-region, sampling is performed according to a set angular resolution to obtain a more detailed viewpoint. The sampling formula is as follows: in For the generated viewpoint; The coordinates and rotation angle of this viewpoint; , , This serves as the starting point for the entire sampling space; For location resolution; To obtain an angular resolution, a series of initial viewpoints are obtained.

3. The UAV 3D coverage path planning method based on Monte Carlo tree search according to claim 1, characterized in that, Eliminating viewpoints located inside the model involves the following steps: The viewpoint is compared with the boundary of the object to be detected. Viewpoints within the boundary are filtered out, and viewpoints outside the boundary are retained, as shown in the following formula: in, Indicate viewpoint The output of the boundary filtering function, if the viewpoint If not within the boundary, then A value of 1 indicates that the viewpoint meets the requirements; if the viewpoint... Within the boundary, then A value of 0 indicates that the viewpoint needs to be filtered out.

4. The UAV 3D coverage path planning method based on Monte Carlo tree search according to claim 1, characterized in that, The specific criteria for eliminating viewpoints based on safe distance and sensor detection distance are as follows: The remaining viewpoints after removing internal viewpoints are filtered using the safe distance of the drone and the detection range of the camera. Viewpoints that are too close to the object to be detected or are outside the camera's detection range are removed, and only viewpoints that meet the requirements are retained as the final feasible viewpoints. Specifically: in: This is the minimum safe distance between the drone and the object to be detected. The value of the minimum safe distance is determined based on the drone's motion characteristics. This is the maximum distance between the drone and the object to be detected. The value of the maximum distance is based on the detection range of the sensor. From the perspective The actual distance of the distance model; a set of viewpoints uniformly distributed around the object to be detected is obtained, the distance between the viewpoints and the object to be detected meeting both the safety requirements of the UAV and the detection requirements of the sensor.

5. The UAV 3D coverage path planning method based on Monte Carlo tree search according to claim 2, characterized in that, Calculate the angle between the normal vector of each point and the camera's line of sight corresponding to the viewpoint. Specifically as follows: The normal vector of the fitted plane and the Z-axis of the camera coordinate system are both expressed in the world coordinate system, as follows: in, This represents the rotation transformation matrix between the body coordinate system and the sensor coordinate system. The direction of the Z-axis in the camera coordinate system is given by the following value: ; The Z-axis of the camera coordinate system is the vector in the world coordinate system; Next, the angle between the normal vector of each point and the Z-axis in the camera coordinate system is calculated using the following formula. : 。 6. The UAV 3D coverage path planning method based on Monte Carlo tree search according to claim 1, characterized in that, The sub-viewpoint sequence is generated for each viewpoint based on distance relationships as follows: Connect the initial viewpoints and determine the distance threshold. For each viewpoint In other words, if there exists a viewpoint Make the distance between the two less than the distance threshold. Then the viewpoint As a viewpoint One of the subviewpoint sequences, specifically represented as: in, It is a distance function; for Subview sequence, viewpoint Distance threshold All other viewpoints within will be added to the viewpoint. In the sub-viewpoint sequence, distance threshold Adjustments can be made based on specific application scenarios; When extending a path, a viewpoint is selected from the sequence of sub-viewpoints of the current viewpoint as the path extension until the target point is reached or the maximum number of iterations is reached.

7. The UAV 3D coverage path planning method based on Monte Carlo tree search according to claim 1, characterized in that, Performing expansion, simulation, backtracking, and selection on the current viewpoint based on the idea of ​​Monte Carlo tree search includes the following steps: a) Expansion set up From the current viewpoint The sub-viewpoint sequence, the expansion process is from Selecting nodes that have not yet been scored prepares them for the next step of simulation scoring, as shown below: in, To broaden perspectives; Indicate viewpoint The scoring status during the expansion process of the current viewpoint; if the viewpoint If a viewpoint has not been scored during the expansion process of the current viewpoint, it is true; otherwise, it is false. The viewpoint sequence is generated by a distance threshold. A viewpoint may be in the sub-viewpoint sequence of multiple viewpoints. Regardless of whether the sub-viewpoint has been expanded or scored during the expansion process of other viewpoints, it needs to be expanded during the expansion process of the current viewpoint. b) Simulation During the viewpoint selection process, after the corresponding sub-viewpoints are selected through expansion, the sub-viewpoints are simulated. In each step of the simulation, a greedy algorithm is used to select the sub-viewpoint with the largest increase in coverage to continue the simulation. The simulation process begins by setting the initial viewpoint to the expanded viewpoint obtained in the previous step: Then, from the sub-viewpoints of the simulated viewpoint, select the sub-viewpoint with the largest increase in coverage as the next simulated viewpoint: in, The function for calculating the coverage growth rate simulates the current sub-viewpoint a set number of times before proceeding to the next step of expansion or selection. c) Backtracking After simulating the sub-viewpoints, the simulation results are traced back up the simulation path to the root node, with each node having a number of visits. and reward value Two parameters: during the backtracking process, the number of visits to each node will be incremented by 1; if the simulation result is a victory, the reward value of the node will also be incremented by 1; otherwise, the reward value remains unchanged. Viewpoint visits are equated to node visits. During backtracking, the viewpoint visit count for each viewpoint on the path is incremented by 1. The viewpoint reward value is equated to the node reward value. During backtracking, the viewpoint reward value is increased based on the increase in coverage of the current viewpoint relative to the initial simulated viewpoint. At the end of the simulation, the total path coverage corresponding to the next viewpoint is greater than the total path coverage of the corresponding viewpoint at the start of the simulation. The increase in total path coverage during the simulation represents the reward value for simulating child nodes. A greedy algorithm is used to perform full-coverage path planning for different models, and the average coverage rate of each viewpoint is calculated. The average coverage rate is then used as the standard coverage rate. d) Selection After all viewpoints have been expanded and simulated, the number of visits for each viewpoint is [number missing]. and reward value All are updated based on the number of visits to each sub-viewpoint. and reward value The specific numerical values ​​for selecting sub-viewpoints are used to form a full-coverage path. An improved UCT value is used to calculate the selection probability of each sub-viewpoint, resulting in a path with higher coverage. The path is extended using the sub-viewpoints of the value as the current viewpoint until the path coverage reaches the algorithm's convergence condition: 。 8. The UAV 3D coverage path planning method based on Monte Carlo tree search according to claim 7, characterized in that, The viewpoint reward value is calculated as follows: in, The viewpoint reward value for traversing the path upwards from the last viewpoint after the simulation ends; The total path coverage of the last viewpoint after the simulation ends; The total path coverage of the initial viewpoint at the start of the simulation; The number of simulations.

9. The UAV 3D coverage path planning method based on Monte Carlo tree search according to claim 7, characterized in that, The improved UCT value calculation modifies the exploration constant. The improved UCT value calculation formula is as follows, taking into account both the numerical value and the coverage growth factor: in To explore constants, This is the sub-viewpoint coverage coefficient. This represents the increase in coverage of the current child node compared to its parent node.

Images