Unmanned aerial vehicle path planning and obstacle avoidance method based on digital twin technology

Abstract

The invention provides an unmanned aerial vehicle path planning and obstacle avoidance method based on a digital twin technology. The method comprises the following steps: S1, constructing an initial live-action three-dimensional model of an unmanned aerial vehicle flight area; s2, constructing an unmanned aerial vehicle digital twin platform by using the initial live-action three-dimensional model of the unmanned aerial vehicle flight area and the unmanned aerial vehicle three-dimensional model; s3, establishing a position mapping relation between the virtual space and the physical space by adopting a UTM projection technology, and constructing a corresponding technical scheme; s4, dividing an obstacle space of the initial live-action three-dimensional model in the unmanned aerial vehicle digital twin platform, and constructing an octree map based on the obstacle space; s5, according to the starting point and the target point, performing path planning in a virtual space of the unmanned aerial vehicle digital twin platform to obtain a planned path result; and S6, performing coordinate conversion on the planned path result according to the position mapping relation between the virtual space and the physical space, and then transmitting the result to the physical unmanned aerial vehicle in real time through a data transmission channel between the virtual unmanned aerial vehicle and the physical unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the technical fields of UAV obstacle avoidance and digital twin, and specifically provides a UAV path planning and obstacle avoidance method based on digital twin technology. Background Technique

[0002] In the field of UAV technology, the obstacle avoidance function is one of the key technologies to ensure the safe flight of UAVs. With the wide application of UAVs in fields such as logistics distribution, inspection, and aerial photography, their autonomous flight ability in low-altitude environments has become particularly important. As the basis for task execution, UAV path planning needs to plan the optimal path between the starting position and the target position in a specific environment. However, existing three-dimensional path planning algorithms are often tested in simplified virtual environments and are difficult to meet the actual flight requirements in real and complex environments.

[0003] The emergence of digital twin technology provides a new idea for the system-level development of UAVs. This technology digitally presents physical entities through multi-dimensional, multi-space, and multi-scale models, and uses multi-source heterogeneous data to achieve real-time connection, mapping, and update between physical entities and virtual entities. The high precision, safety, reliability, and cost-effectiveness of digital twin technology make it of great value and application prospects in the research of UAV obstacle avoidance technology.

[0004] Although digital twin technology has potential in the field of UAV obstacle avoidance, there are some limitations in the currently disclosed technologies. For example, some digital twin-based UAV tactical path planning and obstacle avoidance methods simplify the terrain and battlefield threats and do not accurately model the real three-dimensional environment, resulting in difficulties in ensuring the real-time performance and practicality of the path planning algorithm. In addition, although the UAV three-dimensional path planning and obstacle avoidance method based on the oblique photography model can perform three-dimensional reconstruction of the real environment, its processing process requires multiple tool software, which complicates the application process, and there is a lack of public technical solutions on how to combine the oblique photography model with the UAV for real-time path planning.

[0005] Therefore, the present invention aims to provide a UAV path planning and obstacle avoidance method based on digital twin technology to solve the problems in the existing technology and improve the obstacle avoidance ability and path planning practicality of UAVs in complex real environments. Summary of the Invention

[0006] The purpose of the present invention is to provide a UAV path planning and obstacle avoidance method based on digital twin technology to solve the problems in the existing technology.

[0007] To achieve the above purpose, the present invention provides the following technical solutions: The present invention provides a UAV path planning and obstacle avoidance method based on digital twin technology, including the following steps:

[0008] Including the following steps:

[0009] S1. Construct an initial real - scene 3D model of the UAV flight area;

[0010] S2. Use the initial real - scene 3D model of the UAV flight area and the UAV 3D model to build a UAV digital twin platform;

[0011] S3. Adopt the UTM projection technology to establish the position mapping relationship between the virtual space and the physical space, and construct the corresponding technical solution;

[0012] S4. Divide the obstacle space of the initial real - scene 3D model in the UAV digital twin platform, and construct an octree map based on this obstacle space;

[0013] S5. According to the starting point and the target point, conduct path planning in the virtual space of the UAV digital twin platform to obtain the planned path result;

[0014] S6. Convert the coordinates of the planned path result according to the position mapping relationship between the virtual space and the physical space, and then transmit it to the physical UAV in real - time through the data transmission channel between the virtual UAV and the physical UAV, so as to realize the flight control of the physical UAV along the planned path.

[0015] Preferably, in step S3, the UTM projection technology is adopted to establish the position mapping relationship between the virtual space and the physical space, and the specific operation is as follows:

[0016] First, select several ground control points from the initial real - scene 3D model and record their 3D coordinates. Subsequently, locate the corresponding observation points of these selected ground control points in the real environment and record their longitude and latitude coordinates. Then, use the UTM projection forward solution formula to convert the longitude and latitude coordinates containing height information recorded by the observation points into 3D coordinates in the Cartesian coordinate system;

[0017] Next, based on the 3D coordinates of the selected control points and the 3D coordinates after conversion of the longitude and latitude coordinates of the corresponding observation points, use the least - squares method to estimate the coordinate deviation of the center point of the real - scene 3D model from the longitude and latitude coordinates to the origin of the Cartesian coordinate system. After the longitude and latitude coordinates of each point in the physical space are converted into 3D coordinates through the UTM projection forward solution formula and added with the coordinate deviation, the 3D coordinates of the corresponding points in the virtual space in the Cartesian coordinate system can be obtained. Finally, subtract the coordinate deviation from the 3D coordinates of each point in the virtual space and convert it through the UTM projection inverse solution formula to obtain the longitude and latitude coordinates of the corresponding points in the physical space.

[0018] Preferably, in S4, the specific steps are as follows:

[0019] S4.1. Extract the mesh data of all triangular patches in the initial real - scene 3D model;

[0020] S4.2. Use the mesh data of all triangular patches to expand the obstacle surface and generate a 3D mesh expansion model;

[0021] S4.3. Construct an octree structure containing the 3D mesh expansion model in the virtual space. In this octree structure, use the octree to divide the obstacle space depicted by the 3D mesh expansion model, thereby obtaining all the node information of the octree and recording it as the octree map;

[0022] S4.4. For each node of the octree, if the cube grid represented by the current node intersects with the triangular patches in the 3D mesh expansion model, mark this node as containing an obstacle and create eight child nodes for it; then check whether these eight child nodes also intersect with the triangular patches in the 3D mesh expansion model. If so, continue to create their respective child nodes for these child nodes until the preset maximum depth is reached.

[0023] Preferably, in S5, the specific steps are as follows:

[0024] S5.1. Convert the octree map into a directed graph. This directed graph is composed of the leaf nodes of the octree map that are not marked as containing obstacles. These leaf nodes are converted into graph nodes, and their connectivity is represented by directed edges;

[0025] S5.2. Identify the leaf nodes closest to the starting point and the target point in the octree map respectively, and name them the starting leaf node and the target leaf node. Then further refine these starting and target leaf nodes in the directed graph;

[0026] S5.3. Remove the graph node with the minimum cost value from the container, set it as the current graph node, and mark it as visited;

[0027] S5.4. Check whether there is a line of sight between the current graph node and its parent node in the directed graph. If there is no line of sight, change the parent node of the current graph node and update its cost value;

[0028] S5.5. Traverse and calculate the cost values of all unvisited successor neighbor graph nodes of the current graph node. If the cost value decreases, mark these successor neighbor graph nodes as in the to - be - visited state and add them to the container;

[0029] S5.6. Repeat S5.3 to S5.5 until the current graph node visits the target graph node or the container is empty. Take all the graph nodes marked as in the visited state and the target graph node as the initial graph nodes of the path, thereby obtaining the initial path graph;

[0030] S5.7. Conduct a line-of-sight check on each initial path graph node in the initial path graph to obtain an optimized path graph, and use it as the final path result of the planning.

[0031] Preferably, in S5 or S5.2, the starting point and the target point specifically refer to the starting point and the target point selected in the virtual space, or the starting point and the target point received through the data transmission channel between the virtual drone and the physical drone and processed through coordinate transformation.

[0032] Preferably, in S5.7, for the starting graph node of each path, it is necessary to determine whether there is an obstacle between the current graph node and the parent node's parent node of the current graph node in the initial path graph. If there is an obstacle, the parent node of the current graph node in the initial path graph is abandoned, and the current graph node is directly connected to the parent node's parent node of the current graph node in the initial path graph;

[0033] The cost value of each graph node is calculated through the following cost function:

[0034] f(n, s, t) = g(s, n) + h(n, t)

[0035] Among them, (n, s, t) represents the cost function value, g(s, n) represents the cost value from the starting graph node s to the current graph node n, and h(n, t) represents the heuristic function, specifically representing the estimated cost value from the current graph node n to the target graph node t. This estimated cost value is obtained by calculating the Euclidean distance between the current graph node n and the target graph node t and multiplying it by a proportionality coefficient. The proportionality coefficient is inversely proportional to the density of obstacles within a certain field of view in the direction from the current graph node to the target graph node.

[0036] The present invention has at least the following beneficial effects:

[0037] A method for unmanned aerial vehicle path planning and obstacle avoidance based on digital twin technology provided by the present invention. The present invention utilizes an unmanned aerial vehicle digital twin platform developed by Unity, and controls the loading and display of the real-scene three-dimensional model and the construction of the octree map by writing script programs. This enables the path planning algorithm to conduct a large number of tests in a virtual space highly similar to the real environment during the development and testing stage, thereby significantly improving the safety of the unmanned aerial vehicle running in the real world.

[0038] By expanding and processing the surface of obstacles in the real-scene three-dimensional model and constructing an octree map and saving it to an offline file, the present invention greatly saves the online processing time during the construction and search graph stages of path planning. In addition, the designed heuristic function takes into account the distance to the target point and the density of obstacles, effectively avoiding selecting paths blocked by obstacles, thereby improving the efficiency and quality of path search.

[0039] The present invention realizes the real-time synchronization between the virtual space and the physical space, enabling the drone to obtain the path planning results between any two points in the virtual space of the drone digital twin platform in real time. This not only effectively reduces the processing pressure on the on-board computer of the drone, but also improves the efficiency of the drone in performing tasks, showing broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flowchart of the present invention;

[0041] Figure 2 is a flowchart of the real scene three-dimensional modeling of the present invention;

[0042] Figure 3 is a rendering of the real scene three-dimensional model of the present invention;

[0043] Figure 4 is a connection block diagram of the drone digital twin platform of the present invention;

[0044] Figure 5 is a schematic diagram of the software interface of the drone digital twin platform of the present invention;

[0045] Figure 6 is a schematic diagram of the data transmission channel between the virtual drone and the physical drone of the present invention;

[0046] Figure 7 is a flowchart of the octree map for constructing the real scene three-dimensional model of the present invention;

[0047] Figure 8 is a rendering of the drone digital twin platform software of the present invention loading the octree map file to visualize the cube grid;

[0048] Figure 9 is the path search algorithm flow of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs. The terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0050] Embodiment

[0051] As Figures 1-9 shown, a method for path planning and obstacle avoidance of a drone based on digital twin technology includes the following steps:

[0052] S1. Construct an initial real scene three-dimensional model of the drone flight area;

[0053] To construct an initial real - scene 3D model of the UAV flight area, it is necessary to collect geographical information data of the area. This can be achieved by aerial photography using a high - resolution camera carried by the UAV to obtain a series of high - definition pictures. Then, a structured light scanner or LiDAR is used to scan the ground to obtain accurate 3D data of the terrain. The collected data is processed through image processing and point cloud processing software to generate a real - scene 3D model. The model should include detailed information such as terrain, buildings, and vegetation to ensure the accuracy and practicality of the model.

[0054] S2. Use the initial real - scene 3D model of the UAV flight area and the UAV 3D model to build a UAV digital twin platform;

[0055] After obtaining the real - scene 3D model, it is necessary to combine it with the 3D model of the UAV to build a digital twin platform. The digital twin platform is a virtual environment that can simulate the flight environment of the UAV in the real world. On this platform, the 3D model of the UAV can be loaded and placed in the real - scene 3D model to simulate the flight path and obstacle avoidance behavior of the UAV. The digital twin platform should have the ability of real - time data exchange, be able to receive sensor data from the physical UAV, and feedback the path planning results to the physical UAV in real time.

[0056] S3. Adopt UTM projection technology to establish the position mapping relationship between the virtual space and the physical space, and construct the corresponding technical solution;

[0057] UTM (Universal Transverse Mercator) projection technology is a map projection method used to convert points on the Earth's surface into plane coordinates. In the present invention, UTM projection technology is used to establish the position mapping relationship between the virtual space and the physical space to ensure that the path planning of the UAV in the virtual space can be accurately mapped into the physical space. First, several ground control points need to be selected in the real - scene 3D model and their 3D coordinates are recorded. Then, the corresponding observation points of these control points are found in the real environment and their longitude and latitude coordinates are recorded. Through the UTM forward solution formula, the longitude and latitude coordinates of the observation points are converted into 3D coordinates in the Cartesian coordinate system. Then, the least - squares method is used to estimate the coordinate deviation of the center point of the real - scene 3D model, thereby establishing an accurate mapping relationship between the virtual space and the physical space.

[0058] S4. Divide the obstacle space of the initial real - scene 3D model within the UAV digital twin platform, and construct an octree map based on this obstacle space;

[0059] Within the digital twin platform, it is necessary to identify and classify obstacles in the real - scene 3D model. Extract the mesh data of all triangular patches and use this data to expand the obstacle surface to generate a 3D mesh expansion model. Then, construct an octree structure containing the 3D mesh expansion model in the virtual space. An octree is a spatial partitioning data structure that recursively divides the 3D space into eight sub - spaces until a preset maximum depth is met or there are no obstacles in the sub - space. The octree map records all node information for subsequent path planning and obstacle avoidance.

[0060] S5. Based on the starting point and the target point, perform path planning in the virtual space of the UAV digital twin platform to obtain the planned path result.

[0061] The path planning is based on the octree map. First, convert the octree map into a directed graph, where the nodes not marked as obstacles form the graph nodes, and the connectivity between them is represented by directed edges. Then, identify the leaf nodes closest to the starting point and the target point, and further refine these nodes in the directed graph. Through a heuristic search algorithm (such as the A* algorithm), starting from the starting point, gradually select the node with the minimum cost value for expansion until the target point is reached. During the search process, a line - of - sight check is required to ensure that the selected path is feasible. Finally, the obtained path graph is the planned path result.

[0062] Specifically, the cost value of each graph node is calculated through the following cost function:

[0063] f(n, s, t) = g(s, n)+h(n, t)

[0064] where (n, s, t) represents the cost function value, g(s, n) represents the cost value from the starting graph node s to the current graph node n, and h(n, t) represents the heuristic function, which specifically represents the estimated cost value from the current graph node n to the target graph node t. This estimated cost value is obtained by calculating the Euclidean distance between the current graph node n and the target graph node t and multiplying it by a scaling factor. The scaling factor is inversely proportional to the density of obstacles within a certain field of view in the direction from the current graph node to the target graph node.

[0065] S6. Convert the planned path result according to the position mapping relationship between the virtual space and the physical space, and then transmit it to the physical UAV in real - time through the data transmission channel between the virtual UAV and the physical UAV, so as to realize the flight control of the physical UAV along the planned path.

[0066] The planned path result needs to be converted from the virtual space to the physical space so that the physical drone can fly along the planned path. Using the previously established position mapping relationship between the virtual space and the physical space, the coordinates of each node in the path result are converted from the virtual space to the longitude and latitude coordinates in the physical space. Then, through the data transmission channel between the virtual drone and the physical drone, the converted path information is transmitted to the physical drone in real time. The physical drone executes the flight mission and flies along the planned path according to the received path information through its on-board computer and flight control system.

[0067] A method for drone path planning and obstacle avoidance based on digital twin technology provided by the present invention utilizes a drone digital twin platform developed by Unity, and controls the loading and display of the real-scene three-dimensional model and the construction of the octree map by writing script programs. This enables the path planning algorithm to conduct a large number of tests in a virtual space highly similar to the real environment during the development and testing stages, thereby significantly enhancing the safety of the drone's operation in the real world.

[0068] By expanding and processing the surface of obstacles in the real-scene three-dimensional model and constructing an octree map and saving it to an offline file, the present invention greatly saves the online processing time during the construction and search graph stages of path planning. In addition, the designed heuristic function takes into account the distance to the target point and the density of obstacles, effectively avoiding selecting paths blocked by obstacles, thereby improving the efficiency and quality of path search.

[0069] The present invention realizes the real-time synchronization between the virtual space and the physical space, enabling the drone to obtain the path planning result between any two points from the virtual space of the drone digital twin platform in real time. This not only effectively reduces the processing pressure on the drone's on-board computer but also improves the efficiency of the drone's mission execution, demonstrating broad application prospects.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; under the concept of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and the steps can be implemented in any order, and there are many other variations in different aspects of the present invention as described above. For the sake of brevity, they are not provided in detail; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A UAV path planning and obstacle avoidance method based on digital twin technology, characterized in that: The following steps are involved: S1. Construct an initial real-scene 3D model of the drone flight area; S2. Use the initial real-life 3D model of the drone flight area and the 3D model of the drone to build a drone digital twin platform; S3. Use UTM projection technology to establish the position mapping relationship between virtual space and physical space, and build corresponding technical solutions; S4, dividing the obstacle space of the initial real-scene 3D model in the UAV digital twin platform, and constructing an octree map based on the obstacle space; S5. Based on the starting point and the target point, path planning is performed in the virtual space of the UAV digital twin platform to obtain the planned path result; S6. The planned path result is converted into coordinates according to the position mapping relationship between the virtual space and the physical space, and then transmitted to the physical drone in real time through the data transmission channel between the virtual drone and the physical drone, thereby realizing the flight control of the physical drone along the planned path.

2. The method for unmanned aerial vehicle path planning and obstacle avoidance based on digital twin technology according to claim 1, characterized in that: In step S3, the UTM projection technology is used to establish a position mapping relationship between the virtual space and the physical space. The specific operations are as follows: First, several ground control points are selected from the initial real-life 3D model and their 3D coordinates are recorded. Then, the corresponding observation points of these selected ground control points are located in the real environment and their longitude and latitude coordinates are recorded. Then, the longitude and latitude coordinates containing height information recorded at the observation points are converted into 3D coordinates in the Cartesian coordinate system using the UTM projection forward solution formula. Next, based on the three-dimensional coordinates of the selected control point and the three-dimensional coordinates of the corresponding observation point after conversion from the longitude and latitude coordinates, the least squares method is used to estimate the coordinate system deviation of the center point of the real-life three-dimensional model from the longitude and latitude coordinates to the origin of the Cartesian coordinate system. The longitude and latitude coordinates of each point in the physical space are converted into three-dimensional coordinates through the UTM projection forward solution formula, and then the coordinate system deviation is added to obtain the three-dimensional coordinates of the corresponding point in the virtual space in the Cartesian coordinate system. Finally, the coordinate system deviation is subtracted from the three-dimensional coordinates of each point in the virtual space and converted through the UTM projection inverse solution formula to obtain the longitude and latitude coordinates of the corresponding point in the physical space.

3. The UAV path planning and obstacle avoidance method based on digital twin technology according to claim 1 is characterized in that: In S4, the specific steps are as follows: S4.1, extracting mesh data of all triangular facets in the initial real-scene 3D model; S4.2, using the mesh data of all triangular facets to expand the obstacle surface and generate a three-dimensional mesh expansion model; S4.3, constructing an octree structure including the three-dimensional mesh extension model in the virtual space, wherein the obstacle space depicted by the three-dimensional mesh extension model is segmented by using the octree, thereby obtaining all node information of the octree and recording it as an octree map; S4.

4. For each node of the octree, if the cube mesh represented by the current node intersects with the triangular face in the three-dimensional mesh extension model, the node is marked as containing an obstacle and eight child nodes are created for it; then, it is checked whether these eight child nodes also intersect with the triangular face in the three-dimensional mesh extension model. If so, continue to create their own child nodes for these child nodes until the preset maximum depth is reached.

4. The method for unmanned aerial vehicle path planning and obstacle avoidance based on digital twin technology according to claim 1, characterized in that: In S5, the specific steps are as follows: S5.

1. converting the octree map into a directed graph, the directed graph consisting of leaf nodes of the octree map that are not marked as containing obstacles, converting these leaf nodes into graph nodes, and representing the connectivity between them through directed edges; S5.2, identifying the leaf nodes closest to the starting point and the target point in the octree map, respectively, naming them the starting leaf node and the target leaf node, and then further refining these starting and target leaf nodes in the directed graph; S5.

3. Remove the graph node with the smallest cost value from the container, set it as the current graph node, and mark it as visited; S5.

4. Check whether there is a line of sight between the current graph node and its parent node in the directed graph. If there is no line of sight, change the parent node of the current graph node and update its cost value. S5.5, traverse and calculate the cost values ​​of all unvisited successor neighbor graph nodes of the current graph node. If the cost value is reduced, mark these successor neighbor graph nodes as pending access and add them to the container; S5.6, repeat S5.3 to S5.5 until the current graph node accesses the target graph node or the container is empty, and use all graph nodes marked as visited and the target graph node as the path initial graph nodes, thereby obtaining the initial path graph; S5.

7. Perform a sight check on each node of the initial path graph in the initial path graph to obtain an optimized path graph, and use it as the final path result of the planning.

5. The UAV path planning and obstacle avoidance method based on digital twin technology according to claim 4 is characterized in that: In S5 or S5.2, the starting point and the target point specifically refer to the starting point and the target point selected in the virtual space, or the starting point and the target point received through the data transmission channel between the virtual drone and the physical drone and processed by coordinate transformation.

6. The UAV path planning and obstacle avoidance method based on digital twin technology according to claim 4 is characterized in that: In S5.7, for the starting graph node of each path, it is necessary to determine whether there is an obstacle between the current graph node and the parent node of its parent node in the initial path graph. If there is an obstacle, the parent node of the current graph node in the initial path graph is abandoned, and the current graph node is directly connected to the parent node of its parent node in the initial path graph; The cost of each graph node is calculated using the following cost function: f(n,s,t)=g(s,n)+h(nt) Among them, (n, s, t) represents the cost function value, g(s, n) represents the cost value from the starting graph node s to the current graph node n, and h(n, t) represents the heuristic function, which specifically represents the estimated cost value from the current graph node n to the target graph node t. This estimated cost value is obtained by calculating the Euclidean distance between the current graph node n and the target graph node t and multiplying it by a proportional coefficient. The proportional coefficient is inversely proportional to the density of obstacles within a certain field of view in the direction from the current graph node to the target graph node.

Images