Unmanned aerial vehicle three-dimensional low-altitude city operation guiding method based on lightweight task template

Abstract

The invention discloses an unmanned aerial vehicle low-altitude city operation guiding method based on a lightweight task template, and the method comprises the steps: a UAV-Agent collects environment information after entering a geofence, and constructs a three-dimensional rasterized graph structure, isomorphic nodes represent plane path key points, and heterogeneous nodes represent a three-dimensional space path relation; dynamically updating the task template based on mobile edge calculation, and storing guide path information; and the user machine requests a path from the navigation station through lightweight interaction, the navigation station matches the optimal guide path and feeds back the optimal guide path, and the user machine autonomously navigates along the key point. The method is characterized in that a heterogeneous node graph structure is adopted to model an urban three-dimensional airspace, and multi-layer path planning is achieved; based on a dynamic updating mechanism of a lightweight task template, the communication overhead is reduced; and in combination with a key point optimization algorithm of geometric topology, the navigation safety is improved. Compared with traditional route planning, the scheme can significantly reduce calculation and communication resource consumption.

Description

Technical Field

[0001] The invention relates to unmanned aerial vehicle control, and in particular to a three-dimensional low-altitude urban operation guidance method for unmanned aerial vehicles based on a lightweight mission template. Background Art

[0002] In existing drone control solutions, drone flight path planning usually requires static map data to obtain basic geographic information such as latitude, longitude and altitude information for each flight point in the initial path. However, in complex urban environments, drones used for logistics, express delivery, inspection and regional coverage require dynamic drone control and dynamic path planning solutions. Therefore, the drone control system needs to periodically interact with multiple map data APIs through mobile edge servers to obtain basic geographic information different from the flight points on the initial path. For ground unmanned vehicles, this information interaction may solve existing problems in path planning. However, for drones in complex three-dimensional urban environments, this information interaction requires a large amount of communication and computing resources; further, accurate drone planning requires the creation and access of a larger-scale high-precision dynamic map basic platform. The high-precision dynamic map basic platform is considered to be the core support for the safe and efficient operation of drone logistics and express delivery. There is currently no fully mature high-precision dynamic map platform dedicated to drones.

[0003] In order to meet the needs of logistics, express delivery, inspection and regional coverage in complex urban environments, UAV flight path planning requires static and dynamic temporal and spatial data. However, the existing high-precision dynamic map platform and bird's-eye view (BEV) database are not yet mature, which limits the efficiency and safety of UAV low-altitude urban planning. Summary of the invention

[0004] Purpose of the invention: In view of the above shortcomings, the present invention provides a three-dimensional low-altitude urban operation guidance method for UAVs based on lightweight mission templates with high control efficiency.

[0005] Technical solution: To solve the above problems, the present invention adopts a three-dimensional low-altitude urban operation guidance method of a UAV based on a lightweight mission template, comprising the following steps: (1) The UAV-Agent enters the geo-fence and sends a starting point collection command to the navigation station within the geo-fence; (2) After receiving the start point collection command, the navigation station feeds back the start point and end point information within the geographic fence and the correct reception information ACK to the UAV-Agent; (3) The UAV-Agent plans the initial path from the starting point to the end point based on the feedback information. During the operation, it collects environmental information within the geographic fence and builds a preliminary task template based on the collected environmental information. The task template is then updated through the mobile edge server. The task template includes a graph structure obtained by rasterizing a three-dimensional low-altitude area mapping, a number of key points on a path, and a number of guide paths planned according to the graph structure and the key points on the path; (4) Sending the updated task template to the navigation station, the navigation station uses a path planning algorithm based on the key points on the guidance path in the task template to plan the guidance path and obtain a smooth navigation path; (5) The user machine entering the starting point of the geographic fence sends a task request to the navigation station; (6) After receiving the task request, the navigation station selects guidance path data from the task template and feeds back the navigation path to the user machine; (7) After receiving the guidance path data and the navigation path, the user machine starts the subsequent process of autonomous navigation and passes through the key points on the guidance path in sequence; (8) When the navigation task is completed, the user machine notifies the navigation station and the navigation station sends an ACK to the user machine.

[0006] Furthermore, the task template includes the task type and corresponding process, the longitude, latitude and altitude of the navigation station, the geographic fence boundary, the longitude, latitude and altitude of the starting point and end point within the geographic fence, the number of guidance paths, the longitude, latitude and altitude of several key points of the path, several guidance paths, the area ID set after rasterization of the three-dimensional low-altitude area, the area ID index and the area topological relationship.

[0007] Furthermore, the steps for constructing the graph structure in the task template are: (31) Divide the three-dimensional low-altitude urban structure within the geographic fence into multiple grids with geometric topological relationships; (32) Mapping the grid into multiple virtual roads, abstracting a virtual road into a graph structure of homogeneous nodes, each homogeneous node graph structure stores a single path and its key points on the path and implements graph calculations related to path planning; the graph structure describes the connection relationship between different areas in the environment; virtual roads of different heights are abstracted into a graph structure of heterogeneous nodes, and the graph structure of heterogeneous nodes includes several paths, and each path has a different height; (33) Assign weights to the edges of the graph structure.

[0008] Furthermore, the path planning algorithm includes an improved RRT-star algorithm; the improved RRT-star algorithm assists the sampling of sampling points in the RRT-star algorithm through path key points, specifically: The key points of the path on the guidance path are used as prior positions in sequence; For the interval between the starting point and the key point of the path, obtain the prior position, find the node closest to the prior position, confirm the coordinates of the target point, use the rectangular area formed by the nearest node and the target point as the sampling area, perform target biased sampling in the sampling area, and sample to obtain a new node; For the interval between two key points of the path, the new local target position is calculated by the scaling factor, the prior position and the coordinates of the nearest node, the node closest to the new target position is found, and the rectangular area formed by the node closest to the new target position and the local target position is used as the sampling area. Target biased sampling is performed within the sampling area to obtain a new node.

[0009] Furthermore, the synthesis cost of the trajectory planned by the improved RRT-star algorithm is calculated, and the weight of the edge in the graph structure is updated by the synthesis cost; the synthesis cost The calculation formula is: ; in, For Node The height value of For Node The height value of is the energy consumption function caused by altitude change, is the distance between the trajectory and the nearest obstacle, , and is the weight coefficient.

[0010] Furthermore, the path planning method used for the initial path in step (3) includes: A* algorithm, Dijkstra algorithm, RRT algorithm; the environmental information collected within the geographic fence during the operation includes image collection, location information request and geographic API call to obtain data of key points of the path; After the UAV-Agent arrives at the destination, it generates an initial task template with the collected information and uploads it to the database of the mobile edge server MEC server. The MEC server calibrates and updates the initial task template and feeds back the updated task template to the UAV-Agent. The UAV-Agent transmits the updated task template to the navigation station. After receiving the ACK sent by the navigation station, the UAV-Agent leaves the service area involved in the geographic fence.

[0011] Furthermore, the task template also includes spatial restriction boundaries, traffic data and meteorological data.

[0012] Furthermore, in step (6), after the navigation station receives the task request, when the navigation station collects real-time congestion data, the navigation station will perform graph calculation to obtain a suboptimal guidance path.

[0013] The present invention also adopts a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0014] The present invention also adopts a computer-readable storage medium on which a computer program is stored, and the computer program implements the steps of the above method when executed by a processor.

[0015] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: through the navigation station containing lightweight mission templates, it effectively supports low-altitude UAV navigation and path planning. The user machine needs to interact with the navigation station once, rather than periodically and dynamically receiving instructions from the dynamic map API, which helps save communication and computing resources and improve the efficiency of UAV control. Path planning is carried out using a priori key points to improve the safety of UAV low-altitude urban planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of information collection and processing in the lightweight mechanism of the present invention.

[0017] Figure 2 It is a schematic diagram of the information transmission process in the navigation task of the present invention.

[0018] Figure 3 It is a schematic diagram of the flight path of a low-altitude UAV or the driving path of a ground vehicle in the present invention.

[0019] Figure 4 It is a schematic diagram of the simulation process of the graph structure in the present invention.

[0020] Figure 5 It is a schematic diagram of the graph structure of heterogeneous nodes in the present invention.

[0021] Figure 6 It is a flow chart of the single key point guided sampling method 1 in the present invention.

[0022] Figure 7 It is a flow chart of the dual key point guided sampling method 2 in the present invention. DETAILED DESCRIPTION

[0023] In this embodiment, a method for guiding UAV three-dimensional low-altitude urban operations based on a lightweight mission template includes the following steps: like Figure 1 As shown in the figure, the process of lightweight geographic information collection and processing is not used to build a high-precision dynamic map, but to build a low-altitude UAV navigation knowledge base for applications such as logistics, express delivery, and inspection. Figure 1The information flow of each entity in is represented by arrows.

[0024] (1) After the UAV-Agent that enters the geographic fence sends a starting point acquisition instruction to the navigation station, the navigation station feeds back the correct reception information (ACK) to the UAV-Agent. Then, the UAV-Agent will fly from the starting point to the end point along an initial path. The initial path can come from a path planning method such as the A* algorithm, the Dijkstra algorithm, the RRTs algorithm, etc. At the same time, the UAV-Agent with conventional image processing or specific embodied intelligence will perform image acquisition, location information requests and geographic API calls, and assign the key point data in the path to the BEV image attributes to complete the attribute labeling of the key points in the graph structure. ID2 is 0, indicating that the UAV-Agent turns on the camera for BEV image acquisition. The map API call accesses the remote geographic database through the MEC server.

[0025] (2) After the UAV-Agent reaches the destination, it ends image acquisition and generates an initial task template, which is uploaded to the MECserver database. Optionally, if the UAV-Agent has specific embodied intelligence, it can complete the visual language navigation function in real time, such as using computer vision to achieve environmental perception and target recognition, parsing navigation intent in text, and uploading updated task templates. When ID2 is 3, its navigation intent is summarized as: determine the first path key point, and the first path key point is within the starting line of sight. Optionally, the visual language navigation function can convert voice instructions into text to facilitate the generation of path key points for other conditions by combining visual and language information.

[0026] The task template is a subclass of the standardized interface. It defines the standardized interface to ensure interoperability between modules and reduce the difficulty of integration. The task template facilitates the interoperability of various modules in the UAV control system of the large language model (LLMs) and Agent. The UAV control system includes: user machine, navigation station, UAV intelligent agent (UAV-Agent) and mobile edge server (MEC Server).

[0027] User machines are used to represent autonomous navigation vehicles on the ground and UAVs (Unmanned Aerial Vehicles) at low altitudes, and they can communicate with the navigation station through a wireless interface. The navigation station (The Navigation Station) with integrated wireless and wired interfaces can communicate with the UAV-Agent and the MEC Server. Table 1 is a designed semantic entity involving the semantic understanding of navigation scenes and the data collection and processing process, namely the task template.

[0028] Table 1 Initial task template

[0029] The set of ID1 of the acquisition and navigation tasks is {0, 1, ..., M_ID1}. When ID1 is 0 and 1, it represents the acquisition task and navigation task respectively.

[0030] The ID2 set of navigation and operation tasks is {0,1,…,M_ID2}. When ID2 is 0 and 1, it indicates the process 1 adopted in the acquisition task (see Figure 1 ) and navigation tasks using the process 2 (see Figure 2 ). When ID2 is 1, the UAV-Agent turns on the camera for environmental perception to obtain the BEV and calculate the building height. Optionally, the UAV-Agent turns on the lidar sensor to more accurately estimate the building outline and height value. When ID2 is 2, based on process 2, the UAV in the navigation mission turns on the camera for environmental perception, data processing and dynamic autonomous navigation.

[0031] In this embodiment, the case where ID2 is 3 is discussed in detail, because it requires the UAV-Agent to have a specific visual language navigation function, which depends on which LLMs dense model it adopts. When ID2 is 3, based on process 2, in the geo-fence, the first path key point is located on the starting line of sight (see Figure 3 ). The ID2 takes 4 as an optional situation. The navigation station in the navigation task interacts with the UAV-Agent, which can dynamically update the knowledge base in the navigation station to deal with sudden low-altitude traffic congestion events.

[0032] ID3 contains the latitude, longitude and altitude of the navigation station, the latitude, longitude and altitude of the starting and ending points, the geographic fence boundary and region ID set and the number of guided paths. The region here is an extension of the traditional three-dimensional grid concept. See Figure 4 , the projections of multiple consecutive 3D grids on a guidance path are multiple rectangular areas with the same height parameters; see Figure 5 , a three-dimensional grid with the same height parameter is defined as a region.,The number of guidance paths within the geo-fence is M_ID3, the latitude, longitude and altitude of the navigation station are tuples (x, y, z), and the guidance path represents the path from the starting point to the end point.

[0033] ID4 contains the guide path i, the area ID index and the area topology relationship.

[0034] ID5 contains the number of key points in the guided path i.

[0035] ID6 contains multiple types of key points, such as path key points, longitude and latitude and altitude of low-altitude obstacles (such as wires and trees) and ground building type obstacles. On a guidance path, these key points are in order.

[0036] ID7 contains airspace restriction (such as no-fly zone) boundaries, traffic data and corresponding meteorological data. It should be noted here that airspace restriction (such as no-fly zone) boundaries and geofence boundaries are conceptually interchangeable. When the geofence boundary is physically present, it is the airspace restriction boundary; when the geofence boundary is virtual, a boundary is added to the BEV diagram for area division, annotation and subsequent RRT-star search interval definition.

[0037] Optionally, the definitions of ID8 and thereafter are similar to those of ID4-ID7. However, the ID8 guided path i+1 may correspond to other path planning requirements, such as drone path planning at other altitudes, drone path planning for low-altitude emergency roads, etc. LLMs and Agents can be used to help understand different path planning requirements because the inputs of these requirements may be text and images. Therefore, the geo-fence boundary definition of ID3 in Table 1 can adopt a hybrid linked list. This linked list can contain annotations of each area and key point annotations in each area; it can also contain a link to associate to a BEV graph with semantic annotations.

[0038] UAV-Agent is responsible for basic low-altitude geographic data collection and partial data processing. When the number of guidance paths is greater than 1, guidance path i and guidance path i+1 will use different path trajectories, which makes the length of the linked list in Table 1 increase continuously, and also leads to great complexity in the storage and search of guidance paths and key points. In order to facilitate data processing and analysis in geographic information collection, UAV-Agent needs to use LLMs and Agent to effectively extract geographic information parameters from the linked list and ensure interoperability between modules in the involved UAV navigation system.

[0039] Table 1 divides the low sky into multiple grids with geometric topological relationships. Each rectangular grid is represented by the latitude and longitude of the lower left vertex, the upper right vertex, and the height of the lower left vertex. For the convenience of evaluation, it is assumed that the vertical range d_h of each grid is the same. Therefore, d_h is the distance between two neighbors in the vertical direction or the lane safety interval. In this way, a virtual lane located in the low sky of the city is formed.

[0040] The grid is mapped to a graph structure. Table 1 mainly defines the semantic information of low-altitude urban planning. The graph structure is a data structure that facilitates the storage, search and calculation of geographic information data for low-altitude urban planning. Figure 4As shown, from top to bottom, the simulation of grid, graph structure and discovery guidance path or trajectory in low-altitude urban planning is represented.

[0041] The RRT-star algorithm quickly calculates the shortest path with the help of key points stored in the graph structure; the output of the RRT-star algorithm will update the edge weights in the graph structure; then the graph calculation is performed to recalculate the synthetic cost of path planning and evaluate the synthetic cost of switching drones at adjacent altitudes. Furthermore, since the LLM agent at the navigation station can process the collected measured data, the static graph structure update and graph calculation in the navigation knowledge base are performed in sequence, and the latter output will include the shortest path and the switching prediction of the adjacent low-altitude road.

[0042] In short, the above expressions form a drone navigation method proposed in this embodiment that is different from the previous ones, that is, a process that allows the navigation system to simulate, monitor and optimize performance in real time, and its closed-loop advantage allows the static graph structure to adapt to dynamic spatiotemporal traffic.

[0043] The global access order defines the geometric relationship of four adjacent grids. For height i, the order of rectangular areas from left to right is area 0, area 1 and area 2, and area 3. The left edge lines of area 1 and area 2 are the same, which means that area 1 and area 2 are right adjacent to area 0. The semantics on the map is that there is an intersection at the left edge line of area 0, and the left branch and right branch (of area 0) reach area 1 and area 2 respectively. The roads at height i and height i+1 are in an up-down topological relationship.

[0044] Implementation of isomorphic graph: In a static isomorphic graph structure, each node represents a rectangular area (a projection of a three-dimensional grid) (with lane safety intervals). The node attributes refer to the following constructor __init__. def __init__(self, rect_id, attr0=Node((0,0)), attr1=Node((0,0)),attr2=0, attr3=0, attr4=0, attr5=Node((0,0)), attr6=Node((0,0)), attr7=Node((0,0))): The constructor __init__ is used to initialize an instance of the Node_rect class. Its definition can be further optimized according to actual needs. It accepts parameters such as: rect_id: A unique identifier for the rectangular region.

[0045] attr0: The latitude and longitude of the lower left corner of the rectangular area. The default value is (0,0).

[0046] attr1: The latitude and longitude of the upper right corner of the rectangular area. The default value is (0,0).

[0047] attr2: The number of reference points in the rectangular area. The default value is 0.

[0048] attr3: The height of the rectangular area. The default value is 0.

[0049] attr4: congestion index of the rectangular area, such as the number of obstacles. The default value is 0.

[0050] attr5: The latitude and longitude of reference point 0. The default value is (0,0).

[0051] attr6: latitude and longitude of reference point 1. The default value is (0,0).

[0052] attr7: latitude and longitude of reference point 2. The default value is (0,0).

[0053] The definition of edge types in dynamic graph structures usually includes connection relationships (such as the connection between roads and intersections), traffic flow, speed limits, and congestion levels. In the evaluation experiments, synthetic costs are used to extend the definition of static isomorphic graphs to dynamic heterogeneous graphs.

[0054] Implementation of heterogeneous graph: A directed acyclic graph (DAG) is constructed by using a similar method to homogeneous graph to simulate the rectangular area relationship (with lane safety intervals) in three-dimensional geographic space. Figure 5 ,An implementation example of evaluating road switching in a heterogeneous graph is outlined as follows, which includes defining node class and rectangle node class, creating nodes and building connection relationships, using NetworkX to create graph structure, merging graph structures, using Matplotlib to draw graphs, and calculating the shortest path.

[0055] The functional description of creating nodes and building connection relationships is as follows: two groups of nodes, node0_0, node1_0, node2_0, node3_0 and node0_1, node1_1, node2_1, node3_1, are created, representing rectangular areas on two height layers respectively.

[0056] The functional description of using NetworkX to create a graph structure is as follows: Two directed graphs G0 and G1 are created, representing the graph structures at two height levels. Nodes and edges are added to the graphs, and weights are assigned to the edges.

[0057] The functional description of the merged graph structure is as follows: Create a new directed graph G and add the nodes and edges of G0 and G1 to G. Add edges between G0 and G1 to represent the switching of the drone between different altitude layers.

[0058] The function description of drawing graphics using Matplotlib is as follows: Defines the node position pos and label labels. Use different shapes and colors to represent common nodes and heterogeneous nodes. Draw nodes and edges, and display the label of the node center.

[0059] The function description of calculating the shortest path is as follows: Use the path planning algorithm to calculate the shortest path and path length from node 0 to node 4, node 4 to node 7, and node 7 to node 3. Merge these paths to get the complete path and total length from node 0 to node 3 through nodes 4 and 7.

[0060] Optionally, the heterogeneous graph-based path planning can consider multiple factors, such as: shortest path, minimum time, minimum congestion, etc. Note that in the actual path planning process of a UAV greater than 10 meters, the local environment information discovered by the camera and the lidar is usually fused to further improve the performance of the path planning scheme based on a single feature (Euclidean distance).

[0061] Path planning based on heterogeneous graphs can effectively reduce the reliance on predicted trajectories based on prior knowledge, i.e. one or more key points on the guidance path, while the usual predicted trajectory contains a large number of sampling points. If the time-varying nature of the low-altitude obstacle position cannot be ignored, the design of the UAV navigation system should also consider how to deal with the problem of failed reference points in the predicted trajectory.

[0062] In particular, the output of the a priori RRT-star simulation shown in this embodiment will be fed back into the graph structure so that the edge weights can be updated according to the measured data.

[0063] Considering that the accuracy of BEV scene understanding based on conventional image processing is relatively poor, this embodiment proposes a key point selection strategy. Considering that BEV scene understanding can roughly identify the four edge lines of a rectangular area, such as the top, bottom, left, and right, selecting a position with a horizontal coordinate slightly larger than the midpoint of the left edge line of the rectangular area as the first key point of each rectangular area can avoid the recognition error of BEV scene understanding to a certain extent. Of course, more reference points can be taken for each rectangular area. Figure 3 The horizontal range of area 0 is greater than 150 meters, so area 0 contains 3 reference points (starting point 1, path key point 9, navigation station 10), and area 0 contains building type obstacles (3, 4, 5, 6), low-altitude drone suspension type obstacle 7 and low-altitude wire type obstacle 8. In addition, LLMs and Agent can be used to parse the navigation intent in the text and improve the accuracy of BEV scene understanding.

[0064] exist Figure 3In the above example, the a priori RRT-star algorithm can find the trajectory from start point 1 to end point 2, and the synthetic cost of its output (see the following definition) is fed back into the graph structure. Assume that the height of the drone is constant at h 1 , the three-dimensional grid is projected into two-dimensional rectangular areas, and the rectangular areas that the trajectory passes through are region 0, region 1, and region 3, because it is assumed that the congestion level in region 2 is high. In a future urban ultra-low altitude scene, such as in the range of 1 meter to 45 meters in altitude, in addition to static obstacles, drones will also encounter a large number of unpredictable dynamic obstacles, which challenges the timeliness of the output trajectory of the static path planning algorithm, and also makes it difficult to achieve the path planning goals accurate to the sub-meter and sub-second levels. Therefore, according to actual requirements, the actual path planning algorithm needs to run regularly to obtain the latest navigation trajectory.

[0065] To quickly demonstrate the use of a mission template (only when ID2 is 3), in the geo-fence, the first path keypoint is located on the starting line of sight. The first path keypoint and the second path keypoint are called drone prior position 1 and prior position 2 respectively, and the navigation station is located at prior position 2.

[0066] Compared with the baseline RRT-star (see RRT-star and its improvements below), the example is used to modify the baseline RRT-star (2) to (4) steps, using the key points or prior positions from the guided path data to reduce the sampling random points. Secondly. It also integrates target biased sampling. It is a key point guided RRT-star path planning method, and its detailed design is as follows.

[0067] Considering the autonomous navigation scenarios of low-altitude UAVs in urban streets or urban canyons, UAVs face challenges such as lack of high-precision dynamic maps, weak GPS and cellular communication signals, and temporal and spatial low-altitude interference that is different from ground navigation. A UAV path planning framework combining graph structure and RRT-star algorithm is considered. This framework not only provides UAVs with key points on suboptimal paths, but also generates temporary paths in time when pre-planned paths cannot be used. Even at the starting point, the user machine only needs to interact with the navigation station once, instead of periodically and dynamically receiving instructions from the dynamic map API, which helps save communication and computing resources. Using the prior key points, both the UAV and the navigation station can simulate path planning, thereby calculating the shortest path before navigation. However, since the navigation station can obtain real-time road conditions beyond line of sight, the simulation at the navigation station can obtain more effective trajectory prediction, thus playing the role of traffic lights at low altitudes in the city.

[0068] The steps of the benchmark RRT-star (Rapidly Expanding Random Tree Star Algorithm) include: (1) Initializing the tree, that is, taking the starting point as the root node of the tree; (2) Sampling random points, that is, randomly generating a point in the search space; (3) Expanding the tree, that is, finding the nearest node in the tree and expanding a new node toward the random point; (4) Reconnecting, that is, checking the nodes within a certain range around the new node and trying to reconnect through the new node to reduce the path length; (5) Updating the path cost, that is, updating the path cost of all affected nodes in the tree. The steps (2) to (4) are iterated repeatedly, and finally an optimal path from the starting point to the target is found as the navigation path.

[0069] The advantage of the Euclidean distance-based, probabilistic sampling, and incremental sampling baselines RRT-star is that they can quickly find an initial path and then continuously optimize it as the number of samples increases. They have moderate computational complexity and are able to find solutions without using explicit information about obstacles in the configuration space. They rely on collision checking modules and build a roadmap of feasible trajectories by connecting together a set of points sampled from the obstacle-free space.

[0070] In general, the benchmark RRT-star still needs to be expanded in terms of obstacle avoidance requirements, flight dynamic constraints, environmental complexity and real-time performance to effectively handle the safety requirements and flight constraints of UAVs in complex urban three-dimensional environments. In particular, the idea of ​​visual language navigation is introduced into the information flow and device of the UAV navigation system, and an example of keypoint-assisted RRT-star path planning is proposed. Its innovative strategy lies in the following single keypoint-assisted sampling method 1 and double keypoint-assisted sampling method 2. See Figure 3 At the beginning of the navigation task, the navigation station selects one or more guidance path data according to the requirements and sends them to the user machine. See the case where ID2 is 3 in Table 1, and its guidance path data contains the definition of key points. From the starting point to the first key point, the sampling-based method 1 is used, and from the first key point to the second key point, the sampling-based method 2 is used. When the number of key points is greater than 2, method 2 is used cyclically.

[0071] For navigation tasks, a keypoint-guided sampling method is proposed1 to provide a more direct straight path for the UAV with a certain probability. Using the prior provided by a single keypoint (SingleKeypoint), this local node expansion method can quickly find the initial path to reduce random space exploration and avoid the redundancy of the benchmark RRT-star when sampling locally. This strategy usually needs to be combined with the output of a camera or lidar to quickly find routes greater than 10 meters.

[0072] The prior position is used to guide the sampling process, which is different from the step of sampling random points in the benchmark RRT-star algorithm. By sampling near the prior position, the user's machine can more effectively explore the area of interest and improve the efficiency of path planning or graphics processing. When ID2 takes the value of 3, prior position 1 is key point 1, and prior position 2 is key point 2 (navigation station position). When ID2 takes the value of 4, it is a functional extension of the case when ID2 takes the value of 3, which will not be discussed in detail here. The functions of sampling method 1 with single key point guidance and sampling method 2 with double key point guidance are described in sequence as follows. A sampling strategy for crossing the boundary between two regions is also mentioned at the end of the specification.

[0073] Function parameters of sampling method 1 with single key point guidance: There is only one parameter self in the definition of the method. It is used in this function to access and operate on the instance variables and methods of the RrtStar class.

[0074] Method implementation: 1) Method definition: This is an instance method. It has no parameters (except for the above-mentioned self) and will operate using the attributes of the class.

[0075] 2) Calculate the prior index self.prior_locations is a list containing prior positions, and self.index_ref is a reference index (in the implementation of this patent, it refers to the key point index). By calculating len(self.prior_locations) - self.index_ref through subtraction, an index of a prior position is obtained. When self.index_ref < 0, it means that all key points have been used up, and the proposed key point-guided RRT-star will execute (2), (3), (4), and (5) of the benchmark RRT-star.

[0076] 3) Obtain the prior position Take out the element with index index_prior from the list of prior positions and pass its coordinates to the instance node_rand of the Node class.

[0077] 4) Find the nearest node Use the nearest_neighbor method to find the node node_near closest to node_rand. self.vertex may be a list or set containing all nodes.

[0078] 5) Define the target point and the boundary of the sampling area target_local is the coordinate of the target point, and bounds_local defines the boundary of the sampling area. Here the boundary is the rectangular area formed by the nearest node and the target point.

[0079] 6) Perform target bias sampling Use the goal_biased_sampling method to perform target biased sampling in the defined sampling area. goal_bias_prob is the probability of target biased sampling, provided by self.direct_prob. The sampling result sample is used to create a new node node_new.

[0080] 7) Return results The method returns the prior position prior_pos, the nearest node node_near, and the newly sampled node node_new.

[0081] The flowchart of the single keypoint guided sampling method 1 is shown in Figure 6 .

[0082] It is used to perform a priori sampling in a specific local area, that is, to perform target biased sampling and generate new sampling points based on dual key points and local scale parameters.

[0083] Function parameters of dual keypoint guided sampling method 2: self: usually refers to the instance of the class, indicating that this is a class method.

[0084] prior_pos: Prior position, indicating the dual key points (latitude, longitude and altitude) referenced during sampling.

[0085] node_near: The node closest to a target point.

[0086] local_scale: Local scale parameter used to control the range of the sampling area.

[0087] Method implementation: 1) Initialize a new node list Create an empty list node_new_list to store newly generated nodes.

[0088] 2) Cyclic sampling Loop local_scale times, generating a new sampling point each time.

[0089] 3) Calculate the new local target position A scaling factor scale is calculated based on the current loop number fruit and local_scale, and then the new local target position target_local_new is calculated using the factor and the prior position prior_pos and the coordinates of the nearest node node_near.

[0090] 4) Find the node closest to the new target Use the nearest_neighbor method to find the node closest to the new target position target_local_new and update node_near.

[0091] 5) Define the sampling range Define the sampling range bounds_local, that is, the x and y ranges, which are the coordinates of node_near and target_local_new respectively. As mentioned above, the example usage of the task template omits the height parameter z.

[0092] 6) Perform target bias sampling Use the goal_biased_sampling method to perform target biased sampling within the defined range and generate new sampling points sample. goal_bias_prob is the probability parameter of target biased sampling.

[0093] 7) Add new nodes to the list Convert the generated sampling point sample into a Node object and add it to the node_new_list list.

[0094] 8) Return the new node list Returns a list node_new_list containing all newly generated nodes.

[0095] Supplement to the above path planning method: 1) nearest_neighbor and goal_biased_sampling are two methods defined in the relevant code, which need to be implemented in advance. nearest_neighbor is used to find the nearest node.

[0096] 2) goal_biased_sampling is used to perform target biased sampling within a specified range.

[0097] 3) self.vertex is a list or data structure containing all nodes.

[0098] 4) self.direct_prob is the probability parameter of target biased sampling, which is used to control the degree of bias towards the target during sampling.

[0099] 5) This method assumes that Node is a custom class that contains x and y attributes to represent the coordinates (latitude and longitude) of the node. As mentioned above, the example usage of the task template omits the height parameter z.

[0100] The flowchart of the dual keypoint guided sampling method 2 or strategy can be found in Figure 7 The evaluation experiment uses a synthetic cost. By comprehensively considering distance, height, energy consumption and obstacle distance, an RRT-star cost function that meets actual needs is designed, and the selection of weight coefficients needs to be adjusted according to specific application scenarios. The synthetic cost is: ; Among them, h 1 and h 2 The nodes q are 1 and q 2 Height value; E(h 1 ,h 2 ) is the energy consumption function caused by height change, which is usually proportional to the height difference; in the obstacle distance penalty term, d(q 1 ,q 2 ) is the distance between the trajectory and the nearest obstacle; λ 1 , 2 and λ 3 is the weight coefficient, which is used to balance the influence of each item.

[0101] (3) MEC Server feeds back the updated task template to UAV-Agent.

[0102] Due to the complex environment of low-altitude cities, the UAV-Agent's environmental perception and target recognition information may be biased, airspace restrictions may be updated in real time, and new navigation visual language inputs may be required. Therefore, the MEC Server needs to be calibrated and updated.

[0103] (4) The UAV-Agent downloads the updated mission template to the navigation station.

[0104] Through UAV-Agent, the navigation station will obtain a low-altitude UAV navigation knowledge base.

[0105] (5) After receiving the ACK sent by the navigation station, the UAV-Agent leaves the service area involved in the geo-fence.

[0106] The UAV-Agent cruises in several geo-fences of the urban grid to transmit the low-altitude UAV navigation knowledge base to the distributed navigation stations.

[0107] Figure 1The key is to show the information transfer process between various entities and how to collaboratively complete the construction of the low-altitude UAV navigation knowledge base. This is necessary to understand the working mechanism of the UAV navigation system and the lightweight interaction between various components.

[0108] like Figure 2 As shown in the figure, the information transmission process between the user machine, navigation station, UAV-Agent and MEC Server during the navigation task, where the four vertical boundary lines represent the user machine, navigation station, UAV-Agent and MEC Server respectively. There are multiple arrows between each boundary line, indicating different message transmission processes.

[0109] (1) The user machine that enters the starting point of the geo-fence sends a navigation task request to the navigation station. Here, the user machine has a low-level visual language navigation function and can issue navigation task requests in different forms, such as instructions, text, or even voice memos.

[0110] (2) After receiving the request, the navigation station selects the guidance path data from the task template and feeds back the navigation path to the user machine. When the navigation station collects real-time congestion data, it will perform graph calculation to obtain the suboptimal navigation path. When the navigation task request of the user machine is a text or even a voice prompt, the navigation station uses the local embodied intelligence module to understand the navigation task request. Therefore, the navigation station is a device with medium or high-level visual language navigation functions.

[0111] (3) After receiving the guidance path data, the user machine begins the subsequent process of autonomous navigation. It passes through the key points on the guidance path one by one and uses low-level control to fine-tune the (virtual) lane within 10 meters to cope with interference in the three-dimensional space, such as static ground buildings, power lines and branches moving in the wind, and surrounding drones.

[0112] (4) When the navigation task is completed, the user machine notifies the navigation station and the navigation station sends an ACK to the user machine.

[0113] (5) Finally, the UAV-Agent communicates with the MEC Server, stores path trajectories, and updates task templates.

[0114] Figure 2 The information transmission process between the user machine, navigation station, UAV-Agent and MEC Server is shown, and how the low-altitude UAV and its ground autonomous navigation vehicle share information with the navigation station through a wireless interface. The dynamic interaction between the navigation station and the UAV-Agent (when ID2 is 4) is not reflected here, which is optional. This is important for understanding the workflow and communication mechanism of the autonomous navigation system.

[0115] When ID2 is 1, 2, 3, and 4, the drone control system involved uses the already built drone navigation knowledge base. When ID2 is 3, the key point is also called the prior position. With the assistance of other components or sensors (such as cameras, lidar, and millimeter-wave radar, etc.), the navigation station will be responsible for coordinating the navigation and path planning of the drone within the geofence. The geofence may contain one or more navigation stations, and the drone is not specifically located within the visual range of the navigation station.

[0116] In the path planning experiment, a class is used to define environmental parameters and obstacles. By setting the coordinate ranges x_range=[-20,250] (meters) and y_range=[-40,40] (meters) related to longitude and latitude, the height z_range is set to 15 to 45 meters; secondly, the boundary obstacles, rectangular obstacles and circular obstacles are initialized through the obs_boundary, obs_circle and obs_rectangle functions to configure the environmental constraints that need to be considered when planning the path. The geographic fence takes multiple rectangles of different heights. See Figure 3 In addition to the boundary obstacles that simulate geo-fences, each rectangular area in the benchmark test includes one or more rectangular obstacles. The navigation station is located between two rectangular obstacles and it is not within the line of sight of the starting point.

[0117] The following two paragraphs will explain the task templates and Figure 4 and Figure 5 The graph structure involved is further explained and supplemented. Table 1 is a semantic entity involving data collection and processing procedures suitable for LLMs and Agents to perform semantic understanding of navigation scenes. The graph structure is a data structure that is convenient for modeling three-dimensional (virtual) roads and lanes. The latter uses heterogeneous nodes to model the three-dimensional airspace of the city, thereby facilitating multi-layer path planning at different altitudes. UAV-Agent is responsible for the first mission template construction. The semantic entity it outputs is the initial entity, but the low-altitude flight management system considered can dynamically update the mission template based on mobile edge computing. The mission template and graph structure are transmitted to the navigation station. The navigation station can not only perform graph calculations, but also RRT-star simulations. As mentioned earlier, the guidance path in Table 1 contains several path key point parameters, but the nodes of the graph structure are not directly equivalent to the path key points in Table 1. See Figure 3In the two-dimensional plane in the figure, the midpoint of the left boundary of an area is only one of the key points of the path. The guidance path involved in Table 1 is only a static, initial set of path planning parameters. It does not necessarily contain the coordinates of a large number of points in the trajectory, nor does it consider kinematic constraints and dynamic obstacles during local range navigation. In short, the demand for safe and smooth paths is inseparable from the RRT-star algorithm, and the guidance path is only a sampling of the trajectory output by RRT-star. Secondly, the definition of the initial graph structure requires the use of the parameters in Table 1. After executing RRT-star, the edge weights of the initial graph structure will be reset to the edge weights of the new graph structure.

[0118] Continue with the above explanations and supplements. Using the key points of the guidance path in Table 1, the navigation station uses the RRT-star algorithm for path planning. The purpose of setting the guidance path in the mission template is to plan across multiple areas. Therefore, the mission template focuses on semantic regional guidance, and the graph structure facilitates the evaluation of multi-layer path planning to adapt to dynamic low-altitude traffic, while RRT-star focuses on local navigation planning. Because the boundaries of the two areas may be different, the traditional RRT-star algorithm is very sensitive to boundary parameters and is prone to local traps at the boundary of the two areas. At this time, using the prior reference points provided by the mission template, another target-biased RRT-star algorithm can be implemented, that is, a sampling strategy that crosses the boundary of the two areas.

[0119] Summary of the above two paragraphs: Aiming at the low-altitude flight management system in the low-altitude economy, the present invention innovatively designs the semantic entity, graph structure implementation and target-biased RRT-star, and makes the three stages complement each other.

Claims

1. A three-dimensional low-altitude urban operation guidance method for unmanned aerial vehicles based on lightweight mission templates, characterized in that: The following steps are involved: (1) The UAV-Agent enters the geo-fence and sends a starting point collection command to the navigation station within the geo-fence; (2) After receiving the start point collection command, the navigation station feeds back the start point and end point information within the geographic fence and the correct reception information ACK to the UAV-Agent; (3) The UAV-Agent plans the initial path from the starting point to the end point based on the feedback information. During the operation, it collects environmental information within the geographic fence and builds a preliminary task template based on the collected environmental information. and updating the task template via the mobile edge server; The task template includes a graph structure obtained by rasterizing a three-dimensional low-altitude area mapping, a number of key points on a path, and a number of guide paths planned according to the graph structure and the key points on the path; (4) Sending the updated task template to the navigation station, the navigation station uses a path planning algorithm based on the key points on the guidance path in the task template to plan the guidance path and obtain a smooth navigation path; (5) The user machine entering the starting point of the geographic fence sends a task request to the navigation station; (6) After receiving the task request, the navigation station selects guidance path data from the task template and feeds back the navigation path to the user machine; (7) After receiving the guidance path data and the navigation path, the user machine starts the subsequent process of autonomous navigation and passes through the key points on the guidance path in sequence; (8) When the navigation task is completed, the user machine notifies the navigation station and the navigation station sends an ACK to the user machine.

2. The method for guiding UAV three-dimensional low-altitude urban operations based on lightweight mission templates according to claim 1 is characterized in that: The task template includes the task type and corresponding process, the longitude, latitude and altitude of the navigation station, the geographic fence boundary, the longitude, latitude and altitude of the starting point and end point within the geographic fence, the number of guidance paths, the longitude, latitude and altitude of several key points of the path, several guidance paths, a set of regional IDs after rasterization of the three-dimensional low-altitude area, the regional ID index and the regional topological relationship.

3. The method for guiding UAV three-dimensional low-altitude urban operations based on lightweight mission templates according to claim 2 is characterized in that: The steps for constructing the graph structure in the task template are as follows: (31) Divide the three-dimensional low-altitude urban structure within the geographic fence into multiple grids with geometric topological relationships; (32) Mapping the grid into multiple virtual roads, abstracting a virtual road into a graph structure of homogeneous nodes, each homogeneous node graph structure stores a single path and its key points on the path and implements graph calculations related to path planning; the graph structure describes the connection relationship between different areas in the environment; virtual roads of different heights are abstracted into a graph structure of heterogeneous nodes, and the graph structure of heterogeneous nodes includes several paths, and each path has a different height; (33) Assign weights to the edges of the graph structure.

4. The method for guiding UAV three-dimensional low-altitude urban operations based on lightweight mission templates according to claim 3 is characterized in that: The path planning algorithm includes an improved RRT-star algorithm; the improved RRT-star algorithm assists the sampling of sampling points in the RRT-star algorithm through path key points, specifically: The key points of the path on the guidance path are used as prior positions in sequence; For the interval between the starting point and the key point of the path, obtain the prior position, find the node closest to the prior position, confirm the coordinates of the target point, use the rectangular area formed by the nearest node and the target point as the sampling area, perform target biased sampling in the sampling area, and sample to obtain a new node; For the interval between two key points of the path, the new local target position is calculated by the scaling factor, the prior position and the coordinates of the nearest node, the node closest to the new target position is found, and the rectangular area formed by the node closest to the new target position and the local target position is used as the sampling area. Target biased sampling is performed within the sampling area to obtain a new node.

5. The method for guiding UAV three-dimensional low-altitude urban operations based on lightweight mission templates according to claim 4 is characterized in that: Calculate the synthesis cost of the trajectory planned by the improved RRT-star algorithm, and update the weight of the edge in the graph structure through the synthesis cost; the synthesis cost The calculation formula is: ； in, For Node The height value of For Node The height value of is the energy consumption function caused by altitude change, is the distance between the trajectory and the nearest obstacle, , and is the weight coefficient.

6. The method for guiding UAV three-dimensional low-altitude urban operations based on lightweight mission templates according to claim 5 is characterized in that: The path planning method used for the initial path in step (3) includes: A* algorithm, Dijkstra algorithm, RRT algorithm; The environmental information collected within the geo-fence during the operation includes image acquisition, location information request and geographic API call to obtain data of key points of the path; After the UAV-Agent arrives at the destination, it generates an initial task template with the collected information and uploads it to the database of the mobile edge server MEC server. The MEC server calibrates and updates the initial task template and feeds back the updated task template to the UAV-Agent. The UAV-Agent transmits the updated task template to the navigation station. After receiving the ACK sent by the navigation station, the UAV-Agent leaves the service area involved in the geographic fence.

7. The method for guiding UAV three-dimensional low-altitude urban operations based on lightweight mission templates according to claim 6 is characterized in that: The mission template also includes spatial restriction boundaries, traffic data and meteorological data.

8. The method for guiding UAV three-dimensional low-altitude urban operations based on lightweight mission templates according to claim 7 is characterized in that: In step (6), after the navigation station receives the task request, when the navigation station collects real-time congestion data, the navigation station will perform graph calculation to obtain a suboptimal guidance path.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images