Unmanned aerial vehicle autonomous navigation system based on multi-source vision assistance

Abstract

The invention discloses an unmanned aerial vehicle autonomous navigation system based on multi-source vision assistance, and relates to the technical field of unmanned aerial vehicle autonomous navigation, and the system comprises the following modules: a terrain recognition module. A camera and a laser radar are used for synchronously collecting environment data, a visual odometer and a three-dimensional point cloud analysis technology are combined, a dynamically updated grating map is constructed, obstacle and topographic features are recognized through point cloud clustering, an improved Dijkstra algorithm is combined with an information entropy evaluation mechanism to plan a global optimal path, a B spline curve is used for eliminating path mutation, and the optimal path is obtained. The method comprises the following steps that: a dynamic speed decision-making module adjusts a flight parameter in real time on the basis of an obstacle distance-response model, and finally realizes trajectory tracking and dynamic deviation correction through closed-loop PID control and visual pose correction, so that full-link closed-loop control of'perception-planning-execution 'is formed, and the method has high precision, strong robustness and real-time response capability.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous navigation of unmanned aerial vehicles (UAVs), and in particular to an autonomous navigation system of UAVs based on multi-source vision assistance. Background Art

[0002] The autonomous navigation system of a drone refers to a system that can intelligently perceive and determine its cruising position and environment in the absence of satellite navigation (GPS Beidou system, etc.) and ground navigation systems, such as in indoor tunnels, mines, underground fortifications, open battlefields, or electronic confrontation situations. It can rely on its own machine vision system, precise inertial navigation sensor system, and artificial intelligence system to intelligently complete path planning, flight control, and cruise obstacle avoidance based on the initial target, and ultimately complete the predetermined cruise mission.

[0003] Traditional drone navigation technology mostly relies on GPS and inertial measurement units (IMU) for positioning. However, in complex urban canyons, indoors or disaster scenarios, GPS signals are easily blocked or interfered with, resulting in positioning drift or even failure. At the same time, navigation solutions based on a single sensor (such as pure vision or lidar) have obvious shortcomings: visual SLAM (simultaneous localization and mapping) is easily affected by lighting changes and texture loss, and although lidar has accurate ranging, it is difficult to recognize semantic information and cannot work stably in low-visibility environments (such as thick smoke and haze). In addition, although traditional path planning algorithms (such as A* and RRT) can generate obstacle avoidance paths, they are not suitable for autonomous driving. However, due to the neglect of environmental dynamics and sudden changes in path curvature, UAV flight becomes discontinuous or emergency obstacle avoidance responses lag. Existing speed control strategies are mostly based on fixed thresholds or simple linear adjustments, lacking adaptive optimization of obstacle distance and motion inertia, making it difficult to balance efficiency and safety. To address these pain points, an autonomous navigation system that integrates multi-source perception, possesses dynamic decision-making capabilities, and is highly robust is urgently needed. Through multi-sensor data complementation, real-time path optimization, and closed-loop control mechanisms, it can break through the navigation bottleneck in complex environments and meet the needs of high-precision autonomous UAV operations in fields such as industrial inspection, agricultural plant protection, and disaster relief. To this end, we propose an autonomous UAV navigation system based on multi-source vision assistance. Summary of the Invention

[0004] The purpose of the present invention is to provide a multi-source vision-assisted unmanned aerial vehicle autonomous navigation system to solve the problems raised in the above background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an autonomous navigation system for unmanned aerial vehicles based on multi-source vision assistance, the system comprising the following modules:

[0006] Terrain recognition module, which uses a high-definition camera to recognize terrain features, obtains environmental structure information through visual odometry (VO), generates three-dimensional point cloud data using a lidar, accurately measures the distance to obstacles and terrain height, identifies independent obstacles or terrain undulations through point cloud clustering, unifies the data of the camera and lidar into the UTM coordinate system and establishes a grid map;

[0007] Path planning module, which brings the grid map data into the Dijkstra algorithm, searches for the shortest path from the starting point to the ending point, and uses the entropy method to verify the optimal path. The lower the entropy value, the higher the reliability of the path;

[0008] Speed calculation module, which uses a wheel speed sensor to obtain the current flight speed and calculates the maximum allowable speed according to the characteristics of the path nodes;

[0009] Control module, which obtains the speed data and the optimal path data to control the flight speed and steering of the drone.

[0010] Preferably, before unifying the data of the camera and lidar into the UTM coordinate system, the median filtering method is used to remove noise from the acquired data.

[0011] Preferably, the grid map divides the environment into regular grids of 1m×1m, and each regular grid is marked as passable, an obstacle, or unknown, and the grid map is refreshed in real time.

[0012] Preferably, when substituting the grid map data into the Dijkstra algorithm in the path planning module, the starting point is the position of the current drone in the grid map, and the ending point is the regular grid marked as passable on the edge of the grid map closest to the starting point.

[0013] Preferably, the calculation formula of the Dijkstra algorithm is:

[0014] Initialization:

[0015]

[0016] where d[s] represents the starting point, d[v] represents other nodes, S represents the set of nodes with the shortest path determined, the distance from the starting point to itself is 0, and the initial distance of other nodes is infinity.

[0017] Select the node u with the smallest d[v] among the unvisited nodes and add it to S:

[0018]

[0019] Update the distance of the adjacent node v of u:

[0020] d[v] = min(d[v], d[u] + w(u, v))

[0021] The termination condition is: all nodes are visited (S = v) or the shortest path to the target node t has been determined (t ∈ S).

[0022] Preferably, the entropy method calculates the information entropy of the path to evaluate the uncertainty, and the formula is:

[0023]

[0024] where H represents the information entropy of the path, which is used to evaluate the uncertainty of the path, n represents the total number of nodes in the path, and p i represents the probability of the i-th node or state occurring. The lower the entropy value, the higher the reliability of the path.

[0025] Preferably, the calculation of the node speed has the formula:

[0026]

[0027] where v max represents the maximum allowable speed, v limit represents the speed limit, d obstacle represents the distance to the obstacle, and τ represents the reaction time.

[0028] Preferably, for the path generated by the Dijkstra algorithm, the acute angles generated by the path are smoothed by the B-spline curve algorithm.

[0029] Preferably, the path smoothing algorithm formula is:

[0030]

[0031] where C(u) is a point on the curve, the parameter u varies within the range of the node vector, P i is the control point, N i,k (u) is the i-th k-th order B-spline basis function, n is the number of control points minus 1, and k is the order of the B-spline.

[0032] Preferably, the nodes in the Dijkstra algorithm are each regular grid.

[0033] Compared with the prior art, the beneficial effects of the present invention are

[0034] By setting up this system, the present invention synchronously collects environmental data using a camera and a lidar, combines visual odometry and 3D point cloud analysis technologies to construct a dynamically updated grid map, accurately identifies obstacles and terrain features through point cloud clustering, uses an improved Dijkstra algorithm combined with an information entropy evaluation mechanism to plan the globally optimal path, eliminates path mutations using B-spline curves to ensure flight coherence, and a dynamic speed decision module adjusts flight parameters in real time based on an obstacle distance-response model. Finally, trajectory tracking and dynamic deviation correction are achieved through closed-loop PID control and visual pose correction, forming a full-link closed-loop control of "perception - planning - execution", with high precision, strong robustness, and real-time response capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flowchart of the system of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0037] Embodiment: Please refer to Figure 1 , the present invention provides a technical solution: an unmanned aerial vehicle autonomous navigation system based on multi-source vision assistance, and the system includes the following modules:

[0038] A terrain recognition module that uses a high-definition camera to recognize terrain features, obtains environmental structure information through visual odometry (VO), generates 3D point cloud data using a lidar, accurately measures the distance to obstacles and the terrain height, identifies independent obstacles or terrain undulations through point cloud clustering, and unifies the data of the camera and the lidar into the UTM coordinate system and establishes a grid map;

[0039] A path planning module that brings the grid map data into the Dijkstra algorithm to search for the shortest path from the starting point to the ending point, and uses the entropy method to verify the optimal path. The lower the entropy value, the higher the reliability of the path;

[0040] A speed calculation module that uses a wheel speed sensor to obtain the current flight speed and calculates the maximum allowable speed according to the characteristics of the path nodes;

[0041] A control module that obtains the speed data and the optimal path data to control the flight speed and steering of the unmanned aerial vehicle.

[0042] As a further limitation of the present invention, before unifying the data of the camera and the lidar into the UTM coordinate system, the median filtering method is used to denoise the acquired data.

[0043] The grid map divides the environment into regular grids of 1m×1m, and each regular grid is marked as passable, an obstacle, or unknown, and the grid map is refreshed in real time.

[0044] When bringing the grid map data into the Dijkstra algorithm in the path planning module, the starting point is the position of the current drone in the grid map, and the end point is the regular grid marked as passable on the edge of the grid map closest to the starting point.

[0045] The calculation formula of the Dijkstra algorithm is:

[0046] Initialization:

[0047]

[0048] Among them, d[s] represents the starting point, d[v] represents other nodes, S represents the set of nodes with the shortest path determined, the distance from the starting point to itself is 0, and the initial distance of other nodes is infinite.

[0049] Select the node u with the smallest d[v] among the unvisited nodes and add it to S:

[0050]

[0051] Update the distance of the adjacent node v of u:

[0052] d[v] = min(d[v], d[u] + w(u, v))

[0053] The termination condition is: all nodes are visited (S = v) or the shortest path of the target node t has been determined (t ∈ S).

[0054] The entropy method calculates the information entropy of the path to evaluate the uncertainty. The formula is:

[0055]

[0056] Among them, H represents the information entropy of the path, which is used to evaluate the uncertainty of the path, n represents the total number of nodes in the path, and p i represents the probability of the i-th node or state appearing. The lower the entropy value, the higher the reliability of the path.

[0057] The calculation of the node speed, the formula is:

[0058]

[0059] Among them, v max represents the maximum allowable speed, vlimit represents the speed limit, d obstacle represents the distance to the obstacle, and τ represents the reaction time.

[0060] The path generated by the Dijkstra algorithm is smoothed for the acute angles generated by the path through the B-spline curve algorithm.

[0061] The path smoothing algorithm formula is:

[0062]

[0063] Among them, C(u) is the point on the curve, the parameter u varies within the range of the knot vector, P i is the control point, N i,k (u) is the i-th k-th order B-spline basis function, n is the number of control points minus 1, and k is the order of the B-spline.

[0064] The nodes in the Dijkstra algorithm are each regular grid.

[0065] The specific implementation of this embodiment is as follows: Terrain perception and mapping: The system first perceives the environment through the fusion of a high-definition camera and a lidar. The visual data collected by the camera is combined with visual odometry (VO) to extract environmental feature points and motion trajectories, while the lidar synchronously generates high-precision three-dimensional point clouds for identifying terrain undulations, obstacle contours and distances. The point cloud clustering algorithm is used to distinguish independent obstacles (such as trees and buildings) from continuous terrain changes, and the multi-source data is uniformly registered to the UTM coordinate system to construct a dynamically updated 1m×1m grid map. This map provides a global spatial representation for path planning by real-time annotating grid attributes (passable, obstacle, unknown area);

[0066] Path planning and optimization: Based on the grid map, the system uses an improved Dijkstra algorithm to search for the global optimal path. The algorithm starts from the current position of the UAV and plans the shortest path to the passable area at the edge of the map by iteratively selecting the minimum-cost node and updating the weights of the neighborhood grids. To ensure the reliability of the path, the system introduces an information entropy evaluation mechanism to quantify the path risk by calculating the environmental feature stability of the path nodes (such as obstacle distribution density and terrain change gradient), and preferentially selects the candidate path with the lowest entropy value. For the problem of sudden motion at the path turning points, the B-spline curve algorithm is used to smooth the original path to eliminate acute-angle broken lines and improve flight coherence;

[0067] Dynamic speed decision-making: During the path execution phase, the system dynamically adjusts flight parameters by integrating real-time sensor data. The wheel speed sensor feeds back the current motion state, and path node attributes (such as the distance to adjacent obstacles and preset safety thresholds) trigger speed constraint calculations. By establishing an obstacle distance-speed response model, the maximum allowable speed for each flight segment is derived to ensure that the UAV has sufficient braking distance in case of sudden obstacles. Meanwhile, the median filtering algorithm denoises the sensor data to suppress the interference of outliers on speed decision-making;

[0068] Motion control and execution: Finally, the control module converts the planned path and speed commands into flight control signals, adjusts the rotor speed and rudder surface deflection through a PID controller to achieve the dual goals of heading tracking and speed maintenance. The system continuously monitors the deviation between the actual trajectory and the planned path through a closed-loop feedback mechanism, and uses the pose correction function of the visual odometer for dynamic deviation correction, forming a complete closed-loop of "perception - planning - control", and finally achieving the goal of autonomous navigation in complex environments.

[0069] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device.

[0070] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An unmanned aerial vehicle autonomous navigation system based on multi-source vision assistance, characterized in that: The system includes the following modules: A terrain recognition module that uses a high-definition camera to recognize terrain features, obtains environmental structure information through visual odometry (VO), generates three-dimensional point cloud data using a lidar, accurately measures the distance to obstacles and terrain height, identifies independent obstacles or terrain undulations through point cloud clustering, and unifies the data of the camera and lidar into the UTM coordinate system and creates a grid map; A path planning module that inputs the grid map data into the Dijkstra algorithm, searches for the shortest path from the starting point to the ending point, and uses the entropy method to verify the optimal path. The lower the entropy value, the higher the reliability of the path; A speed calculation module that uses a wheel speed sensor to obtain the current flight speed and calculates the maximum allowable speed according to the characteristics of the path nodes; A control module that obtains the speed data and the optimal path data to control the flight speed and steering of the drone.

2. The autonomous navigation system for unmanned aerial vehicles based on multi-source vision assistance according to claim 1, characterized in that: Before unifying the data of the camera and lidar into the UTM coordinate system, the median filtering method is used to remove noise from the acquired data.

3. The autonomous navigation system for an unmanned aerial vehicle based on multi-source vision assistance according to claim 1, wherein: The grid map divides the environment into regular grids of 1m×1m, and each regular grid is marked as passable, an obstacle, or unknown, and the grid map is refreshed in real time.

4. The autonomous navigation system for unmanned aerial vehicle based on multi-source vision assistance according to claim 1, characterized in that: When inputting the grid map data into the Dijkstra algorithm in the path planning module, the starting point is the position of the current drone in the grid map, and the ending point is the regular grid marked as passable on the grid map edge closest to the starting point.

5. The autonomous navigation system for unmanned aerial vehicles based on multi-source vision assistance according to claim 1, wherein: The calculation formula of the Dijkstra algorithm is: Initialization: Where d[s] represents the starting point, d[v] represents other nodes, S represents the set of nodes with the shortest path determined, the distance from the starting point to itself is 0, and the initial distance of other nodes is infinity. Select the node u with the smallest d[v] among the unvisited nodes and add it to S: Update the distance of the adjacent node v of u: d[v] = min(d[v], d[u] + w(u, v)) The termination condition is: all nodes are visited (S = v) or the shortest path of the target node t has been determined (t ∈ S).

6. The autonomous navigation system for unmanned aerial vehicle based on multi-source vision assistance according to claim 1, characterized in that: The entropy method calculates the information entropy of the path and evaluates the uncertainty. The formula is: Among them, H represents the information entropy of the path, which is used to evaluate the uncertainty of the path, n represents the total number of nodes in the path, and p i represents the probability of the occurrence of the i-th node or state. The lower the entropy value, the higher the path reliability.

7. The autonomous navigation system for unmanned aerial vehicles based on multi-source vision assistance according to claim 1, characterized in that: The calculation formula of the node speed is: where v max represents the maximum allowable speed, v limit represents the speed limit, d obstacle represents the distance to the obstacle, and τ represents the reaction time.

8. The autonomous navigation system for unmanned aerial vehicles based on multi-source vision assistance according to claim 1, characterized in that: For the path generated by the Dijkstra algorithm, the B-spline curve algorithm is used to smooth the acute angles generated by the path.

9. The multi-source vision-assisted unmanned aerial vehicle autonomous navigation system according to claim 1, characterized in that: The formula of the path smoothing algorithm is: Among them, C(u) is a point on the curve, the parameter u varies within the range of the knot vector, and P i is a control point, and N i,k (u) is the i-th B-spline basis function of degree k, n is the number of control points minus 1, and k is the degree of the B-spline.

10. The autonomous navigation system for unmanned aerial vehicle based on multi-source vision assistance according to claim 1, characterized in that: The nodes in the Dijkstra algorithm are each regular grid.

Images