Unmanned aerial vehicle flight control method and apparatus, unmanned aerial vehicle, and storage medium

Abstract

The application discloses a kind of unmanned plane flight control method, device, unmanned plane and storage medium, the method includes: obtaining IMU data, laser radar data, determine unmanned plane current position and attitude information;Obtain the depth map data of camera, convert it into point cloud data, and construct unified environment model;Based on the dynamics constraint of environment model and unmanned plane, the feasible path from current position to target point is planned out using path planning algorithm;According to the path point sequence on feasible path, generate corresponding control instruction, drive unmanned plane to fly along feasible path autonomously;Use visual image to identify target, dynamically adjust the speed and attitude of unmanned plane according to the position deviation between unmanned plane and target point, realize accurate following to target point.The above scheme significantly improves the autonomy and robustness of unmanned plane in complex dynamic environment, so that it can complete positioning, obstacle avoidance, navigation and tracking tasks stably in variable scene.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) flight control technology, specifically to a UAV flight control method, device, UAV, and storage medium. Background Technology

[0002] Currently, drones are widely used in security patrols, logistics transportation, environmental monitoring, and military reconnaissance. To achieve autonomous flight and target tracking for drones, existing technologies employ various methods for positioning. These include optical flow or motion capture, visual odometry (VIO) based on monocular or binocular cameras combined with an IMU to estimate the drone's motion state, or lidar (LIO) combined with an IMU. For obstacle avoidance, ultrasonic or infrared sensors are used for short-range obstacle avoidance, or lidar is used to build an environmental map and combine it with path planning algorithms to generate a flight path for global obstacle avoidance and navigation. Alternatively, a global map can be pre-built before obstacle avoidance planning. For tracking, deep learning models or traditional image processing algorithms are used for target identification and localization, or visual servoing methods are employed for target tracking.

[0003] However, the aforementioned existing technologies have the following problems: 1. Optical flow positioning depends on ground texture. If the ground texture is simple or the lighting conditions are poor, it is easy to fail. Although motion capture (motion capture) systems have high accuracy, they require external base stations or camera arrays, which are complex to deploy and difficult to use on a large scale in actual task environments. Positioning based on visual odometry (VIO) is affected by environmental texture, lighting conditions and other factors, and is prone to drift or failure.

[0004] 2. Short-range obstacle avoidance solutions based on ultrasonic or infrared sensors have limited detection range, only suitable for low-speed, short-range flight scenarios, and cannot meet the obstacle avoidance needs of high-speed flight in complex 3D environments. LiDAR has a limited field of view, and methods based on LiDAR-built maps for obstacle avoidance have potential misplanning risks. LiDAR cannot scan the area diagonally below it, which may lead to misjudgments of feasible space during path planning, resulting in collisions. Methods based on pre-built global maps lack environmental adaptability; once the environment changes, the planned results may not match the actual environment, thus reducing the reliability of obstacle avoidance.

[0005] 3. When visual servoing methods are tracking targets, if there are obstacles between the UAV and the target, the UAV cannot effectively perceive and avoid the obstacles, which may lead to collisions during the tracking process. Existing methods are mostly based on a single visual mode and lack fusion with other sensors, resulting in insufficient overall robustness.

[0006] Most existing technologies only improve a single aspect of positioning, obstacle avoidance, or tracking, lacking a systematic integration mechanism; it is difficult to achieve an organic combination of high-precision positioning, stable obstacle avoidance, autonomous navigation, and dynamic tracking, which limits the application effect of UAVs in complex and dynamic environments. Summary of the Invention

[0007] In view of the above problems, the present invention is proposed to provide a method, apparatus, drone, and storage medium for drone flight control that overcomes or at least partially solves the above problems.

[0008] According to one aspect of the present invention, a flight control method for an unmanned aerial vehicle (UAV) is provided, the method comprising: Acquire IMU data and lidar data, and determine the current position and attitude information of the UAV based on the IMU data and lidar data; Acquire depth map data from the camera, convert the depth map data into point cloud data, and construct a unified environment model based on the point cloud data and LiDAR data; Based on the environmental model and the dynamic constraints of the UAV, a path planning algorithm is used to plan a feasible path from the current position to the target point. Based on the sequence of path points on the feasible path, corresponding control commands are generated, and the UAV is driven to fly autonomously along the feasible path. When the drone reaches the target point within a preset range, it identifies the target based on the acquired camera image data and dynamically adjusts the drone's speed and attitude according to the positional deviation between the drone and the target point to achieve precise tracking of the target point.

[0009] In some implementations, acquiring IMU data and lidar data, and determining the current position and attitude information of the UAV based on the IMU data and lidar data includes: Acquire IMU data from the flight controller, pre-integrate the IMU data to predict the motion state of the UAV in the current frame, and align the IMU data with the lidar to provide initial values ​​for the UAV's positioning; Acquire the initial point cloud data of the LiDAR and extract feature points, then insert the extracted feature points into the local map structure; For each frame of laser point cloud scanned, feature points are extracted and nearest neighbor search is performed with feature points in the local map structure to obtain matching constraints. The pose changes of the UAV are obtained by tightly coupling and optimizing the pre-integrated data and feature point data of the IMU, and then transformed into the world coordinate system to obtain the current position of the UAV.

[0010] In some implementations, feature points are extracted for each scanned frame of the laser point cloud, and nearest neighbor searches are performed with feature points in the local map structure to obtain matching constraints, including: Find the corner points in the local map structure's corner feature set that are closest to the corner feature points of the new frame; Find the nearest planar points to the planar feature points in the new frame from the planar feature set of the local map; For planar features, a straight line model is constructed by fitting the nearest corner points, and the distance from the corner points of the new frame to this straight line model is determined as the constraint residual; for planar features, a planar model is constructed by fitting the nearest planar points, and the distance from the planar points of the new frame to this planar model is used as the constraint residual. The aforementioned constraint residuals are used as matching constraints for the LiDAR front-end registration, and are then used for subsequent pose optimization.

[0011] In some implementations, acquiring depth map data from a camera, converting the depth map data into point cloud data, and constructing a unified environment model based on the point cloud data and LiDAR data includes: The depth map is converted into a depth point cloud, and the depth point cloud is fused with LiDAR data to obtain a unified environment model. Using the point cloud in the environmental model as the center and half the maximum length of the drone as the radius, a spherical range is obtained. The drone is then used to perform obstacle avoidance tests based on this spherical range.

[0012] In some embodiments, the method further includes: The detection of obstacles or target points is performed based on the RGB image of the camera, specifically including: Receive RGB image data recorded by a camera, and perform preprocessing on the RGB images including at least one of the following: size scaling, normalization, or color space conversion. The preprocessed RGB image is input into the target detection model, which includes a backbone feature extraction network, a feature fusion network, and a multi-scale detection head, enabling the identification of obstacles or target points in the RGB image. The detection results are sent to the local image structure in message format.

[0013] In some embodiments, the method further includes: Calculating the distance between the drone and an obstacle or target point based on the camera includes at least one of the following methods: calculating the distance using the pinhole imaging principle of a monocular camera; or calculating the distance between the two lenses in a binocular camera; or calculating the distance using pixel coordinates from a monocular depth map; and / or, Set the target point location in the visual interface so that the drone can fly to the desired location even if the target point is not found; and / or, In an environment where the map and the location of the target point are known, a flight path is planned using a planning algorithm based on the target point and the map, and the drone is controlled to reach the target point according to the flight path.

[0014] In some embodiments, the method further includes: If the target point is in motion, its position is dynamically updated, triggering a path planning process to continuously generate new collision-free paths, ensuring the UAV can stably approach and track the moving target; and / or, The following data can be exchanged through the ROS communication mechanism: motion control commands for the flight control system; and / or target status feedback information for the mission layer.

[0015] According to another aspect of the present invention, a flight control device for an unmanned aerial vehicle (UAV) is provided, the device comprising: The current pose determination module is adapted to acquire IMU data and lidar data, and determine the current position and attitude information of the UAV based on the IMU data and lidar data. The environment model building module is suitable for acquiring depth map data from the camera, converting the depth map data into point cloud data, and building a unified environment model based on the point cloud data and lidar data. The feasible path planning module is suitable for planning feasible paths from the current position to the target point based on environmental models and the dynamic constraints of UAVs. The autonomous flight control module is suitable for generating corresponding control commands based on the sequence of waypoints on a feasible path, and driving the UAV to fly autonomously along the feasible path; The target visual tracking module is suitable for recognizing the target based on the acquired camera image data when the drone arrives within a preset range of the target point, and dynamically adjusting the speed and attitude of the drone based on the positional deviation between the drone and the target point to achieve precise tracking of the target point.

[0016] According to another aspect of the present invention, a drone is provided, comprising: a processor and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform a drone flight control method according to any one of the above embodiments.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores one or more programs, which, when executed by a processor, implement the unmanned aerial vehicle flight control method according to any one of the above.

[0018] As can be seen from the above, the UAV flight control scheme disclosed in the embodiments of the present invention can achieve the following beneficial effects: 1. To improve the accuracy and safety of obstacle avoidance, a method combining LiDAR positioning and depth map obstacle avoidance is adopted. This approach not only utilizes LiDAR for high-precision positioning and environmental mapping but also supplements the LiDAR's blind spots (especially the lower area) with depth map information. This solution effectively avoids the path planning errors caused by insufficient coverage in the lower area of ​​traditional LiDAR, thus ensuring that the UAV can safely avoid obstacles in complex 3D environments and reducing the risk of collisions.

[0019] 2. Enhancing the flexibility and adaptability of target point calculation, this invention provides multiple target point acquisition methods. When the target is far away, the target point can be set manually or through a visual interface. After the target is identified, the target center point can be automatically calculated as the target point according to different task requirements, using monocular, binocular, or monocular combined with depth map methods. This improvement enables the UAV to flexibly switch target point calculation methods according to different application scenarios, improving task adaptability, and is especially suitable for working in environments with complex lighting, sparse textures, or diverse target types.

[0020] 3. Achieving an organic combination of global safety and local precision: After detecting a target, this invention first calculates a safe path avoiding obstacles through a path planning module, enabling the UAV to approach the target point globally. Once the UAV enters the vicinity of the target point, it switches to visual servo mode, using visual detection to track the target in real time, achieving precise target tracking. This solution avoids the unsafe flight path problems that may result from relying solely on visual servoing, achieving an organic combination of global safety and local precision.

[0021] 4. A closed-loop control system is formed to improve robustness and dynamic response capabilities. This invention establishes a closed-loop control system through an overall design combining multimodal sensor fusion, dynamic path planning, and visual servo tracking. When the target moves or the environment changes, the system can update the target point in real time and replan the path, while simultaneously performing local corrections using visual servoing. This closed-loop mechanism significantly improves the autonomy and robustness of the UAV in complex dynamic environments, enabling it to stably complete positioning, obstacle avoidance, navigation, and tracking tasks in varied scenarios.

[0022] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0023] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating a drone flight control method according to an embodiment of the present invention is shown; Figure 2(a) shows a schematic diagram illustrating the principle of distance estimation using a monocular camera according to an embodiment of the present invention; Figure 2(b) shows a schematic diagram illustrating the principle of calculating the X-direction coordinates of a target using a monocular camera according to an embodiment of the present invention; Figure 3 A schematic diagram illustrating the principle of distance estimation using a binocular camera according to an embodiment of the present invention is shown; Figure 4 A schematic diagram of a drone flight control device according to an embodiment of the present invention is shown; Figure 5 A schematic diagram of the structure of a drone (controller, flight controller) according to an embodiment of the present invention is shown; Figure 6 A data flow diagram for positioning, navigation, and tracking using multi-sensor fusion according to an embodiment of the present invention is shown; Figure 7 The diagram shows the result of path planning after obtaining the three-dimensional position of the target point according to an embodiment of the present invention. Detailed Implementation

[0024] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0025] Terminology Explanation: ROS: Robot Operating System: An open-source software framework for robot development that provides message communication mechanisms, device drivers, algorithm libraries, and visualization tools. It is widely used in the research and development of autonomous systems such as drones and robots.

[0026] IMU: Inertial Measurement Unit: A sensor consisting of accelerometers, gyroscopes (sometimes including magnetometers) used to measure the acceleration, angular velocity, and attitude information of a UAV, and is an important data source for flight control and positioning.

[0027] MAVROS: A bridging node between ROS and the MAVLink protocol, serving as communication middleware between UAV flight controllers (PX4 / ArduPilot, etc.) and the ROS system. Through MAVROS, users can publish or subscribe to topics in ROS to achieve functions such as flight controller parameter setting, status monitoring, and mission control (position, speed, attitude, etc.).

[0028] Figure 1 A flowchart illustrating a drone flight control method according to an embodiment of the present invention is shown, the method comprising the following steps: Step S110: Acquire IMU data and LiDAR data, and determine the current position and attitude information of the UAV based on the IMU data and LiDAR data; Step S120: Obtain depth map data from the camera, convert the depth map data into point cloud data, and construct a unified environment model based on the point cloud data and lidar data; Step S130: Based on the environmental model and the dynamic constraints of the UAV, a path planning algorithm is used to plan a feasible path from the current position to the target point; Step S140: Based on the sequence of path points on the feasible path, generate corresponding control commands and drive the UAV to fly autonomously along the feasible path; Step S150: When the drone reaches the preset range of the target point, the target is identified based on the acquired camera image data. The speed and attitude of the drone are dynamically adjusted according to the positional deviation between the drone and the target point to achieve precise tracking of the target point. The camera image data includes RGB images and / or depth images from a monocular camera. The target point may include the target object itself, the target location, or a reference object near the target location.

[0029] The UAV flight control method disclosed in the above embodiments proposes a positioning and obstacle avoidance navigation scheme that combines laser positioning with depth mapping. High-precision global positioning and mapping are achieved through laser radar, while depth images generated by a depth camera supplement obstacle information, especially in areas below where the laser radar's field of view is insufficient. This method solves the blind spot problem of laser radar in local environmental perception and avoids misjudgments caused by missing information about the lower environment during path planning, thereby significantly improving the obstacle avoidance safety and environmental adaptability of the UAV.

[0030] In some embodiments, obtaining IMU data and lidar data in step S110, and determining the current position and attitude information of the UAV based on the IMU data and lidar data, includes: Acquire IMU data from the flight controller, pre-integrate the IMU data to predict the motion state of the UAV in the current frame, and align the IMU data with the lidar to provide initial values ​​for the UAV's positioning; Acquire the initial point cloud data of the LiDAR and extract feature points, then insert the extracted feature points into the local map structure; For each frame of laser point cloud scanned, feature points are extracted and nearest neighbor search is performed with feature points in the local map structure to obtain matching constraints. The pose changes of the UAV are obtained by tightly coupling and optimizing the pre-integrated data and feature point data of the IMU, and then transformed into the world coordinate system to obtain the current position of the UAV.

[0031] In some embodiments, feature points are extracted once for each scanned frame of laser point cloud, and nearest neighbor search is performed with feature points in the local map structure to obtain matching constraints, including: Find the corner points in the local map structure's corner feature set that are closest to the corner feature points of the new frame; Find the nearest planar points to the planar feature points in the new frame from the planar feature set of the local map; For planar features, a straight line model is constructed by fitting the nearest corner points, and the distance from the corner points of the new frame to this straight line model is determined as the constraint residual; for planar features, a planar model is constructed by fitting the nearest planar points, and the distance from the planar points of the new frame to this planar model is used as the constraint residual. The aforementioned constraint residuals are used as matching constraints for the LiDAR front-end registration, and are then used for subsequent pose optimization.

[0032] In some embodiments, step S120 involves acquiring depth map data from the camera, converting the depth map data into point cloud data, and constructing a unified environment model based on the point cloud data and LiDAR data, including: The depth map is converted into a depth point cloud, and the depth point cloud is fused with LiDAR data to obtain a unified environment model. Using the point cloud in the environmental model as the center and half the maximum length of the drone as the radius, a spherical range is obtained. The drone is then used to perform obstacle avoidance tests based on this spherical range.

[0033] In some embodiments, the method further includes: The detection of obstacles or target points is performed based on the RGB image of the camera, specifically including: Receive RGB image data recorded by a camera, and perform preprocessing on the RGB images including at least one of the following: size scaling, normalization, or color space conversion. The preprocessed RGB image is input into the target detection model, which includes a backbone feature extraction network, a feature fusion network, and a multi-scale detection head, enabling the identification of obstacles or target points in the RGB image; the design and selection of bounding boxes in the specific identification process are described in the following specific embodiments. The detection results are sent to the local image structure in message format.

[0034] In some embodiments, the method further includes: The distance between the drone and the obstacle or target point is calculated based on the camera, including at least one of the following methods: combining Figure 2 and Figure 3 As shown, the distance between the UAV and the obstacle or target point can be calculated using the pinhole imaging principle of a monocular camera, or by using the distance between the two lenses in a binocular camera, or by using the pixel coordinates in a monocular depth map. The calculation process for the monocular and binocular cameras is described in the specific embodiments below.

[0035] The method of this invention further includes: setting the target point location on a visual interface, so that the drone can fly to the desired location when the target point is not found; and / or, The method of this embodiment further includes: in an environment where the map and the location of the target point are known, using a planning algorithm to plan a flight path based on the target point and the map, and controlling the UAV to reach the target point according to the flight path.

[0036] In some embodiments, the method further includes: If the target point is in motion, its position is dynamically updated, triggering a path planning process to continuously generate new collision-free paths, ensuring the drone can stably approach and track the moving target. Specifically, when the drone approaches the target point, it switches to visual servo mode. The visual detection module identifies and tracks the target in real time, acquiring its motion information (position, speed, direction). The control module dynamically adjusts the drone's speed and attitude based on the position deviation to achieve precise target tracking. If the target moves, the target point is recalculated and sent to the obstacle avoidance module to trigger new path planning, forming a closed-loop control process.

[0037] It should be noted that the embodiments of the present invention form a closed-loop control system through the overall design of multimodal sensor fusion, dynamic path planning and visual servo tracking. The modules interact with each other through the ROS communication mechanism to achieve the following data interaction: motion control commands for the flight control system; and / or, target status feedback information for the mission layer.

[0038] In summary, the method provided by the embodiments of the present invention has the following advantages compared with the prior art: Improvements in Localization and Obstacle Avoidance: Existing technologies mostly rely on single sensors, such as LiDAR alone for localization and obstacle avoidance. This invention proposes an obstacle avoidance method combining LiDAR localization and depth mapping. High-precision global localization and mapping are achieved through LiDAR, while depth images generated by a depth camera supplement obstacle information, especially in areas below where the LiDAR's field of view is insufficient. This method solves the blind spot problem of LiDAR in local environmental perception, avoids misjudgments caused by missing information about the lower environment during path planning, and thus significantly improves the obstacle avoidance safety and environmental adaptability of the UAV.

[0039] The diversity and flexibility of target point calculation: Existing technologies typically employ a single fixed scheme for target localization. This invention, however, proposes multiple target point calculation methods based on the characteristics of different application scenarios: In scenarios requiring lightweight design and low cost, a monocular camera combined with depth information can be used to estimate the target position; in scenarios requiring higher precision, binocular vision can be used for target point calculation; alternatively, depth maps can be combined with monocular vision for fusion calculation, improving robustness under weak texture or complex lighting conditions. It also supports user selection of the UAV's target position within the rviz (RobotVisualization) visualization interface. This diverse range of target point calculation methods allows the UAV to flexibly select appropriate target detection and localization methods according to different mission requirements and environmental conditions, enhancing the system's adaptability and versatility.

[0040] A Fusion Mechanism for Path Planning and Visual Servoing: In existing technologies, visual servoing methods typically adjust the UAV's flight trajectory directly based on the target's positional deviation in the image, easily overlooking obstacles in the environment and thus posing potential collision risks. This invention establishes a fusion mechanism between path planning and visual servoing: First, a safe and feasible path is generated using the path planning module, allowing the UAV to approach the target point globally; after entering the vicinity of the target point, visual servoing control is switched to achieve precise target tracking. This fusion mechanism balances the global safety of path planning with the local accuracy of visual servoing, not only improving the UAV's target tracking capability but also effectively avoiding the collision risks that easily occur in visual servoing methods.

[0041] System integrity and closed-loop operation: This invention is not a single improvement to positioning, obstacle avoidance, or tracking, but rather a holistic solution that achieves a closed-loop control system through multimodal sensor fusion, dynamic path planning, and visual servo tracking. When the target moves or the environment changes, the system can dynamically update the target point, replan the path, and make real-time corrections through visual servoing in local stages, thereby ensuring the robustness and autonomy of the UAV in complex dynamic environments.

[0042] The above embodiments will be described in detail below through a specific example: This specific embodiment discloses a multimodal sensor fusion positioning, obstacle avoidance, autonomous navigation, and tracking scheme for unmanned aerial vehicles (UAVs), combining... Figure 6 As shown, the system can be executed through the following modules: localization module, obstacle avoidance module, detection module, and control module. Each module can run independently and communicates data via the ROS communication mechanism. The following sections will elaborate on each module in turn: The features of the positioning module include: 1.1 The localization module first receives flight control IMU data, then performs pre-integration on the IMU data to predict the robot's motion state in the current frame, and aligns the IMU data with the LiDAR to provide an initial value for localization.

[0043] 1.2 After acquiring the initial point cloud data collected by the Livox Mid-360 LiDAR, feature point extraction is performed on the point cloud; wherein, the feature points are planar feature points, corner feature points, or curvature feature points extracted from the point cloud. The feature extraction method includes: based on the curvature calculation of the local neighborhood of the point cloud, marking points with high curvature as corner features, and marking points with low curvature and continuous distribution as planar feature points; subsequently, the extracted feature points of various types are inserted into the local map structure in chronological order for subsequent matching and pose optimization.

[0044] 1.3 For each newly acquired LiDAR point cloud frame, the localization module extracts feature points and performs a nearest neighbor search between the new frame feature points and the feature points already stored in the local map structure; wherein, the nearest neighbor search includes: 1.3.1 For the corner feature points of the new frame, find the nearest corner points in the corner feature set of the local map; for the planar feature points of the new frame, find the nearest planar points in the planar feature set of the local map.

[0045] 1.3.2 Construct matching constraints based on the above nearest neighbor point set. For corner point features, by fitting a straight line model formed by the nearest neighbor corner points, the distance from the new frame corner point to the straight line is used as the constraint residual; for plane features, by fitting a plane model formed by the nearest neighbor plane points, the distance from the new frame plane point to the plane is used as the constraint residual.

[0046] 1.3.3 The point-to-line and point-to-surface residuals obtained from the above two steps are used as matching constraints for the LiDAR front-end registration and are used for subsequent pose optimization.

[0047] 1.4 The pre-integrated data and feature point data of the IMU are tightly coupled and optimized to obtain the current pose change, which includes the UAV's position change and attitude change. Then, the pose change is transformed into the world coordinate system to obtain the current position, with the origin of the world coordinate system being the position when the localization algorithm is enabled.

[0048] The obstacle avoidance module features include: The principle behind converting a depth map into a point cloud is as follows: (1) (2) (3) Where X, Y, and Z represent the three-dimensional coordinates of a pixel in the depth map within the camera coordinate system. The camera coordinate system is a three-dimensional coordinate system established with the camera optical center as the origin, the Z-axis along the positive direction of the camera optical axis, the X-axis to the right, and the Y-axis downwards. (u, v) represents the coordinates of the pixel. , Indicates the camera's focal length. , Represents the coordinates of the principal point of the optical center.

[0049] 2.2 Once the point cloud is obtained, it can be used as an obstacle. To prevent the drone from colliding with the obstacle, the point cloud is expanded to half the size of the drone. This effectively ensures that the drone does not collide with the obstacle.

[0050] 2.3 After obtaining the target point's location, the obstacle avoidance module can plan an optimal path in real time and divide the optimal path into multiple path points. The UAV sends these path points sequentially to the control module for flight control. The target point's location can be set manually or manually by clicking on it in the rviz (Robot Visualization) visual interface. More importantly, the target point's location can be automatically calculated by the target detection module in conjunction with the control module.

[0051] The features of the target detection module include: 3.1 The target detection module first receives image data from the UAV's onboard camera. After receiving the image data, it performs preprocessing on the image, including size scaling, normalization, and color space conversion, to meet the input requirements of the deep learning network.

[0052] 3.2 Preprocessed image input target detection model, the model includes a backbone feature extraction network, a feature fusion network and a multi-scale detection head, which can realize the rapid identification of target objects in the input image.

[0053] 3.3 The model outputs the target category information and its bounding box position in the image coordinate system. The bounding box is represented by the center point coordinates and width and height (x, y, w, h), along with the target confidence score. To improve the stability of the detection results, non-maximum suppression (NMS) is applied to the candidate targets output by the model to filter out overlapping and redundant bounding boxes, retaining only the detection results with the highest confidence score.

[0054] 3.4 The final detection results are sent to the control module in a custom message format for target point calculation.

[0055] The features of the control module include: 4.1 The control module first receives the current position and attitude information of the UAV output by the positioning module through the ROS communication mechanism, and sends the information to the flight control system in real time as the basis for the UAV's status feedback.

[0056] 4.2 The control module receives the target category and the target's position in the image coordinate system or world coordinate system from the target detection module. Combined with the UAV's current position, it calculates the desired target point position information and sends this target point to the obstacle avoidance module. The control module calculates the UAV's target point using five methods: 4.2.1 Monocular Target Point Calculation. Referring to Figure 2, for a fixed, regularly sized object, a monocular image can accurately calculate the position of the target's center point. Based on the pinhole imaging principle, after the object is mapped onto the camera through the camera's optical center, the object and the camera can form similar triangles through their optical centers. Therefore, the formula for calculating the target point is: (4) (5) (6) 4.2.2 Target point calculation for binocular cameras. (Combined with...) Figure 3 As shown, assuming the aircraft is equipped with a stereo camera, the position of the target's center point can be calculated using the stereo camera. For an object, it is necessary to calculate the distance between the pixel coordinates of the target's center point in the left eye and the center point itself, and the distance between the pixel coordinates of the target's center point in the right eye and the center point itself. This is then derived using similar triangles.

[0057] (7) in Indicates the baseline between the two eyes, and This represents the horizontal distance from the center point of the target in the binocular vision, which is represented by the pixel coordinates of the center point. Indicates focal length. Represents depth. Equation (7) is transformed to obtain... (8) After obtaining the depth of the object, the three-dimensional coordinates (X and Y) of point P relative to the left camera can be obtained.

[0058] (9) 4.2.3 Target point calculation using a monocular depth map. For solid objects, the target center point can be detected in real time using a deep learning model, and the pixel coordinates of this center point can be aligned to the depth map. This method allows for the real-time determination of the target's 3D position relative to the depth camera.

[0059] 4.2.4 When encountering complex environments, it is necessary to set the target point location in the visualization interface so that the drone can fly to the desired location even if the target is not found.

[0060] 4.2.5 When added to an environment with a known map and a known target location, a preset path can be planned using preset target points, enabling the UAV to fly along the preset path.

[0061] 4.3 The control module receives the sequence of local collision-free path points returned by the obstacle avoidance module, converts the path points into control commands that the UAV can execute, and sends them to the flight control system so that the UAV gradually approaches the target along the planned path.

[0062] 4.4 When the UAV approaches the target area, the control module switches to visual servo mode. By receiving the detection results from the target detection module in real time, it generates speed or attitude control commands based on image errors (such as the deviation between the target center point and the image center) to directly drive the UAV to autonomously track the target.

[0063] 4.5 If the target is in motion, the control module will dynamically update the target position and re-trigger the path planning process. Combining the results of the positioning module and the obstacle avoidance module, it will continuously generate new collision-free path points to ensure that the UAV can stably approach and track the moving target.

[0064] 4.6 The output of the control module includes two types: one is the low-level motion control command for the flight control system; the other is the target status feedback information for the mission layer. Both of them interact with other modules through the ROS communication mechanism.

[0065] See also Figure 6 In this embodiment, each module performs the following steps: Step 1: The drone is equipped with a LiDAR and an inertial measurement unit, and achieves autonomous localization in the environment through multi-sensor data fusion. Simultaneously, depth data collected by the depth camera is converted into point cloud data to construct obstacle distribution information.

[0066] Step 2: The target point for the drone's flight can come from multiple sources. In a known environment, the target can be reached by pre-setting a target point; the target point can also be specified in the visual interface; finally, a visual detection module (such as a deep learning-based target detection algorithm) can identify and detect targets in the environment. The detection results are combined with the camera's intrinsic and extrinsic parameters to calculate the target's two-dimensional image coordinates and depth information in the camera coordinate system. Through coordinate transformation, the target position is mapped to the drone's body coordinate system to obtain the target point's three-dimensional spatial coordinates. These target point coordinates are then used as input for the desired destination in the navigation task.

[0067] The target point can be calculated using the object's own choice of a 3D calculation method for the target center.

[0068] 1. For objects with regular shapes and known sizes, according to the pinhole imaging principle (see Figures 2(a) and 2(b)), the image height of an object with height H in the camera is L. The distance from the center point of the image in the X direction to the center of the pixel coordinate system is xc, and the distance in the Y direction to the center of the pixel coordinate system is yc. Then, according to the principle of similar triangles, we can use the ratio between the sides to obtain: (10) (11) (12) 2. For objects with regular shapes, please refer to the schematic diagram. Figure 3 L represents the optical center of the left camera, R represents the optical center of the right camera, f represents the focal length, b represents the camera baseline (the distance between the optical centers of the left and right cameras), and P represents the center point of the object. This represents the distance in the X direction between the imaging point of P on the left camera and the center of the image. Z represents the distance in the X direction between the imaging point of point P on the right camera and the center of the image, and Z is the depth of point P. Suppose and Let the distance be d, where d is the disparity between the two pixels. Then we have Therefore, according to the similarity of triangles, we have: (13) After conversion, we can obtain... (14) After obtaining the depth of the object, the three-dimensional coordinates (X and Y) of point P relative to the left camera can be obtained.

[0069] (15) in, , These represent the pixel coordinates of point P in the left camera. , , This indicates the internal parameters of the left camera.

[0070] If the object is solid, the pixel coordinates of the RGB image of the object's center point can be obtained through object detection. Next, the RGB image is aligned with the depth image to obtain the depth Z of the object's center point, and then the X and Y coordinates are obtained.

[0071] (16) in, , , This represents the intrinsic parameters of the depth camera.

[0072] The 3D coordinates obtained through vision are based on the camera coordinate system. Therefore, we need to first transform them to the body coordinate system, and then to the global coordinate system. Assume the object's position in the camera coordinate system is X, and we know the rotation matrix from the left camera coordinate system to the body coordinate system. Translation matrix Then the position Y of the object in the body coordinate system can be represented as (17) After converting to the body coordinate system, it is necessary to convert to the global coordinate system, as shown in the following formula.

[0073] (18) Where W represents the world coordinate system, This represents the rotation matrix used to transform the body coordinate system to the world coordinate system. This represents the translation matrix used to transform the machine coordinate system to the world coordinate system.

[0074] Step 3: Input the target point obtained in Step 2 into the obstacle avoidance module. The obstacle avoidance module uses the LiDAR positioning information and the obstacle point cloud generated by the depth camera to construct a unified environment model. Based on the environment model and the dynamic constraints of the UAV, a path planning algorithm is used to calculate the feasible path from the current position to the target point. During the path planning process, obstacles are automatically avoided. The generated path point sequence is sent to the UAV's control module, which then uses MAVROS or other control algorithms to control the aircraft. See [link to obstacle avoidance module path generation]. Figure 7 .

[0075] Step 4: The control module receives the path point sequence output by the obstacle avoidance module and generates corresponding control commands, including position control commands or speed control commands, based on the path point sequence to drive the UAV to fly autonomously along the planned path. During flight, the positioning module continuously updates the UAV's attitude to ensure that the flight path remains consistent with the planned path. If the deviation between the path point and the actual flight trajectory exceeds the limit, the control module performs trajectory correction.

[0076] Step 5: When the drone approaches the target point, switch to visual servo mode. The visual detection module identifies and tracks the target in real time, acquiring the target's motion information (position, speed, direction). The control module dynamically adjusts the drone's speed and attitude based on the position deviation to achieve precise target following. If the target moves, the target point is recalculated and sent to the obstacle avoidance module to trigger new path planning, forming a closed-loop control process.

[0077] In summary, the key points and beneficial effects of the embodiments of the present invention include: 1. Improve the accuracy and safety of obstacle avoidance. This approach combines lidar positioning with depth map obstacle avoidance. LiDAR not only enables high-precision positioning and environmental mapping, but also supplements the lidar's blind spots (especially the lower regions) with depth maps. This solution effectively avoids the path planning errors caused by insufficient coverage in the lower regions, a problem inherent in traditional lidar systems. This ensures the UAV can safely avoid obstacles in complex 3D environments, reducing the risk of collisions.

[0078] 2. Enhance the flexibility and adaptability of target point calculation. This invention provides multiple methods for acquiring target points. When the target is far away, the target point can be manually specified or set through a visual interface. After the target is identified, the target center point can be automatically calculated as the target point by selecting monocular, binocular, or monocular combined with depth map methods according to different task requirements. This improvement enables the UAV to flexibly switch target point calculation methods according to different application scenarios, enhancing task adaptability, and is especially suitable for working in environments with complex lighting, sparse textures, or diverse target types.

[0079] 3. Achieving an organic combination of global security and local precision. After detecting a target, this embodiment of the invention first calculates a safe path that avoids obstacles through a path planning module, enabling the UAV to approach the target point globally. Once the UAV enters the area near the target point, it switches to visual servo mode, using visual detection to track the target in real time, achieving precise target tracking. This solution avoids the unsafe flight path problems that may result from relying solely on visual servo, achieving an organic combination of global safety and local accuracy.

[0080] 4. Form a closed-loop control system to improve robustness and dynamic response capability. This invention employs a holistic design combining multimodal sensor fusion, dynamic path planning, and visual servo tracking to form a closed-loop control system. When the target moves or the environment changes, the system can update the target point in real time and replan the path, while simultaneously performing local corrections using visual servoing. This closed-loop mechanism significantly improves the autonomy and robustness of the UAV in complex dynamic environments, enabling it to stably complete positioning, obstacle avoidance, navigation, and tracking tasks in varied scenarios.

[0081] Combination Figure 4 As shown, the present invention also protects a drone flight control device 400, the drone flight control device 400 comprising: The current pose determination module 410 is adapted to acquire IMU data and lidar data, and determine the current position and attitude information of the UAV based on the IMU data and lidar data. The environment model building module 420 is adapted to acquire depth map data from the camera, convert the depth map data into point cloud data, and build a unified environment model based on the point cloud data and lidar data. The feasible path planning module 430 is suitable for planning feasible paths from the current position to the target point based on environmental models and the dynamic constraints of UAVs, using path planning algorithms. The autonomous flight control module 440 is adapted to generate corresponding control commands based on the sequence of waypoints on a feasible path and drive the UAV to fly autonomously along the feasible path. The target visual following module 450 is suitable for identifying the target point based on the acquired camera image data when the drone arrives within a preset range of the target point, and dynamically adjusting the speed and attitude of the drone based on the positional deviation between the drone and the target point to achieve precise tracking of the target point.

[0082] Based on the UAV flight control device in this embodiment, a positioning and obstacle avoidance navigation scheme combining laser positioning and depth mapping is proposed. High-precision global positioning and mapping are achieved through laser radar, while depth images generated by a depth camera supplement obstacle information, especially in areas below where the laser radar's field of view is insufficient. This solves the blind spot problem of laser radar in local environmental perception and avoids misjudgments caused by missing information about the lower environment during path planning, thereby significantly improving the obstacle avoidance safety and environmental adaptability of the UAV.

[0083] In some embodiments, the current pose determination module 410 is specifically adapted to: Acquire IMU data from the flight controller, pre-integrate the IMU data to predict the motion state of the UAV in the current frame, and align the IMU data with the lidar to provide initial values ​​for the UAV's positioning; Acquire the initial point cloud data of the LiDAR and extract feature points, then insert the extracted feature points into the local map structure; For each frame of laser point cloud scanned, feature points are extracted and nearest neighbor search is performed with feature points in the local map structure to obtain matching constraints. The pose changes of the UAV are obtained by tightly coupling and optimizing the pre-integrated data and feature point data of the IMU, and then transformed into the world coordinate system to obtain the current position of the UAV.

[0084] In some embodiments, feature points are extracted once for each scanned frame of laser point cloud, and nearest neighbor search is performed with feature points in the local map structure to obtain matching constraints, including: Find the corner points in the local map structure's corner feature set that are closest to the corner feature points of the new frame; Find the nearest planar points to the planar feature points in the new frame from the planar feature set of the local map; For planar features, a straight line model is constructed by fitting the nearest corner points, and the distance from the corner points of the new frame to this straight line model is determined as the constraint residual; for planar features, a planar model is constructed by fitting the nearest planar points, and the distance from the planar points of the new frame to this planar model is used as the constraint residual. The aforementioned constraint residuals are used as matching constraints for the LiDAR front-end registration, and are then used for subsequent pose optimization.

[0085] In some embodiments, the environment model building module 420 is specifically adapted to: The depth map is converted into a depth point cloud, and the depth point cloud is fused with LiDAR data to obtain a unified environment model. Using the point cloud in the environmental model as the center and half the maximum length of the drone as the radius, a spherical range is obtained. The drone is then used to perform obstacle avoidance tests based on this spherical range.

[0086] In some embodiments, the device is further adapted to: The detection of obstacles or target points is performed based on the RGB image of the camera, specifically including: Receive RGB image data recorded by a camera, and perform preprocessing on the RGB images including at least one of the following: size scaling, normalization, or color space conversion. The preprocessed RGB image is input into the target detection model, which includes a backbone feature extraction network, a feature fusion network, and a multi-scale detection head, enabling the identification of obstacles or target points in the RGB image. The detection results are sent to the local image structure in message format.

[0087] In some embodiments, the device is further adapted to: Calculating the distance between the drone and an obstacle or target point based on the camera includes at least one of the following methods: calculating the distance using the pinhole imaging principle of a monocular camera; or calculating the distance between the two lenses in a binocular camera; or calculating the distance using pixel coordinates from a monocular depth map; and / or, Set the target point location in the visual interface so that the drone can fly to the desired location even if the target point is not found; and / or, In an environment where the map and the location of the target point are known, a flight path is planned using a planning algorithm based on the target point and the map, and the drone is controlled to reach the target point according to the flight path.

[0088] In some embodiments, the device is further adapted to: If the target point is in motion, its position is dynamically updated, triggering a path planning process to continuously generate new collision-free paths, ensuring the UAV can stably approach and track the moving target; and / or, The following data can be exchanged through the ROS communication mechanism: motion control commands for the flight control system; and / or target status feedback information for the mission layer.

[0089] It should be noted that the specific implementation methods of the above-mentioned device embodiments can be referred to the specific implementation methods of the corresponding methods described above, and will not be repeated here.

[0090] It should be noted that: The algorithms and displays provided herein are not inherently related to any particular computer, virtual device, or other equipment. Various general-purpose devices can also be used in conjunction with the teachings herein. The required structure for constructing such devices is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0091] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0092] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.

[0093] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0094] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the invention and form different embodiments.

[0095] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the UAV flight control device according to embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0096] This invention provides a non-volatile computer storage medium storing at least one executable instruction that can execute the UAV flight control method described in any of the above method embodiments.

[0097] Figure 5 The diagram shows a structural schematic of an embodiment of the UAV of the present invention. The UAV may be a vertical take-off and landing fixed-wing UAV. The specific embodiments of the present invention do not limit the specific structure of the UAV.

[0098] like Figure 5 As shown, the UAV (including controller, control node or flight controller) may include: processor 502, communication interface 504, memory 506, and communication bus 508.

[0099] The processor 502, communication interface 504, and memory 506 communicate with each other via communication bus 508. Communication interface 504 is used to communicate with other network elements such as clients or other servers. Processor 502 executes program 510, specifically performing the relevant steps in the above-described UAV flight control method embodiment.

[0100] Specifically, program 510 may include program code that includes computer operation instructions.

[0101] Processor 502 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in the drone may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0102] Memory 506 is used to store program 510. Memory 506 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0103] Specifically, program 510 can be used to cause processor 502 to perform the operations corresponding to the above-described UAV flight control method embodiments.

[0104] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

Claims

1. A method for controlling the flight of an unmanned aerial vehicle (UAV), characterized in that, The method includes: Acquire IMU data and lidar data, and determine the current position and attitude information of the UAV based on the IMU data and lidar data; Acquire depth map data from the camera, convert the depth map data into point cloud data, and construct a unified environment model based on the point cloud data and LiDAR data; Based on the environmental model and the dynamic constraints of the UAV, a path planning algorithm is used to plan a feasible path from the current position to the target point. Based on the sequence of path points on the feasible path, corresponding control commands are generated, and the UAV is driven to fly autonomously along the feasible path. When the drone reaches the preset range of the target point, the target is identified based on the acquired camera image data. The speed and attitude of the drone are dynamically adjusted according to the positional deviation between the drone and the target point to achieve precise tracking of the target point. The process of acquiring IMU data and lidar data, and determining the current position and attitude information of the UAV based on the IMU data and lidar data, includes: Acquire IMU data from the flight controller, pre-integrate the IMU data to predict the motion state of the UAV in the current frame, and align the IMU data with the lidar to provide initial values ​​for the UAV's positioning; Acquire the initial point cloud data of the LiDAR and extract feature points, then insert the extracted feature points into the local map structure; For each frame of laser point cloud scanned, feature points are extracted and nearest neighbor search is performed with feature points in the local map structure to obtain matching constraints. The pose changes of the UAV are obtained by tightly coupling and optimizing the pre-integrated data and feature point data of the IMU, and then transformed into the world coordinate system to obtain the current position of the UAV. In each scan of a frame of laser point cloud, feature points are extracted and nearest neighbor searches are performed with feature points in the local map structure to obtain matching constraints, including: Find the corner points in the local map structure's corner feature set that are closest to the corner feature points of the new frame; Find the nearest planar points to the planar feature points in the new frame from the planar feature set of the local map; For planar features, a straight line model is constructed by fitting the nearest corner points, and the distance from the corner points of the new frame to this straight line model is determined as the constraint residual; for planar features, a planar model is constructed by fitting the nearest planar points, and the distance from the planar points of the new frame to this planar model is used as the constraint residual. The aforementioned constraint residuals are used as matching constraints for the LiDAR front-end registration, and are then used for subsequent pose optimization.

2. The method according to claim 1, characterized in that, Acquiring depth map data from the camera, converting the depth map data into point cloud data, and constructing a unified environment model based on the point cloud data and LiDAR data includes: The depth map is converted into a depth point cloud, and the depth point cloud is fused with LiDAR data to obtain a unified environment model. Using the point cloud in the environmental model as the center and half the maximum length of the drone as the radius, a spherical range is obtained. The drone is then used to perform obstacle avoidance tests based on this spherical range.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Obstacle or target point detection is performed based on the camera's RGB image, specifically including: Receive RGB image data from the camera, and perform preprocessing on the RGB image including at least one of the following: size scaling, normalization, or color space conversion. The preprocessed RGB image is input into the target detection model, which includes a backbone feature extraction network, a feature fusion network, and a multi-scale detection head, enabling the identification of obstacles or target points in the RGB image. The detection results are sent to the local image structure in message format.

4. The method according to claim 1 or 2, characterized in that, The method further includes: Calculating the distance between the drone and an obstacle or target point based on the camera includes at least one of the following methods: calculating the distance using the pinhole imaging principle of a monocular camera; or, calculating the distance between the two lenses in a binocular camera; or, calculating the distance using pixel coordinates from a monocular depth map; or... Set the target location in the visual interface so that the drone can fly to the desired location even if the target location cannot be found; or, In an environment where the map and the location of the target point are known, a flight path is planned using a planning algorithm based on the target point and the map, and the drone is controlled to reach the target point according to the flight path.

5. The method according to claim 1 or 2, characterized in that, The method further includes: If the target point is in motion, its position is dynamically updated, triggering a path planning process to continuously generate new collision-free paths, ensuring the drone can stably approach and track the moving target; or, The following data are exchanged through the ROS communication mechanism: motion control commands for the flight control system; and target status feedback information for the mission layer.

6. A flight control device for unmanned aerial vehicles (UAVs), characterized in that, The device includes: The current pose determination module is adapted to acquire IMU data and lidar data, and determine the current position and attitude information of the UAV based on the IMU data and lidar data. The environment model building module is suitable for acquiring depth map data from the camera, converting the depth map data into point cloud data, and building a unified environment model based on the point cloud data and lidar data. The feasible path planning module is suitable for planning feasible paths from the current position to the target point based on environmental models and the dynamic constraints of UAVs. The autonomous flight control module is suitable for generating corresponding control commands based on the sequence of waypoints on a feasible path, and driving the UAV to fly autonomously along the feasible path; The target vision tracking module is suitable for recognizing the target based on the acquired camera image data when the drone reaches the preset range of the target point, and dynamically adjusting the speed and attitude of the drone based on the positional deviation between the drone and the target point to achieve precise tracking of the target point; The current pose determination module is also adapted to: Acquire IMU data from the flight controller, pre-integrate the IMU data to predict the motion state of the UAV in the current frame, and align the IMU data with the lidar to provide initial values ​​for the UAV's positioning; Acquire the initial point cloud data of the LiDAR and extract feature points, then insert the extracted feature points into the local map structure; For each frame of laser point cloud scanned, feature points are extracted and nearest neighbor search is performed with feature points in the local map structure to obtain matching constraints. The pose changes of the UAV are obtained by tightly coupling and optimizing the pre-integrated data and feature point data of the IMU, and then transformed into the world coordinate system to obtain the current position of the UAV. In each scan of a frame of laser point cloud, feature points are extracted and nearest neighbor searches are performed with feature points in the local map structure to obtain matching constraints, including: Find the corner points in the local map structure's corner feature set that are closest to the corner feature points of the new frame; Find the nearest planar points to the planar feature points in the new frame from the planar feature set of the local map; For planar features, a straight line model is constructed by fitting the nearest corner points, and the distance from the corner points of the new frame to this straight line model is determined as the constraint residual; for planar features, a planar model is constructed by fitting the nearest planar points, and the distance from the planar points of the new frame to this planar model is used as the constraint residual. The aforementioned constraint residuals are used as matching constraints for the LiDAR front-end registration, and are then used for subsequent pose optimization.

7. A drone, characterized in that, It includes a processor and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the unmanned aerial vehicle flight control method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs that, when executed by a processor, implement the UAV flight control method according to any one of claims 1-5.

Images