Unmanned Aerial Vehicle (UAV) contour flight system and method based on multi-sensor fusion and adaptive planning

Abstract

This invention discloses a UAV contouring flight system and method based on multi-sensor fusion and adaptive planning. The system includes a multi-sensor module for simultaneously acquiring 3D point cloud data, image data, and UAV inertial measurement data of the environment; a localization and mapping module for processing the acquired data through a tightly coupled fusion algorithm to output the UAV's real-time pose and a 3D map of the environment; an adaptive path planning module for identifying target objects based on the 3D environmental map and adaptively selecting and executing a matching path generation strategy to generate a contouring flight path that conforms to the target object's surface; and a flight control module for controlling the UAV according to the contouring flight path. This invention solves the problems of low positioning accuracy, inaccurate environmental modeling, and inability to generate high-precision 3D contouring paths for UAVs in environments lacking GPS signals, achieving efficient contouring for regular targets and high-precision contouring flight for complex targets.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) flight control technology, specifically, it relates to a UAV contour flight system and method based on multi-sensor fusion and adaptive planning. Background Technology

[0002] Unmanned aerial vehicles (UAVs) have been widely used in various fields such as agricultural plant protection, power line inspection, topographic mapping, and emergency rescue due to their advantages of flexibility, efficiency, and low cost. In many applications, UAVs are required to fly close to the surface of target objects and automatically track their outlines—a process known as "contour-following flight." For example, in power line inspection, UAVs need to fly along power lines or tower structures to capture close-up details; in agricultural plant protection, UAVs need to follow the contours of crop canopies to achieve uniform spraying. Contour-following flight is a key technology for achieving automated inspection, detection, and operations, relying on accurate environmental perception, self-positioning, and intelligent flight path planning.

[0003] Currently, there are various technical approaches to achieving contour-following flight for UAVs in terms of perception and localization. These include relying on a combination of GPS and inertial navigation, or using sensors such as vision and LiDAR for simultaneous localization and map building in environments without GPS. Path planning, on the other hand, largely depends on preset flight modes (such as fixed orbits) or general motion planning algorithms. However, these existing solutions have significant system-level limitations when dealing with complex and diverse real-world operational scenarios. First, the perception and planning modules are fragmented: the perception module often outputs a general environmental map for obstacle avoidance or display, without optimization or organization for the specific task of contour-following. This makes it difficult for the planning module to directly and efficiently extract the key information needed to generate a high-precision contour-following path. Second, the planning strategies are rigid and lack understanding of the target: existing planning algorithms cannot autonomously select the optimal generation strategy based on the geometric characteristics (such as regularity and complexity) of the target, resulting in low efficiency for regular targets and insufficient accuracy for complex targets. Summary of the Invention

[0004] The primary objective of this invention is to overcome the shortcomings and deficiencies of existing technologies and provide a drone contouring flight system based on multi-sensor fusion and adaptive planning. This system solves the problems of low positioning accuracy, inaccurate environmental modeling, and inability to generate high-precision three-dimensional contouring paths in existing technologies when drones are in environments lacking GPS signals. It achieves efficient contouring of regular targets and high-precision flight to follow complex targets.

[0005] The second objective of this invention is to provide a method for UAV contour flight based on multi-sensor fusion and adaptive planning.

[0006] The objective of this invention is achieved through the following technical solution: a UAV contour-following flight system based on multi-sensor fusion and adaptive planning, comprising:

[0007] A multi-sensor module is used to simultaneously acquire 3D point cloud data, image data, and inertial measurement data of the UAV from the environment.

[0008] The positioning and mapping module is connected to the multi-sensor module and is used to process the three-dimensional point cloud data, image data and inertial measurement data through a tightly coupled fusion algorithm to output the real-time pose and three-dimensional map of the environment of the UAV.

[0009] An adaptive path planning module, connected to the positioning and mapping module, is used to identify the target object to be operated based on the environmental 3D map, and adaptively select and execute a path generation strategy that matches the geometric features of the target object to generate a contour-following flight path that conforms to the surface of the target object.

[0010] The flight control module, connected to the adaptive path planning module, is used to control the UAV according to the contour flight path.

[0011] Preferably, the multi-sensor module includes a lidar, a vision sensor, and an inertial measurement unit.

[0012] Preferably, the localization and mapping module implements the tightly coupled fusion algorithm through an iterative error state Kalman filter, and the processing includes:

[0013] State prediction stage: Predicting the system state based on the inertial measurement data;

[0014] LiDAR update phase: The three-dimensional point cloud data is matched with the local planar features in the three-dimensional environmental map, and the predicted state is updated for the first time based on the geometric residual generated by the matching.

[0015] Visual update phase: The point cloud data used for map matching in the LiDAR update phase is used as visual map points, and the associated reference image blocks are projected onto the current image data. By minimizing the direct photometric error, the state after the first update is updated a second time.

[0016] Preferably, the adaptive path planning module includes:

[0017] An instance segmentation unit is used to process the 3D map of the environment to separate the point cloud of the target object;

[0018] An adaptive path generation unit is used to determine whether the target object's point cloud belongs to a regular geometric body or a complex freeform surface based on its geometric features: if it is determined to be a regular geometric body, a first path generation strategy is adopted; if it is determined to be a complex freeform surface, a second path generation strategy is adopted.

[0019] Preferably, the first path generation strategy includes: performing three-dimensional shape fitting on the point cloud of the target object, and generating a contour-following flight path based on the fitted simplified geometric model;

[0020] The second path generation strategy includes: performing serialized slicing, cross-sectional contour curve fitting, and 3D synthesis on the point cloud of the target object to generate a contour flight path.

[0021] Preferably, the adaptive path planning module further includes: a trajectory optimization unit, used to model the three-dimensional map of the environment based on an improved octree structure, and to parameterize the contour flight path using B-spline curves, and to jointly optimize the control point and time allocation by introducing a repulsion vector mechanism to generate a smooth and collision-free flight trajectory.

[0022] The UAV contour-following flight method based on multi-sensor fusion and adaptive planning includes the following steps:

[0023] The system simultaneously collects 3D point cloud data, image data, and inertial measurement data of the UAV from multiple sensors.

[0024] The data is processed by a tightly coupled fusion algorithm to calculate the pose of the UAV in real time and construct a 3D map of the environment.

[0025] Identify the target object to be worked on based on the aforementioned 3D environmental map;

[0026] Based on the geometric features of the target object, an adaptive path generation strategy is selected and executed to generate a contour flight path that conforms to the surface of the target object.

[0027] The drone is controlled to fly along the contour flight path according to the contour flight path.

[0028] Preferably, the adaptive selection and execution of the matching path generation strategy includes:

[0029] The target object is determined to be either a regular geometric shape or a complex free-form surface;

[0030] If the object is a regular geometric shape, then the point cloud of the target object is fitted with a 3D shape, and a contouring path is generated based on the fitted model.

[0031] If the target object's point cloud is a complex freeform surface, then the point cloud is serialized, sliced, fitted with cross-sectional curves, and synthesized in 3D to generate a contour path.

[0032] Preferably, after generating the contour flight path that conforms to the surface of the target object, the method further includes:

[0033] Real-time detection of obstacles in the current flight environment;

[0034] When an obstacle is detected, a local path replanning algorithm is used to dynamically correct the contour flight path to generate a local safe trajectory.

[0035] Preferably, the local path replanning algorithm is implemented through the following steps: parameterizing the path segment to be corrected as a B-spline curve; constructing a repulsion vector based on an improved octree map containing obstacle information; and constructing and solving an optimization problem that simultaneously optimizes trajectory smoothness and collision penalty term to generate the local safe trajectory.

[0036] The present invention has the following advantages and effects compared with the prior art:

[0037] (1) This invention provides a UAV contouring flight system based on multi-sensor fusion and adaptive planning. The system is based on an overall architecture comprising a multi-sensor module, a localization and mapping module, an adaptive path planning module, and a flight control module. Through a tightly coupled multi-sensor fusion algorithm, it achieves high-precision localization and 3D environment reconstruction in complex environments without GPS, and generates contouring flight paths that conform to the object's surface based on an adaptive planning strategy using the target's geometric features. This system solves the technical problems of low localization accuracy, incomplete environmental modeling, and inability to generate high-precision 3D contouring paths in existing UAV technologies in environments lacking GPS. It enables UAVs to complete stable and reliable autonomous contouring flight in complex environments such as indoors, canyons, and underground facilities, greatly expanding their application boundaries.

[0038] (2) The adaptive path planning module of the present invention, through instance segmentation and geometric feature discrimination, can intelligently switch and execute a path generation strategy that matches the target object, whether it is a regular geometric object or a complex free-form surface. For regular targets, shape fitting is used to generate an efficient coverage path; for complex surfaces, point cloud slicing, cross-sectional curve fitting and three-dimensional synthesis are used to achieve high-precision fitting, effectively overcoming the defects of traditional single or fixed planning strategies, and improving the overall operation efficiency while ensuring the accuracy of the operation.

[0039] (3) The system of this invention constructs a two-layer guarantee mechanism that combines global pre-optimization and online local replanning. The trajectory optimization unit, based on the improved octree map and B-spline parameterization, smooths the global path and optimizes it for dynamic feasibility and static collision avoidance. During flight, the local path replanning algorithm can quickly respond to sudden obstacles and correct the local trajectory, thereby improving flight safety and mission completion reliability in complex dynamic environments. Attached Figure Description

[0040] Figure 1 This is a block diagram illustrating the logical implementation of the UAV contour flight system based on multi-sensor fusion and adaptive planning in Embodiment 1 of the present invention.

[0041] Figure 2 This is a schematic diagram of the hardware implementation of the UAV contour flight system based on multi-sensor fusion and adaptive planning in Embodiment 1 of the present invention.

[0042] Figure 3 This is a schematic diagram of the multi-sensor data synchronization process in Embodiment 1 of the present invention.

[0043] Figure 4 This is a schematic diagram of the tightly coupled algorithm workflow of the positioning and mapping module in Embodiment 2 of the present invention.

[0044] Figure 5 This is a flowchart illustrating the UAV contour flight method based on multi-sensor fusion and adaptive planning in Embodiment 2 of the present invention.

[0045] Figure 6 This is a comparison diagram of the actual flight trajectory and the ideal flight path of the UAV when flying around the tree in Embodiment 2 of the present invention.

[0046] Among them, 1 is the lidar, 2 is the computer, 3 is the binocular camera, and 4 is the flight control device. Detailed Implementation

[0047] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0048] Example 1

[0049] This invention provides a UAV contour-following flight system based on multi-sensor fusion and adaptive planning. The system includes a multi-sensor module, a localization and mapping module, an adaptive path planning module, and a flight control module connected sequentially. For example... Figure 1The diagram shows the logical implementation of the system of this invention, illustrating the complete workflow from environmental perception, data fusion, path planning to flight control. Logically, the system is divided into a perception layer, a data fusion layer, and a control execution layer. The perception layer includes a LiDAR, a visual sensor, and an inertial measurement unit (IMU), forming a multi-sensor module responsible for synchronously acquiring 3D point cloud data, image data, and UAV inertial measurement data of the environment. The data fusion layer first uses Kalman filter pose estimation (corresponding to the localization and mapping module) to tightly fuse multi-source perception data, achieving high-precision real-time pose calculation for the UAV and constructing a 3D grid map (3D environmental map). Subsequently, based on the map and pose information, target objects are identified, and a contour-following flight path conforming to the target surface is adaptively generated (corresponding to the adaptive path planning module). The control execution layer corresponds to the flight control module, responsible for converting the planned path into control commands to drive the UAV to complete contour-following flight.

[0050] Specifically, the raw data collected by the multi-sensor module is aggregated and transmitted via a communication bus and data synchronization nodes. After receiving this data, the positioning and mapping module uses a state estimation algorithm (such as an error state iterative Kalman filter) to deeply fuse inertial measurement data, visual features, and laser point clouds, thereby calculating the precise pose of the UAV in real time in complex environments. Simultaneously, this module constructs a 3D environment map containing traversable areas and obstacle distributions by fusing semantic information from the laser point cloud and visual images. Based on the pose and 3D map, the adaptive path planning module identifies target objects and generates a continuous contour-following flight path that conforms to the complex 3D surface of the target object through point cloud slicing and contour curve fitting (such as using RANSAC or B-spline fitting algorithms). Finally, the flight control module converts this path into control commands to drive the UAV to perform contour-following flight.

[0051] The multi-sensor module includes a lidar, a vision sensor, and an inertial measurement unit (IMU). The lidar is used for the UAV's positioning and environmental perception, acquiring real-time 3D point cloud data of the surrounding environment to provide accurate spatial information for the UAV's subsequent flight planning. The vision sensor can be a multi-view camera or a monocular camera. In this embodiment, a multi-view camera (such as a binocular camera) is preferred, which directly acquires environmental depth information through stereo vision, used to calculate the distance between the UAV and target objects in real time to control the contouring distance, and simultaneously provides multi-view image data, which is tightly coupled and fused with the lidar point cloud to enhance positioning accuracy and the integrity of the 3D map. The inertial measurement unit tracks the UAV's pose, three-axis angular velocity, and three-axis linear acceleration in real time to prevent the UAV from rolling or losing control during contouring flight. This module preprocesses the raw data to provide a high-quality data source for subsequent fusion processing.

[0052] The drone uses a 5G communication module to connect sensing units such as visual sensors, lidar, and inertial measurement units to the drone's communication network, preparing for subsequent multi-sensor data fusion.

[0053] As a preferred implementation method for achieving multi-sensor data synchronization, such as Figure 3 As shown, the system can use a microcontroller (such as the STM32 series) to generate precise timing control signals. This microcontroller outputs raw satellite navigation data (e.g., data conforming to the GPRMC protocol) through one pin and a low-frequency PWM signal (e.g., 1Hz) through another pin. These two signals are input to the lidar frequency divider module, which increases the lidar's trigger frequency (e.g., to 10Hz), thereby enabling higher-frequency environmental 3D point cloud acquisition. Simultaneously, the microcontroller can also output another high-frequency PWM signal (e.g., 10Hz) to control the operating frequency of the vision sensor, driving it to acquire color image information of the environment. The acquired point cloud data and image data are published to a unified communication node for data sharing. Synchronization is achieved through timestamps during data interaction to ensure temporal consistency, providing time-consistent data for subsequent multi-source data fusion in contour flight.

[0054] The positioning and mapping module is connected to the multi-sensor module and is used to process the three-dimensional point cloud data, image data and inertial measurement data through a tightly coupled fusion algorithm to output the real-time pose and three-dimensional map of the environment of the UAV.

[0055] Specifically, the localization and mapping module is the core localization unit of the system. It achieves tight coupling and fusion of the LiDAR, vision sensor, and inertial measurement unit through a sequentially updated error state iterative Kalman filter (ESIKF). Based on the prior information propagated by the IMU, the system state is updated sequentially first through LiDAR measurements, and then through image measurements. Both are based on a single unified voxel map. The LiDAR update phase constructs the geometry for registering new LiDAR scans, while the vision update phase appends image patches to the LiDAR points.

[0056] Specifically, in the LiDAR update phase, the system registers the original points into a unified voxel map to construct and update their geometry. In the visual update phase, the system reuses the point cloud data used for map matching in the LiDAR update phase as visual map points without extracting ORB or FAST corner features, triangulation, or optimizing any visual features from the image. Visual map points selected in the map are associated with previously observed reference image patches and then projected onto the current image, adjusting their pose by minimizing the total photometric error function. To improve image alignment accuracy, the reference image patches are dynamically updated, using planar priors obtained from the LiDAR points. To improve computational efficiency, LiDAR points are used to identify visible visual map points in the current image, and on-demand voxel ray casting is performed in the absence of LiDAR points, with exposure time estimated in real-time to handle illumination variations.

[0057] An adaptive path planning module, connected to the positioning and mapping module, is used to identify the target object to be operated based on the environmental 3D map, and adaptively select and execute a path generation strategy that matches the geometric features of the target object to generate a contour-following flight path that conforms to the surface of the target object.

[0058] Specifically, the adaptive path planning module is the decision-making core of the system. It can intelligently select the optimal path generation strategy based on the three-dimensional geometric features of the target operation to achieve high-precision, high-efficiency contour-following flight or full-coverage operations. The adaptive path planning module includes an instance segmentation unit, an adaptive path generation unit, and a trajectory optimization unit. It receives the real-time UAV pose and the environmental three-dimensional map information provided by the positioning and mapping module, and performs the following operations:

[0059] (1) Instance object segmentation: The instance segmentation unit first preprocesses the global point cloud map generated by the localization and mapping module to separate different operable objects. This process includes two key steps:

[0060] Ground point cloud filtering: Advanced algorithms such as Cloth Simulation Filter (CSF) are used to effectively separate ground point clouds from non-ground point clouds, laying the foundation for subsequent target recognition.

[0061] Instance object extraction: Apply algorithms such as Euclidean clustering to non-terrestrial point clouds to aggregate spatially continuous point clouds into independent instance objects (such as individual buildings, equipment, trees, etc.), thereby generating an independent point cloud dataset for each target.

[0062] (2) Adaptive path generation: Based on the geometric features of the segmented instance object point cloud, it is determined whether it belongs to a regular geometric body or a complex freeform surface. The adaptive path generation unit provides two adaptive path generation strategies to deal with targets with different geometric complexities: if it is determined to be a regular geometric body, the first path generation strategy is adopted; if it is determined to be a complex freeform surface, the second path generation strategy, which is different from the first path generation strategy, is adopted.

[0063] The first path generation strategy (simplified contour path generation based on regular shape fitting): This method is suitable for target objects with relatively regular geometric shapes that can be approximated by basic geometric bodies (such as cubes, cylinders, spheres) (such as substations, regular warehouses, wind turbine nacelles).

[0064] Shape fitting: Perform planar fitting on the point cloud of the instance object or directly calculate its 3D bounding box to represent the outer contour of the target object with a simplified geometric model.

[0065] Path generation: Based on the simplified geometric model, generate a spiral path or parallel scan path that wraps around its surface.

[0066] This method is computationally efficient and suitable for large-scale, regular objects where absolute precision is not critical but operational efficiency is paramount.

[0067] The second path generation strategy (accurate 3D contouring path generation based on point cloud slices and cross-sectional curve fitting): This method is suitable for target objects with complex and irregular free-form surfaces (such as sculptures, irregular buildings, ship hulls, aircraft shells, and complex plant canopies), which require high-fidelity contouring of their real shape.

[0068] Point cloud slicing: Along a predetermined contouring direction (such as the vertical direction), the point cloud of the instance object is continuously and precisely sliced ​​horizontally to obtain a series of dense cross-sectional point cloud data.

[0069] Cross-sectional curve fitting: For the point cloud within each slice, an algorithm (such as RANSAC or B-spline curve fitting) is used to extract its precise outer contour boundary and fit a smooth cross-sectional curve. During the fitting process, the fitted curves corresponding to adjacent slices are constrained to smoothly connect at the connection points to ensure the continuity of the synthesis path. This step can accurately restore the complex shape of the target object at each cross-section.

[0070] 3D path synthesis: The series of fitted cross-sectional curves are connected in an orderly manner through path connection algorithms (such as spiral ascent and descent, smooth interpolation) to finally form a continuous, high-precision contour flight path that can closely fit the complex 3D surface of the target object.

[0071] (3) Trajectory optimization (generating the globally optimal path):

[0072] The trajectory optimization unit is used to model the 3D map of the environment based on an improved octree structure. This improved octree structure can efficiently complete octree initialization, point cloud data allocation, occupancy information storage, and support real-time updates and maintenance. Based on this, B-spline curves are used to parameterize the contour flight path. This unit executes a complete trajectory optimization process, including: initial trajectory generation and collision detection, control point generation, gradient optimization, and spatiotemporal refinement. To effectively avoid obstacles identified in the 3D map of the environment, this invention introduces a repulsion vector mechanism, which performs joint optimization by adjusting control points and time allocation, ultimately outputting a smooth, safe, and dynamically feasible globally optimal trajectory as the reference flight path for the UAV.

[0073] (4) Local path replanning (dynamic obstacle avoidance during flight):

[0074] In actual flight, the adaptive path planning module also possesses local path replanning capabilities. When an obstacle not recorded in the 3D map of the current flight environment is detected in real time, the system immediately triggers a local path replanning algorithm. This algorithm reuses the core technologies of the trajectory optimization unit (such as B-spline parameterization and repulsion vector mechanism), but performs rapid optimization for local path segments: based on the current flight path, the path segment to be corrected is parameterized as a B-spline curve; a repulsion vector is constructed based on the real-time perceived obstacle information; a lightweight optimization problem that simultaneously optimizes trajectory smoothness and collision penalty terms is constructed and solved; finally, a local safe trajectory is quickly generated to ensure that the UAV can dynamically avoid sudden obstacles and smoothly return to the original globally optimal trajectory after obstacle avoidance.

[0075] Optionally, for applications that require overall scanning and mapping of the area to be surveyed (such as regional topographic mapping and search and rescue), this adaptive path planning module provides a global full-coverage path generation function. This function projects the three-dimensional work area onto a two-dimensional plane, and generates a complete coverage path (such as a bow-shaped path) on this basis. It can also be optimized in combination with the field of view of the sensor to ensure the integrity and efficiency of data acquisition.

[0076] The flight control module is used to convert the planned path into control commands to drive the UAV to complete contour-following flight.

[0077] Example 2

[0078] like Figure 5 The diagram shows a flowchart of a UAV contour flight method based on multi-sensor fusion and adaptive planning, including the following steps:

[0079] S1. Simultaneously collect 3D point cloud data, image data, and inertial measurement data of the UAV from multiple sensors;

[0080] S2. The data is processed by a tightly coupled fusion algorithm to calculate the pose of the UAV in real time and construct a 3D map of the environment.

[0081] Specifically, such as Figure 4 The diagram shows the tightly coupled algorithm workflow of the localization and mapping module in this embodiment of the invention. The specific workflow includes:

[0082] S201, Data Synchronization and Preprocessing: Receive synchronized lidar frames, image frames and IMU data packets, preprocess them and then transmit the data forward to provide raw data for subsequent state prediction;

[0083] S202. State Prediction: Based on preprocessed data, predict the state changes of the UAV from the previous moment to the current moment (including position, velocity, and attitude).

[0084] S203, LiDAR Update (Geometric Update): Match the current frame's laser point cloud with local planar features in the voxel map, construct the geometric residual based on the distance from the point to the plane, optimize the pose by constructing and solving the least squares problem, calculate the pose error between the predicted state and the geometric observation, and complete the first state update.

[0085] S204, Visual Update (Photometric Update): The system directly reuses the LiDAR points that were successfully matched with the map plane in step S203. The 3D coordinates of these points are used as visual map points, and associated with the small image patches stored in historical keyframes. These image patches are projected onto the current visual sensor image based on the current pose estimation. The direct photometric error between the projected area and the corresponding area of ​​the current image is calculated for all pixels. By iteratively optimizing the pose and minimizing the total photometric error, the second state update is completed.

[0086] The error information obtained by the above two methods is used to optimize and correct the previously predicted pose, resulting in a high-precision current pose estimate. Through continuous iterative state estimation, a highly robust real-time positioning is obtained.

[0087] S205. Map Update: The LiDAR point cloud is downsampled, and then the voxel map is updated based on the sampled point cloud. This involves dividing the 3D space into voxel units, marking the occupancy state of each unit, and constructing a 3D geometric map of the environment. Simultaneously, features are extracted from the visual image to generate visual map points. These map points are then optimized (e.g., removing mismatched points and optimizing the spatial position of points) to construct a visual feature map of the environment, providing a long-term stable feature reference for visual positioning and error calculation. State estimation relies on the environmental reference provided by the map to complete its own accurate pose calculation, while the map is continuously updated and optimized through the pose information of state estimation, ultimately forming a closed loop of localization-mapping, ensuring the accuracy of localization and the real-time nature of environmental perception during contour flight.

[0088] S3. Identify the target object to be operated based on the three-dimensional environmental map; based on the geometric features of the target object, adaptively select and execute a path generation strategy that matches it, and generate a contour flight path that conforms to the surface of the target object.

[0089] Specifically, such as Figure 5 As shown, for regular-shaped targets, point clouds belonging to the same regular-shaped target are grouped together to achieve shape fitting of the regular object. The outline of the target object is replaced with a simplified geometry, and then a spiral flight path covering the surface of its fitted geometry is generated.

[0090] For irregular target objects, the point cloud belonging to the irregular curved surface target is first separated by segmenting the point cloud region. Then, the point cloud is sliced ​​according to the direction of the contouring by using point cloud slicing. The cross-sectional point cloud of each slice is extracted. Finally, the contour curve is fitted by the algorithm. Through the above fitting contour curve, a continuous high-precision contouring flight path that can closely fit the complex three-dimensional surface of the target object is obtained.

[0091] Furthermore, B-spline optimization is performed on the spiral path generated from the regular-shaped target and the contour curve trajectory obtained from the irregular surface target to obtain a smooth, safe, and dynamically feasible optimal trajectory.

[0092] S4. Local Path Replanning: During actual flight, obstacles in the current flight environment are detected in real time. When an obstacle is detected, a local path replanning algorithm is used to dynamically correct the contour flight path generated in step S3 for obstacle avoidance. Specifically, this includes: parameterizing the path segment to be corrected as a B-spline curve; constructing a repulsion vector based on an improved octree map containing obstacle information; and constructing and solving an optimization problem that simultaneously optimizes trajectory smoothness and collision penalty terms to quickly generate a locally safe trajectory.

[0093] S5. Flight Control: The optimized trajectory (or local safe trajectory) is input into the flight controller, which then controls the UAV to perform contour flight according to the planned trajectory.

[0094] Figure 6 This diagram illustrates a comparison between the actual flight trajectory and the ideal flight path of the UAV when it flies around a tree, as shown in this embodiment of the invention. The blue line represents the actual flight trajectory, and the green line represents the theoretical flight path (generated by the adaptive path planning module).

[0095] In actual flight, physical factors such as airframe structural oscillations and dynamic fluctuations can cause slight deviations in the inertial measurement unit (IMU) perception. As shown in the figure, despite dynamic errors, the actual flight trajectory can be stably controlled within the vicinity of the theoretical path. This demonstrates that the present invention achieves high-precision state estimation and online IMU error compensation through a tightly coupled fusion algorithm in the positioning and mapping module, while the local path replanning algorithm in the adaptive path planning module is triggered in real time: when the system senses state deviations or sudden obstacles, it can quickly optimize online and generate local adjustment commands. This result verifies the robustness of the system in the present invention in coping with uncertainties and maintaining operational accuracy in complex real-world environments.

[0096] The above embodiments are preferred embodiments of the present invention and are not intended to limit the present invention. Any other changes or equivalent substitutions made without departing from the technical solution of the present invention are included within the protection scope of the present invention.

Claims

1. A UAV contour-following flight system based on multi-sensor fusion and adaptive planning, characterized in that, include: A multi-sensor module is used to simultaneously acquire 3D point cloud data, image data, and inertial measurement data of the UAV from the environment. The positioning and mapping module is connected to the multi-sensor module and is used to process the three-dimensional point cloud data, image data and inertial measurement data through a tightly coupled fusion algorithm to output the real-time pose and three-dimensional map of the environment of the UAV. An adaptive path planning module, connected to the positioning and mapping module, is used to identify the target object to be operated based on the environmental 3D map, and adaptively select and execute a path generation strategy that matches the geometric features of the target object to generate a contour-following flight path that conforms to the surface of the target object. The flight control module, connected to the adaptive path planning module, is used to control the UAV according to the contour flight path.

2. The UAV contour-following flight system based on multi-sensor fusion and adaptive planning according to claim 1, characterized in that, The multi-sensor module includes a lidar, a vision sensor, and an inertial measurement unit.

3. The system according to claim 1 or 2, characterized in that, The localization and mapping module implements the tightly coupled fusion algorithm through an iterative error state Kalman filter, and the processing includes: State prediction stage: Predicting the system state based on the inertial measurement data; LiDAR update phase: The three-dimensional point cloud data is matched with the local planar features in the three-dimensional environmental map, and the predicted state is updated for the first time based on the geometric residual generated by the matching. Visual update phase: The point cloud data used for map matching in the LiDAR update phase is used as visual map points, and the associated reference image blocks are projected onto the current image data. By minimizing the direct photometric error, the state after the first update is updated a second time.

4. The UAV contour-following flight system based on multi-sensor fusion and adaptive planning according to claim 1, characterized in that, The adaptive path planning module includes: An instance segmentation unit is used to process the 3D map of the environment to separate the point cloud of the target object; An adaptive path generation unit is used to determine whether the target object's point cloud belongs to a regular geometric body or a complex freeform surface based on its geometric features: if it is determined to be a regular geometric body, a first path generation strategy is adopted; if it is determined to be a complex freeform surface, a second path generation strategy is adopted.

5. The UAV contour-following flight system based on multi-sensor fusion and adaptive planning according to claim 4, characterized in that, The first path generation strategy includes: performing three-dimensional shape fitting on the point cloud of the target object, and generating a contour-following flight path based on the fitted simplified geometric model; The second path generation strategy includes: performing serialized slicing, cross-sectional contour curve fitting, and 3D synthesis on the point cloud of the target object to generate a contour flight path.

6. The UAV contour-following flight system based on multi-sensor fusion and adaptive planning according to claim 1, characterized in that, The adaptive path planning module further includes a trajectory optimization unit, which is used to model the 3D map of the environment based on an improved octree structure, parameterize the contour flight path using B-spline curves, and jointly optimize the control point and time allocation by introducing a repulsion vector mechanism to generate a smooth and collision-free flight trajectory.

7. A UAV contour-following flight method based on multi-sensor fusion and adaptive planning, characterized in that, Includes the following steps: The system simultaneously collects 3D point cloud data, image data, and inertial measurement data of the UAV from multiple sensors. The data is processed by a tightly coupled fusion algorithm to calculate the pose of the UAV in real time and construct a 3D map of the environment. Identify the target object to be worked on based on the aforementioned 3D environmental map; Based on the geometric features of the target object, an adaptive path generation strategy is selected and executed to generate a contour flight path that conforms to the surface of the target object. The drone is controlled to fly along the contour flight path according to the contour flight path.

8. The UAV contour-following flight method based on multi-sensor fusion and adaptive planning according to claim 7, characterized in that, The adaptive selection and execution of the matching path generation strategy includes: The target object is determined to be either a regular geometric shape or a complex free-form surface; If the object is a regular geometric shape, then the point cloud of the target object is fitted with a 3D shape, and a contouring path is generated based on the fitted model. If the target object's point cloud is a complex freeform surface, then the point cloud is serialized, sliced, fitted with cross-sectional curves, and synthesized in 3D to generate a contour path.

9. The UAV contour-following flight method based on multi-sensor fusion and adaptive planning according to claim 7, characterized in that, After generating the contour-following flight path that conforms to the surface of the target object, the process also includes: Real-time detection of obstacles in the current flight environment; When an obstacle is detected, a local path replanning algorithm is used to dynamically correct the contour flight path to generate a local safe trajectory.

10. The UAV contour-following flight method based on multi-sensor fusion and adaptive planning according to claim 9, characterized in that, The local path replanning algorithm is implemented through the following steps: parameterizing the path segment to be corrected as a B-spline curve; constructing a repulsion vector based on an improved octree map containing obstacle information; and constructing and solving an optimization problem that simultaneously optimizes trajectory smoothness and collision penalty term to generate the local safe trajectory.

Images