A multi-sensor fusion low-power obstacle avoidance and safe landing method and system for a hydrogen energy large-load unmanned aerial vehicle and a storage medium

Abstract

The application relates to the technical field of unmanned aerial vehicle autonomous navigation and flight control, and discloses a multi-sensor fusion low-power obstacle avoidance and safe landing method and system for a hydrogen energy large-load unmanned aerial vehicle and a storage medium. The method comprises the following steps: acquiring IMU measurement values, rotor motor rotating speed signals and original LiDAR point cloud data; applying motion distortion compensation based on rotor vibration estimation to the LiDAR point cloud data, and outputting compensated LiDAR point cloud data; acquiring a visual image, uniformly fusing perception of the compensated LiDAR point cloud data, the visual image and the IMU measurement values in a BEV space, and outputting obstacle distribution information and ground attribute information; constructing a trajectory planning model containing a fuel consumption cost index, and outputting an optimal obstacle avoidance trajectory; and selecting an optimal landing point based on the ground attribute information. Through vibration compensation, fuel constraint planning and multi-dimensional landing evaluation, the application realizes low-power autonomous obstacle avoidance and safe landing of a large-load hydrogen energy unmanned aerial vehicle in a complex environment.

Description

Technical Field

[0001] This application relates to the field of autonomous navigation and flight control technology for unmanned aerial vehicles (UAVs), and in particular to a multi-sensor fusion low-power obstacle avoidance and safe landing method, system and storage medium for hydrogen-powered UAVs with large payloads. Background Technology

[0002] Hydrogen-powered drones use hydrogen fuel cells as their power source, offering advantages such as large payload capacity and long endurance; they have broad application prospects in scenarios involving large payloads and long flight times, such as material transportation, emergency rescue, and power line inspection. However, their large payload characteristics also bring a series of technical challenges that are absent or only slightly present in lithium-ion battery drones or micro-drones.

[0003] Currently, multi-sensor fusion obstacle avoidance technology has been applied in the field of unmanned aerial vehicles (UAVs). Existing solutions typically integrate lidar, visual sensors, and inertial measurement units to achieve perception of the surrounding environment and obstacle avoidance. However, these solutions are mainly designed for micro or lightweight UAVs. When directly transplanted to hydrogen-powered UAVs with large payloads, the solutions are difficult to apply due to fundamental differences in airframe dynamics, energy system response characteristics, and mission scenarios.

[0004] Specifically, existing obstacle avoidance solutions have the following main shortcomings: First, under heavy load conditions, the high-frequency vibrations generated by the rotors and motors of hydrogen-powered drones significantly degrade the perception accuracy of the lidar. Within the same scan frame, the continuous vibration of the drone causes small but not negligible changes in the lidar's pose, resulting in motion distortion of the scanned point cloud and blurring of obstacle edges, thus affecting the reliability of subsequent obstacle detection and obstacle avoidance decisions. This problem of vibration-coupled perception accuracy is not significant in micro-drones with low inertia and low amplitude, but it becomes one of the core bottlenecks affecting flight safety in hydrogen-powered drones with heavy loads.

[0005] Secondly, existing obstacle avoidance decisions do not fully consider the energy characteristics of hydrogen fuel cells. During emergency maneuvers, the fuel cell stack needs to output several times the cruising power instantaneously, leading to a sharp increase in hydrogen consumption rate. An inappropriate obstacle avoidance path may cause a large amount of fuel to be consumed in a short period of time, threatening mission completion and safe return. Existing solutions generally assume that the energy supply is sufficient or that consumption changes linearly.

[0006] Third, the existing solutions have a relatively singular assessment dimension for landing safety, lacking a comprehensive consideration of multiple factors such as ground load-bearing capacity, terrain slope, and weather conditions, making it difficult to guarantee the safety and reliability of heavy-payload UAVs when landing on unstructured sites.

[0007] In summary, the long-endurance, high-payload mission requirements of hydrogen-powered high-payload unmanned aerial vehicles (UAVs) place unique demands on their obstacle avoidance and landing systems compared to conventional UAVs. Ensuring perception accuracy under vibration interference, achieving safe obstacle avoidance under energy constraints, and completing reliable landings in unstructured environments have become pressing technical challenges in this field. Summary of the Invention

[0008] To address the aforementioned technical issues, this application provides a multi-sensor fusion low-power obstacle avoidance and safe landing method, system, and storage medium for hydrogen-powered high-payload unmanned aerial vehicles (UAVs). By fusing and sensing data from multiple sources, combined with vibration suppression and fuel consumption constraints, it achieves accurate perception under vibration interference, safe obstacle avoidance under fuel constraints, and reliable landing in complex terrain.

[0009] According to a first aspect of the embodiments disclosed in this application, this application provides a multi-sensor fusion low-power obstacle avoidance and safe landing method for hydrogen-powered high-payload unmanned aerial vehicles, adopting the following technical solution: A multi-sensor fusion low-power obstacle avoidance and safe landing method for hydrogen-powered high-payload unmanned aerial vehicles (UAVs) includes the following steps: Acquire IMU measurements, rotor motor speed signals, and raw LiDAR point cloud data; apply motion distortion compensation based on rotor vibration estimation to the LiDAR point cloud data, and output the compensated LiDAR point cloud data; and simultaneously calculate the true velocity, true acceleration, and current total mass based on rotor lift inversion after removing vibration noise. Visual images are acquired, and a lightweight multimodal fusion network is used to perform unified fusion perception of the compensated LiDAR point cloud data, the visual images, and IMU measurements in the BEV space, and output obstacle distribution information and ground attribute information. Based on the obstacle distribution information, combined with the actual speed, actual acceleration and current total mass, a trajectory planning model including fuel consumption cost index is constructed to optimize the hierarchical obstacle avoidance trajectory, and the obstacle avoidance response threshold is adaptively adjusted according to the current total mass to output the optimal obstacle avoidance trajectory. When responding to a landing command, the system performs a multi-dimensional assessment based on the ground attribute information, taking into account factors such as terrain flatness, ground bearing capacity, obstacle clearance, ground slope, and crosswind influence, and selects the landing point with the best overall assessment to carry out the landing.

[0010] Optionally, in the motion distortion compensation based on rotor vibration estimation, the rotor vibration estimation is achieved in the following way: The rotor motor is operated at a stepper speed, and acceleration response data is collected at the LiDAR mounting location. The rotor passing frequency and its harmonic components at each speed are identified, and the vibration transfer function is fitted. The amplitude-frequency response and phase-frequency response, the vibration transfer function adopts a superposition form of a second-order resonance model: in, It is a positive integer not less than 2. The frequency of vibration. For the first The amplitude coefficient of the first harmonic. For the first k First harmonic angular frequency, For the first Damping ratio of first harmonic ； For harmonic orders, The imaginary unit; Will The parameterization is done by using a lookup table indexed by rotor speed.

[0011] Optionally, applying motion distortion compensation based on rotor vibration estimation to the LiDAR point cloud data specifically includes: Vibration filtering preprocessing is performed on the accelerometer data and gyroscope data in the IMU measurement values. A notch filter is used to filter out the vibration signal components at the rotor passing frequency and its harmonic frequencies to obtain the true angular velocity and true acceleration after removing vibration noise. IMU pre-integration is performed based on the actual angular velocity and actual acceleration to obtain the data from the start of the frame. to the corresponding time of the laser point Transformation of the actual motion posture of the organism between ; Obtain the timestamp of each laser point in the LiDAR point cloud data, and query the lookup table based on the current rotor speed to predict the vibration displacement of the laser point at the corresponding moment. and vibration rotation amount The vibration displacement includes vibration displacement components with three translational degrees of freedom, and the vibration rotation includes vibration angular components with three rotational degrees of freedom. Transform the actual motion posture of the machine body With the vibration displacement and vibration rotation amount Perform synthesis to construct the compensation transformation matrix for the laser point. The coordinates of the laser point are transformed from the instantaneous measurement coordinate system to the frame start unified reference coordinate system using the compensation transformation matrix. The compensated coordinates of the laser point are: in, For the first The compensation transformation matrix corresponding to each laser point at any given time is obtained by synthesizing the true motion pose, vibration displacement, and vibration rotation after vibration removal from the IMU pre-integration. These are the original coordinates of the laser point in the instantaneous measurement coordinate system. The coordinates of the laser point after compensation are given in the unified reference coordinate system at the beginning of the frame.

[0012] Optionally, the current total mass based on rotor lift inversion is obtained as follows: based on the rotor motor speed signal, the real-time total lift is inverted using the mapping relationship between rotor speed and lift: in, This represents the total number of rotor motors. This is the motor serial number. For the first The rotor speed of each motor, This is the lift calibration coefficient; In hovering or constant-speed flight, assuming the real-time total lift equals the current gravity, the current total mass can be inverted using the following formula: in, The current total mass is continuously updated as hydrogen fuel is consumed, with the acceleration due to gravity as the current mass is increased.

[0013] Optionally, the lightweight multimodal fusion network includes a visual feature extraction branch and a LiDAR feature extraction branch, wherein the visual feature extraction branch uses a lightweight convolutional neural network to extract image features, and the LiDAR feature extraction branch uses a sparse convolutional network to extract point cloud features. The two features are projected onto the BEV space and then stitched and fused to output a unified BEV feature map. The BEV feature map is output in parallel through a multi-task output head: the 3D target detection head outputs the three-dimensional position, size, orientation and category of the obstacle; the passable area segmentation head outputs the passability probability of each grid area to construct obstacle distribution information; and the ground attribute head outputs the ground material classification results to constitute ground attribute information.

[0014] Optionally, the trajectory planning model is a model predictive controller, and its cost function consists of multiple weighted terms, including trajectory tracking cost, control smoothing cost, collision cost, and fuel consumption cost; the cost function has the following form: in, For the current moment, To predict the time domain, To predict the state trajectory, For reference trajectory, To control the input, For collision cost terms, For fuel consumption costs, Terminal status. The target state; These are the weighting coefficients for the trajectory tracking cost term. To control the weighting coefficients of the smoothing cost term, These are the weighting coefficients for the collision cost term. These are the weighting coefficients for the terminal cost term. This represents the weighting coefficient for the fuel consumption cost item; This is the weight matrix for the trajectory tracking cost term. To control the weight matrix of the smoothing cost term, This is the weight matrix for the terminal cost term; The fuel consumption cost item The calculation formula is: in, For hovering power, air density, The drag coefficient, For windward area, and These are the actual velocity and the actual acceleration, respectively. For the current total mass, The temperature of the fuel cell stack is collected in real time by a temperature sensor. The current load power is calculated by the flight control system based on the current flight status. The instantaneous efficiency of the fuel cell stack is obtained in real time based on the stack temperature and the current load power. The weighting coefficient of the fuel consumption cost item The weighting coefficient is adaptively adjusted based on the remaining hydrogen quantity: when the remaining hydrogen quantity is higher than a preset threshold, the weighting coefficient takes the nominal value; when the remaining hydrogen quantity is lower than or equal to the preset threshold, the weighting coefficient increases, so that the trajectory planning model prioritizes the selection of low hydrogen consumption trajectories.

[0015] Optionally, the layered obstacle avoidance trajectory optimization adopts a three-layer progressive architecture of long-distance layer, mid-distance layer, and short-distance layer; the obstacle avoidance response threshold is adaptively adjusted in the following manner: in, 、 and These are preset base distance thresholds for the long-range, mid-range, and short-range layers, determined based on the drone model, mission scenario, or preset security level configuration. It is a speed adjustment factor, determined based on the ratio of the current flight speed to the cruise speed, and this ratio is limited to a preset minimum speed ratio and maximum speed ratio. It is a mass adjustment factor, determined based on the ratio of the current total mass to the unloaded mass, and this ratio is limited to a preset minimum mass ratio and maximum mass ratio.

[0016] Optionally, based on the ground attribute information, a multi-dimensional assessment is performed, integrating terrain flatness, ground bearing capacity, obstacle clearance, ground slope, and crosswind influence, and calculated using a weighted sum method: in, , , , and The preset weighting coefficients are for five dimensions: terrain flatness, ground bearing capacity, obstacle clearance, ground slope, and crosswind margin. The terrain smoothness score is calculated based on the fitting residuals of the ground point cloud in the landing area. The ground bearing capacity score is calculated based on the load-bearing capacity mapped from the ground attribute information and the current total mass. This is an obstacle clearance score calculated based on obstacle detection around the landing point. This is a ground slope score calculated based on the angle between the ground normal vector and the direction of gravity. The crosswind margin score is calculated based on the crosswind component; the point with the highest comprehensive score is searched in the landing area using a grid sampling method as the optimal landing point.

[0017] According to a second aspect of the embodiments disclosed in this application, this application provides a multi-sensor fusion low-power obstacle avoidance and safe landing system for hydrogen-powered high-payload unmanned aerial vehicles, adopting the following technical solution: A multi-sensor fusion low-power obstacle avoidance and safe landing system for hydrogen-powered high-payload unmanned aerial vehicles (UAVs) includes: The data acquisition and compensation module is used to acquire IMU measurement values, rotor motor speed signals and raw LiDAR point cloud data, apply motion distortion compensation based on rotor vibration estimation to the LiDAR point cloud data, output the compensated LiDAR point cloud data, and simultaneously calculate the true velocity, true acceleration and current total mass based on rotor lift inversion after removing vibration noise. The fusion perception module is used to acquire visual images. It adopts a lightweight multimodal fusion network to perform unified fusion perception of the compensated LiDAR point cloud data, the visual images, and IMU measurements in the BEV space, and outputs obstacle distribution information and ground attribute information. The obstacle avoidance decision module is used to construct a trajectory planning model that includes a fuel consumption cost index based on the obstacle distribution information, combined with the real speed, real acceleration and current total mass, to perform hierarchical obstacle avoidance trajectory optimization, and to adaptively adjust the obstacle avoidance response threshold according to the current total mass, and output the optimal obstacle avoidance trajectory. The landing assessment module is used to conduct a multi-dimensional assessment based on the ground attribute information when responding to a landing command, taking into account factors such as terrain flatness, ground bearing capacity, obstacle clearance, ground slope, and crosswind influence, and to select the landing point with the best comprehensive assessment to carry out the landing.

[0018] According to a third aspect of the embodiments disclosed in this application, this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the multi-sensor fusion low-power obstacle avoidance and safe landing method for hydrogen-powered high-payload unmanned aerial vehicles as described in any one of the embodiments disclosed in the first aspect of this application.

[0019] In summary, this application provides a multi-sensor fusion low-power obstacle avoidance and safe landing method, system, and storage medium for hydrogen-powered high-payload unmanned aerial vehicles (UAVs); compared with the prior art, this application has the following beneficial effects: (1) By introducing motion distortion compensation based on rotor vibration estimation, the pollution of point cloud data by fuselage vibration under heavy load conditions can be effectively eliminated, thereby improving obstacle perception accuracy.

[0020] (2) A lightweight multimodal fusion network is adopted to achieve low-power real-time inference while ensuring perception accuracy, and adapt to the resource constraints of the airborne computing platform.

[0021] (3) Introduce the fuel consumption cost index into the obstacle avoidance trajectory planning to achieve comprehensive optimization of flight safety and energy consumption, and prevent the inability to return due to excessive maneuvering; adapt to the obstacle avoidance requirements under different flight speeds and different load conditions by dynamically adjusting the adaptive obstacle avoidance response threshold.

[0022] (4) Conduct a multi-dimensional landing assessment by taking into account the flatness of the terrain, the bearing capacity of the ground, the clearance of obstacles, the ground slope and the influence of crosswinds, to ensure safe landing under unstructured ground conditions.

[0023] Other features and advantages disclosed in this application will be described in detail in the following detailed description section. Attached Figure Description

[0024] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of a multi-sensor fusion low-power obstacle avoidance and safe landing method for hydrogen-powered high-payload UAVs, provided as an embodiment of this application.

[0025] Figure 2 This application provides a structural block diagram of a multi-sensor fusion low-power obstacle avoidance and safe landing system for hydrogen-powered high-payload unmanned aerial vehicles. Detailed Implementation

[0026] The specific embodiments disclosed in this application will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of this disclosure.

[0027] Hydrogen-powered high-payload unmanned aerial vehicles (UAVs) have broad application prospects in long-endurance, high-payload operation scenarios such as material transportation, emergency rescue, and power line inspection. However, their high-payload characteristics also bring unique obstacle avoidance and landing technology challenges. On the one hand, the high-frequency vibrations generated by the rotor and motor under high-payload conditions can significantly degrade the perception accuracy of lidar, leading to motion distortion of point clouds and blurring of obstacle edges. On the other hand, the energy characteristics of hydrogen fuel cells mean that the instantaneous high power output during obstacle avoidance maneuvers may cause a sharp increase in hydrogen consumption. If the obstacle avoidance decision does not consider fuel consumption constraints, it may directly threaten the UAV's return safety. In addition, when landing on unstructured sites, a comprehensive assessment of multiple factors such as ground bearing capacity, terrain flatness, and weather conditions is crucial to ensuring the safe landing of high-payload UAVs. To this end, this application provides a multi-sensor fusion low-power obstacle avoidance and safe landing method for hydrogen-powered high-payload UAVs. Through body vibration suppression, fuel constraint planning, and multi-dimensional landing assessment, it achieves accurate perception under vibration interference, safe obstacle avoidance under fuel constraints, and reliable landing in complex terrain.

[0028] like Figure 1 As shown, this application provides a multi-sensor fusion low-power obstacle avoidance and safe landing method for hydrogen-powered high-payload unmanned aerial vehicles (UAVs), specifically including steps S101 to S104. Figure 2 As shown, the system provided in this application embodiment includes a data acquisition and compensation module, a fusion perception module, an obstacle avoidance decision module, and a landing assessment module. Each of the above modules corresponds to steps S101 to S104 and can be deployed on an airborne computing platform (such as Jetson Orin NX) to conduct data communication between modules through shared memory or message queues.

[0029] In step S101, the IMU measurement value, rotor motor speed signal and original LiDAR point cloud data are acquired; motion distortion compensation based on rotor vibration estimation is applied to the LiDAR point cloud data, and the compensated LiDAR point cloud data is output; and the true velocity, true acceleration and current total mass based on rotor lift inversion are calculated simultaneously after removing vibration noise.

[0030] In this embodiment, the hydrogen-powered high-payload UAV adopts a quadcopter configuration, with an empty weight of approximately 125 kg and a maximum payload capacity of approximately 100 kg, resulting in a total fully loaded weight of approximately 225 kg. The onboard sensor system includes a lidar, an inertial measurement unit (IMU), rotor motor speed sensors, and a binocular vision camera. The lidar is a 32-line or 64-line mechanical rotating lidar with a scanning frequency of 10 Hz, outputting no fewer than 100,000 laser points per frame, each containing three-dimensional coordinate information and a timestamp. The IMU has a sampling frequency of no less than 200 Hz, outputting three-axis angular velocity and three-axis specific force data with a range of ±50 g. Each motor is equipped with a speed feedback device, updating the speed signal at a frequency of no less than 100 Hz. The binocular vision camera has an image resolution of 640×480 after downsampling, with a frame rate of 30 Hz. All sensors achieve hardware-level time synchronization via the IEEE 1588 precise time protocol, with a synchronization accuracy better than 1 ms. All data is uniformly timestamped with GPS to ensure consistency in time reference across data sources during subsequent multi-sensor fusion.

[0031] Rotor vibration estimation is achieved through two stages: ground calibration and online compensation. The purpose of the ground calibration stage is to establish a transfer function model between the rotor speed and the vibration response at the LiDAR mount location. During flight, the high-frequency vibrations generated by the rotor and motor of the hydrogen-powered high-payload UAV are mainly concentrated in the 100-300Hz frequency band. This vibration is transmitted to the LiDAR mount location, causing continuous pose changes of the LiDAR within a single frame scan cycle (100ms). Because the LiDAR operates using a point-by-point scanning mode, the sampling time of each laser point is different. The difference in LiDAR pose at different times within the same frame will cause motion distortion in the point cloud. Originally vertical rod-shaped obstacles may appear tilted or bent in the point cloud, and the edges of obstacles will be blurred. To eliminate this distortion, accurate estimation and compensation of rotor vibration are required.

[0032] The specific ground calibration process is as follows: First, a high-precision accelerometer is rigidly mounted on the mounting base next to the LiDAR optical center, ensuring a rigid connection between the accelerometer and the LiDAR, and that their vibration responses are consistent. Simultaneously, reference accelerometers are placed at each of the four motor mounting bases to measure the vibration excitation source. Then, the UAV is fixed on a six-DOF vibration test bench, and the rotor motors are sequentially operated at speeds ranging from 1000 to 6000 RPM in 200 RPM steps, covering typical operating conditions such as hovering, cruising, and maximum power. Each speed point is operated stably for at least 30 seconds, simultaneously collecting accelerometer data and rotor speed feedback signals. Power spectral density analysis is performed on the acceleration data collected at each speed point to identify the dominant frequency component and its harmonic components. The fundamental frequency of rotor vibration is determined by dividing the rotor speed by 60 and then multiplying by the number of blades. After identifying the fundamental frequency, its second, third, and fourth harmonic frequencies are identified as two, three, and four times the fundamental frequency, respectively. Engineering practice shows that the energy of fourth and higher harmonics usually accounts for less than five percent of the total vibration energy, and their contribution to point cloud distortion is negligible. Therefore, modeling within the range of fundamental frequency to fourth harmonic can achieve sufficient compensation accuracy.

[0033] For each identified harmonic component, the amplitude-frequency response and phase-frequency response of the vibration transfer function are fitted. This embodiment uses a superposition of second-order resonance models: ; in, ω is the angular frequency of vibration, which is the independent variable of the transfer function, and its unit is rad / s; The order of harmonic superposition is a positive integer not less than 2, and is preferably 4 in this embodiment, that is, superimposing the fundamental frequency to the fourth harmonic component; For harmonic orders, =1 corresponds to the fundamental frequency component. =2, 3, 4 correspond to the second, third, and fourth harmonic components; For the first The amplitude coefficient of a harmonic reflects the vibration amplitude of that harmonic component and is obtained by fitting calibration experimental data. For the first k First harmonic angular frequency, ,in, The fundamental angular frequency, The mechanical rotation frequency of the drone's rotor / propeller; For the first The damping ratio of a harmonic reflects the attenuation characteristics of that harmonic component and is obtained by fitting calibration experimental data. The unit is the imaginary unit. This second-order resonance model is a complex frequency domain transfer function. The damping term in the denominator characterizes the damping properties of the vibration system. This model can accurately describe the amplitude amplification and phase delay of the vibration at the LiDAR mount position under different excitation frequencies. The calibrated transfer function parameters are stored as a lookup table indexed by rotor speed for real-time lookup during the online compensation phase in flight.

[0034] It should be noted that in actual flight, the rotor speed is limited by the dynamic response characteristics of the motor and the moment of inertia of the blades, and its rate of change is a finite value (usually not exceeding XX rad / s²), and a step change is impossible. Therefore, using linear interpolation or cubic spline interpolation between adjacent calibration speed points to obtain the vibration transfer function parameters at the current speed can ensure sufficient prediction accuracy under normal maneuvering flight conditions.

[0035] To further improve the accuracy of vibration prediction under unsteady conditions, this embodiment also includes an online adaptive correction stage: Accelerometer measurements at the LiDAR mounting base are acquired in real time, and the vibration components within the rotor passage frequency and its harmonic bands are extracted using bandpass filtering. The vibration amplitude predicted by the lookup table is compared with the measured vibration amplitude, and the amplitude coefficients output from the lookup table are adjusted using recursive least squares or Kalman filters. Online correction is performed, and the corrected amplitude coefficients are used to predict the vibration displacement at the next moment. This online correction step can effectively compensate for vibration model mismatch errors caused by sudden changes in rotational speed or aerodynamic environment.

[0036] In the online compensation phase, firstly, the accelerometer data and gyroscope data in the IMU measurements are subjected to vibration filtering preprocessing. A notch filter is used to filter out the vibration signal components at the rotor passing frequency and its harmonic frequencies to obtain the true angular velocity and true acceleration after removing vibration noise.

[0037] IMU pre-integration is performed based on the actual angular velocity and actual acceleration to obtain the data from the start of the frame. to the corresponding time of the laser point Transformation of the actual motion posture of the organism between .

[0038] Next, the timestamps of each laser point in the LiDAR point cloud data are obtained, and the vibration displacement of the laser point at the corresponding moment is predicted by querying the lookup table based on the current rotor speed. and vibration rotation amount The vibration displacement includes vibration displacement components with three translational degrees of freedom, and the vibration rotation includes vibration angular components with three rotational degrees of freedom; the actual motion posture of the machine body is transformed. With the vibration displacement and vibration rotation amount Perform synthesis to construct the compensation transformation matrix for the laser point. The coordinates of the laser point are transformed from the instantaneous measurement coordinate system to the frame start unified reference coordinate system using the compensation transformation matrix. The compensated coordinates of the laser point are: ; in, For the first The compensation transformation matrix corresponding to each laser point at any given time is obtained by synthesizing the true motion pose, vibration displacement, and vibration rotation after vibration removal from the IMU pre-integration. These are the original coordinates of the laser point in the instantaneous measurement coordinate system. The coordinates of the laser point after compensation are given in the unified reference coordinate system at the beginning of the frame.

[0039] After performing the above transformation on all laser points one by one, all points within a frame are corrected to the same reference coordinate system, and the compensated LiDAR point cloud data without motion distortion is output.

[0040] The current total mass based on rotor lift inversion is obtained as follows: The total lift of a multi-rotor UAV is proportional to the sum of the squares of the rotational speeds of each rotor. Based on the rotor motor speed signals, the real-time total lift is inverted using the mapping relationship between rotor speed and lift. ; in, This represents the total lift generated by each rotor at the current moment. This represents the total number of rotor motors; for a quadcopter drone, it is 4, and for a hexcopter drone, it is 6. This is the motor serial number. For the first The rotor speed of each motor, in rad / s, is obtained in real time from the motor speed sensor. The lift calibration coefficient is obtained through pre-calibration via ground test bench experiments. The calibration method involves measuring the rotational speed of each motor during hovering under the condition of a known total mass of the UAV. The lift calibration coefficient is calculated by multiplying the known total mass by the gravitational acceleration and then dividing by the sum of the squares of the hovering rotational speeds of each motor.

[0041] In hovering or constant-speed flight, assuming the real-time total lift equals the current gravity, the current total mass can be inverted using the following formula: ; in, The current total mass is continuously updated as hydrogen fuel is consumed, with the acceleration due to gravity as the current mass is increased.

[0042] When hovering or flying at a constant speed, the drone's acceleration is zero. At this point, the total lift equals the current gravity, which is the current total mass multiplied by the acceleration due to gravity. Therefore, the current total mass equals the total lift divided by the acceleration due to gravity. This mass estimate is continuously updated as hydrogen fuel is consumed.

[0043] In step S102, visual images are acquired, and a lightweight multimodal fusion network is used to perform unified fusion perception of the compensated LiDAR point cloud data, visual images, and IMU measurements in the BEV (Bird's Eye View) space, outputting obstacle distribution information and ground attribute information.

[0044] This embodiment employs a lightweight multimodal fusion network to achieve environmental perception. The network uses a dual-branch architecture to process visual images and LiDAR point cloud data separately, and the features from both paths are fused together in a unified manner within the bird's-eye view space.

[0045] The visual feature extraction branch employs a lightweight convolutional neural network to extract image features. Specifically, this embodiment uses MobileNetV4 as the backbone network, which has the advantages of small parameter count and low computational cost, making it suitable for deployment on resource-constrained airborne computing platforms. For each input frame of stereo image, multi-scale visual feature maps are first extracted using MobileNetV4, and then the image features are projected from the perspective view to the bird's-eye view space using the Lift-Splat-Shoot method. The core idea of ​​the Lift-Splat-Shoot method is to use the camera's intrinsic and extrinsic parameter matrices to elevate each pixel into a truncated cone in three-dimensional space, and then aggregate them into a bird's-eye view feature map through bird's-eye view pooling.

[0046] The LiDAR feature extraction branch employs a sparse convolutional network to extract point cloud features. Because LiDAR point clouds are sparse in 3D space, with most voxels being empty, sparse convolution only performs convolution operations on non-empty voxels, reducing computation by approximately 70% compared to traditional dense convolution. Specifically, the compensated LiDAR point cloud is first voxelized with a grid resolution of 0.2 meters. For each non-empty voxel, the average features (including 3D coordinates and reflection intensity) of all points within it are calculated as the feature vector for that voxel. Then, a multi-layer sparse convolutional network is used to extract the bird's-eye view feature map.

[0047] The bird's-eye view feature maps output from the visual branch and the LiDAR branch are concatenated along the channel dimension to form a fused bird's-eye view feature map. To further improve the fusion effect, two 3×3 convolutional layers and a Squeeze-and-Excitation attention module are introduced after concatenation, enabling the network to adaptively learn the importance weights of different modal features. The fused bird's-eye view feature map has a size of 200 by 200 grids, with each grid corresponding to a 0.2m by 0.2m area in actual space, covering a 40m by 40m range around the drone.

[0048] The fused bird's-eye view feature map is processed by three parallel task output heads. The 3D object detection head uses an anchor-point-based or center-point-based detection method to output the 3D position (i.e., the x, y, z coordinates of the obstacle's center point in the world coordinate system), 3D dimensions (i.e., the length, width, and height of the obstacle), orientation angle (i.e., the obstacle's orientation on the horizontal plane), and category (i.e., the semantic category of the obstacle, such as pedestrian, vehicle, tree, building, etc.) of each obstacle. The passable region segmentation head uses a pixel-by-pixel classification method to output the passability probability for each grid in the bird's-eye view feature map. The probability value is a continuous value between 0 and 1. Grids with a probability value higher than a preset threshold (e.g., 0.5) are considered passable regions, and grids with a probability value lower than the threshold are considered obstacle regions. The passability probabilities of all grids together constitute an obstacle cost map for use by the subsequent obstacle avoidance decision module. The ground attribute header outputs ground material classification results, including categories such as gravel, soil, concrete, metal, and grass. These classification results are inferred by a trained classifier based on the reflectance intensity distribution and geometric features of LiDAR point clouds. The ground attribute information is used in the subsequent landing assessment phase.

[0049] To achieve low-power real-time inference, this embodiment employs an optimized strategy of quantization-aware training and INT8 quantization deployment. During training, the effect of INT8 quantization is simulated by inserting pseudo-quantized nodes after each layer, allowing the network to adapt to the accuracy loss caused by quantization during the training phase. After training, TensorRT is used for INT8 quantization deployment, employing a per-tensor symmetric quantization method with an 8-bit quantization width. TensorRT automatically performs graph optimization and layer fusion during deployment, such as fusing convolutional layers, batch normalization layers, and ReLU activation functions into a single layer, further reducing computation and memory access. After these optimizations, the network can achieve an inference speed of at least 25 frames per second on Jetson Orin NX, with an overall power consumption not exceeding 8 watts. Furthermore, this embodiment introduces a temperature-aware adaptive accuracy mechanism. When the stack temperature exceeds 75 degrees Celsius and the onboard computing platform temperature exceeds 80 degrees Celsius, a accuracy degradation mechanism is triggered. The first level of degradation involves switching from INT8 to a mixed precision mode of INT4 and INT8 when the computing platform temperature is between 80 and 85 degrees Celsius. Critical layers maintain INT8 precision, while non-critical layers are reduced to INT4 precision. The second level of degradation involves switching to FP16 mode to reduce power consumption when the computing platform temperature exceeds 85 degrees Celsius, while simultaneously reducing the bird's-eye view mesh resolution to 0.3 meters to reduce computational load. When the computing platform temperature remains below 75 degrees Celsius for 30 consecutive seconds, the system gradually reverts to the high-precision INT8 mode.

[0050] The lightweight multimodal fusion network training implementation method of this application is as follows: (I) Training Dataset: This embodiment uses a combination of simulation and field testing to construct the training dataset. The simulation data is generated using AirSim or CARLA simulators, covering typical scenarios such as urban roads, industrial parks, grasslands, and gravel surfaces, totaling approximately 500,000 frames of image-LiDAR point cloud-label paired data. The field testing data is collected in real-world locations using drones equipped with the same sensor configuration, covering different lighting conditions such as sunny days, cloudy days, and backlighting, totaling approximately 50,000 frames.

[0051] (ii) Labeling method: 3D target detection labels use three-dimensional bounding box labeling, which includes the three-dimensional position (x,y,z), size (length, width, height), orientation angle and category (such as pedestrian, vehicle, tree, building) of the obstacle; passable area segmentation labels use pixel-by-pixel labeling, which indicates whether each BEV grid is passable by the UAV; ground attribute labels use area-level labeling, which indicates the ground type (such as concrete, asphalt, grass, gravel, mud, sand).

[0052] (III) Loss Function: The total loss function for multi-task tasks is: ; in, For 3D object detection loss, Smooth L1 Loss is used to calculate the position and size regression loss, and Focal Loss is used to calculate the classification loss; The loss for segmenting traversable regions is a weighted sum of Dice Loss and cross-entropy loss; For ground attribute classification, cross-entropy loss is used. The weight coefficients are set as follows: =1.0, =0.8, =0.6.

[0053] (iv) Training strategy: The Adam optimizer is used, and the initial learning rate is set to A cosine annealing decay strategy was adopted, with a batch size of 32 and a total of 120 training epochs. Data augmentation strategies, including random rotation, random scaling, random flipping, and color jitter, were used during training to improve the model's generalization ability.

[0054] (v) Deployment and Inference: The trained model is accelerated by FP16 precision quantization using TensorRT and deployed on an airborne embedded platform to achieve end-to-end real-time inference.

[0055] In step S103, based on obstacle distribution information, combined with real speed, real acceleration and current total mass, a trajectory planning model including fuel consumption cost index is constructed to optimize the hierarchical obstacle avoidance trajectory, and the obstacle avoidance response threshold is adaptively adjusted according to the current total mass to output the optimal obstacle avoidance trajectory.

[0056] This embodiment employs a three-layer progressive architecture: a long-range layer, a mid-range layer, and a short-range layer. The long-range layer, the outermost defense, covers distances greater than 50 meters. Based on a global path planning algorithm (such as an improved A algorithm or RRT algorithm), and combined with obstacle distribution information output from the previous steps, it calculates a set of pre-selected paths from the current position to the target point, providing macro-level decision-making for large-scale detours for the UAV. The mid-range layer, the core decision-making layer, covers distances between 10 and 50 meters. It performs local trajectory optimization based on a trajectory planning model, explicitly introducing a fuel consumption cost term into its cost function to achieve joint optimization of flight safety and energy consumption. The short-range layer, the innermost defense, covers distances less than 10 meters. When an obstacle enters the short-range area, it triggers an emergency braking and hovering strategy, serving as the last barrier to avoid collisions. The trigger distance thresholds for each layer are not fixed values ​​but are adaptively adjusted based on the current flight speed and total mass.

[0057] The mid-range layer employs a trajectory planning model for local trajectory optimization. This trajectory planning model is a model predictive controller (MMC). The basic principle of MMC is that in each control cycle, based on the UAV dynamics model, the future trajectory is predicted in the prediction time domain. The optimal control sequence is then solved by optimizing the cost function, and the first control variable is applied to the system. The process is repeated in the next cycle. In this embodiment, the prediction time domain is 3 seconds, and the control cycle is 0.1 seconds, meaning the MMC performs trajectory optimization once per control cycle.

[0058] In this embodiment, the cost function of the model predictive controller consists of multiple weighted terms, including trajectory tracking cost, control smoothing cost, collision cost, and fuel consumption cost; the cost function has the following form: ; in, For the current moment, To predict the time domain, To predict the state trajectory (including motion state quantities such as the position, velocity, attitude angle and angular velocity of the UAV). The reference trajectory is obtained by smoothing the global path provided by the distant layer. For control inputs (including speed commands for each rotor motor or desired acceleration and attitude angle commands). This is the collision cost term (obtained in real-time by looking up the obstacle cost map output from the previous steps; the closer the drone is to the obstacle, the larger the value of this cost term). For fuel consumption costs, Terminal status. The target state; These are the weighting coefficients for the trajectory tracking cost term, used to control the tracking accuracy of the UAV on the reference trajectory. The weighting coefficients of the smoothing cost term are used to control the magnitude of changes in the control input to avoid drastic maneuvers. These are the weighting coefficients for the collision cost term. These are the weighting coefficients for the terminal cost term. This represents the weighting coefficient for the fuel consumption cost item; This is the weight matrix for the trajectory tracking cost term. To control the weight matrix of the smoothing cost term, This is the weight matrix for the terminal cost term.

[0059] In the above cost function, For trajectory tracking cost term, To control the smoothing cost term, the two are respectively with and Multiply and then integrate; and Multiply and then integrate; and Multiplication is followed by integration; terminal cost term Multiply This then serves as the boundary constraint for the entire cost function.

[0060] In the embodiments of this application, a fuel consumption cost term is introduced. Its physical meaning is the rate at which hydrogen fuel is consumed per unit time under the current flight conditions, expressed in the form of equivalent power. Its calculation formula is: ; in, Hovering power is the minimum power required for a drone to maintain flight while hovering, and it is proportional to its current total mass. air density, The drag coefficient, For windward area, and The actual velocity and actual acceleration are respectively obtained from the aforementioned steps; The current total mass is obtained by inversion from the aforementioned steps; the three powers in the molecule correspond to different physical sources: The basic power consumption required to overcome gravity; This is the aerodynamic drag power, which corresponds to the power consumption required to overcome air resistance, and is proportional to the cube of the speed. For inertial power, which corresponds to the power consumption required to change the kinetic energy of the drone, power equals force multiplied by velocity, while inertial force equals mass multiplied by acceleration. In the denominator, The temperature of the fuel cell stack is collected in real time by a temperature sensor. The current load power, i.e., the total electrical power that the fuel cell stack currently needs to output, is calculated by the flight control system based on the current flight status. The instantaneous efficiency of the fuel cell stack represents the efficiency with which the hydrogen fuel cell stack converts the chemical energy of hydrogen fuel into electrical energy. It is obtained in real time based on the stack temperature and the current load power; it can be obtained by interpolation from a pre-calibrated efficiency MAP table.

[0061] The weighting coefficient γ of the fuel consumption cost term is adaptively adjusted based on the remaining hydrogen quantity. When the remaining hydrogen quantity is higher than 30% of the total capacity, γ takes the nominal value (usually set to 1.0). At this point, the weight of fuel consumption in the cost function is moderate, and the model predictive control considers energy consumption optimization while ensuring safety. When the remaining hydrogen quantity is lower than or equal to 30% of the total capacity, γ automatically increases to 2.5 to 3.0. At this point, the weight of fuel consumption in the cost function is significantly increased, and the model predictive control will prioritize obstacle avoidance trajectories with low hydrogen consumption. The core effect of this adaptive mechanism is that when hydrogen fuel is sufficient, the UAV can perform more aggressive obstacle avoidance maneuvers, while when hydrogen fuel is insufficient, the UAV will prioritize more conservative but lower-energy-consumption obstacle avoidance methods (such as slightly slowing down and detouring), even if the latter path is slightly longer. This mechanism effectively prevents situations where excessive hydrogen fuel consumption during obstacle avoidance maneuvers leads to the UAV being unable to return safely.

[0062] In the three-layer obstacle avoidance architecture, the trigger distance threshold for each layer is dynamically adjusted based on the current flight speed and current total mass, using the following formula: , , , in, 、 and These are preset base distance thresholds for the long-range, mid-range, and short-range layers, determined based on the UAV model, mission scenario, or preset security level configuration. For a typical quadcopter hydrogen-powered high-payload UAV, A distance of 50 meters corresponds to a flight time of approximately 3 to 5 seconds, which is sufficient to support global path replanning; The distance is 10 meters, corresponding to a flight time of approximately 1 second, which matches the prediction time domain of the model predictive control. The distance is 3 meters, which corresponds to the emergency braking trigger distance. Combined with the maximum deceleration of the heavy-load drone and the system delay (approximately 200 to 300 milliseconds), this is the safety boundary to ensure no collision. The speed adjustment factor is determined based on the ratio of the current flight speed to the cruise speed, and this ratio is limited to a preset minimum speed ratio and maximum speed ratio. The faster the speed, the earlier the response to obstacles is required, so the threshold increases with the increase of speed. The mass adjustment factor is determined based on the ratio of the current total mass to the empty mass, and this ratio is limited to a preset minimum mass ratio and maximum mass ratio. The larger the load, the greater the inertia and the longer the braking distance; therefore, the threshold increases with increasing mass. The upper and lower limits of the saturation function are set to prevent threshold runaway under extreme conditions: when the speed is too low, the drone approaches a hovering state, and the obstacle avoidance distance should not be further reduced; when the speed is too high, it is an abnormal flight state, and the obstacle avoidance distance should not be infinitely increased. The lower mass limit corresponds to the minimum mass after fuel depletion, and the upper mass limit corresponds to the maximum takeoff mass limited by structural strength. For a typical quadcopter hydrogen-powered high-payload drone, the minimum speed ratio is 0.5, the maximum speed ratio is 2.0, the minimum mass ratio is 0.8, and the maximum mass ratio is 1.8.

[0063] When an obstacle enters the proximity layer trigger range, an emergency braking hovering strategy is executed. The emergency braking strategy has two modes based on the remaining hydrogen level. When the remaining hydrogen level is higher than the safety threshold (20% of total capacity), a full-speed braking mode is executed, with the motors outputting maximum available power to generate maximum deceleration, allowing the UAV to brake and hover within the shortest distance. When the remaining hydrogen level is lower than or equal to the safety threshold, a controlled deceleration braking mode is executed. This mode calculates the maximum allowable deceleration under the current remaining hydrogen level, determined by the remaining hydrogen level, current speed, and obstacle distance. Braking is performed at the minimum required deceleration while ensuring no collision with the obstacle, keeping the power peak within the fuel cell's tolerance range, and simultaneously sending a low fuel warning signal to the ground station. The core idea of ​​controlled deceleration braking is to sacrifice some braking distance to ensure safety when fuel is insufficient, reducing instantaneous power peaks and preventing system failure due to power demand exceeding fuel cell capacity.

[0064] In step S104, when responding to a landing command, a multi-dimensional assessment is conducted based on ground attribute information, taking into account terrain flatness, ground bearing capacity, obstacle clearance, ground slope, and crosswind influence, and the landing point with the best comprehensive assessment is selected for landing.

[0065] When the UAV receives a landing command, it enters the landing assessment phase. This command may be automatically generated by the flight control system based on the mission plan, or it may be a manual command from the ground station or control tower. The assessment uses a circular area with a radius of 20 meters centered on the initial target landing point as the search area.

[0066] The multidimensional assessment includes five dimensions. The terrain smoothness score is calculated as follows: ground point cloud is extracted within a 5-meter radius of the landing area. Outliers are removed using the RANSAC plane fitting algorithm. Principal component analysis is performed on the ground point cloud, and eigenvalues ​​are calculated in the three principal component directions. The minimum eigenvalue is recorded as the smoothness eigenvalue, reflecting the degree of dispersion of the point cloud in the plane normal direction, i.e., the thickness of the ground. The smaller the eigenvalue, the smoother the ground. The smoothness score is defined as an exponential function with the natural constant e as the base and the negative smoothness eigenvalue divided by the smoothness normalization constant (typically 0.01 to 0.05 square meters, calibrated experimentally). The score ranges from 0 to 1, with the closer the eigenvalue is to 0, the closer the score is to 1.

[0067] The ground bearing capacity score is calculated based on the ground attribute information output from the preceding steps. The bearing capacity parameters are mapped according to the ground material classification results. For example, the bearing capacity of concrete surfaces is typically higher than 200 kPa, metal surfaces are typically higher than 500 kPa, gravel surfaces are between 100 and 200 kPa, soil surfaces are between 50 and 150 kPa, and grass surfaces are between 30 and 80 kPa. Combining the current total mass output from the preceding steps, the safety factor is calculated as the bearing capacity divided by the current total mass multiplied by the gravitational acceleration, and then divided by the landing gear contact area. The bearing capacity score is defined as the safety factor divided by the smaller of 2 and 1. A score of 1 is given when the safety factor is not less than 2; otherwise, the score is proportional to the safety factor.

[0068] The obstacle clearance score is calculated as follows: In the point cloud below 2 meters in height around the landing point, all point cloud clusters with a detection height greater than 0.3 meters are considered obstacles. The horizontal distance from the landing point to the nearest obstacle is calculated, and the usable clearance radius is this horizontal distance minus a safety margin of 0.5 meters. The clearance score is defined as the usable clearance radius divided by the smaller of the maximum radius of the UAV fuselage (including the rotor) and 1. When the usable clearance radius is greater than the radius of the UAV fuselage, the score is 1.

[0069] The ground slope score is calculated as follows: The plane normal vector is extracted from the RANSAC plane fitting results, and the angle between it and the unit vector of gravity is calculated. This angle is the ground slope angle. The slope score is defined as an exponential function with the natural constant e as the base and the square of the negative slope angle divided by the square of the slope normalization constant (taken as 5 degrees) as the exponent. The score is 1 when the slope angle is 0 degrees, and the larger the slope, the lower the score.

[0070] The crosswind margin score is calculated as follows: Current wind speed and direction are obtained using an onboard anemometer or a wind speed estimation algorithm based on IMU and GPS. The crosswind component is then calculated, which is the wind speed multiplied by the sine of the angle between the wind direction and the landing direction. The crosswind score is defined as the difference between the maximum crosswind the drone can withstand and the crosswind component, divided by the smaller of 2 and 1. A score of 1 is given when the crosswind margin is greater than 2 meters per second.

[0071] The scores from the five dimensions are weighted and summed to calculate the overall score, as shown in the formula below: in, , , , and The preset weighting coefficients are for five dimensions: terrain flatness, ground bearing capacity, obstacle clearance, ground slope, and crosswind margin. The values ​​of each weighting coefficient are determined according to the safety requirements of the landing scenario. The terrain smoothness score is calculated based on the fitting residuals of the ground point cloud in the landing area. The ground bearing capacity score is calculated based on the load-bearing capacity mapped from the ground attribute information and the current total mass. This is an obstacle clearance score calculated based on obstacle detection around the landing point. This is a ground slope score calculated based on the angle between the ground normal vector and the direction of gravity. This is the crosswind margin score calculated based on the crosswind component.

[0072] For example, when landing a hydrogen-powered, high-payload UAV on an unstructured site, the weights of the five dimensions mentioned above can be set to 0.25, 0.30, 0.20, 0.15, and 0.10, respectively. This weighting is based on the following: high-payload UAVs are most sensitive to ground bearing capacity; if the ground cannot support the weight of the UAV, the landing gear may become stuck or the fuselage may overturn. Therefore, ground bearing capacity is given the highest weight. Terrain flatness is the next most important factor, as uneven ground may lead to excessive landing impact or overturning. Obstacle clearance ensures sufficient unobstructed space around the landing point. Ground slope and crosswind effects are secondary factors.

[0073] Within the landing area, grid sampling is performed in 1-meter increments. A comprehensive score is calculated for each candidate point, and the point with the highest score is selected as the optimal landing point. After determining the optimal landing point, before the final landing, the UAV is controlled to descend to a preset height (e.g., 2 meters) above the landing point and hover. The downward-looking sensor is activated to perform a secondary re-inspection of the landing point. The secondary re-inspection includes: acquiring a high-precision terrain point cloud of the local area of ​​the landing point using a downward-looking laser rangefinder or a downward-looking depth camera to re-inspect the ground flatness and local slope; acquiring a high-resolution image using a downward-looking camera to re-inspect the ground material texture and whether there are any temporary obstacles; if the consistency between the secondary re-inspection result and the ground attribute information is higher than a preset threshold, the final landing is performed; if the deviation between the secondary re-inspection result and the ground attribute information exceeds a preset threshold, the landing point search and evaluation are re-executed, or the ground bearing capacity score of the landing point is multiplied by a reduction factor less than 1 (e.g., 0.5) and the comprehensive score is recalculated.

[0074] If the overall score is lower than the preset threshold (e.g., 0.6), it indicates that there is no landing point that meets the safety requirements within the search area, and the hovering waiting mode (waiting for environmental conditions to improve or manual intervention) or the alternate landing point search mode (expanding the search area or flying to the alternate airport) is triggered.

[0075] It should be noted that the environment is not static during autonomous landing. New personnel, vehicles, or objects blown away by the wind may turn the originally selected optimal landing point into an unsafe area. Therefore, in this embodiment, the fusion perception process in step S102 is continuously run during the landing implementation phase, and the real-time perception results are fed back to the trajectory planning module in step S103 and the landing evaluation module in step S104.

[0076] Specifically, the overall score of the currently selected landing point is recalculated at a preset interval (e.g., 0.5 seconds). .like Below the preset safety threshold If a new dynamic obstacle is detected within a safe radius (e.g., 1.5 times the fuselage diameter) around the landing point, dynamic replanning is triggered: the UAV immediately hovers and ascends to a safe altitude, re-executes the trajectory planning in step S103 and the landing point search in step S104, and selects the optimal landing point under the new environmental conditions. If the number of consecutive replanning failures exceeds a preset limit (e.g., 3 times), a landing abort command is triggered, the UAV switches to a hovering standby state and reports to the ground station.

[0077] like Figure 2 As shown, this application provides a multi-sensor fusion low-power obstacle avoidance and safe landing system 200 for hydrogen-powered high-payload unmanned aerial vehicles (UAVs), including: The data acquisition and compensation module 210 is used to acquire IMU measurement values, rotor motor speed signals and raw LiDAR point cloud data, apply motion distortion compensation based on rotor vibration estimation to the LiDAR point cloud data, output the compensated LiDAR point cloud data, and simultaneously calculate the true velocity, true acceleration and current total mass based on rotor lift inversion after removing vibration noise. The fusion perception module 220 is used to acquire visual images. It adopts a lightweight multimodal fusion network to perform unified fusion perception of the compensated LiDAR point cloud data, the visual images, and IMU measurements in the BEV space, and outputs obstacle distribution information and ground attribute information. The obstacle avoidance decision module 230 is used to construct a trajectory planning model including a fuel consumption cost index based on the obstacle distribution information, combined with the real speed, real acceleration and current total mass, to perform hierarchical obstacle avoidance trajectory optimization, and to adaptively adjust the obstacle avoidance response threshold according to the current total mass, and output the optimal obstacle avoidance trajectory. The landing assessment module 240 is used to perform a multi-dimensional assessment based on the ground attribute information, including terrain flatness, ground bearing capacity, obstacle clearance, ground slope and crosswind influence, in response to a landing command, and select the landing point with the best comprehensive assessment to carry out the landing.

[0078] The above modules correspond to steps S101 to S104 and can be deployed on an airborne computing platform (such as Jetson OrinNX) to communicate data between modules through shared memory or message queues.

[0079] The data flow between the modules is as follows: the data acquisition and compensation module 210 outputs the compensated LiDAR point cloud data, actual velocity and acceleration, and current total mass to the fusion perception module 220 and the obstacle avoidance decision module 230, respectively; the fusion perception module 220 outputs obstacle distribution information and ground attribute information to the obstacle avoidance decision module 230 and the landing evaluation module 240, respectively; the obstacle avoidance decision module 230 outputs the optimal obstacle avoidance trajectory to the flight control system; and the landing evaluation module 240 outputs the optimal landing point coordinates to the flight control system after receiving the landing command.

[0080] This application also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the multi-sensor fusion low-power obstacle avoidance and safe landing method for hydrogen-powered high-payload unmanned aerial vehicles as described in any of the above embodiments of this application. The computer-readable storage medium can be any tangible medium capable of storing a computer program, including but not limited to hard disks, solid-state drives, flash drives, read-only memory, random access memory, optical disks, or magnetic disks. The computer program can be stored in the form of source code, object code, or intermediate code between source code and object code. The source code can be written in programming languages ​​such as C++ and Python.

[0081] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0082] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0083] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A multi-sensor fusion low-power obstacle avoidance and safe landing method for hydrogen-powered high-payload unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: Acquire IMU measurements, rotor motor speed signals, and raw LiDAR point cloud data; apply motion distortion compensation based on rotor vibration estimation to the LiDAR point cloud data, and output the compensated LiDAR point cloud data; Simultaneously calculate the true velocity, true acceleration, and current total mass based on rotor lift inversion after removing vibration noise; Visual images are acquired, and a lightweight multimodal fusion network is used to perform unified fusion perception of the compensated LiDAR point cloud data, the visual images, and IMU measurements in the BEV space, and output obstacle distribution information and ground attribute information. Based on the obstacle distribution information, combined with the actual speed, actual acceleration and current total mass, a trajectory planning model including fuel consumption cost index is constructed to optimize the hierarchical obstacle avoidance trajectory, and the obstacle avoidance response threshold is adaptively adjusted according to the current total mass to output the optimal obstacle avoidance trajectory. When responding to a landing command, the system performs a multi-dimensional assessment based on the ground attribute information, taking into account factors such as terrain flatness, ground bearing capacity, obstacle clearance, ground slope, and crosswind influence, and selects the landing point with the best overall assessment to carry out the landing.

2. The method according to claim 1, characterized in that, In the motion distortion compensation based on rotor vibration estimation, the rotor vibration is estimated in the following way: The rotor motor is operated at a stepper speed, and acceleration response data is collected at the LiDAR mounting location. The rotor passing frequency and its harmonic components at each speed are identified, and the vibration transfer function is fitted. The amplitude-frequency response and phase-frequency response, the vibration transfer function adopts a superposition form of a second-order resonance model: in, It is a positive integer not less than 2. The frequency of vibration. For the first The amplitude coefficient of the first harmonic. For the first k First harmonic angular frequency, For the first Damping ratio of first harmonic ； For harmonic orders, The imaginary unit; Will The parameterization is done by using a lookup table indexed by rotor speed.

3. The method according to claim 2, characterized in that, The application of motion distortion compensation based on rotor vibration estimation to LiDAR point cloud data specifically includes: Vibration filtering preprocessing is performed on the accelerometer data and gyroscope data in the IMU measurement values. A notch filter is used to filter out the vibration signal components at the rotor passing frequency and its harmonic frequencies to obtain the true angular velocity and true acceleration after removing vibration noise. IMU pre-integration is performed based on the actual angular velocity and actual acceleration to obtain the data from the start of the frame. to the corresponding time of the laser point Transformation of the actual motion posture of the organism between ; Obtain the timestamp of each laser point in the LiDAR point cloud data, and query the lookup table based on the current rotor speed to predict the vibration displacement of the laser point at the corresponding moment. and vibration rotation amount The vibration displacement includes vibration displacement components with three translational degrees of freedom, and the vibration rotation includes vibration angular components with three rotational degrees of freedom. Transform the actual motion posture of the machine body With the vibration displacement and vibration rotation amount Perform synthesis to construct the compensation transformation matrix for the laser point. The coordinates of the laser point are transformed from the instantaneous measurement coordinate system to the frame start unified reference coordinate system using the compensation transformation matrix. The compensated coordinates of the laser point are: in, For the first The compensation transformation matrix corresponding to each laser point at any given time is obtained by synthesizing the true motion pose, vibration displacement, and vibration rotation after vibration removal from the IMU pre-integration. These are the original coordinates of the laser point in the instantaneous measurement coordinate system. The coordinates of the laser point after compensation are given in the unified reference coordinate system at the beginning of the frame.

4. The method according to claim 1, characterized in that, The current total mass based on rotor lift inversion is obtained as follows: based on the rotor motor speed signal, the real-time total lift is calculated by inversely using the mapping relationship between rotor speed and lift: in, This represents the total number of rotor motors. This is the motor serial number. For the first The rotor speed of each motor, This is the lift calibration coefficient; In hovering or constant-speed flight, assuming the real-time total lift equals the current gravity, the current total mass can be inverted using the following formula: in, The current total mass is continuously updated as hydrogen fuel is consumed, with the acceleration due to gravity as the current mass is increased.

5. The method according to claim 1, characterized in that, The lightweight multimodal fusion network includes a visual feature extraction branch and a LiDAR feature extraction branch. The visual feature extraction branch uses a lightweight convolutional neural network to extract image features, while the LiDAR feature extraction branch uses a sparse convolutional network to extract point cloud features. The two features are projected onto the BEV space and then stitched and fused to output a unified BEV feature map. The BEV feature map is output in parallel through a multi-task output head: the 3D target detection head outputs the three-dimensional position, size, orientation and category of the obstacle; the passable area segmentation head outputs the passability probability of each grid area to construct obstacle distribution information; and the ground attribute head outputs the ground material classification results to constitute ground attribute information.

6. The method according to claim 1, characterized in that, The trajectory planning model is a model predictive controller, and its cost function consists of multiple weighted terms, including trajectory tracking cost, control smoothing cost, collision cost, and fuel consumption cost. The cost function has the following form: in, For the current moment, To predict the time domain, To predict the state trajectory, For reference trajectory, To control the input, For collision cost terms, For fuel consumption costs, Terminal status. The target state; These are the weighting coefficients for the trajectory tracking cost term. To control the weighting coefficients of the smoothing cost term, These are the weighting coefficients for the collision cost term. These are the weighting coefficients for the terminal cost term. This represents the weighting coefficient for the fuel consumption cost item; This is the weight matrix for the trajectory tracking cost term. To control the weight matrix of the smoothing cost term, This is the weight matrix for the terminal cost term; The fuel consumption cost item The calculation formula is: in, For hovering power, air density, The drag coefficient, For windward area, and These are the actual velocity and the actual acceleration, respectively. For the current total mass, The temperature of the fuel cell stack is collected in real time by a temperature sensor. The current load power is calculated by the flight control system based on the current flight status. The instantaneous efficiency of the fuel cell stack is obtained in real time based on the stack temperature and the current load power. The weighting coefficient of the fuel consumption cost item The weighting coefficient is adaptively adjusted based on the remaining hydrogen quantity: when the remaining hydrogen quantity is higher than a preset threshold, the weighting coefficient takes the nominal value; when the remaining hydrogen quantity is lower than or equal to the preset threshold, the weighting coefficient increases, so that the trajectory planning model prioritizes the selection of low hydrogen consumption trajectories.

7. The method according to claim 1, characterized in that, The hierarchical obstacle avoidance trajectory optimization adopts a three-layer progressive architecture: long-range layer, mid-range layer, and short-range layer; the obstacle avoidance response threshold is adaptively adjusted in the following manner: in, 、 and These are preset base distance thresholds for the long-range, mid-range, and short-range layers, determined based on the drone model, mission scenario, or preset security level configuration. It is a speed adjustment factor, determined based on the ratio of the current flight speed to the cruise speed, and this ratio is limited to a preset minimum speed ratio and maximum speed ratio. It is a mass adjustment factor, determined based on the ratio of the current total mass to the unloaded mass, and this ratio is limited to a preset minimum mass ratio and maximum mass ratio.

8. The method according to claim 1, characterized in that, Based on the aforementioned ground attribute information, a multi-dimensional assessment is performed, integrating terrain flatness, ground bearing capacity, obstacle clearance, ground slope, and crosswind influence, and calculated using a weighted sum method: in, , , , and The preset weighting coefficients are for five dimensions: terrain flatness, ground bearing capacity, obstacle clearance, ground slope, and crosswind margin. The terrain smoothness score is calculated based on the fitting residuals of the ground point cloud in the landing area. The ground bearing capacity score is calculated based on the load-bearing capacity mapped from the ground attribute information and the current total mass. This is an obstacle clearance score calculated based on obstacle detection around the landing point. This is a ground slope score calculated based on the angle between the ground normal vector and the direction of gravity. The crosswind margin score is calculated based on the crosswind component; the point with the highest comprehensive score is searched in the landing area using a grid sampling method as the optimal landing point.

9. A multi-sensor fusion low-power obstacle avoidance and safe landing system for hydrogen-powered high-payload unmanned aerial vehicles, characterized in that, include: The data acquisition and compensation module is used to acquire IMU measurement values, rotor motor speed signals and raw LiDAR point cloud data, apply motion distortion compensation based on rotor vibration estimation to the LiDAR point cloud data, output the compensated LiDAR point cloud data, and simultaneously calculate the true velocity, true acceleration and current total mass based on rotor lift inversion after removing vibration noise. The fusion perception module is used to acquire visual images. It adopts a lightweight multimodal fusion network to perform unified fusion perception of the compensated LiDAR point cloud data, the visual images, and IMU measurements in the BEV space, and outputs obstacle distribution information and ground attribute information. The obstacle avoidance decision module is used to construct a trajectory planning model that includes a fuel consumption cost index based on the obstacle distribution information, combined with the real speed, real acceleration and current total mass, to perform hierarchical obstacle avoidance trajectory optimization, and to adaptively adjust the obstacle avoidance response threshold according to the current total mass, and output the optimal obstacle avoidance trajectory. The landing assessment module is used to conduct a multi-dimensional assessment based on the ground attribute information when responding to a landing command, taking into account factors such as terrain flatness, ground bearing capacity, obstacle clearance, ground slope, and crosswind influence, and to select the landing point with the best comprehensive assessment to carry out the landing.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-sensor fusion low-power obstacle avoidance and safe landing method for hydrogen-powered high-payload unmanned aerial vehicles as described in any one of claims 1 to 8.

Images