Unmanned aerial vehicle autonomous obstacle avoidance method and system based on point cloud perception and deep learning

Abstract

The application discloses a UAV autonomous obstacle avoidance method and system based on point cloud perception and deep learning, and belongs to the technical field of UAV autonomous flight control. The UAV six-degree-of-freedom pose is solved in real time by using a laser inertial odometer calculation method, through the collection of environment point clouds and motion states by an airborne 3D LiDAR and an IMU; the high-dimensional point clouds are converted into low-dimensional environment observation vectors by angle partitioning, extreme distance extraction and unknown area modeling; the fused state vectors are input into a lightweight multi-layer perception machine strategy network, a control action instruction is output through nonlinear feature coding, and the control action instruction is mapped into a bottom layer control quantity to drive the UAV to realize real-time obstacle avoidance; the strategy network is trained based on a proximal policy optimization algorithm; the system is composed of a multi-modal perception unit, a state estimation unit, a feature engineering unit, a deep reinforcement learning unit and a flight control interface unit; and the application realizes stable, efficient and low-delay autonomous obstacle avoidance in a complex unknown environment without GPS and dense obstacles.

Description

Technical Field

[0001] This invention relates to the field of autonomous flight control technology for unmanned aerial vehicles (UAVs), specifically to an autonomous obstacle avoidance method and system for UAVs based on point cloud perception and deep learning. Background Technology

[0002] Achieving fully autonomous flight of unmanned aerial vehicles (UAVs) in environments without GPS and with dense obstacles is a core technological challenge in the field of robotics and intelligent unmanned systems. Traditional autonomous flight systems mostly adopt a layered architecture of perception-mapping-planning-tracking, relying on external positioning equipment. This results in problems such as high system latency, easy error accumulation, and poor adaptability to complex environments, making it difficult to meet the real-time requirements of high-speed, high-maneuverability flight. With the rapid development of 3D LiDAR and embedded computing platforms, direct perception control based on LiDAR point clouds has become a research hotspot. However, problems such as low efficiency in high-dimensional point cloud processing, large gap between simulation and real-world domains, and insufficient generalization of end-to-end strategies still restrict engineering implementation.

[0003] In recent years, 3D LiDAR has gradually replaced visual sensors as the mainstream solution for UAVs to perceive complex environments due to its advantages such as strong anti-interference and direct geometric perception. However, the high dimensionality and redundant information of raw point cloud data make it easy for traditional downsampling and feature extraction methods to lose key information about small obstacles, leading to obstacle avoidance failure. At the same time, the lack of key technologies such as point cloud temporal fusion and unknown area modeling significantly increases the decision-making risk of UAVs in some observable environments.

[0004] In academic research, the Fast Lab team at Zhejiang University published a paper on arXiv titled "Flying on PointClouds with Reinforcement Learning" (arXiv:2503.00496), combining 3D LiDAR point clouds with reinforcement learning to construct an end-to-end flight strategy, significantly reducing reliance on intermediate maps and trajectory optimization. The Shanghai Jiao Tong University team proposed a mapless obstacle avoidance method based on dual KD-Trees (arXiv:2503.10141), which rapidly calculates safety constraints using sparse point clouds, improving local planning efficiency. However, these methods still suffer from drawbacks such as redundant point cloud representations, limited training batches, and insufficient robustness in simulation-to-real-world transfer, resulting in a significant decrease in success rate in high-speed dynamic scenarios.

[0005] Among Chinese patent technologies, CN113376658A discloses an autonomous obstacle avoidance method for UAVs based on single-line lidar, which improves obstacle avoidance stability in static environments by using distance calculation and clustering processing to determine obstacles. CN112965517B discloses a UAV inspection safety obstacle avoidance system and method based on binocular vision fusion lidar and electromagnetic field detection, proposing a multi-sensor fusion obstacle avoidance system that integrates binocular vision and lidar data to enhance the reliability of perception in complex electromagnetic environments. However, the former only supports two-dimensional planar perception and cannot cope with small obstacles and dynamic obstacles in three-dimensional space; the latter relies on redundant fusion of multiple sensors, resulting in high computational overhead and high latency, making it difficult to meet the real-time control requirements of airborne embedded platforms, and it does not have specific optimizations for efficient point cloud representation and reinforcement learning transfer. Summary of the Invention

[0006] To address the aforementioned technical issues, this application discloses a method and system for autonomous obstacle avoidance by unmanned aerial vehicles (UAVs) based on point cloud perception and deep learning; the method includes the following steps:

[0007] S1. The airborne 3D LiDAR sensor is used to collect environmental point cloud flow at a set sampling frequency, and combined with the data from the inertial measurement unit, the six-degree-of-freedom pose state of the UAV is calculated in real time using the laser inertial odometry method.

[0008] S2. Construct an efficient perception representation by transforming historical time-series point cloud data to the current machine coordinate system, dividing the space into multiple sectors using an angle partitioning strategy, extracting extreme distance features within each sector, and combining them with unknown region modeling strategies to construct a low-dimensional environmental observation vector.

[0009] S3. The observation vector constructed in S2 is fused with the state estimation vector calculated in S1 and input into the pre-trained lightweight multilayer perceptron policy network. The control action command at the current moment is output through nonlinear feature encoding.

[0010] S4 maps the control action commands output by S3 to the underlying control quantities of the flight control system, driving the UAV to perform corresponding thrust and torque adjustments, thereby achieving real-time obstacle avoidance in unknown environments.

[0011] Preferably, the efficient perception representation construction in S2 specifically includes: defining the space around the machine as... The uniformly distributed sector, for the first Point cloud collection within a sector Extracting feature values The feature extraction process is defined as a piecewise mapping function. When point cloud data exists within a sector, the minimum Euclidean distance is extracted. Saturated nonlinear processing is then performed; when no point cloud data exists within a sector, the distance to the unknown region in that direction is calculated. And map it to extended feature values; construct observation vectors The mathematical formula is: ,in, To maximize the sensing cutoff distance, For unknown regions, the weighting coefficient is denoted as .

[0012] Preferably, the point cloud processing in S2 further includes voxelization downsampling and motion compensation; the resolution vector of the voxel mesh is defined as... A hash table is used to store the occupied voxel cells; for moving drones, the angular velocity is calculated from the IMU. History Motion distortion correction is performed on the frame point cloud, and the corrected point cloud coordinates are obtained. By rotation matrix Perform a rigid transformation: ,in, It is a translation vector to ensure the geometric consistency of the point cloud in the body coordinate system.

[0013] Preferably, the lightweight multilayer perceptron policy network in S3 adopts a fully connected feedforward neural network architecture; the input layer feature dimension is defined as... The number of hidden layers is , No. The dimension of the hidden layer is The forward propagation process of the policy network is represented as a nonlinear function approximation. , its first The feature transformation formula for the layer is: ,in, and The first Layer weight matrix and bias vector, It is a non-linear activation function; the output layer is directly mapped to... 3D control space, containing normalized thrust commands With angular velocity command .

[0014] Preferably, the output layer of the policy network adopts a linear activation strategy, directly regressing the increment of the control action. Define the mapping function for the action space as follows: ,in Let this be the action vector at the current moment. This represents the current system state. The state is the action scaling factor. Including observation vectors Current machine speed Posture Quaternions Target position and direction vector and the actions of the previous moment .

[0015] Preferably, the policy network in S3 is trained based on a proximal policy optimization algorithm, and the training process constructs a Markov decision process in a simulation environment; a reward function is defined. Includes forward reward items Control quantity penalty items Collision penalty items and obstacle distance bonus The optimization objective function of the PPO algorithm The formula is: ,in, For the empirical expectation operator, The probability ratio is given by the formula: , For the current policy network, For old policy networks, This is the estimated value of the dominance function. This is for pruning hyperparameters.

[0016] Preferably, the training process incorporates a dynamic domain randomization strategy to narrow the gap between simulation and reality; in the simulation environment, the dynamic parameters of the UAV are modeled with random perturbations, and the thrust coefficient is defined. Follow the interval Uniform distribution on the surface, air drag coefficient Follow the interval Random sampling; simultaneously, zero-mean Gaussian noise is injected into the state observations. This simulates the measurement error of a real sensor.

[0017] Preferably, the state calculation in S1 and the control execution in S4 are scheduled in real time through an onboard embedded platform; the system control cycle is... The sensing module uses frequency Running, policy network inference at frequency The process employs a timestamp alignment strategy to interpolate and synchronize point cloud data at different frequencies with state estimation data, ensuring the state vector of the input policy network is synchronized. Corresponding to the current time The true physical state.

[0018] The autonomous obstacle avoidance system for drones based on point cloud perception and deep learning includes:

[0019] The multimodal sensing unit, including a 3D LiDAR sensor and an IMU sensor, is used to acquire information about the geometric structure of the environment and the motion state of the UAV.

[0020] The real-time state estimation unit is equipped with a laser inertial odometry calculation method, which integrates the data from the multimodal sensing unit and outputs high-frequency pose, velocity and acceleration state quantities.

[0021] The feature engineering processing unit is used to execute the piecewise mapping function to convert the original point cloud flow into a low-dimensional vector containing features of unknown regions.

[0022] The deep reinforcement learning unit includes a model loading module and an inference engine. The model loading module is used to load the parameters of the MLP neural network trained based on the PPO algorithm, and the inference engine is used to perform nonlinear feature transformation and calculate the control strategy.

[0023] The flight control interface unit is used to decode the abstract action vector output by the strategy network into PWM signals or MAVLINK commands that can be recognized by the flight control system, and drive the actuators to complete obstacle avoidance actions.

[0024] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned autonomous obstacle avoidance method for unmanned aerial vehicles based on point cloud perception and deep learning.

[0025] Compared with the prior art, the technical solution of this application has the following technical effects:

[0026] This invention constructs a low-dimensional point cloud perception representation by angular partitioning and extreme distance feature extraction. It can efficiently preserve environmental geometric details and complete explicit modeling of unknown areas, effectively improving the UAV's ability to identify and judge obstacles in complex spaces, ensuring comprehensive perception and risk prediction of surrounding obstacles during autonomous flight, and providing reliable environmental perception support for stable obstacle avoidance.

[0027] This invention employs a lightweight multilayer perceptron to construct a strategy network, using a simple, fully connected architecture to complete state fusion and action output, reducing onboard computational overhead and inference latency, enabling rapid generation and execution of control commands, ensuring that the UAV responds promptly to obstacle threats in a dynamically changing environment, and maintaining the real-time performance and continuity of flight control.

[0028] This invention enhances the generalization ability and robustness of policy networks in real-world scenarios through domain randomization training in a simulated environment and simulation-to-reality transfer design. This enables UAVs to adapt to different dynamic disturbances and sensor noise, maintain stable decision-making and reliable execution in diverse and complex scenarios, and improve the environmental adaptability of autonomous flight systems.

[0029] This invention enables integrated airborne operation of the entire process of point cloud perception, state estimation, strategy reasoning, and control execution, without the need for external positioning and offline computing support. It can independently complete autonomous obstacle avoidance tasks in GPS-free environments, simplifying system deployment conditions, improving the flexibility and practicality of UAV autonomous operations, and meeting the practical application needs of diverse scenarios.

[0030] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0031] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0033] Based on the description of the figures and their corresponding technical content in the document, the titles of the figures are as follows:

[0034] Figure 1 A schematic diagram of the overall process of an autonomous obstacle avoidance method for UAVs based on point cloud perception and deep learning;

[0035] Figure 2 A diagram illustrating the hierarchical structure and forward propagation architecture of a lightweight multilayer perceptron policy network;

[0036] Figure 3 The overall flowchart of the policy network training and proximal policy optimization algorithm;

[0037] Figure 4 A comparative graph showing how the cumulative reward for different obstacle avoidance strategies changes with the number of training steps;

[0038] Figure 5 A comparative graph showing the change in mission success rate of different obstacle avoidance schemes with flight speed;

[0039] Figure 6 Scatter plot showing the distribution of control error of different obstacle avoidance schemes as a function of environmental disturbance intensity;

[0040] Figure 7 A block diagram of an autonomous obstacle avoidance system for unmanned aerial vehicles (UAVs) based on point cloud perception and deep learning. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0042] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0043] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0044] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0045] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0046] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0047] Example 1

[0048] This embodiment mainly describes an autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs) based on point cloud perception and deep learning, such as... Figure 1 As shown, it specifically includes:

[0049] S1. The airborne 3D LiDAR sensor is used to collect environmental point cloud flow at a set sampling frequency, and combined with the data from the inertial measurement unit, the six-degree-of-freedom body attitude state of the UAV is calculated in real time using the laser inertial odometry method.

[0050] S2. Construct an efficient perception representation by transforming historical time-series point cloud data to the current machine coordinate system, dividing the space into multiple sectors using an angle partitioning strategy, extracting extreme distance features within each sector, and combining them with unknown region modeling strategies to construct a low-dimensional environmental observation vector.

[0051] S3. The observation vector constructed in S2 is fused with the state estimation vector calculated in S1 and input into the pre-trained lightweight multilayer perceptron policy network. The control action command at the current moment is output through nonlinear feature encoding.

[0052] S4 maps the control action commands output by S3 to the underlying control quantities of the flight control system, driving the UAV to perform corresponding thrust and torque adjustments, thereby achieving real-time obstacle avoidance in unknown environments.

[0053] Furthermore, a multimodal sensing system is constructed using an airborne 3D LiDAR sensor and an inertial measurement unit. The two types of sensors achieve precise timestamp alignment through a hardware synchronization module, eliminating timing deviations in the data acquisition process. The 3D LiDAR continuously outputs a three-dimensional point cloud stream at a set sampling frequency. Each sampling point in the point cloud stream contains three-dimensional spatial coordinates and reflection intensity information, fully covering the environmental geometry within a preset sensing radius around the UAV. The inertial measurement unit outputs raw triaxial acceleration, triaxial angular velocity, and attitude data at a sampling frequency much higher than that of the LiDAR, providing a high-frequency motion state reference for subsequent point cloud motion distortion compensation and body posture calculation.

[0054] A tightly coupled data fusion framework based on laser inertial odometry (IMU) is constructed. This framework dynamically fuses feature points from LiDAR point clouds with IMU pre-integration results. Specifically, the relative pose transformation between adjacent time points is first calculated using IMU pre-integration to eliminate motion distortion within short timescales. Then, the iterative nearest-point algorithm of the LiDAR point cloud is used to match adjacent frame point clouds to obtain pose estimates under environmental geometric constraints. Finally, the two types of information are fused through a residual correction mechanism to output the UAV's six-DOF body posture state in real time. This posture state includes a three-dimensional spatial position vector, attitude information represented by quaternions, linear velocity, and angular velocity. The overall state vector dimension is a fixed value, providing a high-precision spatiotemporal reference for subsequent point cloud coordinate transformation and policy network input. The state calculation frequency is synchronized with the control cycle to ensure the physical consistency and temporal continuity of the data.

[0055] Furthermore, for multi-frame point cloud data from historical time series, voxelization downsampling is first performed to reduce data redundancy; voxelization downsampling uses a fixed-resolution 3D voxel mesh, with the mesh resolution vector denoted as... The original point cloud is mapped to a voxel grid space, retaining only the center point coordinates within each voxel unit. This compresses the number of point clouds while preserving the core features of the spatial distribution of the environment, thus avoiding the impact of redundant data on the subsequent construction of perceptual representations and the efficiency of network inference.

[0056] Based on the six-DOF pose states calculated in step one, the historical... The frame point cloud undergoes motion distortion compensation and rigid coordinate transformation, uniformly converting it to the current body coordinate system. This rigid transformation is achieved through a rotation matrix and a translation vector, with the rotation matrix derived from the angular velocity calculated by the inertial measurement unit. In time interval The inner integral yields the result, denoted as The translation vector is the difference between the current position and the historical position of the organism, denoted as . The core formula for point cloud coordinate transformation strictly matches the claims and figures, specifically: In the formula, These are the original coordinates of the historical point cloud in its own coordinate system. To unify the point cloud coordinates to the current body coordinate system after transformation, the transformation process achieves accurate construction of the rotation matrix through integral angular velocity, eliminating point cloud distortion caused by the high-speed movement of the UAV, ensuring the spatial consistency and geometric accuracy of the subsequent perception representation construction, and providing a reliable point cloud data foundation for feature extraction.

[0057] Furthermore, for angular partitioning and extreme distance feature extraction, the three-dimensional space is uniformly partitioned according to azimuth and pitch angles, with the origin of the UAV body coordinate system as the center. The overall space is divided into... Each sector corresponds to a fixed angle range, forming a high-resolution environmental perception grid structure. For the first sector... Each sector, and its corresponding point cloud set denoted as . For each sector, extreme distance feature extraction is performed, and the extraction rules are divided into two cases:

[0058] When there is a valid point cloud in the sector, that is At that time, calculate the Euclidean distance from all points within the sector to the center of the machine, and take the minimum value as the core distance feature of the sector, denoted as . To prevent distance values ​​from exceeding the perception range, a maximum perception cutoff distance is introduced. With saturation constraints applied, the corresponding feature extraction formula is: In the formula, Let be the coordinate vector of a point within the sector. This is the calculated value for Euclidean distance. This represents the obstacle distance feature value corresponding to this sector.

[0059] When there is no valid point cloud in the sector When this direction is determined to be an unknown region, the distance from the unknown region to the center of the machine is calculated and denoted as . Combined with weighting coefficients Maximum sensing cutoff distance The formula for constructing the feature mapping of the unknown region is as follows: The formula uses weighting coefficients. Adjusting the weight ratio of unknown region features ensures that the unknown region features and obstacle distance features form a unified numerical range, thus avoiding network learning bias caused by differences in feature scale.

[0060] Through the above-described angle partitioning and feature extraction process, each sector corresponds to a scalar feature value. The feature values ​​of all sectors are concatenated in sector number order to form a dimension of Environmental observation vector ,Right now: This environmental observation vector fully represents the obstacle distribution and unknown area risk information in the space around the UAV in a compact, low-dimensional form. Compared with traditional occupancy grid maps, its feature dimension is significantly reduced, while retaining the core spatial geometry and risk perception information, providing a foundation for efficient inference of the subsequent lightweight policy network.

[0061] Furthermore, the S2 construction 3D environmental observation vector The six-DOF body pose state vector calculated in step one is concatenated and fused to form the complete system state. The system's state vector includes environmental observation features, linear velocity, angular velocity, attitude quaternions, and the control action vector from the previous moment. The overall dimension is a fixed value to ensure the integrity and consistency of the policy network input. The formula for system state fusion is expressed as: In the formula, It is a three-dimensional spatial position vector. It is the linear velocity vector. It is the angular velocity vector. The system state, which is the control action vector of the previous moment, includes both environmental geometry information and the UAV's own motion information through this fusion method. This provides a comprehensive input basis for the policy network's decision-making, ensuring the comprehensiveness and accuracy of the decision.

[0062] Furthermore, such as Figure 2 As shown, the lightweight multilayer perceptron policy network architecture employs a lightweight multilayer perceptron (MLP) architecture. The overall network lacks convolutional layers, recurrent layers, or attention mechanism modules. Nonlinear feature mapping and decision inference are achieved through fully connected layers, effectively reducing computational overhead and inference latency. The network structure consists of three parts: a feature encoder, a multi-source feature fusion module, and an action output module. The layer structure and parameter settings of each module are as follows:

[0063] The feature encoder consists of three fully connected layers, with the input dimension being the system state. With a fixed dimension, the first fully connected layer maps the input features to a 128-dimensional hidden space, the second fully connected layer compresses the feature dimension to 64 dimensions, and the third fully connected layer further adjusts the feature dimension to 64 dimensions. Each fully connected layer is followed by a ReLU activation function to achieve non-linear feature encoding. The core formula of this encoder is:

[0064]

[0065] In the formula, These are the weight matrices for the three fully connected layers. These are the corresponding bias vectors. The hidden feature vectors output from each layer are listed in order.

[0066] The multi-source feature fusion module consists of four fully connected layers. The input is the 64-dimensional hidden features output by the feature encoder. The first fully connected layer increases the feature dimension to 128 dimensions, and the subsequent three fully connected layers adjust the feature dimensions to 256, 256, and 128 dimensions respectively. This module fuses the encoded environmental features, the UAV's own state features, and historical action features to form a high-dimensional decision feature vector. The formula for the fusion process is as follows:

[0067]

[0068] In the formula, to This is the weight matrix of the fully connected layer of the fusion module. to For the corresponding bias vector, to This refers to the decision feature vectors output by each layer of the fusion module.

[0069] The action output module employs a single-layer linear fully connected layer. Its input is the 128-dimensional decision features output by the fusion module, which are directly mapped to the UAV control action space. The output is a normalized control command. The control action includes a one-dimensional thrust command. With three-dimensional angular velocity commands The overall motion vector has 4 dimensions, and the formula for motion output is:

[0070]

[0071] In the formula, This is the output layer weight matrix. This is the output layer bias vector. This is the control action vector output at the current moment.

[0072] To ensure the continuity and smoothness of control commands, an incremental regression mechanism is introduced. The current action vector is correlated with the previous action vector through a scaling factor, and the corresponding formula is:

[0073]

[0074] In the formula, The action increment output by the policy network. This is the motion scaling factor, which limits the range of changes in control commands to prevent drastic fluctuations in the drone's flight attitude.

[0075] Furthermore, such as Figure 3 As shown, the policy network is trained using the Proximal Policy Optimization (PPO) algorithm, which belongs to the reinforcement learning algorithm category. Its core is to construct a Markov Decision Process (MDP) and optimize the policy by maximizing the cumulative reward. The core of the PPO algorithm is to cut and replace the objective function to avoid excessively large policy updates that could lead to training instability. The corresponding objective function is... The formula is: In the formula, , For the empirical expectation operator, The probability ratio is given by the formula: , For the current policy network, For old policy networks, This is the estimated value of the dominance function. To trim hyperparameters, The pruning function limits the range of the probability ratio to... , The objective function is set as the shearing threshold, and stable iterative optimization of the policy network is achieved through this objective function.

[0076] The reward function design includes multi-dimensional constraints, including forward reward, control penalty, collision penalty, and obstacle distance reward. The forward reward encourages the drone to move continuously in the target direction. The control penalty limits the amplitude of control commands to avoid over-control. The collision penalty provides a negative reward when the drone approaches the risk of collision, guiding obstacle avoidance behavior. The obstacle distance reward provides a positive reward based on the distance between the drone and the obstacle, encouraging the maintenance of a safe distance. Through multi-dimensional rewards, the policy network is guided to learn safe and efficient obstacle avoidance strategies.

[0077] To enhance the generalization ability of the policy network in real-world environments, a dynamics domain randomization and observation noise injection mechanism are introduced during the training process. Dynamics domain randomization involves random sampling of the UAV's physical parameters, including the thrust coefficient, across intervals. air drag coefficient This allows the policy network to adapt to different UAV dynamics; observation noise injection adds zero-mean Gaussian noise to the environmental observation vector. This method simulates the measurement noise of real sensors, thereby narrowing the gap between the simulation environment and the real environment and improving the robustness of the policy network in real-world scenarios.

[0078] The 4D control action vector output by the strategy network is decoded and converted by the flight control interface unit, and mapped into low-level control quantities that the UAV flight control system can recognize, specifically pulse width modulation (PWM) signals or micro air vehicle communication protocol (MAVLink) commands. PWM signals directly correspond to motor speed control, achieving precise thrust adjustment; MAVLink commands correspond to attitude control commands of the flight control system, achieving closed-loop adjustment of angular velocity.

[0079] In the simulation environment, the dynamic parameters of the UAV are modeled with random perturbations, and the thrust coefficient is defined. Follow the interval Uniform distribution on the surface, air drag coefficient Follow the interval Random sampling; simultaneously, zero-mean Gaussian noise is injected into the state observations. To simulate the measurement error of a real sensor;

[0080] The state calculation and control execution are scheduled in real time through an onboard embedded platform; the system control cycle is... The sensing module uses frequency Running, policy network inference at frequency The process employs a timestamp alignment strategy to interpolate and synchronize point cloud data at different frequencies with state estimation data, ensuring the state vector of the input policy network is synchronized. Corresponding to the current time The true physical state;

[0081] The system employs a real-time scheduling mechanism to ensure the timing synchronization of perception, decision-making, and control. The perception module completes point cloud acquisition, coordinate transformation, and observation vector construction at a frequency of 10Hz. The strategy inference module completes system state fusion and action output at a high frequency of 50Hz. The control execution module completes command decoding and actuation at a frequency synchronized with the flight control system. Through timestamp alignment and interpolation synchronization mechanisms, the timing consistency of perception input, strategy decision-making, and control execution is ensured, forming a closed-loop control system.

[0082] This implementation constructs a policy network using a lightweight multilayer perceptron, combining a proximal policy optimization algorithm with advantage function estimation, shear constraint updates, and randomized training in the dynamic domain. While ensuring the accuracy of model decision-making, it significantly reduces the number of parameters and computational overhead, enabling efficient encoding of state features and rapid output of action commands. This improves the system's real-time response capability and control stability in complex dynamic environments, while reducing hardware resource consumption, enhancing model generalization and robustness, and making the overall control process more efficient, lightweight, and easy to deploy in engineering.

[0083] To further verify the authenticity, stability, and engineering feasibility of the UAV autonomous obstacle avoidance method based on point cloud perception and deep learning proposed in this application, the superiority of this application over existing mainstream technologies is fully demonstrated by relying on a standardized simulation platform and a unified comparison benchmark.

[0084] Example 2: This example demonstrates complete policy training based on the PyBullet simulation platform. The platform incorporates a high-precision UAV dynamics model, a Livox Mid-360 lidar simulator, a random obstacle generation module, and a wind field disturbance module, enabling complete reproduction of typical operational scenarios such as no satellite positioning signal, dense obstacles, and a high proportion of unknown areas. Training employs a 1024-group parallel environment, with a single batch size of 70,000 data points. The Adam optimizer is used, the learning rate is kept constant, and the domain randomization parameters are continuously perturbed within a set interval. The total training steps are set to 20 million steps until the policy fully converges. To ensure the verification results are comparable to industry standards and have engineering credibility, six mainstream autonomous obstacle avoidance solutions in the industry were selected as benchmarks for comparison. These are: Occupancy-Grid + PPO, Visual + SAC, PCD-Sub + DDPG, Fast-Planner, Fast-LIO + DWA, Light-Transformer + RL. All comparison solutions were reproduced and trained under the same hardware environment, observation space, action space, and reward function configuration. The verification scenarios covered three typical scenarios: indoor narrow passages, three-dimensional obstacle-dense areas, and dynamic obstacle interference areas. The test indicators included six core parameters: perception input dimension (Input Dim.), single-frame inference time (Infer. Time), average task success rate (Success Rate), control frequency (Ctrl Freq.), memory usage (Mem.), and end-to-end delay (E2E Delay). Perception input dimension represents the length of environmental features received by the policy network; single-frame inference time represents the time from observation input to action output; average task success rate is the proportion of drones that complete tasks along a fixed path without collision; control frequency is the flight control command update frequency; memory usage is the peak usage of the airborne terminal; and end-to-end delay is the total time from point cloud acquisition to control command output. All indicators were averaged after 10 repeated experiments, and the results are recorded in Table 1.

[0085] Table 1. Comparison of basic performance of different autonomous obstacle avoidance schemes in simulation environment

[0086] Occupancy-Grid + PPO 37500 41.2 73.4 24 189 132 Visual + SAC 2048 18.5 77.1 36 124 78 PCD-Sub + DDPG 4096 22.7 75.3 32 146 91 Fast-Planner — 56.8 81.2 18 213 156 Fast-LIO + DWA — 32.4 79.6 30 167 114 Light-Transformer + RL 3200 26.3 83.5 38 131 85 This application proposal 3200 18.1 92.1 50 72 68

[0087] As shown in Table 1, this application's solution achieves lower single-frame inference time and end-to-end latency while maintaining high-resolution environmental perception. The control frequency reaches 50Hz, meeting the real-time requirements of high-speed maneuvering flight. Simultaneously, its memory footprint is only 72MB, making it more compatible with embedded airborne platforms such as Jetson Orin NX. In three types of complex scenarios, the UAV can stably traverse narrow passages with a minimum width of 0.5m, maintaining continuous and reliable risk assessment of small obstacles and unknown areas, with smooth policy output without severe jitter.

[0088] To visually reflect the learning efficiency and convergence stability of each method during training, training data is collected throughout the process to generate a trend of cumulative reward changes with the number of training steps, such as... Figure 4 As shown, the display uses the horizontal axis to represent the number of training steps, ranging from zero to 20 million steps, and the vertical axis to represent the average cumulative reward per step, which is used to quantify the decision quality and flight safety of the strategy. It includes seven trends of change for this application and all six comparative methods. All methods were trained under the same environment, the same reward function, and the same hyperparameter settings. The overall trend shows that the cumulative reward of all methods increases with the number of training steps, but the rate of increase and the final convergence level differ significantly. The Occupancy-Grid + PPO method, due to its high input dimensionality, experiences a slow increase in reward in the early stages of training, only entering a relatively stable range after eight million steps, and its final reward value is at a relatively low level among all methods, indicating that high-dimensional feature input significantly reduces the training efficiency of reinforcement learning. The Visual + SAC and PCD-Sub + DDPG methods have slightly better reward increase rates than the former, but they exhibit significant fluctuations in the mid-term and insufficient convergence stability, reflecting the inherent defects of visual features being susceptible to environmental interference and point cloud downsampling losing key obstacle information. Fast-Planner and Fast-LIO + DWA, as traditional planning methods, do not rely on large-scale reinforcement learning training, so their reward curves do not show a significant upward process and remain in the middle range, making it impossible to improve performance through continuous learning. The Light-Transformer + RL method has relatively better convergence speed and final reward under a lightweight design, but it still lags behind the scheme in this application. The reward curve corresponding to the proposed scheme rises rapidly in the early stage of training and enters a fully convergent state around three million steps, with no significant fluctuations throughout. The final convergence reward is significantly higher than all other methods. This result directly proves that the angle partitioning point cloud representation method adopted in this application can reduce input redundancy while preserving effective environmental features. Combined with a lightweight multilayer perceptron architecture and a large-batch training strategy, it can significantly improve policy learning efficiency and convergence stability, providing a stable and reliable decision model for practical deployment.

[0089] To further clarify the obstacle avoidance reliability of each method at different flight speeds, the flight speed was gradually increased within a standard speed range, and the mission success rate was statistically analyzed to establish a correlation between speed and success rate. Figure 5 As shown, the display uses the horizontal axis to represent the drone's flight speed and the vertical axis to represent the mission success rate. Higher values ​​indicate more stable obstacle avoidance capabilities. The display includes seven trend curves comparing this application and six comparative methods, visually reflecting the impact of high-speed flight on the obstacle avoidance performance of each method. The curves show that the success rate of all methods decreases with increasing flight speed. This is because increased speed shortens the response time for perception and decision-making, placing higher demands on the system's real-time performance. Traditional planning methods Fast-Planner and Fast-LIO + DWA show the most significant decreases, with their success rates rapidly dropping below 70% after speeds exceed 2.0 m / s, failing to meet the demands of high-speed flight. Visual + SAC and PCD-Sub + DDPG, learning methods based on vision and downsampled point clouds, perform relatively stably in the low-to-medium speed range, but also show a significant decline after speeds exceed 2.5 m / s. This is mainly due to the weak anti-interference capability of visual features and the loss of fine obstacle information in downsampled point clouds, further amplifying errors at high speeds. Light-Transformer + While RL methods offer superior high-speed adaptability due to their lightweight structure, they still exhibit significant performance degradation at speeds above 3.0 m / s. The proposed solution shows the most gradual curve decline, maintaining a success rate of over 85% at 3.0 m / s and retaining safe obstacle avoidance capabilities even at the maximum speed of 3.5 m / s. This result is attributed to the proposed solution's low-latency perception representation, lightweight network inference, and high-frequency control output, enabling rapid obstacle perception, stable decision output, and timely adjustment of flight attitude during high-speed flight. This fully validates the practical engineering value of the proposed solution in highly maneuverable flight scenarios.

[0090] This embodiment fully demonstrates, through quantitative comparison and trend analysis under a unified benchmark, that the present application is significantly superior to existing mainstream methods in terms of inference speed, control frequency, memory usage, convergence efficiency, and high-speed adaptability. It can achieve stable and efficient autonomous obstacle avoidance on embedded platforms, providing solid data support and performance assurance for subsequent real-world deployment.

[0091] Example 3, building upon the basic performance verification completed in Example 2, further tests the system's generalization ability and robustness in extreme scenarios and unknown environments. It examines the policy's cross-domain transfer and stable decision-making capabilities under stringent conditions such as untrained scenarios, strong disturbances, large unknown regions, and dense obstacles. This example tests the system's generalization performance in four types of extreme scenarios: unseen obstacle layout scenarios, strong dynamic disturbance scenarios, large field-of-view unknown region scenarios, and densely stacked multi-obstacle scenarios. All scenarios were not encountered during the training phase to verify the policy's cross-scenario transfer capability. The verification metrics were set as Extreme Scene success rate, Disturbance Res., Unknown Region identification accuracy, and Obstacle Miss Rate. Simultaneously, the task completion performance under different flight speed constraints was statistically analyzed. The comparison scheme remained consistent with Example 2. The test results were averaged after 10 repeated experiments and recorded in Table 2.

[0092] Table 2 Comparison of generalization performance of different schemes in extreme scenarios

[0093] Occupancy-Grid + PPO 61.2 65.7 70.3 12.4 Visual + SAC 67.5 70.2 73.6 9.8 PCD-Sub + DDPG 63.3 68.1 71.5 11.2 Fast-Planner 72.4 74.3 67.2 8.3 Fast-LIO + DWA 70.1 72.6 69.4 9.1 Light-Transformer + RL 78.2 80.5 78.7 5.6 This application proposal 89.3 88.7 92.4 2.1

[0094] The data in Table 2 show that the proposed solution is significantly better than the existing solutions in four indicators: adaptation to unknown scenarios, resistance to dynamic disturbances, perception and judgment of unknown areas, and control of obstacle omissions. The accuracy rate of unknown area judgment reaches 92.4%, and the obstacle omission rate is as low as 2.1%, which proves that the closest distance representation of angle partitioning and randomized training of dynamic domain can effectively improve the robustness and generalization ability of the strategy.

[0095] To further quantify the stability of the strategy output and the distribution of control error of each scheme under the combined effects of unknown environment and dynamic disturbance, such as Figure 6As shown, by collecting deviation data between the control command output and the ideal command under different disturbance intensities, a scatter distribution result of the correspondence between control error and environmental disturbance intensity is constructed. The horizontal axis represents the environmental disturbance intensity, which is used to characterize the uncertainty caused by changes in wind field and dynamic parameters. The vertical axis represents the single-step control error value, which is used to characterize the degree of deviation between the strategy output and the ideal safety command. The scatter distribution includes all error data points of this application and six prior art under the same disturbance gradient. Each scheme corresponds to a set of independently distributed scatter points. The overall distribution shows that the error scatter points of existing technologies exhibit a clear trend of discrete diffusion. The Occupancy-Grid + PPO method, due to inference instability caused by high-dimensional input, has the widest range of error scatter points. Under medium to high disturbance intensities, the control error rises rapidly, with overall values ​​concentrated in a large range, failing to maintain stable command output. The Visual + SAC and PCD-Sub + DDPG methods also show obvious discrete characteristics in their error scatter points, with a large number of high error points appearing after the disturbance intensity increases. This is mainly because visual features are easily affected by environmental changes, and downsampled point clouds lose obstacle detail information, leading to a significant increase in decision bias under uncertain conditions. Fast-Planner and Fast-LIO + DWA, as traditional planning methods, have control errors in a medium range overall, but the scatter point distribution is relatively loose. Under strong disturbance scenarios, they cannot quickly correct command bias, and their stability is insufficient to meet the requirements of high-maneuverability flight. Light-Transformer + RL... The error distribution of the previous method was relatively concentrated, but there were still obvious high error points in the strong disturbance range, and the overall control robustness had an upper limit. The error distribution of the proposed solution showed a highly concentrated distribution. Regardless of the low or medium-to-high disturbance intensity, the error value remained in the extremely low range, with no obvious high error discrete points. As the disturbance intensity gradually increased, the control error only showed a slow and slight increase, without any drastic fluctuations or sudden changes. This fully demonstrates that the angle partitioning perception representation, lightweight MLP policy network, and dynamic domain randomized training method proposed in this application can effectively suppress the decision bias caused by environmental disturbances and parameter uncertainties, maintain a highly stable control output in complex and ever-changing unknown scenarios, and provide reliable protection for the safe flight of UAVs in environments with strong disturbances, no GPS, and dense obstacles.

[0096] This embodiment fully verifies the strong generalization ability and high robustness of this application under harsh conditions such as unknown environment, strong disturbance, and dense obstacles through quantitative testing in extreme scenarios and comparison with real-world visualization. It can maintain stable and efficient autonomous obstacle avoidance capability in real-world scenarios that are detached from the training distribution, and meet the actual operation needs of UAVs in environments without GPS and in complex and unknown environments.

[0097] Example 3 describes in detail an autonomous obstacle avoidance system for unmanned aerial vehicles (UAVs) based on point cloud perception and deep learning, such as... Figure 7 As shown, it specifically includes:

[0098] The multimodal sensing unit, including a 3D LiDAR sensor and an IMU sensor, is used to acquire information about the geometric structure of the environment and the motion state of the UAV.

[0099] The real-time state estimation unit is equipped with a laser inertial odometry calculation method, which integrates the data from the multimodal sensing unit and outputs high-frequency pose, velocity and acceleration state quantities.

[0100] The feature engineering processing unit is used to execute the piecewise mapping function to convert the original point cloud flow into a low-dimensional vector containing features of unknown regions.

[0101] The deep reinforcement learning unit includes a model loading module and an inference engine. The model loading module is used to load the parameters of the MLP neural network trained based on the PPO algorithm, and the inference engine is used to perform nonlinear feature transformation and calculate the control strategy.

[0102] The flight control interface unit is used to decode the abstract action vector output by the strategy network into PWM signals or MAVLINK commands that can be recognized by the flight control system, and drive the actuators to complete obstacle avoidance actions.

[0103] Furthermore, the multimodal perception unit, serving as the data source foundation for the entire autonomous obstacle avoidance system, is composed of a 3D LiDAR sensor and an IMU sensor. The 3D LiDAR sensor can scan the environment around the UAV at a set frequency to acquire three-dimensional point cloud data containing obstacle position, distribution, and distance information, thereby fully characterizing the environmental geometry. The IMU sensor can measure the UAV's three-axis acceleration and three-axis angular velocity in real time, outputting high-frequency UAV motion state information, providing reliable support for subsequent state calculation and motion compensation.

[0104] Furthermore, the real-time state estimation unit integrates the laser inertial odometry calculation method, which can perform tight coupling fusion processing on the LiDAR point cloud data and IMU data input by the multimodal sensing unit. Through data association, state update, distortion correction and other steps, it can calculate and output high-frequency state quantities such as the UAV's six degrees of freedom pose, linear velocity, angular velocity and acceleration in real time, providing high-precision self-state support for point cloud coordinate transformation, policy network state input and flight control closed loop.

[0105] Furthermore, the feature engineering processing unit undertakes the function of converting high-dimensional point cloud data into low-dimensional observation features. This unit executes a pre-set piecewise mapping function, divides the space based on the angle partitioning strategy, extracts the extreme distance features in each partition, and performs feature mapping on the direction of the point cloud data in combination with the unknown region modeling method. Finally, it converts the disordered, massive, and high-dimensional original point cloud stream into a fixed-dimensional low-dimensional feature vector containing obstacle information and unknown region information, which is adapted to the input requirements of lightweight neural networks.

[0106] Furthermore, the deep reinforcement learning unit, as the decision-making core of the system, consists of two parts: a model loading module and an inference engine. The model loading module is responsible for importing the parameters of the lightweight multilayer perceptron network trained in the simulation environment based on the proximal policy optimization algorithm, ensuring the integrity of model loading and the stability of inference. The inference engine receives the fused state vector and feature vector, completes nonlinear feature encoding and policy calculation through a multilayer fully connected network, and directly outputs the control action commands required for UAV obstacle avoidance, achieving low latency and high real-time decision output.

[0107] Furthermore, the flight control interface unit undertakes the function of converting decision commands into execution actions. This unit parses and maps the abstract control action vectors output by the deep reinforcement learning unit, converting them into PWM signals or MAVLink protocol commands that the flight control system can directly recognize. This drives the UAV's motors, servos, and other actuators to complete thrust and torque adjustments, controlling the UAV to achieve attitude correction, speed adjustment, and path avoidance, ultimately achieving real-time, safe, and stable autonomous obstacle avoidance behavior in unknown environments.

[0108] Furthermore, this application also includes a computer-readable storage medium storing a computer program that, when executed by a processor, implements an autonomous obstacle avoidance method for unmanned aerial vehicles based on point cloud perception and deep learning.

[0109] A computer-readable storage medium is a non-transitory medium used to carry, transmit, and store computer program instructions. It can be read and executed by an onboard processor of an unmanned aerial vehicle (UAV), an embedded computing platform, a microcontroller, or a general-purpose computer. The medium stores a complete set of program instructions for implementing an autonomous obstacle avoidance method for UAVs based on point cloud perception and deep learning.

[0110] When the program is loaded and executed by an onboard processor or computing platform, the processor can sequentially complete all steps, including point cloud data acquisition, pose state calculation, angle partition feature extraction, unknown region modeling, observation vector construction, state vector fusion, lightweight multilayer perceptron inference, control action mapping, and low-level control output.

[0111] The program instructions include code and data related to laser inertial odometry calculation, point cloud voxelization downsampling, motion distortion correction, piecewise mapping function, lightweight MLP policy network forward inference, near-end policy optimization training logic, dynamic domain randomization processing, timestamp alignment and multi-source data synchronization, control command solving and protocol conversion, etc., and can completely reproduce all the defined processes and functions.

[0112] The computer-readable storage medium includes, but is not limited to, flash memory, ROM, RAM, SD card, solid-state drive, disk storage, embedded storage chip and other media that can persistently store program instructions. It can be directly integrated into the UAV's onboard computing platform or used as an external extended storage device to access the system, providing a standardized software carrier for the deployment, porting, updating and iteration of autonomous obstacle avoidance methods.

[0113] By storing the program in this computer-readable storage medium and loading and executing it, general-purpose processors, embedded chips, or flight control computing units can be equipped with point cloud perception, deep learning inference, and real-time autonomous obstacle avoidance capabilities without relying on dedicated hardware acceleration units, which can effectively improve the system's versatility, portability, and engineering implementation capabilities.

[0114] This embodiment describes in detail how the system can achieve real-time, stable, and robust autonomous obstacle avoidance for UAVs in unknown and complex environments. It has technical advantages such as high perception efficiency, lightweight model, low control latency, strong simulation-to-real-world transferability, convenient deployment, and good versatility and portability. It can significantly improve the autonomous flight safety and environmental adaptability of UAVs in unstructured, dynamically uncertain scenarios.

[0115] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter changes made to these embodiments within the spirit and principles of the present invention, without departing from the principles and spirit of the present invention, through conventional substitutions or to achieve the same function, fall within the scope of protection of the present invention.

Claims

1. An autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs) based on point cloud perception and deep learning, characterized in that, Includes the following steps: S1. The airborne 3D LiDAR sensor is used to collect environmental point cloud flow at a set sampling frequency, and combined with the data from the inertial measurement unit, the six-degree-of-freedom pose state of the UAV is calculated in real time using the laser inertial odometry method. S2. Construct an efficient perception representation by transforming historical time-series point cloud data to the current machine coordinate system, dividing the space into multiple sectors using an angle partitioning strategy, extracting extreme distance features within each sector, and combining them with unknown region modeling strategies to construct a low-dimensional environmental observation vector. S3. The observation vector constructed in S2 is fused with the state estimation vector calculated in S1 and input into the pre-trained lightweight multilayer perceptron policy network. The control action command at the current moment is output through nonlinear feature encoding. S4 maps the control action commands output by S3 to the underlying control quantities of the flight control system, driving the UAV to perform corresponding thrust and torque adjustments, thereby achieving real-time obstacle avoidance in unknown environments.

2. The method according to claim 1, characterized in that, The efficient perception representation construction in S2 specifically includes: defining the space around the machine as... The uniformly distributed sector, for the first Point cloud collection within a sector Extracting feature values The feature extraction process is defined as a piecewise mapping function. When point cloud data exists within a sector, the minimum Euclidean distance is extracted. Saturated nonlinear processing is then performed; when no point cloud data exists within a sector, the distance to the unknown region in that direction is calculated. And map it to extended feature values; construct observation vectors The mathematical formula is: ,in, To maximize the sensing cutoff distance, For unknown regions, the weighting coefficient is denoted as .

3. The method according to claim 1, characterized in that, The point cloud processing in S2 also includes voxelization downsampling and motion compensation; the resolution vector of the voxel mesh is defined as... A hash table is used to store the occupied voxel cells; for moving drones, the angular velocity is calculated from the IMU. History Motion distortion correction is performed on the frame point cloud, and the corrected point cloud coordinates are obtained. By rotation matrix Perform a rigid transformation: ,in, It is a translation vector to ensure the geometric consistency of the point cloud in the body coordinate system.

4. The method according to claim 1, characterized in that, The lightweight multilayer perceptron policy network in S3 adopts a fully connected feedforward neural network architecture; the input layer feature dimension is defined as... The number of hidden layers is , No. The dimension of the hidden layer is The forward propagation process of the policy network is represented as a nonlinear function approximation. , its first The feature transformation formula for the layer is: ,in, and The first Layer weight matrix and bias vector It is a non-linear activation function; the output layer is directly mapped to... 3D control space, containing normalized thrust commands With angular velocity command .

5. The method according to claim 4, characterized in that, The output layer of the policy network adopts a linear activation strategy, directly regressing the increment of the control action. Define the mapping function for the action space as follows: ,in Let this be the action vector at the current moment. This represents the current system state. This is the motion scaling factor; The state Including observation vectors Current machine speed Posture Quaternions Target position and direction vector and the actions of the previous moment .

6. The method according to claim 1 or 5, characterized in that, The policy network in S3 is trained based on a proximal policy optimization algorithm. The training process constructs a Markov decision process in a simulation environment; a reward function is defined. Includes forward reward items Control quantity penalty items Collision penalty items and obstacle distance bonus The optimization objective function of the PPO algorithm The formula is: ,in, For the empirical expectation operator, The probability ratio is given by the formula: , For the current policy network, For old policy networks, This is the estimated value of the dominance function. This is for pruning hyperparameters.

7. The method according to claim 6, characterized in that, The training process incorporates a dynamic domain randomization strategy to narrow the gap between simulation and reality; in the simulation environment, the dynamic parameters of the UAV are modeled with random perturbations, and the thrust coefficient is defined. Follow the interval Uniform distribution on the surface, air drag coefficient Follow the interval Random sampling; simultaneously, zero-mean Gaussian noise is injected into the state observations. This simulates the measurement error of a real sensor.

8. The method according to claim 1, characterized in that, The state calculation in S1 and the control execution in S4 are scheduled in real time through an onboard embedded platform; the system control cycle is... The sensing module uses frequency Run, policy network inference at frequency run; A timestamp alignment strategy is employed to interpolate and synchronize point cloud data with state estimation data at different frequencies, ensuring that the state vector input to the policy network is synchronized. Corresponding to the current time The true physical state.

9. An autonomous obstacle avoidance system for unmanned aerial vehicles (UAVs) based on point cloud perception and deep learning, characterized in that: include: The multimodal sensing unit, including a 3D LiDAR sensor and an IMU sensor, is used to acquire information about the geometric structure of the environment and the motion state of the UAV. The real-time state estimation unit is equipped with a laser inertial odometry calculation method, which integrates the data from the multimodal sensing unit and outputs high-frequency pose, velocity and acceleration state quantities. The feature engineering processing unit is used to execute the piecewise mapping function to convert the original point cloud flow into a low-dimensional vector containing features of unknown regions. The deep reinforcement learning unit includes a model loading module and an inference engine. The model loading module is used to load the parameters of the MLP neural network trained based on the PPO algorithm, and the inference engine is used to perform nonlinear feature transformation and calculate the control strategy. The flight control interface unit is used to decode the abstract action vector output by the strategy network into PWM signals or MAVLINK commands that can be recognized by the flight control system, and drive the actuators to complete obstacle avoidance actions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the UAV autonomous obstacle avoidance method based on point cloud perception and deep learning as described in any one of claims 1 to 8.

Images