Unmanned aerial vehicle obstacle avoidance method based on sensing fusion and reinforcement learning

Abstract

The invention discloses an unmanned aerial vehicle obstacle avoidance method based on sensing fusion and reinforcement learning, and belongs to the technical field of aviation flight control, and the method comprises the following steps: S1, sensor fusion time-space synchronization; s2, sensing fusion obstacle recognition and positioning; s3, obstacle avoidance route planning based on reinforcement learning; the low-resolution global map and a local occupation grid map with the unmanned aerial vehicle as the center are superposed to serve as multi-channel input, and the current state and the historical action sequence of the unmanned aerial vehicle are combined to be input into an intelligent agent model fused by a trained convolutional neural network and a long-short-term memory network; and an obstacle avoidance decision is output through a reward function of the agent model, and the airborne computer executes the obstacle avoidance decision to control the unmanned aerial vehicle to complete an obstacle avoidance action. Through sensor fusion, the obstacle sensing capability under different weather conditions and different flight environments is enhanced, the sensing distance is also increased, and the flight safety under extreme conditions can be ensured.

Description

Technical Field

[0001] This invention relates to the field of aviation flight control technology, and in particular to an obstacle avoidance method for unmanned aerial vehicles (UAVs) based on sensor fusion and reinforcement learning. Background Technology

[0002] Existing obstacle avoidance methods are mostly used in quadcopter drones. They achieve real-time environmental monitoring by using multi-sensor fusion, and use the monitoring results as input to the obstacle analysis and processing module. The obstacle analysis and processing module performs structural perception on the surrounding environmental information to identify obstacles. Finally, the obstacle avoidance decision module determines the obstacle avoidance decision, and the flight control system drives the drone to avoid surrounding obstacles.

[0003] For obstacle avoidance systems, existing methods acquire obstacle location information, invert the obstacle's trajectory based on this location information and a quasi-linear variable parametric model, determine the collision time interval and collision area between the UAV and the obstacle based on the UAV's own flight trajectory, and select an appropriate obstacle avoidance scheme according to the current relative position. Existing technologies are mostly used for quadcopter UAVs, which have hovering capabilities and are relatively flexible and can react quickly; however, they are not suitable for long-endurance, high-aspect-ratio UAVs.

[0004] For long-endurance, high-aspect-ratio aircraft, maneuverability is often relatively weak, mainly manifested in lower climb or descent rates and larger turning radii. When flying in low-altitude environments where the conditions are not well-defined, their ability to avoid obstacles is poor, and if obstacles cannot be avoided in advance, there are flight safety hazards. Therefore, in the flight process, the early perception of obstacles and the autonomous planning of obstacle avoidance routes are particularly important for long-endurance, high-aspect-ratio aircraft.

[0005] Chinese patent application CN119937623A, published on May 6, 2025, discloses an automatic obstacle avoidance point selection and method for photovoltaic stations inspected by unmanned aerial vehicles (UAVs). The method includes: for inspection images captured by UAVs equipped with different sensors, applying image processing to analyze image clarity in real time, classifying and identifying obstacles based on differences in clarity, thereby optimizing the obstacle detection process; dynamically adjusting the UAV's flight speed and altitude when the speed of obstacle identification varies to obtain higher inspection image clarity and obstacle identification efficiency; identifying obstacles from the inspection images and using the identification results to update the obstacle avoidance database during UAV flight; planning the optimal obstacle avoidance path based on the real-time updated obstacle avoidance database and the current position of the UAV, and transmitting the optimal obstacle avoidance path to the UAV for execution in real time; while executing the optimal obstacle avoidance path, the UAV system continuously monitors changes in the surrounding environment, and if new obstacles or obstacle movement are detected, adjusting the obstacle avoidance path in real time.

[0006] This patent application discloses an automatic obstacle avoidance point selection and method for photovoltaic (PV) stations inspected by drones, which improves the efficiency and safety of PV station inspections while reducing the cost and time of manual inspections. However, relying on image clarity for obstacle recognition may lead to misjudgments or missed detections in insufficient lighting, inclement weather, or when sensor performance is limited. In addition, the pure vision solution lacks multi-sensor redundancy and has inherent limitations in its ability to detect transparent or reflective objects, such as glass. Furthermore, the update efficiency of the obstacle avoidance database is limited by the image processing speed, making it difficult to guarantee flight safety in extreme conditions. Summary of the Invention

[0007] To overcome the shortcomings of the prior art, this invention provides an obstacle avoidance method for unmanned aerial vehicles (UAVs) based on sensor fusion and reinforcement learning. By fusion, this invention enhances the ability to perceive obstacles under different weather conditions and flight environments, while also increasing the perception distance, thus ensuring flight safety in extreme situations.

[0008] This invention is achieved through the following technical solution: A drone obstacle avoidance method based on sensor fusion and reinforcement learning, characterized by the following steps: Step S1: Sensor fusion and spatiotemporal synchronization; The intrinsic parameters and spatial consistency of the UAV's camera, lidar, and millimeter-wave radar are calibrated and calibrated respectively. The transformation matrix from the coordinate system of each sensor to the coordinate system of the UAV body is established. The raw data of each sensor is projected onto the coordinate system of the body, and timestamp alignment and delay compensation are performed to achieve spatiotemporal synchronization. Step S2: Sensor fusion obstacle identification and localization; A multi-level fusion architecture is adopted to preprocess the data from each sensor to obtain their respective detection results, and perform attribute fusion to determine the obstacle location information and obstacle category, infer and update the probability of obstacle existence, and generate an occupied grid map based on the fused information and multi-frame data. Step S3: Obstacle avoidance route planning based on reinforcement learning; A low-resolution global map and a local occupancy grid map centered on the drone are overlaid as multi-channel inputs. Combined with the drone's current state and historical action sequences, the inputs are fed into a trained agent model that integrates a convolutional neural network and a long short-term memory network. The agent model's reward function outputs obstacle avoidance decisions, and the onboard computer executes these decisions to control the drone to complete obstacle avoidance maneuvers.

[0009] In step S1, the intrinsic parameter calibration of the camera uses a calibration board to calculate the intrinsic parameter matrix and distortion coefficient by capturing images from multiple angles; the intrinsic parameter calibration of the lidar is performed by scanning a static calibration board to verify the distance accuracy and angular resolution of the point cloud; the intrinsic parameter calibration of the millimeter-wave radar uses a corner reflector to calibrate the distance, velocity and Doppler frequency shift, and fits the phase difference of the antenna array through multiple reflection points.

[0010] In step S1, projecting the raw data of each sensor to the body coordinate system means that for a camera, the image is first distorted, and then the pixel coordinates are transformed to the body coordinate system using an extrinsic parameter matrix; for a lidar, the point cloud data is first distorted, and then the point cloud is projected to the body coordinate system using an extrinsic parameter matrix; for a millimeter-wave radar, the polar coordinate information is first converted to Cartesian coordinates in its own sensor coordinate system, and then projected to the body coordinate system according to the extrinsic parameter matrix.

[0011] In step S1, spatiotemporal synchronization refers to aligning sensor data of different frequencies onto the same time axis through linear interpolation.

[0012] In step S2, preprocessing the data from each sensor refers to performing target detection and semantic segmentation on the image data from the camera using a target detection model to obtain the 2D bounding box, category, and confidence level of the obstacle; performing point cloud segmentation and clustering on the point cloud data from the lidar to obtain the 3D bounding box, position, and size of the obstacle; and performing constant false alarm rate (CFAR) detection and target association on the data from the millimeter-wave radar to obtain the distance, velocity, and azimuth information of the obstacle.

[0013] In step S2, attribute fusion refers to matching the detection results of different sensors using the Hungarian algorithm.

[0014] In step S2, matching the detection results of different sensors means projecting the 3D bounding box of the LiDAR onto the image plane and associating it by calculating the intersection-union ratio of the projected box and the 2D bounding box; and associating the detection results of the millimeter-wave radar with the detection results of the LiDAR by the consistency of position and velocity.

[0015] In step S2, determining the obstacle position information means using the 3D bounding box of the lidar as a reference and combining it with the radial velocity of the millimeter-wave radar for correction.

[0016] In step S2, the obstacle category is confirmed by semantic information provided by the camera.

[0017] In step S2, inferring the probability of the existence of an obstacle refers to inferring the probability of its existence based on the confidence level of the sensor and the historical trajectory of the obstacle using Bayes' theorem.

[0018] The confidence level of the sensor is obtained by weighting the camera, lidar, and millimeter-wave radar by assigning dynamically changing weights to each sensor.

[0019] The dynamic weighting of the camera, LiDAR, and millimeter-wave radar refers to adjusting the weights according to environmental conditions. In rainy or foggy weather, the weight of the camera is set to 0.2, the weight of the millimeter-wave radar is 0.5, and the weight of the LiDAR is 0.3; in sunny weather, the weight of the camera is set to 0.6, the weight of the millimeter-wave radar is 0.1, and the weight of the LiDAR is 0.3.

[0020] The 2D bounding box mentioned in this invention refers to a two-dimensional bounding box.

[0021] The 3D bounding box mentioned in this invention refers to a three-dimensional bounding box.

[0022] The beneficial effects of this invention are mainly reflected in the following aspects: 1. Compared with the prior art, the present invention enhances the ability to perceive obstacles under different weather conditions and different flight environments through sensor fusion, and also improves the perception distance, thus ensuring flight safety in extreme situations.

[0023] 2. In step S1 of this invention, spatiotemporal synchronization refers to aligning sensor data of different frequencies to the same time axis through linear interpolation, which effectively eliminates the time difference caused by the different acquisition frequencies and delays of multiple sensors, and ensures a high degree of consistency of the sensing data.

[0024] 3. This invention preprocesses the data from each sensor to extract key feature information of obstacles, providing accurate input for subsequent fusion and laying a solid foundation for the complementary advantages and high-precision fusion recognition of multi-source sensing data.

[0025] 4. This invention enables drones to anticipate obstacles that may cause accidents and to plan obstacle avoidance routes based on reinforcement learning based on the perception results, effectively solving the obstacle avoidance problem of long-endurance, high-aspect-ratio drones. 5. This invention outputs obstacle avoidance decisions and plans obstacle avoidance routes through the reward function of the intelligent agent model, reducing human intervention, improving the rationality of the planned path, effectively reducing the time for making decisions and planning new routes when facing obstacles, and improving the flight safety of UAVs.

[0026] 6. This invention significantly improves the accuracy and robustness of obstacle identification and localization in complex environments through the spatiotemporal synchronization and fusion perception of cameras, lidar, and millimeter-wave radar. Combined with the intelligent decision-making mechanism of reinforcement learning, it enables UAVs to achieve real-time, smooth, and efficient safe obstacle avoidance in dynamic environments, while effectively reducing computational load and energy consumption. Attached Figure Description The present invention will now be further described in detail with reference to the accompanying drawings and specific embodiments: Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0027] Example 1 See Figure 1 A drone obstacle avoidance method based on sensor fusion and reinforcement learning includes the following steps: Step S1: Sensor fusion and spatiotemporal synchronization; The intrinsic parameters and spatial consistency of the UAV's camera, lidar, and millimeter-wave radar are calibrated and calibrated respectively. The transformation matrix from the coordinate system of each sensor to the coordinate system of the UAV body is established. The raw data of each sensor is projected onto the coordinate system of the body, and timestamp alignment and delay compensation are performed to achieve spatiotemporal synchronization. Step S2: Sensor fusion obstacle identification and localization; A multi-level fusion architecture is adopted to preprocess the data from each sensor to obtain their respective detection results, and perform attribute fusion to determine the obstacle location information and obstacle category, infer and update the probability of obstacle existence, and generate an occupied grid map based on the fused information and multi-frame data. Step S3: Obstacle avoidance route planning based on reinforcement learning; A low-resolution global map and a local occupancy grid map centered on the drone are overlaid as multi-channel inputs. Combined with the drone's current state and historical action sequences, the inputs are fed into a trained agent model that integrates a convolutional neural network and a long short-term memory network. The agent model's reward function outputs obstacle avoidance decisions, and the onboard computer executes these decisions to control the drone to complete obstacle avoidance maneuvers.

[0028] This embodiment is the most basic implementation method. Compared with the prior art, it enhances the ability to perceive obstacles under different weather conditions and different flight environments through sensor fusion, and also improves the perception distance, which can ensure flight safety in extreme situations.

[0029] Example 2 See Figure 1 A drone obstacle avoidance method based on sensor fusion and reinforcement learning includes the following steps: Step S1: Sensor fusion and spatiotemporal synchronization; The intrinsic parameters and spatial consistency of the UAV's camera, lidar, and millimeter-wave radar are calibrated and calibrated respectively. The transformation matrix from the coordinate system of each sensor to the coordinate system of the UAV body is established. The raw data of each sensor is projected onto the coordinate system of the body, and timestamp alignment and delay compensation are performed to achieve spatiotemporal synchronization. Step S2: Sensor fusion obstacle identification and localization; A multi-level fusion architecture is adopted to preprocess the data from each sensor to obtain their respective detection results, and perform attribute fusion to determine the obstacle location information and obstacle category, infer and update the probability of obstacle existence, and generate an occupied grid map based on the fused information and multi-frame data. Step S3: Obstacle avoidance route planning based on reinforcement learning; A low-resolution global map and a local occupancy grid map centered on the drone are overlaid as multi-channel inputs. Combined with the drone's current state and historical action sequences, the inputs are fed into a trained agent model that integrates a convolutional neural network and a long short-term memory network. The agent model's reward function outputs obstacle avoidance decisions, and the onboard computer executes these decisions to control the drone to complete obstacle avoidance maneuvers.

[0030] Preferably, in step S1, the intrinsic parameter calibration of the camera uses a calibration board to calculate the intrinsic parameter matrix and distortion coefficient by capturing images from multiple angles; the intrinsic parameter calibration of the lidar is performed by scanning a static calibration board to verify the distance accuracy and angular resolution of the point cloud; the intrinsic parameter calibration of the millimeter-wave radar uses a corner reflector to calibrate the distance, velocity and Doppler frequency shift, and fits the phase difference of the antenna array through multiple reflection points.

[0031] In step S1, projecting the raw data of each sensor to the body coordinate system means that for a camera, the image is first distorted, and then the pixel coordinates are transformed to the body coordinate system using an extrinsic parameter matrix; for a lidar, the point cloud data is first distorted, and then the point cloud is projected to the body coordinate system using an extrinsic parameter matrix; for a millimeter-wave radar, the polar coordinate information is first converted to Cartesian coordinates in its own sensor coordinate system, and then projected to the body coordinate system according to the extrinsic parameter matrix.

[0032] In step S1, spatiotemporal synchronization refers to aligning sensor data of different frequencies onto the same time axis through linear interpolation.

[0033] This embodiment is a preferred implementation. In step S1, spatiotemporal synchronization refers to aligning sensor data of different frequencies to the same time axis through linear interpolation, which effectively eliminates the time difference caused by the different acquisition frequencies and delays of multiple sensors, and ensures a high degree of consistency of the sensing data.

[0034] Example 3 See Figure 1 A drone obstacle avoidance method based on sensor fusion and reinforcement learning includes the following steps: Step S1: Sensor fusion and spatiotemporal synchronization; The intrinsic parameters and spatial consistency of the UAV's camera, lidar, and millimeter-wave radar are calibrated and calibrated respectively. The transformation matrix from the coordinate system of each sensor to the coordinate system of the UAV body is established. The raw data of each sensor is projected onto the coordinate system of the body, and timestamp alignment and delay compensation are performed to achieve spatiotemporal synchronization. Step S2: Sensor fusion obstacle identification and localization; A multi-level fusion architecture is adopted to preprocess the data from each sensor to obtain their respective detection results, and perform attribute fusion to determine the obstacle location information and obstacle category, infer and update the probability of obstacle existence, and generate an occupied grid map based on the fused information and multi-frame data. Step S3: Obstacle avoidance route planning based on reinforcement learning; A low-resolution global map and a local occupancy grid map centered on the drone are overlaid as multi-channel inputs. Combined with the drone's current state and historical action sequences, the inputs are fed into a trained agent model that integrates a convolutional neural network and a long short-term memory network. The agent model's reward function outputs obstacle avoidance decisions, and the onboard computer executes these decisions to control the drone to complete obstacle avoidance maneuvers.

[0035] In step S1, the intrinsic parameter calibration of the camera uses a calibration board to calculate the intrinsic parameter matrix and distortion coefficient by capturing images from multiple angles; the intrinsic parameter calibration of the lidar is performed by scanning a static calibration board to verify the distance accuracy and angular resolution of the point cloud; the intrinsic parameter calibration of the millimeter-wave radar uses a corner reflector to calibrate the distance, velocity and Doppler frequency shift, and fits the phase difference of the antenna array through multiple reflection points.

[0036] In step S1, projecting the raw data of each sensor to the body coordinate system means that for a camera, the image is first distorted, and then the pixel coordinates are transformed to the body coordinate system using an extrinsic parameter matrix; for a lidar, the point cloud data is first distorted, and then the point cloud is projected to the body coordinate system using an extrinsic parameter matrix; for a millimeter-wave radar, the polar coordinate information is first converted to Cartesian coordinates in its own sensor coordinate system, and then projected to the body coordinate system according to the extrinsic parameter matrix.

[0037] In step S1, spatiotemporal synchronization refers to aligning sensor data of different frequencies onto the same time axis through linear interpolation.

[0038] In step S2, preprocessing the data from each sensor refers to performing target detection and semantic segmentation on the image data from the camera using a target detection model to obtain the 2D bounding box, category, and confidence level of the obstacle; performing point cloud segmentation and clustering on the point cloud data from the lidar to obtain the 3D bounding box, position, and size of the obstacle; and performing constant false alarm rate (CFAR) detection and target association on the data from the millimeter-wave radar to obtain the distance, velocity, and azimuth information of the obstacle.

[0039] This embodiment is another preferred implementation. By preprocessing the data from each sensor, key feature information of obstacles is extracted, providing accurate input for subsequent fusion and laying a solid foundation for the complementary advantages and high-precision fusion recognition of multi-source perception data.

[0040] Example 4 See Figure 1 A drone obstacle avoidance method based on sensor fusion and reinforcement learning includes the following steps: Step S1: Sensor fusion and spatiotemporal synchronization; The intrinsic parameters and spatial consistency of the UAV's camera, lidar, and millimeter-wave radar are calibrated and calibrated respectively. The transformation matrix from the coordinate system of each sensor to the coordinate system of the UAV body is established. The raw data of each sensor is projected onto the coordinate system of the body, and timestamp alignment and delay compensation are performed to achieve spatiotemporal synchronization. Step S2: Sensor fusion obstacle identification and localization; A multi-level fusion architecture is adopted to preprocess the data from each sensor to obtain their respective detection results, and perform attribute fusion to determine the obstacle location information and obstacle category, infer and update the probability of obstacle existence, and generate an occupied grid map based on the fused information and multi-frame data. Step S3: Obstacle avoidance route planning based on reinforcement learning; A low-resolution global map and a local occupancy grid map centered on the drone are overlaid as multi-channel inputs. Combined with the drone's current state and historical action sequences, the inputs are fed into a trained agent model that integrates a convolutional neural network and a long short-term memory network. The agent model's reward function outputs obstacle avoidance decisions, and the onboard computer executes these decisions to control the drone to complete obstacle avoidance maneuvers.

[0041] In step S1, the intrinsic parameter calibration of the camera uses a calibration board to calculate the intrinsic parameter matrix and distortion coefficient by capturing images from multiple angles; the intrinsic parameter calibration of the lidar is performed by scanning a static calibration board to verify the distance accuracy and angular resolution of the point cloud; the intrinsic parameter calibration of the millimeter-wave radar uses a corner reflector to calibrate the distance, velocity and Doppler frequency shift, and fits the phase difference of the antenna array through multiple reflection points.

[0042] In step S1, projecting the raw data of each sensor to the body coordinate system means that for a camera, the image is first distorted, and then the pixel coordinates are transformed to the body coordinate system using an extrinsic parameter matrix; for a lidar, the point cloud data is first distorted, and then the point cloud is projected to the body coordinate system using an extrinsic parameter matrix; for a millimeter-wave radar, the polar coordinate information is first converted to Cartesian coordinates in its own sensor coordinate system, and then projected to the body coordinate system according to the extrinsic parameter matrix.

[0043] In step S1, spatiotemporal synchronization refers to aligning sensor data of different frequencies onto the same time axis through linear interpolation.

[0044] More preferably, in step S2, preprocessing the data from each sensor refers to performing target detection and semantic segmentation on the image data from the camera using a target detection model to obtain the 2D bounding box, category, and confidence level of the obstacle; performing point cloud segmentation and clustering on the point cloud data from the lidar to obtain the 3D bounding box, position, and size of the obstacle; and performing constant false alarm rate (CFAR) detection and target association on the data from the millimeter-wave radar to obtain the distance, velocity, and azimuth information of the obstacle.

[0045] In step S2, attribute fusion refers to matching the detection results of different sensors using the Hungarian algorithm.

[0046] In step S2, matching the detection results of different sensors means projecting the 3D bounding box of the LiDAR onto the image plane and associating it by calculating the intersection-union ratio of the projected box and the 2D bounding box; and associating the detection results of the millimeter-wave radar with the detection results of the LiDAR by the consistency of position and velocity.

[0047] This embodiment is another preferred implementation, which enables the UAV to achieve advance perception when facing obstacles that may cause accidents, and to plan obstacle avoidance routes based on reinforcement learning according to the perception results, effectively solving the obstacle avoidance problem when long-endurance, high aspect ratio UAVs are flying. Example 5 See Figure 1 A drone obstacle avoidance method based on sensor fusion and reinforcement learning includes the following steps: Step S1: Sensor fusion and spatiotemporal synchronization; The intrinsic parameters and spatial consistency of the UAV's camera, lidar, and millimeter-wave radar are calibrated and calibrated respectively. The transformation matrix from the coordinate system of each sensor to the coordinate system of the UAV body is established. The raw data of each sensor is projected onto the coordinate system of the body, and timestamp alignment and delay compensation are performed to achieve spatiotemporal synchronization. Step S2: Sensor fusion obstacle identification and localization; A multi-level fusion architecture is adopted to preprocess the data from each sensor to obtain their respective detection results, and perform attribute fusion to determine the obstacle location information and obstacle category, infer and update the probability of obstacle existence, and generate an occupied grid map based on the fused information and multi-frame data. Step S3: Obstacle avoidance route planning based on reinforcement learning; A low-resolution global map and a local occupancy grid map centered on the drone are overlaid as multi-channel inputs. Combined with the drone's current state and historical action sequences, the inputs are fed into a trained agent model that integrates a convolutional neural network and a long short-term memory network. The agent model's reward function outputs obstacle avoidance decisions, and the onboard computer executes these decisions to control the drone to complete obstacle avoidance maneuvers.

[0048] In step S1, the intrinsic parameter calibration of the camera uses a calibration board to calculate the intrinsic parameter matrix and distortion coefficient by capturing images from multiple angles; the intrinsic parameter calibration of the lidar is performed by scanning a static calibration board to verify the distance accuracy and angular resolution of the point cloud; the intrinsic parameter calibration of the millimeter-wave radar uses a corner reflector to calibrate the distance, velocity and Doppler frequency shift, and fits the phase difference of the antenna array through multiple reflection points.

[0049] In step S1, projecting the raw data of each sensor to the body coordinate system means that for a camera, the image is first distorted, and then the pixel coordinates are transformed to the body coordinate system using an extrinsic parameter matrix; for a lidar, the point cloud data is first distorted, and then the point cloud is projected to the body coordinate system using an extrinsic parameter matrix; for a millimeter-wave radar, the polar coordinate information is first converted to Cartesian coordinates in its own sensor coordinate system, and then projected to the body coordinate system according to the extrinsic parameter matrix.

[0050] In step S1, spatiotemporal synchronization refers to aligning sensor data of different frequencies onto the same time axis through linear interpolation.

[0051] In step S2, preprocessing the data from each sensor refers to performing target detection and semantic segmentation on the image data from the camera using a target detection model to obtain the 2D bounding box, category, and confidence level of the obstacle; performing point cloud segmentation and clustering on the point cloud data from the lidar to obtain the 3D bounding box, position, and size of the obstacle; and performing constant false alarm rate (CFAR) detection and target association on the data from the millimeter-wave radar to obtain the distance, velocity, and azimuth information of the obstacle.

[0052] In step S2, attribute fusion refers to matching the detection results of different sensors using the Hungarian algorithm.

[0053] In step S2, matching the detection results of different sensors means projecting the 3D bounding box of the LiDAR onto the image plane and associating it by calculating the intersection-union ratio of the projected box and the 2D bounding box; and associating the detection results of the millimeter-wave radar with the detection results of the LiDAR by the consistency of position and velocity.

[0054] In step S2, determining the obstacle position information means using the 3D bounding box of the lidar as a reference and combining it with the radial velocity of the millimeter-wave radar for correction.

[0055] In step S2, the obstacle category is confirmed by semantic information provided by the camera.

[0056] In step S2, inferring the probability of the existence of an obstacle refers to inferring the probability of its existence based on the confidence level of the sensor and the historical trajectory of the obstacle using Bayes' theorem.

[0057] This embodiment is another preferred implementation method. By outputting obstacle avoidance decisions through the reward function of the intelligent agent model and planning obstacle avoidance routes, it reduces human intervention, improves the rationality of the planned path, effectively reduces the time for making decisions and planning new routes when facing obstacles, and improves the flight safety of UAVs.

[0058] Example 6 See Figure 1 A drone obstacle avoidance method based on sensor fusion and reinforcement learning includes the following steps: Step S1: Sensor fusion and spatiotemporal synchronization; The intrinsic parameters and spatial consistency of the UAV's camera, lidar, and millimeter-wave radar are calibrated and calibrated respectively. The transformation matrix from the coordinate system of each sensor to the coordinate system of the UAV body is established. The raw data of each sensor is projected onto the coordinate system of the body, and timestamp alignment and delay compensation are performed to achieve spatiotemporal synchronization. Step S2: Sensor fusion obstacle identification and localization; A multi-level fusion architecture is adopted to preprocess the data from each sensor to obtain their respective detection results, and perform attribute fusion to determine the obstacle location information and obstacle category, infer and update the probability of obstacle existence, and generate an occupied grid map based on the fused information and multi-frame data. Step S3: Obstacle avoidance route planning based on reinforcement learning; A low-resolution global map and a local occupancy grid map centered on the drone are overlaid as multi-channel inputs. Combined with the drone's current state and historical action sequences, the inputs are fed into a trained agent model that integrates a convolutional neural network and a long short-term memory network. The agent model's reward function outputs obstacle avoidance decisions, and the onboard computer executes these decisions to control the drone to complete obstacle avoidance maneuvers.

[0059] In step S1, the intrinsic parameter calibration of the camera uses a calibration board to calculate the intrinsic parameter matrix and distortion coefficient by capturing images from multiple angles; the intrinsic parameter calibration of the lidar is performed by scanning a static calibration board to verify the distance accuracy and angular resolution of the point cloud; the intrinsic parameter calibration of the millimeter-wave radar uses a corner reflector to calibrate the distance, velocity and Doppler frequency shift, and fits the phase difference of the antenna array through multiple reflection points.

[0060] In step S1, projecting the raw data of each sensor to the body coordinate system means that for a camera, the image is first distorted, and then the pixel coordinates are transformed to the body coordinate system using an extrinsic parameter matrix; for a lidar, the point cloud data is first distorted, and then the point cloud is projected to the body coordinate system using an extrinsic parameter matrix; for a millimeter-wave radar, the polar coordinate information is first converted to Cartesian coordinates in its own sensor coordinate system, and then projected to the body coordinate system according to the extrinsic parameter matrix.

[0061] In step S1, spatiotemporal synchronization refers to aligning sensor data of different frequencies onto the same time axis through linear interpolation.

[0062] In step S2, preprocessing the data from each sensor refers to performing target detection and semantic segmentation on the image data from the camera using a target detection model to obtain the 2D bounding box, category, and confidence level of the obstacle; performing point cloud segmentation and clustering on the point cloud data from the lidar to obtain the 3D bounding box, position, and size of the obstacle; and performing constant false alarm rate (CFAR) detection and target association on the data from the millimeter-wave radar to obtain the distance, velocity, and azimuth information of the obstacle.

[0063] In step S2, attribute fusion refers to matching the detection results of different sensors using the Hungarian algorithm.

[0064] In step S2, matching the detection results of different sensors means projecting the 3D bounding box of the LiDAR onto the image plane and associating it by calculating the intersection-union ratio of the projected box and the 2D bounding box; and associating the detection results of the millimeter-wave radar with the detection results of the LiDAR by the consistency of position and velocity.

[0065] More preferably, in step S2, determining the obstacle position information means using the 3D bounding box of the lidar as a reference and combining it with the radial velocity of the millimeter-wave radar for correction.

[0066] In step S2, the obstacle category is confirmed by semantic information provided by the camera.

[0067] In step S2, inferring the probability of the existence of an obstacle refers to inferring the probability of its existence based on the confidence level of the sensor and the historical trajectory of the obstacle using Bayes' theorem.

[0068] The confidence level of the sensor is obtained by weighting the camera, lidar, and millimeter-wave radar by assigning dynamically changing weights to each sensor.

[0069] The dynamic weighting of the camera, LiDAR, and millimeter-wave radar refers to adjusting the weights according to environmental conditions. In rainy or foggy weather, the weight of the camera is set to 0.2, the weight of the millimeter-wave radar is 0.5, and the weight of the LiDAR is 0.3; in sunny weather, the weight of the camera is set to 0.6, the weight of the millimeter-wave radar is 0.1, and the weight of the LiDAR is 0.3.

[0070] This embodiment represents the optimal implementation method. By using spatiotemporal synchronization and fusion perception of cameras, LiDAR, and millimeter-wave radar, the accuracy and robustness of obstacle recognition and localization in complex environments are significantly improved. Combined with the intelligent decision-making mechanism of reinforcement learning, the UAV achieves real-time, smooth, and efficient safe obstacle avoidance in dynamic environments, while effectively reducing computational load and energy consumption.

[0071] The specific process by which this invention achieves spatiotemporal synchronization is as follows: First, timestamp alignment is performed to compensate for inherent sensor and transmission delays. This is achieved by measuring the camera's exposure delay, the LiDAR's scan cycle time deviation, and the millimeter-wave radar's echo time to obtain the delay times of different sensors. A fixed delay is then subtracted from the timestamp of each data packet to correct the timestamp. Finally, linear interpolation is used to align sensor data from different frequencies onto the same timeline. Through spatiotemporal synchronization, the perceived information from different sensors at a given moment can be acquired as the output for obstacle recognition and localization.

[0072] The obstacle avoidance decision output through the reward function of the agent model specifically includes: Target Reward: The maximum positive reward is given when the target point is reached, and the penalty is given proportionally based on the final distance from the target point if the target point is not reached; Collision penalty: A severe negative reward is applied when a collision occurs, and a penalty is applied proportionally to the distance to the nearest obstacle when no collision occurs; Progress Reward: A proportional reward is given based on the amount of distance shortened from the target point with each step; Smoothness penalty: When the drone's heading angle, speed, or altitude changes abruptly, a penalty is imposed based on the severity of the change; Energy consumption penalty: Set the optimal cruising speed, and apply a penalty based on the degree of change when the speed changes.

[0073] After completing environmental training and deployment optimization, sensors such as cameras, lidar, and millimeter-wave radar are installed on the drone to collect flight data. The obtained data is then preprocessed to obtain individual detection results, and attribute fusion is performed to determine obstacle location information and obstacle category, inferring and updating the probability of obstacle presence. Based on the fused information, multi-frame data is fused to generate an occupancy grid map. The high-resolution occupancy grid map and the low-resolution global map are superimposed as a multi-channel input. At the same time, the aircraft state is set, including current position, heading angle, speed, and relative coordinates with the target point, and past actions such as linear velocity and angular velocity are recorded to ensure the continuity of aircraft actions. Then, a convolutional neural network is used to process the occupancy grid map, extracting spatial features, including obstacle edges and channel shapes. Long short-term memory networks are used to fuse the aircraft's historical state and actions, learn motion patterns, and output obstacle avoidance decisions, i.e., obstacle avoidance routes. The trained network is pruned by removing redundant neurons and compressing the model size to 100MB, so that it can be deployed to the airborne computer and the inference time is less than 500ms. Finally, the obstacle avoidance route planning results are used as input, and the airborne computer controls the aircraft actuators to complete the obstacle avoidance action.

Claims

1. A drone obstacle avoidance method based on sensor fusion and reinforcement learning, characterized in that, Includes the following steps: Step S1: Sensor fusion and spatiotemporal synchronization; The intrinsic parameters and spatial consistency of the UAV's camera, lidar, and millimeter-wave radar are calibrated and calibrated respectively. The transformation matrix from the coordinate system of each sensor to the coordinate system of the UAV body is established. The raw data of each sensor is projected onto the coordinate system of the body, and timestamp alignment and delay compensation are performed to achieve spatiotemporal synchronization. Step S2: Sensor fusion obstacle identification and localization; A multi-level fusion architecture is adopted to preprocess the data from each sensor to obtain their respective detection results, and perform attribute fusion to determine the obstacle location information and obstacle category, infer and update the probability of obstacle existence, and generate an occupied grid map based on the fused information and multi-frame data. Step S3: Obstacle avoidance route planning based on reinforcement learning; A low-resolution global map and a local occupancy grid map centered on the drone are overlaid as multi-channel inputs. Combined with the drone's current state and historical action sequences, the inputs are fed into a trained agent model that integrates a convolutional neural network and a long short-term memory network. The agent model's reward function outputs obstacle avoidance decisions, and the onboard computer executes these decisions to control the drone to complete obstacle avoidance maneuvers.

2. The UAV obstacle avoidance method based on sensor fusion and reinforcement learning according to claim 1, characterized in that: In step S1, the intrinsic parameter calibration of the camera uses a calibration board to calculate the intrinsic parameter matrix and distortion coefficient by capturing images from multiple angles; the intrinsic parameter calibration of the lidar is performed by scanning a static calibration board to verify the distance accuracy and angular resolution of the point cloud; the intrinsic parameter calibration of the millimeter-wave radar uses a corner reflector to calibrate the distance, velocity and Doppler frequency shift, and fits the phase difference of the antenna array through multiple reflection points.

3. The UAV obstacle avoidance method based on sensor fusion and reinforcement learning according to claim 2, characterized in that: In step S1, projecting the raw data of each sensor to the body coordinate system means that for a camera, the image is first distorted, and then the pixel coordinates are transformed to the body coordinate system using an extrinsic parameter matrix; for a lidar, the point cloud data is first distorted, and then the point cloud is projected to the body coordinate system using an extrinsic parameter matrix; for a millimeter-wave radar, the polar coordinate information is first converted to Cartesian coordinates in its own sensor coordinate system, and then projected to the body coordinate system according to the extrinsic parameter matrix.

4. The UAV obstacle avoidance method based on sensor fusion and reinforcement learning according to claim 1, characterized in that: In step S1, spatiotemporal synchronization refers to aligning sensor data of different frequencies onto the same time axis through linear interpolation.

5. The UAV obstacle avoidance method based on sensor fusion and reinforcement learning according to claim 1, characterized in that: In step S2, preprocessing the data from each sensor refers to performing target detection and semantic segmentation on the image data from the camera using a target detection model to obtain the 2D bounding box, category, and confidence level of the obstacle; performing point cloud segmentation and clustering on the point cloud data from the lidar to obtain the 3D bounding box, position, and size of the obstacle; and performing constant false alarm rate (CFAR) detection and target association on the data from the millimeter-wave radar to obtain the distance, velocity, and azimuth information of the obstacle.

6. The UAV obstacle avoidance method based on sensor fusion and reinforcement learning according to claim 1, characterized in that: In step S2, attribute fusion refers to matching the detection results of different sensors using the Hungarian algorithm.

7. The UAV obstacle avoidance method based on sensor fusion and reinforcement learning according to claim 6, characterized in that: In step S2, matching the detection results of different sensors means projecting the 3D bounding box of the LiDAR onto the image plane and associating it by calculating the intersection-union ratio of the projected box and the 2D bounding box; and associating the detection results of the millimeter-wave radar with the detection results of the LiDAR by the consistency of position and velocity.

8. The UAV obstacle avoidance method based on sensor fusion and reinforcement learning according to claim 7, characterized in that: In step S2, determining the obstacle position information means using the 3D bounding box of the lidar as a reference and combining it with the radial velocity of the millimeter-wave radar for correction.

9. The UAV obstacle avoidance method based on sensor fusion and reinforcement learning according to claim 1, characterized in that: In step S2, the obstacle category is confirmed by semantic information provided by the camera.

10. The UAV obstacle avoidance method based on sensor fusion and reinforcement learning according to claim 5, characterized in that: In step S2, inferring the probability of the existence of an obstacle refers to inferring the probability of its existence based on the confidence level of the sensor and the historical trajectory of the obstacle using Bayes' theorem.

11. The UAV obstacle avoidance method based on sensor fusion and reinforcement learning according to claim 10, characterized in that: The confidence level of the sensor is obtained by weighting the camera, lidar, and millimeter-wave radar by assigning dynamically changing weights to each sensor.

12. The UAV obstacle avoidance method based on sensor fusion and reinforcement learning according to claim 11, characterized in that: Assigning dynamically changing weights to cameras, LiDAR, and millimeter-wave radar means adjusting the weights according to environmental conditions. In rainy or foggy weather, the weight of the camera is set to 0.2, the weight of the millimeter-wave radar is 0.5, and the weight of the LiDAR is 0.3; in sunny weather, the weight of the camera is set to 0.6, the weight of the millimeter-wave radar is 0.1, and the weight of the LiDAR is 0.3.

Images