Unmanned aerial vehicle autonomous obstacle avoidance system based on artificial intelligence

Abstract

The invention discloses an unmanned aerial vehicle autonomous obstacle avoidance system based on artificial intelligence, relates to the technical field of unmanned aerial vehicle obstacle avoidance, and aims to solve the problem that the existing obstacle avoidance technology is difficult to meet the requirement of safe flight of an unmanned aerial vehicle. The system comprises a multi-modal sensor fusion module, an environment information preprocessing module, a dynamic obstacle prediction module based on a space-time convolutional neural network, a path planning module and a flight control execution module, wherein the multi-modal sensor fusion module collects environment data by integrating a visual sensor, a radar sensor and an infrared sensor; the collected image, distance and thermal radiation data are subjected to fusion calculation and then output; and the environment information preprocessing module is used for receiving the calculation data output by the fusion module. The method has the advantages of comprehensively and accurately sensing the environment and providing reliable environment information support for safe flight of the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) obstacle avoidance, and more particularly to an autonomous obstacle avoidance system for UAV based on artificial intelligence. Background Art

[0002] With the widespread application of drones in various fields, such as aerial photography, logistics distribution, agricultural plant protection, and power inspection, their flight safety issues have received increasing attention. During flight, drones often face safety threats from various tangible obstacles (such as mountains, buildings, trees, power lines, etc.) and intangible obstacles (such as no-fly zones, danger zones, etc.). Therefore, having efficient and reliable autonomous obstacle avoidance capabilities has become a key requirement for the development of drone technology. However, existing technologies generally struggle with accurate autonomous obstacle avoidance, are not adaptable to environmental changes, and suffer from poor accuracy and real-time performance. In complex environments, such as densely populated cities and forests, existing obstacle avoidance technologies struggle to meet the requirements for safe UAV flight. To address this, we propose an artificial intelligence-based autonomous obstacle avoidance system for UAVs. Summary of the Invention

[0003] The purpose of the present invention is to provide an autonomous obstacle avoidance system for drones based on artificial intelligence, aiming to solve the problem that existing obstacle avoidance technologies are difficult to meet the requirements of safe drone flight.

[0004] To solve the above technical problems, the present invention provides the following technical solutions: an artificial intelligence-based autonomous obstacle avoidance system for unmanned aerial vehicles, which includes a multimodal sensor fusion module, an environmental information preprocessing module, a dynamic obstacle prediction module based on a spatiotemporal convolutional neural network, a path planning module, and a flight control execution module: The multimodal sensor fusion module collects environmental data by integrating visual sensors, radar sensors, and infrared sensors, and outputs the collected image, distance, and thermal radiation data after fusion calculation; The environmental information preprocessing module is used to receive the calculated data output by the fusion module, then process the data to generate standardized environmental information, and transmit it to the dynamic obstacle prediction module; The dynamic obstacle prediction module is used to perform convolution operations on the time and space dimensions of the standardized environmental information, extract obstacle motion characteristics, optimize the prediction model using the mean square error loss function, and output future trajectory prediction data; The path planning module is used to receive the future trajectory prediction data output by the dynamic obstacle prediction module and output the optimal obstacle avoidance action instruction after combining it with the drone status data; The flight control execution module is used to receive the optimal obstacle avoidance action instructions output by the path planning module, drive the UAV to perform obstacle avoidance actions, and provide real-time feedback to the multimodal sensor fusion module and the path planning module to form a closed-loop control.

[0005] Preferably, the multimodal sensor fusion module is implemented by the formula Perform fusion calculation, where To fuse data, is the number of sensor types, is the adaptive weight, Single sensor data.

[0006] Preferably, the dynamic obstacle prediction module uses a mean square error loss function to optimize the prediction accuracy, and its formula is: , where is the loss value, is the number of samples, is the true trajectory, To predict the trajectory.

[0007] Preferably, in the multimodal sensor fusion module, the visual sensor is a binocular camera, the radar sensor is a millimeter-wave radar, and the infrared sensor is a thermal imager, and the three are synchronously sampled at 100 Hz through a clock synchronization circuit.

[0008] Preferably, the adaptive weight pass Calculate, where is the sensor reliability evaluation value.

[0009] Preferably, the sensor reliability evaluation value The calculation of introduces ambient light intensity and obstacle material reflectivity as correction factors, and the formula is , where is the revised reliability value, is the ambient light intensity, is the reflectivity of the obstacle material, 、 is the correction factor.

[0010] Preferably, the data processing by the environmental information preprocessing module includes data cleaning, denoising and normalization.

[0011] Preferably, it also includes a ground monitoring module, which receives drone status data through 5G communication and is used to monitor the drone status data in real time.

[0012] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention integrates a binocular camera, millimeter-wave radar, and thermal imager through a multimodal sensor fusion module, uses a clock synchronization circuit to achieve high-frequency synchronous sampling, and combines it with an adaptive weight fusion mechanism to comprehensively process multi-dimensional data such as images, distances, and thermal radiation. It can effectively cope with changes in light and diverse obstacle materials in complex environments, comprehensively and accurately perceive the environment, and provide reliable environmental information support for the safe flight of drones.

[0013] 2. The dynamic obstacle prediction module in the present invention uses a spatiotemporal convolutional neural network to perform convolution operations on standardized environmental information in the time and space dimensions to extract the motion characteristics of obstacles. At the same time, it uses the mean square error loss function to optimize the prediction model, which can accurately predict the future trajectory of obstacles. This enables the drone to obtain the motion information of obstacles in advance, providing more sufficient time and more accurate data for subsequent path planning, and effectively avoiding collisions with dynamic obstacles.

[0014] 3. After receiving the optimal obstacle avoidance action instruction, the flight control execution module in the present invention drives the UAV to execute the obstacle avoidance action, and feeds back the real-time information to the multimodal sensor fusion module and the path planning module to form a closed-loop control, so that the system can adjust the strategy in time according to the actual execution situation to ensure that the obstacle avoidance action is accurate and effective. In addition, the system uses 5G communication to receive UAV status data through the ground monitoring module to achieve real-time monitoring. The staff can grasp the operation status of the UAV in a timely manner and deal with abnormalities quickly, further enhancing the reliability and safety of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Schematic diagram of the system architecture of the present invention. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0017] Example 1 An AI-based autonomous obstacle avoidance system for drones, comprising a multimodal sensor fusion module, an environmental information preprocessing module, a dynamic obstacle prediction module based on a spatiotemporal convolutional neural network, a path planning module, and a flight control execution module: The multimodal sensor fusion module collects environmental data by integrating visual sensors, radar sensors, and infrared sensors. It then fuses and calculates the collected image, distance, and thermal radiation data for output. This multi-sensor data fusion compensates for the limitations of a single sensor in environmental perception, such as the limitations of visual sensors in low-light environments and the inability of radar sensors to recognize small obstacles. This data complementation improves the integrity and accuracy of environmental information, providing a more reliable perception foundation for obstacle avoidance. The environmental information preprocessing module receives the calculated data output by the fusion module, processes the data to generate standardized environmental information, and transmits it to the dynamic obstacle prediction module to unify data in different formats into a standard format. This allows subsequent modules to more efficiently extract effective features and avoids obstacles prediction accuracy affected by data quality issues. The dynamic obstacle prediction module is used to perform convolution operations on standardized environmental information in the time and space dimensions, extract obstacle motion characteristics, optimize the prediction model using the mean square error loss function, and output future trajectory prediction data. It uses a spatiotemporal convolutional neural network to analyze the movement patterns of obstacles in time series and their position distribution in space. Combined with the mean square error loss function to optimize model parameters, it can accurately predict the future movement trajectory of obstacles, providing sufficient decision-making time for drones to plan obstacle avoidance paths in advance. The path planning module is used to receive the future trajectory prediction data output by the dynamic obstacle prediction module and combine it with the drone's status data to output the optimal obstacle avoidance action instructions. This allows for rapid search and generation of obstacle avoidance action sequences with minimal energy consumption and highest safety in complex environments, achieving a balance between obstacle avoidance efficiency and flight stability. The flight control execution module is used to receive the optimal obstacle avoidance action instructions output by the path planning module, drive the UAV to perform obstacle avoidance actions, and provide real-time feedback to the multimodal sensor fusion module and the path planning module to form a closed-loop control. After the UAV performs the obstacle avoidance action, the closed-loop control enables the actual operating status to be fed back to the multimodal sensor fusion module and the path planning module in real time, so as to adjust the subsequent strategies in time to ensure the accuracy of the obstacle avoidance action and the system's adaptability to environmental changes.

[0018] Furthermore, the multimodal sensor fusion module is constructed by formula Perform fusion calculation, where To fuse data, is the number of sensor types, is the adaptive weight, Single sensor data.

[0019] Furthermore, the dynamic obstacle prediction module uses the mean square error loss function to optimize the prediction accuracy, and its formula is: , where is the loss value, is the number of samples, is the true trajectory, To predict the trajectory.

[0020] Furthermore, in the multimodal sensor fusion module, the visual sensor is a binocular camera, the radar sensor is a millimeter-wave radar, and the infrared sensor is a thermal imager. The three are synchronously sampled at 100Hz through a clock synchronization circuit. The binocular camera provides high-resolution visual information, the millimeter-wave radar achieves all-weather ranging, and the thermal imager identifies thermal radiation targets. The 100Hz synchronous sampling of the three ensures that multi-source data is strictly aligned in the time dimension, avoids obstacle position deviation caused by asynchronous sampling, and improves the real-time and consistency of environmental perception.

[0021] Furthermore, adaptive weight pass Calculate, where is the sensor reliability evaluation value.

[0022] Furthermore, the sensor reliability evaluation value The calculation of introduces ambient light intensity and obstacle material reflectivity as correction factors, and the formula is , where is the revised reliability value, is the ambient light intensity, is the reflectivity of the obstacle material, 、 is the correction factor.

[0023] Furthermore, the environmental information preprocessing module processes the data including data cleaning, denoising and normalization to ensure data quality and avoid obstacle feature extraction errors caused by data deviation.

[0024] Furthermore, it also includes a ground monitoring module, which receives drone status data through 5G communication and is used to monitor the drone status data in real time, so that staff can promptly understand the drone's operating status and quickly deal with any drone anomalies, further enhancing system reliability and safety.

[0025] The embodiments disclosed in the present invention are preferred embodiments, but are not limited to them. Ordinary technicians in this field can easily understand the spirit of the present invention based on the above embodiments and make different extensions and changes. As long as they do not deviate from the spirit of the present invention, they are all within the scope of protection of the present invention.

Claims

1. An autonomous obstacle avoidance system for drones based on artificial intelligence, characterized in that: The system includes a multimodal sensor fusion module, an environmental information preprocessing module, a dynamic obstacle prediction module based on a spatiotemporal convolutional neural network, a path planning module, and a flight control execution module: The multimodal sensor fusion module collects environmental data by integrating visual sensors, radar sensors, and infrared sensors, and outputs the collected image, distance, and thermal radiation data after fusion calculation; The environmental information preprocessing module is used to receive the calculated data output by the fusion module, then process the data to generate standardized environmental information, and transmit it to the dynamic obstacle prediction module; The dynamic obstacle prediction module is used to perform convolution operations on the time and space dimensions of the standardized environmental information, extract obstacle motion characteristics, optimize the prediction model using the mean square error loss function, and output future trajectory prediction data; The path planning module is used to receive the future trajectory prediction data output by the dynamic obstacle prediction module and output the optimal obstacle avoidance action instruction after combining it with the drone status data; The flight control execution module is used to receive the optimal obstacle avoidance action instructions output by the path planning module, drive the UAV to perform obstacle avoidance actions, and provide real-time feedback to the multimodal sensor fusion module and the path planning module to form a closed-loop control.

2. The artificial intelligence-based autonomous obstacle avoidance system for unmanned aerial vehicles according to claim 1, characterized in that: The multimodal sensor fusion module is expressed by the formula Perform fusion calculation, where To fuse data, is the number of sensor types, is the adaptive weight, Single sensor data.

3. The artificial intelligence-based autonomous obstacle avoidance system for unmanned aerial vehicles according to claim 1, characterized in that: The dynamic obstacle prediction module uses the mean square error loss function to optimize the prediction accuracy, and its formula is: , where is the loss value, is the number of samples, is the true trajectory, To predict the trajectory.

4. The artificial intelligence-based autonomous obstacle avoidance system for unmanned aerial vehicles according to claim 1, characterized in that: In the multimodal sensor fusion module, the visual sensor is a binocular camera, the radar sensor is a millimeter-wave radar, and the infrared sensor is a thermal imager. The three are synchronously sampled at 100 Hz through a clock synchronization circuit.

5. The artificial intelligence-based autonomous obstacle avoidance system for unmanned aerial vehicles according to claim 2, characterized in that: The adaptive weight pass Calculate, where is the sensor reliability evaluation value.

6. The artificial intelligence-based autonomous obstacle avoidance system for unmanned aerial vehicles according to claim 5, characterized in that: The sensor reliability evaluation value The calculation of introduces ambient light intensity and obstacle material reflectivity as correction factors, and the formula is , where is the revised reliability value, is the ambient light intensity, is the reflectivity of the obstacle material, 、 is the correction factor.

7. The artificial intelligence-based autonomous obstacle avoidance system for unmanned aerial vehicles according to claim 1, characterized in that: The environmental information preprocessing module processes data including data cleaning, denoising and normalization.

8. The artificial intelligence-based autonomous obstacle avoidance system for unmanned aerial vehicles according to claim 1, characterized in that: It also includes a ground monitoring module that receives drone status data through 5G communication and is used to monitor the drone status data in real time.

Images