Unmanned aerial vehicle autonomous obstacle avoidance method and system based on multi-modal fusion and deep reinforcement learning

Abstract

The application discloses a kind of multi-modal fusion and depth reinforcement learning's unmanned aerial vehicle autonomous obstacle avoidance method and system.The method is first by unmanned aerial vehicle to collect laser radar point cloud and visible light image in forbidden environment, respectively carry out voxel downsampling, outlier removal and contrast enhancement etc.Preprocessing, obtain high-quality multi-source data;Subsequently, the depth features of point cloud and image are extracted using deep convolutional neural network, and the environment feature vector is generated by attention weighted fusion;Finally, the obstacle avoidance task is constructed as a Markov decision process, and the TD3 deep reinforcement learning algorithm is used to train the policy network based on historical experience, to realize the accurate mapping from environment feature vector to real-time obstacle avoidance strategy.The system works collaboratively through multi-source data acquisition and preprocessing, feature extraction and fusion, real-time obstacle avoidance strategy acquisition, significantly improving the robustness of unmanned aerial vehicle perception, the intelligence of decision-making and the accuracy of obstacle avoidance in weak satellite signal or complex environment without signal.

Description

Technical Field

[0001] This invention relates to the field of drone obstacle avoidance technology, and more specifically, to a method and system for autonomous obstacle avoidance of drones based on multimodal fusion and deep reinforcement learning. Background Technology

[0002] With the continuous development of drone technology, its application in various fields is becoming increasingly widespread, especially in reconnaissance, logistics, and rescue missions, where more and more drones are being used to perform high-risk and complex tasks. However, the application of drones in certain environments still faces many challenges, particularly in "no-fly zones" such as those with no GPS signal, insufficient lighting, or extreme weather. To achieve efficient autonomous flight and obstacle avoidance in these environments, it is urgent to solve several problems in existing technologies.

[0003] Currently, in UAV obstacle avoidance missions, LiDAR and visible light cameras are two commonly used airborne sensor devices, acquiring 3D point cloud data and 2D image data respectively. The data acquired by these two sensors differ fundamentally in data structure and physical meaning. LiDAR point cloud data is an unordered, sparse set of 3D points, while camera image data is an ordered, dense matrix of 2D pixels. How to effectively and robustly fuse these two heterogeneous data to obtain more accurate and comprehensive environmental perception information is a major challenge facing current technology. Existing data fusion methods often suffer from poor fusion results and insufficient robustness, resulting in perception accuracy that cannot meet the requirements of high-precision obstacle avoidance.

[0004] Furthermore, drones face numerous uncertainties when flying in complex environments, such as GPS signal loss, changing lighting conditions, and severe weather. In these complex denied environments, traditional obstacle avoidance methods lack the intelligent decision-making capabilities to cope with changing environments. Existing obstacle avoidance methods based on single sensors or rule-driven methods often fail to make efficient decisions by integrating historical experience with the current environmental state, especially under dynamic obstacles or unforeseen emergencies, where they often exhibit low flexibility and adaptability. Summary of the Invention

[0005] The main objective of this invention is to provide a method and system for autonomous obstacle avoidance of unmanned aerial vehicles (UAVs) based on multimodal fusion and deep reinforcement learning, thereby addressing at least the problems of poor cross-modal data fusion performance and low accuracy in intelligent obstacle avoidance in existing technologies. It enables precise flight and intelligent obstacle avoidance of UAVs in denied environments.

[0006] To achieve the above objectives, a method and system for autonomous obstacle avoidance of unmanned aerial vehicles (UAVs) based on multimodal fusion and deep reinforcement learning is provided.

[0007] In a first aspect, the present invention provides an autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs) that combines multimodal fusion and deep reinforcement learning. The obstacle avoidance method includes:

[0008] The raw multi-source data in the denied environment was collected by drone and preprocessed to obtain processed multi-source data. The raw multi-source data includes raw lidar point cloud data and raw visible light image data; the processed multi-source data includes clean point cloud data and high-contrast visible light image data.

[0009] Point cloud feature vectors are extracted from clean point cloud data using PointNet++ network, and image feature vectors are extracted from high-contrast visible light image data using ResNet-34 network. The point cloud feature vectors and image feature vectors are then fused together using an attention weighting mechanism to generate environmental feature vectors.

[0010] The denied environment in which the UAV is located is constructed as an observable Markov decision process. The TD3 algorithm is trained based on historical obstacle avoidance data under the Markov decision process, and the environmental feature vector is mapped into a real-time obstacle avoidance strategy in real time using the TD3 algorithm.

[0011] Specifically, the raw multi-source data is preprocessed to obtain processed multi-source data, including:

[0012] A method that simultaneously performs voxel grid downsampling and statistical outlier removal on the original lidar point cloud data yields clean point cloud data. The setting of the voxel size in voxel grid downsampling and the selection of the k nearest neighbor number in statistical outlier removal are optimized in coordination based on the farthest point sampling mechanism of the PointNet++ network and the requirements for local feature extraction.

[0013] High-contrast visible light image data is obtained by using a contrast-limited adaptive histogram equalization algorithm to enhance the contrast of the original visible light image data.

[0014] Specifically, a method that simultaneously performs voxel grid downsampling and statistical outlier removal on the original lidar point cloud data yields clean point cloud data, including:

[0015] To reduce the density of raw LiDAR point cloud data, a voxel grid downsampling method is used. The space of the raw LiDAR point cloud data is divided into multiple voxels according to the size of the voxels, and the centroid of each voxel is calculated. The calculation formula is as follows:

[0016]

[0017] in, Indicates the center of mass. Represents the set of all possible values ​​of a given voxel. Indicates the number of voxels. This represents a point in the original lidar point cloud data. Voxel representation The Middle Index of points;

[0018] The statistical outlier removal method is used to remove outliers from the original LiDAR point cloud data that do not conform to statistical characteristics. This involves finding each point... The k-nearest neighbor set is used to calculate the average distance from a point to its nearest neighbor to obtain clean point cloud data. The calculation formula is as follows:

[0019]

[0020] in, Indicates the average distance. Describes the set of k-nearest neighbors. Point The number of neighboring points, Represents a point set One point in the middle, Indicates the current point cloud being processed. The index of each point, express The Middle The index of each point.

[0021] Specifically, a contrast-limited adaptive histogram equalization algorithm is used to enhance the contrast of the original visible light image data to obtain high-contrast visible light image data, including:

[0022] The original visible light image data is processed using a contrast-limited adaptive histogram equalization algorithm to equalize the histogram of the original visible light image data. The calculation formula is as follows:

[0023]

[0024] in, This represents the histogram after equalization. This represents the original histogram. This represents the minimum gray value of the original histogram. This represents the maximum gray value of the original histogram. Indicates the index of grayscale level.

[0025] Specifically, the PointNet++ network is used to extract point cloud feature vectors from clean point cloud data, and the ResNet-34 network is used to extract image feature vectors from high-contrast visible light image data. An attention-weighted mechanism is then used to fuse the point cloud feature vectors and image feature vectors to generate an environment feature vector, including:

[0026] The ResNet-34 network was used to extract visible light image feature vectors from high-contrast visible light image data; the PointNet++ network was used to extract point cloud feature vectors from clean point cloud data.

[0027] The feature vector of the visible light image is concatenated with the feature vector of the point cloud to obtain a preliminary fusion vector;

[0028] The correlation between different dimensions of the initial fusion vector is calculated using a feature fusion network to obtain an attention weight vector. The initial fusion vector is then weighted using the attention weight vector to obtain an environmental feature vector.

[0029] Specifically, the visible light image feature vector of high-contrast visible light image data is extracted using the ResNet-34 network, including:

[0030] The image feature extraction network is obtained by removing the fully connected layers from the pre-trained ResNet-34 network.

[0031] High-contrast visible light image data is standardized and then input into an image feature extraction network for convolution and nonlinear transformation to obtain high-level semantic feature maps.

[0032] The high-level semantic feature map is compressed using a global average pooling layer to obtain the feature vector of the visible light image.

[0033] Specifically, the PointNet++ network is used to extract point cloud feature vectors from clean point cloud data, including:

[0034] Step 1: Input the clean point cloud data into the PointNet++ network;

[0035] Step 2: Use the farthest point sampling method to sample the clean point cloud data to select a set of center points. The farthest point sampling method is used to ensure that the center points can uniformly cover the entire point cloud.

[0036] Step 3: Using each center point as a reference, group the points in the surrounding neighborhood of the center point to form multiple local point sets;

[0037] Step 4: Use the PointNet module to extract features from each local point set to obtain multiple local point features, and use the max pooling function to aggregate the local point features to obtain a comprehensive feature vector representing different local point sets;

[0038] Step 5: Stack multiple extraction layers to extract the global structure of the clean point cloud data to obtain a set of point features. The extraction layer steps include steps 2 to 4.

[0039] Step 6: Use the global max pooling function to process point features to obtain a fixed-dimensional point cloud feature vector.

[0040] Specifically, the denial environment in which the drone operates is constructed as an observable Markov decision process, including:

[0041] Define the environmental state of the denied environment. At the current moment, the environmental state is composed of the current environmental feature vector and the historical action sequence to compensate for observability. Define the UAV's actions. Define the actions as the basic flight actions of the UAV. Design a reward function to evaluate the success of the UAV obstacle avoidance. The design of the reward function integrates the smoothness constraint based on the UAV dynamics model and the spatial guidance based on the safe flight corridor.

[0042] A policy network is constructed to be responsible for decision-making, and two value networks are constructed to evaluate the long-term expected return of the environmental state-action pair and to calculate the value of the reward function.

[0043] Create an experience replay buffer to store historical obstacle avoidance data, which includes the current environmental state, the actions taken by the drone, the calculated value of the reward function, and the next environmental state.

[0044] Specifically, the TD3 algorithm is trained based on historical obstacle avoidance data under the Markov decision process. The TD3 algorithm is then used to map environmental feature vectors into real-time obstacle avoidance strategies, including:

[0045] Execute the current action according to the existing strategy and store the generated experience data into the experience replay buffer;

[0046] Sampling experience data is obtained by randomly sampling from the experience replay buffer;

[0047] Update the two value networks by minimizing the Bellman error;

[0048] Delayed update policy network, utilizing deterministic policy gradients to maximize expected return;

[0049] The environmental feature vector is input into the policy network to obtain the real-time obstacle avoidance policy;

[0050] The reward function specifically includes:

[0051] Basic obstacle avoidance and navigation rewards, smoothness rewards based on dynamic models, safe corridor guidance rewards, and a comprehensive reward function.

[0052] Secondly, this invention provides an autonomous obstacle avoidance system for unmanned aerial vehicles (UAVs) based on multimodal fusion and deep reinforcement learning. The obstacle avoidance system is applied to the obstacle avoidance method of the first aspect, and includes:

[0053] The multi-source data acquisition and preprocessing unit is used to acquire raw multi-source data in the denied environment using UAV and preprocess the raw multi-source data to obtain processed multi-source data.

[0054] The feature extraction unit is connected to the multi-source data acquisition and preprocessing unit. The feature extraction unit is used to extract point cloud feature vectors from clean point cloud data using the PointNet++ network, extract image feature vectors from high-contrast visible light image data using the ResNet-34 network, and fuse the point cloud feature vectors and image feature vectors through an attention weighting mechanism to generate environmental feature vectors.

[0055] The real-time obstacle avoidance strategy acquisition unit is connected to the feature extraction unit. The real-time obstacle avoidance strategy acquisition unit is used to construct the denial environment in which the UAV is located into an observable Markov decision process. Under the Markov decision process, the TD3 algorithm is trained using the historical obstacle avoidance data of the UAV and the environmental feature vector is processed by the TD3 algorithm to obtain the real-time obstacle avoidance strategy.

[0056] This application provides a multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance method and system for unmanned aerial vehicles (UAVs). The method first collects raw LiDAR point cloud and visible light image data from a denied environment using the UAV, and preprocesses them with voxel downsampling, outlier removal, and contrast enhancement. Then, a multimodal convolutional neural network is used to extract feature vectors from the point cloud and image data, which are then weighted and fused through a feature fusion network to generate an environmental feature vector. Finally, the environment is modeled as an observable Markov decision process, and a composite reward function incorporating dynamic constraints and safe corridor guidance is introduced. The TD3 algorithm is trained using historical obstacle avoidance data, enabling the UAV to generate a safe, smooth, and efficient obstacle avoidance strategy in real time based on the environmental feature vector. Attached Figure Description

[0057] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0058] Figure 1 A flowchart illustrating a multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance method for unmanned aerial vehicles provided in this application;

[0059] Figure 2 A schematic diagram of the structure of an autonomous obstacle avoidance system for unmanned aerial vehicles (UAVs) based on multimodal fusion and deep reinforcement learning, provided in this application;

[0060] Figure 3 This is a schematic diagram of the structure of an autonomous obstacle avoidance system for unmanned aerial vehicles (UAVs) based on multimodal fusion and deep reinforcement learning, provided in this application, after online deployment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein.

[0063] In this invention, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0064] This application provides a method and system for autonomous obstacle avoidance of unmanned aerial vehicles (UAVs) based on multimodal fusion and deep reinforcement learning. The core of this method lies in the fusion of multimodal perception and deep reinforcement learning. Specifically, the UAV first acquires lidar point clouds and visible light images. After preprocessing, a specialized convolutional neural network is used to extract features from both, and then they are further fused into a comprehensive environmental feature vector. Subsequently, by constructing a Markov decision process and applying the TD3 reinforcement learning algorithm, the UAV can learn from historical obstacle avoidance experience based on this environmental feature vector and output the optimal flight maneuver in real time, thereby achieving precise and intelligent obstacle avoidance.

[0065] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0066] Figure 1 A flowchart illustrating a multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs) provided in this application is shown below. Figure 1 As shown in this embodiment, a multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs) is provided. This method includes:

[0067] The raw multi-source data in the denied environment is collected by drone and preprocessed to obtain processed multi-source data. The raw multi-source data includes raw lidar point cloud data and raw visible light image data; the processed multi-source data includes clean point cloud data and high-contrast visible light image data.

[0068] Point cloud feature vectors are extracted from clean point cloud data using the PointNet++ network, and image feature vectors are extracted from high-contrast visible light image data using the ResNet-34 network. The point cloud feature vectors and image feature vectors are then fused together using an attention weighting mechanism to generate environmental feature vectors.

[0069] The denied environment in which the UAV is located is constructed as an observable Markov decision process. The TD3 algorithm is trained based on historical obstacle avoidance data under the Markov decision process, and the environmental feature vector is mapped into a real-time obstacle avoidance strategy in real time using the TD3 algorithm.

[0070] This application provides a multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs). The method first utilizes the UAV to simultaneously acquire raw LiDAR point clouds and visible light images in a denied environment. Next, through preprocessing steps such as voxel downsampling, outlier removal, and image contrast enhancement, clean point clouds and high-contrast visible light images are obtained. Then, a multimodal convolutional neural network is used to extract the depth features of the two types of data and fuse them into a unified environmental feature vector. Finally, the UAV obstacle avoidance task is constructed as a Markov decision process, and the TD3 deep reinforcement learning algorithm is used to train the policy network using historical experience data, thereby enabling the generation of accurate autonomous obstacle avoidance commands based on the real-time environmental feature vector.

[0071] This method utilizes multimodal perception fusion to comprehensively leverage the precise geometric information of lidar and the rich texture information of visible light images, greatly enhancing the perception capability and robustness in complex environments. Simultaneously, it introduces a deep reinforcement learning framework, enabling the UAV to autonomously learn and optimize obstacle avoidance strategies through continuous interaction with the environment, ultimately achieving efficient, intelligent, and stable autonomous navigation and obstacle avoidance in complex and unknown environments such as GPS denial.

[0072] Specifically, the raw multi-source data is preprocessed to obtain processed multi-source data, including:

[0073] A method that simultaneously performs voxel grid downsampling and statistical outlier removal on the original LiDAR point cloud data yields clean point cloud data. The setting of the voxel size in voxel grid downsampling and the selection of the k-nearest neighbor number in statistical outlier removal are optimized in coordination based on the farthest point sampling mechanism of the PointNet++ network and the requirements for local feature extraction.

[0074] High-contrast visible light image data is obtained by using a contrast-limited adaptive histogram equalization algorithm to enhance the contrast of the original visible light image data.

[0075] This application provides a multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs). The method first utilizes multiple sensors on the UAV to simultaneously acquire raw LiDAR point cloud data and raw visible light image data in a denied environment. Then, the raw point cloud data undergoes a two-stage purification process: first, voxel grid downsampling reduces data density while preserving structural features; then, a statistical outlier removal algorithm filters out noise points, ultimately yielding clean point cloud data that accurately represents the geometric structure of the environment. Simultaneously, a contrast-limited adaptive histogram equalization algorithm is applied to the raw visible light image to enhance the visibility of details in unevenly lit or low-contrast scenes, thereby obtaining high-quality, high-contrast visible light image data.

[0076] This method specifically improves the quality and reliability of data from different modalities. The point cloud processing step significantly reduces the amount of data while preserving key geometric information and effectively eliminating measurement noise, providing stable and accurate input for subsequent feature extraction. The image enhancement step adaptively improves the visual quality of the image, making detailed features more prominent and greatly enhancing the robustness of the visual perception system under harsh lighting conditions. This parallel optimization process lays a solid and high-quality data foundation for subsequent multimodal feature fusion.

[0077] This application has the following significant advantages:

[0078] 1. Perception layer: Enables deep cross-modal collaboration and supports all-weather, high-precision environmental perception.

[0079] Multi-source information complementarity: By fusing 3D spatial data from LiDAR with texture semantic information from visible light cameras, a more comprehensive and reliable environmental model is constructed than that of a single sensor. In low-light conditions, LiDAR plays a primary role; in areas with complex textures, camera information assists in improving recognition accuracy. This complementary mechanism effectively avoids the performance limitations of a single sensor under specific conditions such as low light.

[0080] Adaptive Feature Fusion: Employing an attention-based model, the system can autonomously learn and dynamically weight key components of different modalities. The system automatically adjusts based on scene content: emphasizing precise distance information from the LiDAR when approaching obstacles, and relying more on camera features when identifying targets such as materials and transparent objects. This content-aware fusion approach surpasses traditional data stitching or simple fusion, significantly improving overall perception efficiency.

[0081] 2. Decision-making level: Intelligent decision-making mechanism with high adaptability and strong generalization ability.

[0082] The leap from reactive to planning: Unlike traditional methods based on fixed rules, this system trains an autonomous decision-making model through data-driven training, which can directly generate continuous flight control commands based on current multi-dimensional perception information.

[0083] Coping with unknown and dynamic scenarios: By leveraging the accumulation of historical interaction data and learning mechanisms, when encountering untrained obstacles or environmental changes, the system can refer to similar scenario experience to deduce feasible obstacle avoidance strategies, thereby maintaining good adaptability in dynamic and unknown complex environments.

[0084] Innovative Reward Function: A composite reward function integrating dynamic constraints and safe space guidance was designed. This design embeds the physical motion characteristics of the UAV and intuitive geometric safety information into the deep reinforcement learning process, guiding the agent to learn physically feasible, smooth, and spatially safe obstacle avoidance strategies, fundamentally improving the safety, smoothness, and training efficiency of decision-making.

[0085] 3. System performance: rapid response and stable operation.

[0086] Low latency response: Tests show that the average decision latency is only 85 milliseconds. This is thanks to the simple and efficient computing process during deployment: sensor data is directly output as commands after being propagated forward through the fusion network and decision network, meeting the real-time requirements of high-speed drone flight.

[0087] High stability and robustness: The objective function guides the system to learn safe, stable, and efficient flight strategies. The decision-making process considers not only immediate obstacle avoidance but also the smoothness of the flight path and overall energy consumption optimization, resulting in a significant improvement in range stability.

[0088] Specifically, a method that simultaneously performs voxel grid downsampling and statistical outlier removal on the original lidar point cloud data yields clean point cloud data, including:

[0089] To reduce the density of raw LiDAR point cloud data, a voxel grid downsampling method is used. The space of the raw LiDAR point cloud data is divided into multiple voxels according to the size of the voxels, and the centroid of each voxel is calculated. The calculation formula is as follows:

[0090]

[0091] in, Indicates the center of mass. Represents the set of all possible values ​​of a given voxel. Indicates the number of voxels. This represents a point in the original lidar point cloud data. Voxel representation The Middle The index of each point.

[0092] The statistical outlier removal method is used to remove outliers from the original LiDAR point cloud data that do not conform to statistical characteristics. This involves finding each point... The k-nearest neighbor set is used to calculate the average distance from a point to its nearest neighbor to obtain clean point cloud data. The calculation formula is as follows:

[0093]

[0094] in, Indicates the average distance. Describes the set of k-nearest neighbors. Represents the point The number of neighboring points, Represents a point set One point in the middle, Indicates the current point cloud being processed. The index of each point, express The Middle The index of each point.

[0095] This application provides a multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs). This method performs a two-step processing of raw LiDAR point cloud data to obtain a clean point cloud. First, voxel grid downsampling is used to divide the point cloud space into a uniform voxel grid, and the centroid of all points within each voxel is calculated. Each voxel is represented by a single centroid point, thus significantly reducing the point cloud density while preserving the original shape. Subsequently, statistical outlier removal is performed by calculating the average distance between each point and its k-nearest neighbor and filtering out discrete noise points whose distances far exceed the statistical average. Finally, clean point cloud data that clearly and accurately reflects the geometric structure of the environment is obtained.

[0096] The raw LiDAR point cloud data is first processed using voxel grid downsampling technology. By specifying the voxel size, the density of the point cloud data is reduced, thus lowering the computational burden. Let the point cloud data be... Each of them It is a point in a point cloud, containing three-dimensional coordinates. Voxel grid downsampling, according to voxel size The space is divided into multiple voxels. The centroid of each point within a voxel is calculated.

[0097] The Statistical Outlier Removal method is used to remove outliers that do not conform to statistical characteristics. This helps to filter out inaccurate data introduced by sensor noise or environmental interference. For each point... Find its k-nearest neighbor set Calculate the average distance from a point to its neighboring points to obtain cleaner point cloud data. .

[0098] This method achieves a balance between efficiency and accuracy. Voxel downsampling significantly reduces the amount of data and computation time in subsequent calculations through data compression, while effectively preserving the main structural features of the environment by retaining centroids, avoiding excessive loss of details. Statistical outlier removal accurately identifies and filters random noise points caused by sensor errors or environmental interference, significantly improving the quality and reliability of point cloud data. The combination of these two methods provides a lightweight and clean high-quality point cloud model for subsequent feature extraction and navigation decisions.

[0099] Specifically, a contrast-limited adaptive histogram equalization algorithm is used to enhance the contrast of the original visible light image data to obtain high-contrast visible light image data, including:

[0100] The original visible light image data is processed using a contrast-limited adaptive histogram equalization algorithm to equalize the histogram of the original visible light image data. The calculation formula is as follows:

[0101]

[0102] in, This represents the histogram after equalization. This represents the original histogram. This represents the minimum gray value of the original histogram. This represents the maximum gray value of the original histogram. Indicates the index of grayscale level.

[0103] This application provides a multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs). This method employs a contrast-limited adaptive histogram equalization (CLAHE) algorithm to enhance the original visible light image. Its core operation involves dividing the image into multiple small blocks and performing histogram equalization independently within each block, thereby redistributing the local gray levels and stretching the contrast. Through a normalization calculation formula, the gray range of the original histogram is mapped to the entire usable range. Simultaneously, a contrast limit is introduced to prevent excessive amplification of noise, ultimately generating a high-quality visible light image with clear details and significantly improved contrast.

[0104] For the original visible light image The algorithm employs Limiting Contrast Adaptive Histogram Equalization (CLAHE), which enhances image contrast through histogram equalization in local regions, significantly improving image quality, especially under low-light conditions. It avoids the over-enhancement problem that can occur with global equalization, thus making it more suitable for the image acquisition needs of UAVs under complex lighting conditions.

[0105] Compared to traditional global histogram equalization, CLAHE adaptively enhances the contrast of different regions in an image, particularly improving hidden details in unevenly lit or shadowed areas, making texture features more prominent. Simultaneously, its built-in contrast limiting mechanism effectively suppresses noise amplification in uniform areas (such as the sky and walls), improving the overall visual effect of the image while ensuring the quality and usability of the enhanced image, providing a more reliable data foundation for subsequent visual feature extraction.

[0106] Specifically, the PointNet++ network is used to extract point cloud feature vectors from clean point cloud data, and the ResNet-34 network is used to extract image feature vectors from high-contrast visible light image data. An attention-weighted mechanism is then used to fuse the point cloud feature vectors and image feature vectors to generate an environment feature vector, including:

[0107] The ResNet-34 network was used to extract visible light image feature vectors from high-contrast visible light image data; the PointNet++ network was used to extract point cloud feature vectors from clean point cloud data.

[0108] The feature vectors of the visible light image and the feature vectors of the point cloud are concatenated to obtain a preliminary fusion vector.

[0109] The correlation between different dimensions of the initial fusion vector is calculated using a feature fusion network to obtain an attention weight vector. The initial fusion vector is then weighted using the attention weight vector to obtain an environmental feature vector.

[0110] This application provides a multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs). The method first extracts features through two independent dedicated network branches: a modified ResNet-34 network extracts visible light image feature vectors containing rich texture and semantic information from high-contrast visible light images; simultaneously, a PointNet++ hierarchical network extracts point cloud feature vectors representing spatial geometry from clean point cloud data. These two heterogeneous feature vectors are then concatenated to form a preliminary fusion vector. Finally, a feature fusion network (acting similarly to an attention mechanism) is introduced to analyze the intrinsic correlations between different feature dimensions in the preliminary fusion vector and generate corresponding attention weight vectors to weight it, thereby integrating a comprehensive and complementary environmental feature vector.

[0111] This method achieves efficient complementarity and intelligent integration of multimodal information. The dual-branch structure ensures that the features of each modality (texture of the image and geometry of the point cloud) can be extracted in a deep and specialized manner. Furthermore, the weighted fusion mechanism, which goes beyond simple stitching, adaptively evaluates and highlights the cross-modal features most relevant to the current obstacle avoidance task, suppressing redundant or noisy information, thereby generating a more information-dense and discriminative environmental feature representation. This significantly enhances the UAV's comprehensive understanding of complex environments, providing a more reliable and comprehensive perception foundation for subsequent reinforcement learning decisions.

[0112] Specifically, the visible light image feature vector of high-contrast visible light image data is extracted using the ResNet-34 network, including:

[0113] The image feature extraction network is obtained by removing the fully connected layers from the pre-trained ResNet-34 network.

[0114] High-contrast visible light image data is standardized and then input into an image feature extraction network for convolution and nonlinear transformation to obtain high-level semantic feature maps.

[0115] The high-level semantic feature map is compressed using a global average pooling layer to obtain the feature vector of the visible light image.

[0116] This application provides a multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs). This method constructs an image feature extraction network based on a pre-trained ResNet-34 architecture. First, the top fully connected layer is removed, transforming the network into a feature extractor. Then, a normalized high-contrast visible light image is input into the network, and through a series of convolutions and non-linear activation operations, it is converted into a feature map rich in high-level semantic information (such as edges, shapes, and textures). Finally, a global average pooling layer is used to compress and aggregate this feature map spatially, transforming it into a compact and highly condensed visible light image feature vector.

[0117] Obtain a visible light image enhanced by the CLAHE algorithm. Standardize the pixel values ​​of the image (e.g., scale to the [0,1] range or normalize using the mean and standard deviation of the ImageNet dataset) to meet the input requirements of the pre-trained model. Use a ResNet-34 model pre-trained on a large image dataset (such as ImageNet) as the feature extractor. Remove the last fully connected layer of the original ResNet-34, retaining all preceding convolutional and pooling layers. This way, the network no longer performs final classification but outputs a feature map rich in spatial and semantic information. Input the standardized image into this modified ResNet-34 network. The image undergoes layers of convolutions and non-linear transformations, finally outputting a high-level semantic feature map of size [C,H,W] at the last convolutional layer. Here, C is the number of channels (typically 512 for ResNet-34), and H and W are the height and width of the feature map. Input the resulting [C,H,W] feature map into a global average pooling layer. This pooling layer averages the feature values ​​at all H x W spatial locations in each channel C of the feature map, calculating a single value representing the global information of that channel. After global average pooling, the feature map is compressed into a C-dimensional (i.e., 512-dimensional) feature vector. This vector is the final image feature vector F_img, which aggregates the most salient semantic information of the entire image.

[0118] This method utilizes a pre-trained ResNet-34 network, enabling rapid and powerful feature extraction capabilities and effectively addressing the variability of images captured by UAVs in the field, demonstrating excellent generalization performance. Removing fully connected layers and using global average pooling not only significantly reduces model parameters and lowers the risk of overfitting, but also efficiently compresses the semantic information of the entire image into a fixed-dimensional vector, ensuring the efficiency and stability of subsequent fusion and decision-making steps.

[0119] Specifically, the PointNet++ network is used to extract point cloud feature vectors from clean point cloud data, including:

[0120] Step 1: Input the clean point cloud data into the PointNet++ network.

[0121] Step 2: Use the farthest point sampling method to sample the clean point cloud data to select a set of center points. The farthest point sampling method is used to ensure that the center points can uniformly cover the entire point cloud.

[0122] Step 3: Using each center point as a reference, group the points in the surrounding neighborhood of the center point to form multiple local point sets.

[0123] Step 4: Use the PointNet module to extract features from each local point set to obtain multiple local point features. Then, use the max pooling function to aggregate the local point features to obtain a comprehensive feature vector representing different local point sets.

[0124] Step 5: Stack multiple extraction layers to extract the global structure of the clean point cloud data to obtain a set of point features. The extraction layer steps include steps 2 to 4.

[0125] Step 6: Use the global max pooling function to process point features to obtain a fixed-dimensional point cloud feature vector.

[0126] This application provides a multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs). This method utilizes a PointNet++ network to extract point cloud features hierarchically. First, a set of center points that can uniformly represent the overall point cloud are selected through farthest point sampling, and a local neighborhood point set is constructed with each center point as the core. Next, the PointNet module independently learns features for each local point set, and max pooling is used to aggregate the comprehensive feature vector of that local region. By stacking multiple such extraction layers, the network gradually expands its receptive field, learning the global hierarchical features of the point cloud from the local geometric structure. Finally, global max pooling is used to aggregate the feature vectors of all points into a fixed-dimensional point cloud feature vector to represent the 3D structure of the entire environment.

[0127] Point cloud data after voxel downsampling and outlier removal is acquired, and the PointNet++ network is used as the feature extractor. The core idea of ​​this network is to learn features from local to global in a hierarchical manner through ensemble abstraction layers. The network executes multiple ensemble abstraction layers sequentially, each layer typically containing three sub-steps:

[0128] Sampling: Selecting a set of center points from the input point set. These points define the center of the local region. Sampling methods typically use the farthest point sampling to ensure that the center points uniformly cover the entire point cloud.

[0129] Grouping: Using each center point as a reference, multiple local point sets are formed within its surrounding neighborhood (e.g., a sphere based on Euclidean distance or K-nearest neighbors). This step is to construct local regions in order to learn the local structure.

[0130] Feature Encoding: A simplified PointNet module is used to extract features from each grouped local point set. This module processes each point independently through a shared multilayer perceptron, and then aggregates the features of the entire local point set using symmetric functions such as max pooling, outputting a comprehensive feature vector representing the local region.

[0131] By stacking multiple such ensemble abstraction layers, the network's processing scope expands layer by layer. The first layer learns small-scale local geometric features (such as edges and corners), while subsequent layers, based on these local features, learn larger-scale and more abstract combined features, ultimately capturing the global structure of the entire point cloud.

[0132] After passing through the final ensemble abstraction layer of the PointNet++ network, a set of information-rich point features, significantly reduced in number, is obtained. A global max pooling or similar symmetric function is then applied to this final set of point features to aggregate the information from all points, forming a single, fixed-dimensional global feature vector. This global feature vector is the final point cloud feature vector. Its dimension is typically determined by the number of channels in the network's final layer; in this scheme, it is a 512-dimensional vector.

[0133] This method incorporates powerful hierarchical structure understanding capabilities and robust handling of point cloud disorder. PointNet++'s hierarchical design effectively abstracts high-level global semantics (such as object shape) from low-level local geometric features (such as surface curvature), thereby accurately characterizing the spatial structure of the environment. Simultaneously, the symmetric functions used (such as max pooling) ensure the network's invariance to the order of point inputs. This approach enables UAVs to efficiently learn the most relevant 3D environmental structure information for obstacle avoidance decisions directly from the raw point cloud, proving particularly effective for recognizing complex obstacles.

[0134] Specifically, the denial environment in which the drone operates is constructed as an observable Markov decision process, including:

[0135] Define the environmental state of the denied environment. At the current moment, the environmental state is composed of the current environmental feature vector and the historical action sequence to compensate for observability. Define the UAV's actions. Define the actions as the basic flight actions of the UAV. Design a reward function to evaluate the success of the UAV obstacle avoidance. The design of the reward function integrates the smoothness constraint based on the UAV dynamics model and the spatial guidance based on the safe flight corridor.

[0136] A policy network is constructed to be responsible for decision-making, and two value networks are constructed to evaluate the long-term expected return of the environmental state-action pair and to calculate the value of the reward function.

[0137] Create an experience replay buffer to store historical obstacle avoidance data, which includes the current environmental state, the actions taken by the drone, the calculated value of the reward function, and the next environmental state.

[0138] Create an experience replay buffer to store historical obstacle avoidance data, which includes the current environmental state, the actions taken by the drone, the calculated value of the reward function, and the next environmental state.

[0139] This application provides a multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs). This method formalizes the UAV's obstacle avoidance task in a denied environment as an observable Markov decision process. Its core components include: defining an environmental state composed of the current environmental feature vector and historical action sequences to compensate for some observability issues; defining the UAV's action space as its basic flight actions; and designing a reward function to quantify the success of obstacle avoidance behavior. Based on this, a policy network is constructed to generate decisions, and two value networks are paired to collaboratively evaluate the long-term value of state-action pairs. Simultaneously, an experience replay buffer is created to systematically store and manage historical states, actions, rewards, and new states generated by the UAV during interactions.

[0140] The denied environment in which the UAV operates is modeled as a partially observable Markov decision process (POMDP). At time t, the state is composed of the current multimodal fused feature vector and a certain length of historical action sequence to compensate for partial observability. Actions are defined as basic flight maneuvers of the UAV, such as {forward, left turn, right turn, ascent, descent}. These actions can be discrete or continuous (such as specific speed or angle commands). A reward function is designed to guide the agent's learning. Rewards typically include: positive rewards: successful obstacle avoidance, approaching the target point, smooth flight; negative rewards (penalties): collision with obstacles, prolonged stagnation, excessively violent actions, excessive energy consumption.

[0141] Construct a policy network whose input is the state and output is the action; this policy network is responsible for decision-making. Construct two value networks (Critic), whose input is the state and action, and output is the long-term expected reward of that state-action pair. Using two Critics avoids overestimation of value. Create target networks for the policy and value networks respectively, and initialize their parameters to be the same as the main network. The target networks provide a stable training objective. Create an experience replay buffer to store a large number of historical interaction experiences. Each experience is a tuple representing the current state, the action taken, the immediate reward obtained, and the next state.

[0142] In decision-making, the essence of a policy network is to learn a function that quickly matches and maps the current state (i.e., a "mirror" of the environment) to a large number of successful decision experiences in a buffer. When encountering new obstacles or environmental layouts not seen during training, the agent generalizes and infers reasonable obstacle avoidance strategies by matching historical experiences similar to the current state, rather than following rigid rules.

[0143] This method provides a complete and practical mathematical framework for solving autonomous decision-making problems. By incorporating historical information to construct environmental states, it effectively overcomes the partial observability challenges common in real-world environments, making decision-making more holistic. The architecture of the policy network and dual-value network enhances the stability and convergence of the learning process. Furthermore, the experience replay mechanism breaks down the temporal correlation between data points, allowing the model to learn repeatedly from historical experience, greatly improving data utilization efficiency and laying a solid foundation for the stable training of the core reinforcement learning algorithm.

[0144] Specifically, the TD3 algorithm is trained based on historical obstacle avoidance data under the Markov decision process. The TD3 algorithm is then used to map environmental feature vectors into real-time obstacle avoidance strategies, including:

[0145] The current action is executed according to the existing strategy, and the resulting experience data is stored in the experience replay buffer.

[0146] Sampling experience data is obtained by randomly sampling from the experience playback buffer.

[0147] Update the two value networks by minimizing the Bellman error.

[0148] Delayed update policy network, utilizing deterministic policy gradients to maximize expected return.

[0149] The environmental feature vector is input into the policy network to obtain the real-time obstacle avoidance policy;

[0150] The reward function specifically includes:

[0151] Basic obstacle avoidance and navigation rewards, smoothness rewards based on dynamic models, safe corridor guidance rewards, and a comprehensive reward function.

[0152] This application provides a multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs). This method uses the TD3 algorithm for training and decision-making. First, the UAV interacts with the environment based on its current policy, storing the generated experience data in a replay buffer. Then, by randomly sampling small batches of data from the buffers to minimize temporal difference errors, the two value networks are updated, thereby accurately estimating long-term rewards. During the policy update phase, a delayed and low-frequency update approach is adopted, utilizing a deterministic policy gradient method. Guided by the value networks, the policy network is optimized to ensure its output maximizes the expected reward. Finally, the trained policy network can directly map real-time perceived environmental feature vectors to the optimal real-time obstacle avoidance policy.

[0153] Actions are performed in the environment according to the current policy (with added exploratory noise), and the resulting experience is stored in a buffer. A small batch of experience is randomly sampled from the buffer. Two Critic networks are updated by minimizing the Bellman error. The loss function is:

[0154]

[0155] in, Indicates the first The loss function of a Critic network. Indicates the expected value. Indicates the first The state-action value function output by the Critic network is the state-action value function for each state. Next action Long-term return estimates, Indicates parameters, Indicates the target value, target for:

[0156]

[0157] in, Add truncated noise to the action output by the target policy network. , This represents the immediate reward obtained after performing an action in the current state. This represents the state-action value function output by the target Critic network. This indicates the next state entered after an action is performed. Add noise to the output of the packet-targeted policy network. This represents the discount factor.

[0158] The delayed update policy network uses a deterministic policy gradient to maximize the expected reward. Its gradient formula is:

[0159]

[0160] in, The objective function of the policy network is represented by For parameters gradient, This represents the expected cumulative return of the policy network. Representation function Action gradient, Indicates the policy network in its current state. The action of outputting below, This indicates that the policy network outputs actions in relation to its parameters. The gradient.

[0161] That is, update the parameters of the policy network along the action direction that can improve the Q value.

[0162] With a smaller update rate Slowly update the parameters of all target networks to gradually align them with the main network, thus stabilizing the training process.

[0163]

[0164]

[0165] in, This represents the parameters of the i-th target Critic network. The parameters represent the target policy network. Indicates the soft update rate. and This represents the parameters of the main Critic network and the policy network. The fused environment feature vector is input into the pre-trained policy network, which then propagates forward directly, outputting the current real-time obstacle avoidance policy.

[0166] The reward function includes: basic obstacle avoidance and navigation reward, smoothness reward based on dynamic model, safe corridor guidance reward, and comprehensive reward function.

[0167] Basic Obstacle Avoidance and Navigation Rewards: Defines basic reward items used to encourage obstacle avoidance and approach to the target point, and its calculation formula is as follows:

[0168]

[0169] in, This indicates basic obstacle avoidance and navigation rewards. This indicates a positive reward for successfully avoiding an obstacle or a negative reward for a collision. The reward represents the reduction in distance to the target point. This indicates a time-sensitive reward that encourages quick achievement of goals. , , These represent the weight coefficients of the corresponding reward items.

[0170] Smoothness Reward Based on Dynamic Model: A smoothness constraint reward term based on a simplified kinematic model of the UAV is introduced to penalize actions that may lead to excessive acceleration, angular velocity, or approaching the dynamic limit. Its calculation formula is as follows:

[0171]

[0172] in, Indicates smooth reward, and They represent the times respectively. Estimates of linear and angular acceleration of the UAV. Represents the state variables related to the dynamic limits of the UAV (such as maximum speed and maximum thrust). This indicates an indicator function that outputs a penalty value when the trend of an action approaches or exceeds the safety limit. , , These represent the corresponding penalty weight coefficients.

[0173] Safe Corridor Guidance Reward: During the training phase, a conservative 3D safe flight corridor is generated in real time using preprocessed point cloud data, and a guidance reward is defined to encourage the UAV to fly in the central area of ​​the safe corridor. The calculation formula is as follows:

[0174]

[0175] in, This indicates a reward for guiding passengers through safe corridors. This indicates the distance from the drone's current location to the center line of the safety corridor. Indicates the shortest distance to the corridor boundary. and This indicates the distance adjustment parameter. This indicates an indicator function that outputs a penalty when a drone gets too close to or goes beyond the corridor boundary. and These represent the corresponding reward and penalty weight coefficients, respectively.

[0176] The comprehensive reward function is the weighted sum of the above reward items, calculated using the following formula:

[0177]

[0178] Where represents the comprehensive reward function.

[0179] This method effectively overcomes the overestimation bias and training instability problems commonly found in traditional deep reinforcement learning algorithms by introducing key technologies such as dual-value networks and delayed policy updates. This enables UAVs to learn policies more efficiently and robustly from historical experience, avoiding drastic fluctuations in policy performance. The resulting trained policy network can quickly respond to environmental changes and output smooth and reliable control commands, thereby ensuring stable and continuous autonomous obstacle avoidance for UAVs in complex denied environments.

[0180] The reward function design provided in this application addresses two major problems commonly encountered in traditional deep reinforcement learning for UAV obstacle avoidance applications: First, the learned policy may include violent or impossible actions that do not meet the physical and dynamic limits of the UAV; second, it is prone to dangerous probing behaviors in the early stages of exploration or when facing complex environments, resulting in low training efficiency and poor safety. By introducing a smoothness reward based on a dynamic model, physical feasibility knowledge is injected into the learning process, guiding the policy to converge toward an action space that conforms to the flight characteristics of the UAV. At the same time, through a safety corridor-guided reward, an intuitive and safe geometric guidance domain is provided for the UAV's exploration in complex spaces, significantly accelerating the acquisition of a safe policy and improving the robustness and reliability of the final policy.

[0181] Figure 2 This application provides a connection diagram for an autonomous obstacle avoidance system for unmanned aerial vehicles based on multimodal fusion and deep reinforcement learning. Figure 2 As shown in this embodiment, a multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance system for unmanned aerial vehicles (UAVs) is provided. The system includes:

[0182] The multi-source data acquisition and preprocessing unit is used to acquire raw multi-source data in the restricted environment using drones and preprocess the raw multi-source data to obtain processed multi-source data.

[0183] The feature extraction unit is connected to the multi-source data acquisition and preprocessing unit. The feature extraction unit is used to extract point cloud feature vectors from clean point cloud data using the PointNet++ network and image feature vectors from high-contrast visible light image data using the ResNet-34 network. The point cloud feature vectors and image feature vectors are fused together through an attention weighting mechanism to generate environmental feature vectors.

[0184] The real-time obstacle avoidance strategy acquisition unit is connected to the feature extraction unit. The real-time obstacle avoidance strategy acquisition unit is used to construct the denial environment in which the UAV is located into an observable Markov decision process. Under the Markov decision process, the TD3 algorithm is trained using the historical obstacle avoidance data of the UAV and the environmental feature vector is processed by the TD3 algorithm to obtain the real-time obstacle avoidance strategy.

[0185] This application provides an autonomous obstacle avoidance system for unmanned aerial vehicles (UAVs) based on multimodal fusion and deep reinforcement learning. The system first acquires and purifies lidar point clouds and visible light images through a multi-source data acquisition and preprocessing unit. Then, a feature extraction unit uses a multimodal convolutional neural network to extract depth features from the two types of data and performs adaptive weighted fusion to generate a comprehensive environmental feature vector describing the surrounding environment. Finally, a real-time obstacle avoidance strategy acquisition unit models the obstacle avoidance task as a Markov decision process and uses an offline-trained TD3 deep reinforcement learning algorithm to map the real-time generated environmental feature vector into optimal flight control commands, achieving real-time obstacle avoidance.

[0186] This system achieves tight coupling between perception and decision-making through modular design. Multimodal fusion perception greatly enhances the understanding and robustness of complex denied environments, overcoming the limitations of single sensors. Meanwhile, the deep reinforcement learning-based decision-making mechanism enables the UAV to apply advanced strategies learned from historical experience to generate accurate and smooth obstacle avoidance maneuvers in real time. This end-to-end intelligent processing ultimately allows the system to achieve highly autonomous, stable, and reliable navigation and obstacle avoidance in challenging scenarios with missing GPS signals and unknown environmental dynamics.

[0187] Figure 3 This application provides a schematic diagram of the structure of an online-deployed autonomous obstacle avoidance system for unmanned aerial vehicles (UAVs) based on multimodal fusion and deep reinforcement learning. Figure 3 As shown, this embodiment provides a structure of an autonomous obstacle avoidance system for unmanned aerial vehicles (UAVs) based on multimodal fusion and deep reinforcement learning after online deployment. The structure includes: collecting multi-source heterogeneous data from multiple sensors; spatiotemporally aligning the multi-source heterogeneous data and extracting features; fusing the extracted features and feeding them into a policy network to match a suitable policy network; and updating the policy network to obtain the final real-time obstacle avoidance strategy to achieve autonomous navigation and obstacle avoidance.

[0188] To verify the effectiveness of this invention, we conducted a comparative experiment with two existing typical methods on a self-built test dataset containing typical rejection environments such as urban canyons and nighttime mountain forests. The results are shown in Table 1:

[0189] Table 1

[0190]

[0191] Experimental results show that the method of the present invention is significantly superior to existing typical methods in several key indicators such as obstacle recognition rate, obstacle avoidance success rate, real-time performance (decision delay) and flight stability, which fully demonstrates its superiority and reliability in denied environments.

Claims

1. A multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs), characterized in that, The method includes: The raw multi-source data in the denied environment is collected by using a drone and then preprocessed to obtain processed multi-source data. The raw multi-source data includes raw lidar point cloud data and raw visible light image data; the processed multi-source data includes clean point cloud data and high-contrast visible light image data. Point cloud feature vectors are extracted from the clean point cloud data using the PointNet++ network, and image feature vectors are extracted from the high-contrast visible light image data using the ResNet-34 network. The point cloud feature vectors and the image feature vectors are then fused together using an attention weighting mechanism to generate an environmental feature vector. The denied environment in which the UAV is located is constructed as an observable Markov decision process. Under the Markov decision process, the TD3 algorithm is trained based on historical obstacle avoidance data. The TD3 algorithm is then used to map the environmental feature vector into a real-time obstacle avoidance strategy in real time. The reward function designed during the training of the TD3 algorithm includes: basic obstacle avoidance and navigation reward, smoothness reward based on dynamic model, safe corridor guidance reward, and comprehensive reward function; The basic obstacle avoidance and navigation reward: Defines a basic reward item to encourage obstacle avoidance and approaching the target point. The calculation formula for the basic obstacle avoidance and navigation reward is as follows: in, This refers to the basic obstacle avoidance and navigation reward. This indicates a positive reward for successfully avoiding an obstacle or a negative reward for a collision. The reward represents the reduction in distance to the target point. This indicates a time-sensitive reward that encourages quick achievement of goals. , , These represent the weight coefficients of the corresponding reward items; The smoothness reward based on the dynamic model: A smoothness constraint reward term based on a simplified kinematic model of the UAV is introduced to penalize actions that may lead to excessive acceleration, angular velocity, or approaching the dynamic limits. The calculation formula for the smoothness reward based on the dynamic model is as follows: in, This represents the smoothness reward. and They represent the times at time 1 and 2 respectively. The estimated linear and angular acceleration values ​​of the UAV, This represents the state variables related to the dynamic limits of the UAV. This indicates an indicator function that outputs a penalty value when the trend of an action approaches or exceeds the safety limit. , , These represent the corresponding penalty weight coefficients; The safe corridor guidance reward: During the training phase, a conservative 3D safe flight corridor is generated in real time using preprocessed point cloud data, and a guidance reward item is defined to encourage the UAV to fly in the central area of ​​the safe corridor. The calculation formula for the safe corridor guidance reward is as follows: in, This indicates a reward for guiding people through the safety corridor. This indicates the distance from the current position of the drone to the center line of the safety corridor. Indicates the shortest distance to the corridor boundary. and This indicates the distance adjustment parameter. This indicates an indicator function that outputs a penalty when the drone gets too close to or beyond the corridor boundary. and These represent the corresponding reward and penalty weight coefficients, respectively. The comprehensive reward function, calculated using the weighted sum of the aforementioned reward items, is as follows: wherein, represents the combined reward function.

2. The method of claim 1, wherein, The preprocessing of the original multi-source data to obtain processed multi-source data includes: The clean point cloud data is obtained by simultaneously performing voxel grid downsampling and statistical outlier removal on the original lidar point cloud data. The setting of the voxel size in the voxel grid downsampling and the selection of the k nearest neighbor number in the statistical outlier removal are optimized in coordination based on the farthest point sampling mechanism of the PointNet++ network and the requirements for local feature extraction. The original visible light image data is contrast-enhanced using a contrast-limited adaptive histogram equalization algorithm to obtain the high-contrast visible light image data.

3. The method of claim 2, wherein, The method of simultaneously performing voxel grid downsampling and statistical outlier removal on the original lidar point cloud data to obtain the clean point cloud data includes: The original lidar point cloud data is processed using the voxel grid downsampling method to reduce its density. The space of the original lidar point cloud data is divided into multiple voxels according to their size, and the centroid of each voxel is calculated using the following formula: in, Denotes the centroid point, Represents the set of all voxels described above. This indicates the number of voxels. This represents a point in the original lidar point cloud data. The voxel represents The Middle Index of points; The statistical outlier removal method is used to remove outliers from the original lidar point cloud data that do not conform to statistical characteristics, and to find each of the points. The clean point cloud data is obtained by calculating the average distance from the k-nearest neighbor set of the points. The calculation formula is as follows: in, This represents the average distance. Denotes the set of k-nearest neighbors. Represents the point The number of neighboring points, Represents a point set One point in the middle, Indicates the current point cloud being processed. The index of each point, express The Middle The index of each point.

4. The method of claim 2, wherein, The process of enhancing the contrast of the original visible light image data using a contrast-limited adaptive histogram equalization algorithm to obtain the high-contrast visible light image data includes: The original visible light image data is processed using the contrast-limited adaptive histogram equalization algorithm to equalize the histogram of the original visible light image data. The calculation formula is as follows: in, This represents the histogram after equalization. This represents the original histogram. This represents the minimum gray value of the original histogram. This represents the maximum gray value of the original histogram. Indicates the index of grayscale level.

5. The method of claim 1, wherein, The process involves extracting point cloud feature vectors from the clean point cloud data using a PointNet++ network, extracting image feature vectors from the high-contrast visible light image data using a ResNet-34 network, and fusing the point cloud feature vectors and image feature vectors through an attention weighting mechanism to generate an environment feature vector. This includes: The visible light image feature vector of the high-contrast visible light image data is extracted using the ResNet-34 network; the point cloud feature vector of the clean point cloud data is extracted using the PointNet++ network. The visible light image feature vector and the point cloud feature vector are concatenated to obtain a preliminary fusion vector; The correlation between different dimensions of the preliminary fusion vector is calculated using a feature fusion network to obtain an attention weight vector. The preliminary fusion vector is then weighted using the attention weight vector to obtain the environmental feature vector.

6. The method of claim 5, wherein, The step of extracting visible light image feature vectors from the high-contrast visible light image data using the ResNet-34 network includes: The image feature extraction network is obtained by removing the fully connected layers from the pre-trained ResNet-34 network. The high-contrast visible light image data is standardized and then input into the image feature extraction network for convolution and nonlinear transformation to obtain a high-level semantic feature map. The visible light image feature vector is obtained by compressing the high-level semantic feature map using a global average pooling layer.

7. The method of claim 5, wherein, The step of extracting point cloud feature vectors from the clean point cloud data using the PointNet++ network includes: Step 1: Input the clean point cloud data into the PointNet++ network; Step 2: The clean point cloud data is sampled using the farthest point sampling method to select a set of center points. The farthest point sampling method is used to ensure that the center points can uniformly cover the entire point cloud. Step 3: Using each center point as a reference, group multiple local point sets within the surrounding neighborhood of each center point; Step 4: Use the PointNet module to extract features from each of the local point sets to obtain multiple local point features, and use the max pooling function to aggregate the local point features to obtain a comprehensive feature vector representing different local point sets; Step 5: Stack multiple extraction layers to extract the global structure of the clean point cloud data to obtain a set of point features. The extraction layer steps include steps 2 to 4. Step 6: Process the point features using the global max pooling function to obtain the point cloud feature vector with fixed dimensions.

8. The method of claim 1, wherein, The process of constructing the denial environment in which the UAV is located as an observable Markov decision process includes: Define the environmental state of the denied environment. At the current moment, the environmental state is composed of the current environmental feature vector and the historical action sequence to compensate for observability. Define the actions of the UAV. Define the actions as the basic flight actions of the UAV. Design the reward function to evaluate the success of the UAV obstacle avoidance. The design of the reward function integrates the smoothness constraint based on the UAV dynamics model and the spatial guidance based on the safe flight corridor. A policy network is constructed to be responsible for decision-making, and two value networks are constructed to evaluate the long-term expected return of the environmental state-action pair and to calculate the value of the reward function. An experience replay buffer is created to store the historical obstacle avoidance data, which includes the current environmental state, the actions taken by the drone, the calculated value of the reward function, and the next environmental state.

9. The method of claim 8, wherein, The step of training the TD3 algorithm based on historical obstacle avoidance data under the Markov decision process, and using the TD3 algorithm to map the environmental feature vector into a real-time obstacle avoidance strategy in real time, includes: Execute the current action according to the existing strategy and store the resulting experience data into the experience playback buffer. Sampling experience data is obtained by randomly sampling from the experience playback buffer; Update the two value networks by minimizing the Bellman error; The policy network is updated with a delay, and the deterministic policy gradient is used to maximize the expected return. The environmental feature vector is input into the policy network to obtain the real-time obstacle avoidance policy; The reward function specifically includes: Basic obstacle avoidance and navigation rewards, smoothness rewards based on dynamic models, safe corridor guidance rewards, and a comprehensive reward function.

10. A multimodal fusion and deep reinforcement learning-based autonomous obstacle avoidance system for unmanned aerial vehicles, wherein the system applies the method described in any one of claims 1 to 9, and the system comprises: A multi-source data acquisition and preprocessing unit is used to acquire the original multi-source data in the denied environment using the UAV and perform the preprocessing on the original multi-source data to obtain the processed multi-source data. The feature extraction unit is connected to the multi-source data acquisition and preprocessing unit. The feature extraction unit is used to extract the point cloud feature vector of the clean point cloud data using the PointNet++ network, extract the image feature vector of the high-contrast visible light image data using the ResNet-34 network, and fuse the point cloud feature vector and the image feature vector through the attention weighting mechanism to generate the environment feature vector. A real-time obstacle avoidance strategy acquisition unit is connected to the feature extraction unit. The real-time obstacle avoidance strategy acquisition unit is used to construct the denial environment in which the UAV is located into an observable Markov decision process. Under the Markov decision process, the TD3 algorithm is trained using the historical obstacle avoidance data of the UAV, and the environmental feature vector is processed by the TD3 algorithm to obtain the real-time obstacle avoidance strategy. The reward function designed during the training of the TD3 algorithm includes: the basic obstacle avoidance and navigation reward, the smoothness reward based on the dynamic model, the safe corridor guidance reward, and the comprehensive reward function; The basic obstacle avoidance and navigation reward: Defines a basic reward item to encourage obstacle avoidance and approaching the target point. The calculation formula for the basic obstacle avoidance and navigation reward is as follows: in, This refers to the basic obstacle avoidance and navigation reward. This indicates a positive reward for successfully avoiding an obstacle or a negative reward for a collision. The reward represents the reduction in distance to the target point. This indicates a time-sensitive reward that encourages quick achievement of goals. , , These represent the weight coefficients of the corresponding reward items; The smoothness reward based on the dynamic model: A smoothness constraint reward term based on a simplified kinematic model of the UAV is introduced to penalize actions that may lead to excessive acceleration, angular velocity, or approaching the dynamic limits. The calculation formula for the smoothness reward based on the dynamic model is as follows: in, This represents the smoothness reward. and They represent the times at time 1 and 2 respectively. The estimated linear and angular acceleration values ​​of the UAV, This represents the state variables related to the dynamic limits of the UAV. This indicates an indicator function that outputs a penalty value when the trend of an action approaches or exceeds the safety limit. , , These represent the corresponding penalty weight coefficients; The safe corridor guidance reward: During the training phase, a conservative 3D safe flight corridor is generated in real time using preprocessed point cloud data, and a guidance reward item is defined to encourage the UAV to fly in the central area of ​​the safe corridor. The calculation formula for the safe corridor guidance reward is as follows: in, This indicates a reward for guiding people through the safety corridor. This indicates the distance from the current position of the drone to the center line of the safety corridor. Indicates the shortest distance to the corridor boundary. and This indicates the distance adjustment parameter. This indicates an indicator function that outputs a penalty when the drone gets too close to or beyond the corridor boundary. and These represent the corresponding reward and penalty weight coefficients, respectively. The comprehensive reward function, calculated using the weighted sum of the aforementioned reward items, is as follows: wherein, represents the combined reward function.

Images