An unmanned aerial vehicle intelligent obstacle avoidance method and system based on artificial intelligence
By improving the DCE-Net model and dynamic memory guidance mechanism, and combining it with multimodal perception technology, the problems of inaccurate information fusion and unstable path planning in UAV obstacle avoidance methods are solved, achieving high robustness and high safety in obstacle avoidance in complex environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 南京威翔科技有限公司
- Filing Date
- 2025-11-27
- Publication Date
- 2026-07-31
AI Technical Summary
Existing drone obstacle avoidance methods suffer from poor robustness, inaccurate path planning, and repeated entry into high-risk areas in terms of multimodal information fusion, complex interference background recognition, and dynamic path correction, which affect mission efficiency and safety.
Employing multimodal perception and dynamic path optimization techniques, this paper improves the DCE-Net model and uses a dynamic memory-guided negative incentive reversal mechanism. By combining visual images, infrared images, and TOF images, time synchronization and size unification processing are performed to generate multimodal image tensors. Interference masking maps and consensus feature maps are constructed, and dynamic attention masks are used to adjust the modal feature maps to generate multi-semantic label maps. Finally, a failure memory table is introduced for path planning optimization.
It improves obstacle recognition accuracy and obstacle avoidance path stability, achieves highly robust and safe autonomous flight obstacle avoidance in complex environments, and enhances the obstacle avoidance capability of UAVs under interference conditions such as strong light, obstruction, and smoke.
Smart Images
Figure CN121411469B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of unmanned aerial vehicle (UAV) navigation and control technology, and in particular to an intelligent obstacle avoidance method and system for UAVs based on artificial intelligence. Background Technology
[0002] With the increasing demand for intelligent unmanned systems and autonomous navigation in complex environments, multimodal environmental perception and obstacle avoidance in complex scenarios have become one of the key research directions in intelligent UAV technology. Existing UAV obstacle avoidance methods mostly rely on single-modal information such as vision or lidar for environmental mapping and path planning, but they still have the following problems in terms of heterogeneous fusion of multi-source information, identification of complex interference backgrounds, and dynamic path correction: Existing obstacle avoidance methods based on single-modal images exhibit poor robustness under conditions of strong light, occlusion, smoke, rain, and snow, making it difficult to accurately identify obstacles and passable areas, which can easily lead to path deviation or flight interruption. Some perception systems that fuse infrared and Time-of-Flight (TOF) modalities suffer from problems such as inaccurate image frame alignment and inconsistent spatial scales between modalities, resulting in poor feature fusion and an inability to effectively extract key semantic information. Existing multimodal network models have complex structures and insufficient feedback mechanisms, lacking the ability to dynamically adjust to divergent regions and potential interference information, making it difficult to achieve self-correction and optimization after path failure. Furthermore, traditional path planning algorithms have limitations in remembering failure experiences and dynamically adjusting paths, making it impossible to avoid repeatedly entering high-risk areas, which affects the overall task efficiency and safety.
[0003] Therefore, how to provide an intelligent obstacle avoidance method and system for drones based on artificial intelligence is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0004] One objective of this invention is to propose an intelligent obstacle avoidance method and system for unmanned aerial vehicles (UAVs) based on artificial intelligence. This invention integrates multimodal perception and dynamic path optimization technologies, and meticulously constructs a comprehensive obstacle avoidance method encompassing image preprocessing, feature extraction, interference suppression, semantic understanding, path planning, and command output. By improving the DCE-Net model and employing a dynamic memory-guided negative excitation inversion mechanism, the accuracy of obstacle recognition in complex environments and the stability of the obstacle avoidance path are enhanced. This method possesses advantages such as strong robustness in recognition, high path reliability, and strong adaptability, and can be widely applied in the field of low-altitude intelligent flight.
[0005] An intelligent obstacle avoidance method for unmanned aerial vehicles (UAVs) based on artificial intelligence, according to an embodiment of the present invention, includes the following steps: Step 1: Acquire visual images, infrared images, and TOF images, and perform time synchronization and size unification processing to generate multimodal image tensors; Step 2: Calculate the pixel value difference between modes based on the multimodal image tensor to generate an interference masking image; Step 3: Construct an improved DCE-Net model, which includes a three-branch feature extraction network, a consensus convolution module, a decoupled stimulus feedback path, and an interference guidance sub-network. The three-branch feature extraction network receives multimodal image tensors and generates a set of modal feature maps. Step 4: Stack the modal feature map set and input it into the consensus convolution module to obtain the consistency region between modalities and generate a consensus feature map; Step 5: Input the modal feature map set into the decoupled excitation feedback path to construct a bifurcation feature map, and generate a dynamic attention mask based on the bifurcation feature map. Use the dynamic attention mask to adjust the modal feature map to obtain an adjusted modal feature map set. Step 6: The interference guidance sub-network fuses the interference masking map, consensus feature map, and adjusted modality feature map set to generate a multi-semantic label map; Step 7: Generate a path planning graph based on the multi-semantic label graph, construct a failure memory table, introduce a dynamic memory-guided negative incentive reversal mechanism, and generate an updated path planning graph; Step 8: Perform logical consistency analysis on the updated path planning graph and output the final obstacle avoidance path and UAV flight control command sequence.
[0006] Optionally, step one specifically includes: The timestamp information corresponding to each frame of the visual image, infrared image and TOF image is extracted respectively. The visual image is used as a reference image. The target image frame that is closest to the timestamp of each frame of the reference image is found in the infrared image sequence and the TOF image sequence respectively to achieve intermodal frame-level correspondence matching, and image frames whose time difference exceeds the set time difference threshold are deleted. Each modal image is processed using an image resampling method to unify the resolution, so that each modal image has the same width, height and channel dimension, and then they are stitched together to form a three-channel multimodal image tensor.
[0007] Optionally, step two specifically includes: Pixel matrices corresponding to the visual image channel, infrared image channel and TOF image channel are extracted from the multimodal image tensor. For each pixel position, the pixel value difference between the infrared image channel and the visual image channel, and the pixel value difference between the TOF image channel and the visual image channel are calculated respectively, and a first difference map and a second difference map are generated respectively. The first difference map and the second difference map are averaged point by point according to the corresponding pixel positions to generate a fused difference map. The fused difference map is then subjected to local averaging within a sliding window of a set size to generate a smooth difference map. The locations in the smooth difference map where the pixel value is greater than a set pixel threshold are marked as interference regions, and the locations where the pixel value is less than or equal to the set pixel threshold are marked as non-interference regions. An interference mask map with the same size as the original image is generated. In the interference mask map, the pixel value of the interference region is 1, and the pixel value of the non-interference region is 0.
[0008] Optionally, the three-branch feature extraction network includes a visual image branch, an infrared image branch, and a TOF image branch. Each branch is composed of a first convolutional layer, a batch normalization layer, a ReLU activation function layer, a second convolutional layer, and a max pooling layer connected in sequence. The visual image channel in the multimodal image tensor is input to the visual image branch, the infrared image channel is input to the infrared image branch, and the TOF image channel is input to the TOF image branch; The input image channel of each branch is fed into the first convolutional layer to extract the first convolutional feature map. The first convolutional feature map is fed into the batch normalization layer for standardization processing. The standardization method with a mean of 0 and a variance of 1 is used to obtain the standardized feature map. The standardized feature map is input into the ReLU activation function layer for nonlinear transformation to generate an activation feature map. The activation feature map is then input into the second convolutional layer, which outputs a second convolutional feature map. The second convolutional feature map is input to the max pooling layer, and pooling is performed by taking the maximum value within a set window to output a modal feature map. The visual image branch, infrared image branch and TOF image branch each output modal feature maps of the same size, forming a modal feature map set, which includes visual modal feature maps, infrared modal feature maps and TOF modal feature maps.
[0009] Optionally, step four specifically involves: The visual modal feature map, infrared modal feature map, and TOF modal feature map in the modal feature map set are stacked in the channel dimension to form a modal stacking tensor; The modality stacking tensor is input into the consensus convolution module, which includes a cross-modality one-dimensional convolutional layer, a feature reconstruction layer, and a three-channel fusion layer connected in sequence. The cross-modal one-dimensional convolutional layer uses a convolutional kernel with a stride of 1 that slides along the modal channel direction to perform a one-dimensional convolution operation on the modal channel value corresponding to each spatial position, thereby obtaining the modal response tensor. The modal response tensor is input into the feature reconstruction layer, and the feature reconstruction layer performs variance calculation and mean normalization on the modal response value corresponding to each spatial location, and outputs a consistency score map. The locations where the pixel values in the consensus scoring map are lower than the set divergence threshold are marked as consensus regions. A point-by-point averaging operation is performed on the pixel values of each modality in the modal stacking tensor within the consensus region to generate a consensus feature map.
[0010] Optionally, step five specifically includes: The modal feature map set is input into the decoupled stimulus feedback path, which includes a local difference extraction unit, a global difference extraction unit, a fusion attention generation unit, and a feedback reweighting adjustment unit. The local difference extraction unit performs difference calculation on any two modal feature maps in the modal feature map set at the same pixel position, and subtracts the second modal pixel value from the first modal pixel value and takes the absolute value to obtain a local difference map; The visual modal feature map, infrared modal feature map, and TOF modal feature map are calculated sequentially to obtain multiple local difference maps. All local difference maps are summed along the channel dimension and divided by the total number of local difference maps to generate an average local difference map. The global difference extraction unit calculates the average value of all pixel values for each modal feature map, subtracts the average value from each pixel value and takes the absolute value to obtain the global deviation map of the modal feature map; the global deviation maps of all modal feature maps are added point by point at the corresponding pixel positions and divided by the total number of modal feature maps to obtain the global difference map; The fusion attention generation unit performs an addition operation on the average local difference map and the global difference map at the corresponding pixel positions to generate a divergence feature map. Subtract the minimum pixel value of the bifurcation feature map from each pixel value in the bifurcation feature map, and divide by the difference between the maximum and minimum pixel values in the bifurcation feature map to obtain a normalized bifurcation map; input each pixel value in the normalized bifurcation map into the Sigmoid function to output a dynamic attention mask; The feedback reweighting adjustment unit replicates the dynamic attention mask in the channel dimension, multiplies the pixel value at each position in each modal feature map by the corresponding dynamic attention mask, and adds the result to the original pixel value multiplied by half of the dynamic attention mask, outputting the adjusted set of modal feature maps.
[0011] Optionally, the interference guidance subnetwork specifically includes a feature fusion coding unit, a guidance enhancement unit, and a label generation unit; The feature fusion coding unit concatenates the interference mask map, consensus feature map and adjusted modality feature map in the channel dimension to form a fusion input tensor. The fusion input tensor is then sequentially input into a coding module consisting of two convolutional layers and a ReLU activation function layer to extract the fusion deep feature map. The guidance enhancement unit performs element-wise multiplication on the interference masking map and the fused deep feature map at corresponding positions to obtain the interference guidance feature map; the interference guidance feature map and the fused deep feature map are concatenated along the channel dimension and input into a convolution to extract the semantic fusion feature map; The label generation unit performs a pixel-by-pixel Softmax operation on the semantic fusion feature map, outputting the category probability distribution corresponding to each pixel to form a multi-semantic label map.
[0012] Optionally, step seven specifically includes: Based on the semantic category corresponding to each pixel in the multi-semantic label map, pixels marked as passable areas are selected to construct a passable area map; based on the current spatial position of the UAV and the pixel coordinates of the target position in the image, the shortest path from the starting point to the end point is searched using the cost function-driven Dijkstra algorithm, and a path planning map composed of continuous pixels is output. During the process of the UAV performing the task according to the path planning map, the start and end pixel positions, corresponding timestamps and execution status of each path segment are recorded in real time; when a path segment is interfered with by obstacles, deviates from the preset distance threshold or the flight is interrupted, the path segment and execution status are written into the failure memory table. During the path generation process, the path segments in the current path planning map are extracted, and the Euclidean distance between the start and end coordinates and the coordinates of all path segments in the failure memory table is calculated. If the Euclidean distance between the starting point of the current path segment and the starting point of any path segment in the failure memory table is less than a preset tolerance threshold, and the Euclidean distance between the ending point of the current path segment and the ending point of the failure memory table is also less than a preset tolerance threshold, then the path segment is determined to have a failure risk, triggering the dynamic memory-guided negative incentive reversal mechanism. The dynamic memory-guided negative incentive reversal mechanism reverses the direction vector of the path segment with failure risk, that is, performs a negative direction operation, and adds a set disturbance angle offset to the direction vector to correct the path to avoid high-risk areas, generates an updated path segment, and splices the updated path segment into the original path planning map to output the updated path planning map.
[0013] Optionally, step eight specifically includes: The direction vectors between adjacent path segments in the updated path planning graph are calculated to obtain the direction angle. When the direction angle between a path segment and the previous path segment is greater than a preset angle threshold, the direction vectors of the two path segments are weighted and averaged to obtain a smooth direction vector. Starting from the end coordinates of the previous path segment based on the smooth direction vector, the end coordinates of the path segment are repositioned to replace the original path segment, forming a path sequence with continuous directional changes less than or equal to a set angle threshold, thus constituting a logically consistent smooth path. Convert the coordinates of each pixel in the smooth path into actual 3D spatial coordinates; calculate the coordinate difference between two consecutive 3D spatial coordinates in turn. If the difference on the X-axis is positive, a forward command is generated; if the difference on the X-axis is negative, a backward command is generated; if the difference on the Y-axis is positive, a right shift command is generated; if the difference on the Y-axis is negative, a left shift command is generated; if the difference on the Z-axis is positive, an upward command is generated; if the difference on the Z-axis is negative, a downward command is generated. By connecting all the three-dimensional spatial coordinate points in the smooth path in sequence, the final obstacle avoidance path is obtained. The control commands generated in sequence are arranged in the path order to form the UAV flight control command sequence, which serves as the control signal output for the UAV to perform the final mission.
[0014] An artificial intelligence-based intelligent obstacle avoidance system for unmanned aerial vehicles (UAVs) according to an embodiment of the present invention includes the following modules: The multimodal image preprocessing module is used to acquire visual images, infrared images, and TOF images, and perform time synchronization and size unification processing to generate multimodal image tensors. The interference masking image generation module is used to calculate the pixel value difference between modes based on the multimodal image tensor and generate an interference masking image. Improved DCE-Net model modules, including a three-branch feature extraction network, a consensus convolution module, a decoupled stimulus feedback path, and a perturbation guidance subnetwork; The three-branch feature extraction network is used to receive multimodal image tensors and generate a set of modal feature maps; The consensus convolution module is used to obtain the intermodal consistency region and generate a consensus feature map; The decoupled excitation feedback path is used to construct a divergence feature map, generate a dynamic attention mask based on the divergence feature map, and use the dynamic attention mask to adjust the modal feature map to obtain an adjusted set of modal feature maps. The interference guidance subnetwork is used to fuse the interference masking map, consensus feature map, and adjusted modality feature map set to generate a multi-semantic label map; The path planning graph generation module is used to generate path planning graphs based on multi-semantic label graphs, construct a failure memory table, and introduce a dynamic memory-guided negative incentive reversal mechanism to generate updated path planning graphs. The instruction output module is used to perform logical consistency analysis on the updated path planning map, smooth the path segment direction, convert it into three-dimensional spatial coordinate points, generate forward, backward, left and right movement and up and down flight control instructions based on the coordinate difference, and output the final obstacle avoidance path and UAV flight control instruction sequence.
[0015] The beneficial effects of this invention are: This invention constructs a complete intelligent obstacle avoidance system, including a multimodal image preprocessing module, an interference mask map generation module, an improved DCE-Net model module, a path planning map generation module, and a command output module. Addressing the problems of inaccurate modal information fusion, weak interference recognition, and repeated path failures in existing UAV obstacle avoidance methods, this invention employs time synchronization and size uniformity processing of visual images, infrared images, and TOF images with a unified time base to generate highly consistent multimodal image tensors, improving the accuracy of spatial correspondence between modalities. In the interference mask map generation stage, high-precision positioning of interference areas is achieved through the fusion and smoothing of pixel value differences between modalities, effectively avoiding perception errors. An improved DCE-Net model is constructed, using a three-branch feature extraction network to obtain multi-source... Modal feature maps are extracted using a consensus convolutional module to extract consistent region features. A decoupled incentive feedback path is introduced to construct a divergence feature map and generate a dynamic attention mask, enabling fine-tuning of modal features. Furthermore, an interference-guided sub-network is used to fuse the interference mask map with multi-source features, outputting a multi-semantic label map to enhance semantic perception capabilities. During path planning map generation, a failure memory table and a dynamic memory-guided negative incentive reversal mechanism are introduced to avoid repeated path failures and achieve dynamic obstacle avoidance. Finally, in the command output module, logical consistency analysis smooths path segment direction changes, and combined with 3D coordinate differences, precise forward, backward, left / right movement, and up / down flight control command sequences for the UAV are generated, achieving highly safe and robust autonomous flight obstacle avoidance path output. Attached Figure Description
[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of an artificial intelligence-based intelligent obstacle avoidance method for unmanned aerial vehicles proposed in this invention; Figure 2 This is a schematic diagram of the structure of an artificial intelligence-based intelligent obstacle avoidance system for unmanned aerial vehicles proposed in this invention. Detailed Implementation
[0017] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0018] refer to Figure 1 An artificial intelligence-based intelligent obstacle avoidance method for drones includes the following steps: Step 1: Acquire visual images, infrared images, and TOF images, and perform time synchronization and size unification processing to generate multimodal image tensors; Step 2: Calculate the pixel value difference between modes based on the multimodal image tensor to generate an interference masking image; Step 3: Construct an improved DCE-Net model, which includes a three-branch feature extraction network, a consensus convolution module, a decoupled stimulus feedback path, and an interference guidance sub-network. The three-branch feature extraction network receives multimodal image tensors and generates a set of modal feature maps. Step 4: Stack the modal feature map set and input it into the consensus convolution module to obtain the consistency region between modalities and generate a consensus feature map; Step 5: Input the modal feature map set into the decoupled excitation feedback path to construct a bifurcation feature map, and generate a dynamic attention mask based on the bifurcation feature map. Use the dynamic attention mask to adjust the modal feature map to obtain an adjusted modal feature map set. Step 6: The interference guidance sub-network fuses the interference masking map, consensus feature map, and adjusted modality feature map set to generate a multi-semantic label map; Step 7: Generate a path planning graph based on the multi-semantic label graph, construct a failure memory table, introduce a dynamic memory-guided negative incentive reversal mechanism, and generate an updated path planning graph; Step 8: Perform logical consistency analysis on the updated path planning graph and output the final obstacle avoidance path and UAV flight control command sequence.
[0019] In this embodiment, step one specifically includes: The timestamp information corresponding to each frame of the visual image, infrared image and TOF image is extracted respectively. The visual image is used as a reference image. The target image frame that is closest to the timestamp of each frame of the reference image is found in the infrared image sequence and the TOF image sequence respectively to achieve intermodal frame-level correspondence matching, and image frames whose time difference exceeds the set time difference threshold are deleted. Each modal image is processed using an image resampling method to unify the resolution, so that each modal image has the same width, height and channel dimensions, and then they are stitched together to form a three-channel multimodal image tensor; To achieve time synchronization processing of three modalities of data—visual images, infrared images, and Time-of-Flight (TOF) images—this invention first extracts the timestamp information corresponding to the acquisition of each modality of image, establishing three sets of time series. Using the visual image as the primary reference modality, the target image frame closest to the timestamp of each visual image frame is searched in both the infrared image series and the TOF image series. This search process is based on the principle of minimizing the absolute value of the time difference; that is, the image frame with the timestamp closest to the current reference image is selected from the image series of the non-reference modality as the synchronization frame corresponding to that reference image.
[0020] To ensure temporal consistency between paired images across modalities, a time difference threshold is set. If the time difference between a modal image and a reference image exceeds this threshold, the entire image combination corresponding to that reference image frame is discarded. This method constructs strictly frame-level corresponding multimodal image combinations, providing a consistent temporal basis for feature extraction and tensor fusion, and improving the overall system's spatiotemporal perception accuracy in dynamic scenes.
[0021] In this embodiment, step two specifically includes: Pixel matrices corresponding to the visual image channel, infrared image channel and TOF image channel are extracted from the multimodal image tensor. For each pixel position, the pixel value difference between the infrared image channel and the visual image channel, and the pixel value difference between the TOF image channel and the visual image channel are calculated respectively, and a first difference map and a second difference map are generated respectively. The first difference map and the second difference map are averaged point by point according to the corresponding pixel positions to generate a fused difference map. The fused difference map is then subjected to local averaging within a sliding window of a set size to generate a smooth difference map. The locations in the smooth difference map where the pixel value is greater than a set pixel threshold are marked as interference regions, and the locations where the pixel value is less than or equal to the set pixel threshold are marked as non-interference regions. An interference mask map with the same size as the original image is generated. In the interference mask map, the pixel value of the interference region is 1, and the pixel value of the non-interference region is 0.
[0022] In this embodiment, the three-branch feature extraction network includes a visual image branch, an infrared image branch, and a TOF image branch. Each branch is composed of a first convolutional layer, a batch normalization layer, a ReLU activation function layer, a second convolutional layer, and a max pooling layer connected in sequence. The visual image channel in the multimodal image tensor is input to the visual image branch, the infrared image channel is input to the infrared image branch, and the TOF image channel is input to the TOF image branch; The input image channel of each branch is fed into the first convolutional layer to extract the first convolutional feature map. The first convolutional feature map is fed into the batch normalization layer for standardization processing. The standardization method with a mean of 0 and a variance of 1 is used to obtain the standardized feature map. The standardized feature map is input into the ReLU activation function layer for nonlinear transformation to generate an activation feature map. The activation feature map is then input into the second convolutional layer, which outputs a second convolutional feature map. The second convolutional feature map is input to the max pooling layer, and pooling is performed by taking the maximum value within a set window to output a modal feature map. The visual image branch, infrared image branch and TOF image branch each output modal feature maps of the same size, forming a modal feature map set, which includes visual modal feature maps, infrared modal feature maps and TOF modal feature maps.
[0023] In this embodiment, step four specifically includes: The visual modal feature map, infrared modal feature map, and TOF modal feature map in the modal feature map set are stacked in the channel dimension to form a modal stacking tensor; The modality stacking tensor is input into the consensus convolution module, which includes a cross-modality one-dimensional convolutional layer, a feature reconstruction layer, and a three-channel fusion layer connected in sequence. The cross-modal one-dimensional convolutional layer uses a convolutional kernel with a stride of 1 that slides along the modal channel direction to perform a one-dimensional convolution operation on the modal channel value corresponding to each spatial position, thereby obtaining the modal response tensor. The modal response tensor is input into the feature reconstruction layer, and the feature reconstruction layer performs variance calculation and mean normalization on the modal response value corresponding to each spatial location, and outputs a consistency score map. The locations where the pixel values in the consensus scoring map are lower than the set divergence threshold are marked as consensus regions. A point-by-point averaging operation is performed on the pixel values of each modality in the modal stacking tensor within the consensus region to generate a consensus feature map. This invention introduces a structured recognition mechanism for modal consistency regions in the consensus convolution module. By stacking multimodal feature maps along the channel dimension, a unified modal stacking tensor is formed. Addressing the issue that traditional multimodal fusion processes easily overlook local modal differences, the consensus convolution module includes a cross-modal one-dimensional convolutional layer. This layer performs one-dimensional convolution operations on different modal pixel values at each spatial location along the modal channel direction, extracting local response relationships between modalities and generating a modal response tensor. Based on this, a feature reconstruction layer performs statistical analysis on the modal response tensor, generating a consistency score map using variance calculation combined with mean normalization. Specifically: for each spatial location, the values at that location across all modal channels are summed and divided by the number of modalities to obtain the mean at that location; then, the mean is subtracted from each modal value, and the squares are calculated. The sum of these results is then divided by the number of modalities to obtain the variance at that location; the ratio of this variance to the maximum variance among all pixel locations is used to quantitatively describe the degree of feature difference between modalities.
[0024] By setting a divergence threshold, low-difference regions in the consistency score map are marked as consistent regions, and point-by-point averaging is performed in these regions to fuse multimodal feature information. Finally, a consensus feature map with modal consistency expression capability is generated, which significantly improves the robustness of downstream path planning in recognizing real obstacles and effectively suppresses the impact of intermodal interference information on obstacle avoidance accuracy.
[0025] In this embodiment, step five specifically includes: The modal feature map set is input into the decoupled stimulus feedback path, which includes a local difference extraction unit, a global difference extraction unit, a fusion attention generation unit, and a feedback reweighting adjustment unit. The local difference extraction unit performs difference calculation on any two modal feature maps in the modal feature map set at the same pixel position, and subtracts the second modal pixel value from the first modal pixel value and takes the absolute value to obtain a local difference map; The visual modal feature map, infrared modal feature map, and TOF modal feature map are calculated sequentially to obtain multiple local difference maps. All local difference maps are summed along the channel dimension and divided by the total number of local difference maps to generate an average local difference map. The global difference extraction unit calculates the average value of all pixel values for each modal feature map, subtracts the average value from each pixel value and takes the absolute value to obtain the global deviation map of the modal feature map; the global deviation maps of all modal feature maps are added point by point at the corresponding pixel positions and divided by the total number of modal feature maps to obtain the global difference map; The fusion attention generation unit performs an addition operation on the average local difference map and the global difference map at the corresponding pixel positions to generate a divergence feature map. Subtract the minimum pixel value of the bifurcation feature map from each pixel value in the bifurcation feature map, and divide by the difference between the maximum and minimum pixel values in the bifurcation feature map to obtain a normalized bifurcation map; input each pixel value in the normalized bifurcation map into the Sigmoid function to output a dynamic attention mask; The feedback reweighting adjustment unit replicates the dynamic attention mask in the channel dimension, multiplies the pixel value at each position in each modal feature map by the corresponding dynamic attention mask, and adds the result to the original pixel value multiplied by half of the dynamic attention mask (i.e., the adjusted pixel value = the pixel value before adjustment × 1.5 × dynamic attention mask), and outputs the adjusted modal feature map set. In this invention, a decoupled excitation feedback path is constructed, consisting of a local difference extraction unit, a global difference extraction unit, a fusion attention generation unit, and a feedback reweighting adjustment unit. This significantly improves the robustness and accuracy of multimodal feature fusion for UAVs in complex environments. The local difference extraction unit effectively characterizes the significant differences in fine-grained regions between modalities through pixel-level absolute difference calculations. The global difference extraction unit further calculates the overall deviation of the modal feature maps, capturing the overall distribution differences of modalities and enhancing feature stability. By fusing the average local difference map and the global difference map to generate a divergence feature map, and then normalizing and mapping with the sigmoid function to generate a dynamic attention mask with a continuous value domain, dynamic guidance and adjustment of multimodal difference regions are achieved.
[0026] In this invention, a dynamic feedback adjustment of the modal feature map is achieved by multiplying the pixel value at each location in the modal feature map by the corresponding dynamic attention mask, and then adding the result to the original pixel value multiplied by half the mask value. This process significantly enhances the performance of high-discrepancy regions. The larger the dynamic attention mask value, the stronger the modal inconsistency in that region, requiring the model to increase its attention. The product of the dynamic attention mask and the original pixel value enhances the feature response of that region. Furthermore, multiplying the original pixel value by half the mask value again and weighting the result allows for the preservation of some original feature information while improving the response intensity. This prevents semantic distortion caused by forced suppression or amplification, enabling a more refined adjustment strategy. This adjustment method exhibits good controllability and continuity, avoids the information truncation problem caused by traditional binary masking mechanisms, enhances the ability to discriminate modal conflict regions in complex environments, significantly improves the fusion expression effect, and ultimately improves the obstacle avoidance robustness and decision-making accuracy of UAVs in interference environments.
[0027] In this embodiment, the interference guidance subnetwork specifically includes a feature fusion coding unit, a guidance enhancement unit, and a label generation unit; The feature fusion coding unit concatenates the interference mask map, consensus feature map and adjusted modality feature map in the channel dimension to form a fusion input tensor. The fusion input tensor is then sequentially input into a coding module consisting of two convolutional layers and a ReLU activation function layer to extract the fusion deep feature map. The guidance enhancement unit performs element-wise multiplication on the interference masking map and the fused deep feature map at corresponding positions to obtain the interference guidance feature map; the interference guidance feature map and the fused deep feature map are concatenated along the channel dimension and input into a convolution to extract the semantic fusion feature map; The label generation unit performs a pixel-by-pixel Softmax operation on the semantic fusion feature map, outputting the category probability distribution corresponding to each pixel to form a multi-semantic label map. The multi-semantic label map is used to mark the semantic category of each region in the image, including obstacle edge region, passable region, high-risk interference region and background region, providing scene information support with more contextual understanding for UAV obstacle avoidance decision-making.
[0028] In this embodiment, step seven specifically includes: Based on the semantic category corresponding to each pixel in the multi-semantic label map, pixels marked as passable areas are selected to construct a passable area map; based on the current spatial position of the UAV and the pixel coordinates of the target position in the image, the shortest path from the starting point to the end point is searched using the cost function-driven Dijkstra algorithm, and a path planning map composed of continuous pixels is output. In the path planning process of this invention, the cost function is used to measure the path cost of moving from the current pixel position to the adjacent pixel position. The path search aims to minimize this cost. The cost function consists of two parts: spatial distance cost and semantic category cost.
[0029] Spatial distance cost is used to measure the spatial distance from the current pixel to its neighboring pixels, and is calculated based on the Euclidean distance between pixels.
[0030] The semantic category cost is assigned a fixed value based on the semantic category of each pixel in the multi-semantic label map. The cost for passable areas is set to 1, for obstacle edge areas to 5, for high-risk interference areas to 10, and for background areas to ∞ (i.e., impassable and not involved in path search). Different categories correspond to different static values to accurately distinguish semantic safety levels. The final path cost is obtained by weighting the spatial distance cost and the semantic category cost, thus achieving a strategy optimization that simultaneously considers geometric shortest path and semantic safety in path search.
[0031] During the process of the UAV performing the task according to the path planning map, the start and end pixel positions, corresponding timestamps and execution status of each path segment are recorded in real time; when a path segment is interfered with by obstacles, deviates from the preset distance threshold or the flight is interrupted, the path segment and execution status are written into the failure memory table. During the path generation process, the path segments in the current path planning map are extracted, and the Euclidean distance between the start and end coordinates and the coordinates of all path segments in the failure memory table is calculated. If the Euclidean distance between the starting point of the current path segment and the starting point of any path segment in the failure memory table is less than a preset tolerance threshold, and the Euclidean distance between the ending point of the current path segment and the ending point of the failure memory table is also less than a preset tolerance threshold, then the path segment is determined to have a failure risk, triggering the dynamic memory-guided negative incentive reversal mechanism. The dynamic memory-guided negative incentive reversal mechanism reverses the direction vector of the path segment with failure risk, that is, performs a negative direction operation, and adds a set disturbance angle offset to the direction vector to correct the path to avoid high-risk areas, generates an updated path segment, and splices the updated path segment into the original path planning map to output the updated path planning map.
[0032] In this embodiment, step eight specifically includes: The direction vectors between adjacent path segments in the updated path planning graph are calculated to obtain the direction angle. When the direction angle between a path segment and the previous path segment is greater than a preset angle threshold, the direction vectors of the two path segments are weighted and averaged to obtain a smooth direction vector. Starting from the end coordinates of the previous path segment based on the smooth direction vector, the end coordinates of the path segment are repositioned to replace the original path segment, forming a path sequence with continuous directional changes less than or equal to a set angle threshold, thus constituting a logically consistent smooth path. Convert the coordinates of each pixel in the smooth path into actual 3D spatial coordinates; calculate the coordinate difference between two consecutive 3D spatial coordinates in turn. If the difference on the X-axis is positive, a forward command is generated; if the difference on the X-axis is negative, a backward command is generated; if the difference on the Y-axis is positive, a right shift command is generated; if the difference on the Y-axis is negative, a left shift command is generated; if the difference on the Z-axis is positive, an upward command is generated; if the difference on the Z-axis is negative, a downward command is generated. By connecting all the three-dimensional spatial coordinate points in the smooth path in sequence, the final obstacle avoidance path is obtained. The control commands generated in sequence are arranged in the path order to form the UAV flight control command sequence, which serves as the control signal output for the UAV to perform the final mission.
[0033] refer to Figure 2An artificial intelligence-based intelligent obstacle avoidance system for drones includes the following modules: The multimodal image preprocessing module is used to acquire visual images, infrared images, and TOF images, and perform time synchronization and size unification processing to generate multimodal image tensors. The interference masking image generation module is used to calculate the pixel value difference between modes based on the multimodal image tensor and generate an interference masking image. Improved DCE-Net model modules, including a three-branch feature extraction network, a consensus convolution module, a decoupled stimulus feedback path, and a perturbation guidance subnetwork; The three-branch feature extraction network is used to receive multimodal image tensors and generate a set of modal feature maps; The consensus convolution module is used to obtain the intermodal consistency region and generate a consensus feature map; The decoupled excitation feedback path is used to construct a divergence feature map, generate a dynamic attention mask based on the divergence feature map, and use the dynamic attention mask to adjust the modal feature map to obtain an adjusted set of modal feature maps. The interference guidance subnetwork is used to fuse the interference masking map, consensus feature map, and adjusted modality feature map set to generate a multi-semantic label map; The path planning graph generation module is used to generate path planning graphs based on multi-semantic label graphs, construct a failure memory table, and introduce a dynamic memory-guided negative incentive reversal mechanism to generate updated path planning graphs. The instruction output module is used to perform logical consistency analysis on the updated path planning map, smooth the path segment direction, convert it into three-dimensional spatial coordinate points, generate forward, backward, left and right movement and up and down flight control instructions based on the coordinate difference, and output the final obstacle avoidance path and UAV flight control instruction sequence.
[0034] Example 1: To verify the feasibility of this invention in practice, it was applied to an automated inspection task using a drone in a complex urban environment with multiple obstacles. In this task, the drone needed to navigate through densely built-up urban areas to inspect communication towers, power lines, and traffic nodes. The flight path was interspersed with numerous tall buildings, pedestrians, and temporary construction areas, placing extremely high demands on the drone's obstacle avoidance capabilities and path planning. Traditional obstacle avoidance systems based on single vision or depth images often suffer from inaccurate obstacle recognition, path interruptions, or repeated flights when encountering complex conditions such as strong light interference, infrared obstruction, and reflective surface interference, severely impacting inspection efficiency and flight safety.
[0035] In the system provided by this invention, the acquired visual, infrared, and TOF images are first synchronized in time and standardized in size by a multimodal image preprocessing module, forming a three-channel multimodal image tensor with consistent frame rate and resolution, providing an accurate and consistent input basis for subsequent processing. In actual operation, the multimodal camera system on the UAV can simultaneously acquire 30 frames of image data per second, with the image resolution standardized to 640×480 pixels, and the preprocessing delay controlled within 15ms.
[0036] The interference masking map generation module performs pixel-level modal difference analysis on multimodal image tensors. By generating a fused difference map and performing sliding window processing, it removes noise while preserving edge information. Under conditions of direct sunlight during the day and heat source interference at night, this module can effectively extract dynamic interference areas in the environment.
[0037] Multimodal image tensors are input into an improved DCE-Net model module, which includes a three-branch feature extraction network, a consensus convolutional module, a decoupled stimulus feedback path, and a perturbation guidance sub-network. The three-branch feature extraction network can extract modal feature maps from visual, infrared, and TOF images respectively, improving the semantic separation between modalities. The consensus convolutional module obtains consistent regions, and a local and global difference map fusion mechanism is introduced in the decoupled stimulus feedback path to generate a dynamic attention mask, thereby fine-tuning the modal feature maps to obtain a set of adjusted modal feature maps. The perturbation guidance sub-network further fuses the perturbation mask map, the consensus feature map, and the adjusted set of modal feature maps, effectively filtering invalid or erroneous labeled regions and generating multi-semantic label maps.
[0038] The path planning map generation module constructs a passable area map based on the semantic label map, and combines a failure memory table and a dynamic memory-guided negative incentive reversal mechanism to re-plan the direction of historical failed path segments and the disturbance angle offset, thereby generating an updated path planning map.
[0039] The updated path planning graph is analyzed for logical consistency by the instruction output module. The path segment directions are smoothed and converted into three-dimensional spatial coordinate points. Based on the coordinate differences, control commands for forward, backward, left and right movement, and up and down flight are generated, and the final obstacle avoidance path and UAV flight control command sequence are output. In this embodiment, the system of the present invention is compared with a traditional model system based on YOLOv5 visual obstacle avoidance. The results are shown in Table 1 below.
[0040] Table 1. Comparison of obstacle avoidance performance between the system of this invention and traditional model systems in complex environments. Based on the experimental data analysis in Table 1 above, it can be seen that the obstacle avoidance performance of the system of the present invention is significantly better than that of the traditional model system in various complex obstacle areas, specifically reflected in the two core indicators of path pass rate and path adjustment times.
[0041] In static obstacle environments, the system of this invention achieved a path pass rate of 98.5%, far exceeding the 89.4% of the traditional model. Meanwhile, the average number of path adjustments was only 0.3 times, compared to 1.7 times for the traditional model, indicating that the present invention possesses stronger path planning stability and predictive capabilities when facing fixed obstacles.
[0042] In dynamic obstacle scenarios, the system of this invention still performs excellently, with a pass rate of 95.2%, a significant improvement compared to the 77.1% of the traditional model, indicating that it has a superior response mechanism in dynamic target recognition and real-time path correction. Furthermore, the average number of path adjustments is controlled at 0.8 times, far lower than the 2.9 times of the traditional model, indicating that the system adjusts its path less frequently in dynamic environments, resulting in smoother flight.
[0043] In challenging areas of reflective interference and low-light occlusion, the system of this invention, relying on multimodal image fusion and interference masking mechanisms, effectively suppresses perceptual errors and optical noise, achieving pass rates of 93.7% and 94.6% respectively, while traditional models only achieve 69.5% and 73.8%. Furthermore, the system of this invention requires significantly fewer path adjustments in these two types of regions than traditional models, further demonstrating its robustness and intelligence under complex visual interference.
[0044] This embodiment fully verifies the applicability and advancement of the method and system of the present invention in real-world complex environments, effectively improving the UAV's perception and obstacle avoidance decision-making capabilities in various complex environments such as static obstacles, dynamic obstacles, and visual interference areas. Simultaneously, the dynamic memory-guided negative incentive reversal mechanism can learn and adjust from historical failed paths, avoiding repeated errors and improving the reliability and intelligence level of path planning. The path smoothing and command output mechanism ensures the continuity and precise control of the UAV's flight process, reducing the risk of path oscillation and control delay.
[0045] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. An artificial intelligence-based unmanned aerial vehicle intelligent obstacle avoidance method, characterized in that, Includes the following steps: Step 1: Acquire visual images, infrared images, and TOF images, and perform time synchronization and size unification processing to generate multimodal image tensors; Step 2: Calculate the pixel value difference between modes based on the multimodal image tensor to generate an interference masking image; Step 3: Construct an improved DCE-Net model, which includes a three-branch feature extraction network, a consensus convolution module, a decoupled stimulus feedback path, and an interference guidance sub-network. The three-branch feature extraction network receives multimodal image tensors and generates a set of modal feature maps. Step 4: Stack the modal feature map set and input it into the consensus convolution module to obtain the consistency region between modalities and generate a consensus feature map; Step 5: Input the modal feature map set into the decoupled excitation feedback path to construct a bifurcation feature map, and generate a dynamic attention mask based on the bifurcation feature map. Use the dynamic attention mask to adjust the modal feature map to obtain an adjusted modal feature map set. Step 6: The interference guidance sub-network fuses the interference masking map, consensus feature map, and adjusted modality feature map set to generate a multi-semantic label map; Step 7: Generate a path planning graph based on the multi-semantic label graph, construct a failure memory table, introduce a dynamic memory-guided negative incentive reversal mechanism, and generate an updated path planning graph; Step 8: Perform logical consistency analysis on the updated path planning graph and output the final obstacle avoidance path and UAV flight control command sequence. 2.The method of claim 1, wherein, Step one specifically involves: The timestamp information corresponding to each frame of the visual image, infrared image and TOF image is extracted respectively. The visual image is used as a reference image. The target image frame that is closest to the timestamp of each frame of the reference image is found in the infrared image sequence and the TOF image sequence respectively to achieve intermodal frame-level correspondence matching, and image frames whose time difference exceeds the set time difference threshold are deleted. Each modal image is processed using an image resampling method to unify the resolution, so that each modal image has the same width, height and channel dimension, and then they are stitched together to form a three-channel multimodal image tensor. 3.The method of claim 1, wherein, Step two specifically involves: Pixel matrices corresponding to the visual image channel, infrared image channel and TOF image channel are extracted from the multimodal image tensor. For each pixel position, the pixel value difference between the infrared image channel and the visual image channel, and the pixel value difference between the TOF image channel and the visual image channel are calculated respectively, and a first difference map and a second difference map are generated respectively. The first difference map and the second difference map are averaged point by point according to the corresponding pixel positions to generate a fused difference map. The fused difference map is then subjected to local averaging within a sliding window of a set size to generate a smooth difference map. The locations in the smooth difference map where the pixel value is greater than a set pixel threshold are marked as interference regions, and the locations where the pixel value is less than or equal to the set pixel threshold are marked as non-interference regions. An interference mask map with the same size as the original image is generated. In the interference mask map, the pixel value of the interference region is 1, and the pixel value of the non-interference region is 0.
4. The unmanned aerial vehicle intelligent obstacle avoidance method based on artificial intelligence according to claim 1, characterized in that, The three-branch feature extraction network includes a visual image branch, an infrared image branch, and a TOF image branch. Each branch is composed of a first convolutional layer, a batch normalization layer, a ReLU activation function layer, a second convolutional layer, and a max pooling layer connected in sequence. The visual image channel in the multimodal image tensor is input to the visual image branch, the infrared image channel is input to the infrared image branch, and the TOF image channel is input to the TOF image branch; The input image channel of each branch is fed into the first convolutional layer to extract the first convolutional feature map. The first convolutional feature map is fed into the batch normalization layer for standardization processing. The standardization method with a mean of 0 and a variance of 1 is used to obtain the standardized feature map. The standardized feature map is input into the ReLU activation function layer for nonlinear transformation to generate an activation feature map. The activation feature map is then input into the second convolutional layer, which outputs a second convolutional feature map. The second convolutional feature map is input to the max pooling layer, and pooling is performed by taking the maximum value within a set window to output a modal feature map; the visual image branch, infrared image branch and TOF image branch each output modal feature maps of the same size, forming a modal feature map set.
5. The method of claim 1, wherein the method further comprises: Step four specifically involves: The visual modal feature map, infrared modal feature map, and TOF modal feature map in the modal feature map set are stacked in the channel dimension to form a modal stacking tensor; The modality stacking tensor is input into the consensus convolution module, which includes a cross-modality one-dimensional convolutional layer, a feature reconstruction layer, and a three-channel fusion layer connected in sequence. The cross-modal one-dimensional convolutional layer uses a convolutional kernel with a stride of 1 that slides along the modal channel direction to perform a one-dimensional convolution operation on the modal channel value corresponding to each spatial position, thereby obtaining the modal response tensor. The modal response tensor is input into the feature reconstruction layer, and the feature reconstruction layer performs variance calculation and mean normalization on the modal response value corresponding to each spatial location, and outputs a consistency score map. The locations where the pixel values in the consensus scoring map are lower than the set divergence threshold are marked as consensus regions. A point-by-point averaging operation is performed on the pixel values of each modality in the modal stacking tensor within the consensus region to generate a consensus feature map.
6. The unmanned aerial vehicle intelligent obstacle avoidance method based on artificial intelligence according to claim 1, characterized in that, Step five specifically involves: The modal feature map set is input into the decoupled stimulus feedback path, which includes a local difference extraction unit, a global difference extraction unit, a fusion attention generation unit, and a feedback reweighting adjustment unit. The local difference extraction unit performs difference calculation on any two modal feature maps in the modal feature map set at the same pixel position, and subtracts the second modal pixel value from the first modal pixel value and takes the absolute value to obtain a local difference map; The visual modal feature map, infrared modal feature map, and TOF modal feature map are calculated sequentially to obtain multiple local difference maps. All local difference maps are summed along the channel dimension and divided by the total number of local difference maps to generate an average local difference map. The global difference extraction unit calculates the average value of all pixel values for each modal feature map, subtracts the average value from each pixel value and takes the absolute value to obtain the global deviation map of the modal feature map; the global deviation maps of all modal feature maps are added point by point at the corresponding pixel positions and divided by the total number of modal feature maps to obtain the global difference map; The fusion attention generation unit performs an addition operation on the average local difference map and the global difference map at the corresponding pixel positions to generate a divergence feature map. Subtract the minimum pixel value of the bifurcation feature map from each pixel value in the bifurcation feature map, and divide by the difference between the maximum and minimum pixel values in the bifurcation feature map to obtain a normalized bifurcation map; input each pixel value in the normalized bifurcation map into the Sigmoid function to output a dynamic attention mask; The feedback reweighting adjustment unit replicates the dynamic attention mask in the channel dimension, multiplies the pixel value at each position in each modal feature map by the corresponding dynamic attention mask, and adds the result to the original pixel value multiplied by half of the dynamic attention mask, outputting the adjusted set of modal feature maps.
7. The method of claim 1, wherein the method further comprises: The interference guidance subnetwork specifically includes a feature fusion coding unit, a guidance enhancement unit, and a label generation unit; The feature fusion coding unit concatenates the interference mask map, consensus feature map and adjusted modality feature map in the channel dimension to form a fusion input tensor. The fusion input tensor is then sequentially input into a coding module consisting of two convolutional layers and a ReLU activation function layer to extract the fusion deep feature map. The guidance enhancement unit performs element-wise multiplication on the interference masking map and the number of channels of the fused deep feature map at corresponding positions to obtain the interference guidance feature map; The interference-guided feature map and the fused deep feature map are concatenated in the channel dimension and input into a convolution to extract the semantic fusion feature map; The label generation unit performs a pixel-by-pixel Softmax operation on the semantic fusion feature map, outputting the category probability distribution corresponding to each pixel to form a multi-semantic label map. 8.The method of claim 1, wherein, Step seven specifically involves: Based on the semantic category corresponding to each pixel in the multi-semantic label map, pixels marked as passable areas are selected to construct a passable area map; based on the current spatial position of the UAV and the pixel coordinates of the target position in the image, the shortest path from the starting point to the end point is searched using the cost function-driven Dijkstra algorithm, and a path planning map composed of continuous pixels is output. During the process of the UAV executing the task according to the path planning map, the start and end pixel positions, corresponding timestamps and execution status of each path segment are recorded in real time; When a path segment is interfered with by obstacles, deviates from the preset distance threshold, or the flight is interrupted, the path segment and its execution status are written into the failure memory table. During the path generation process, the path segments in the current path planning map are extracted, and the Euclidean distance between the start and end coordinates and the coordinates of all path segments in the failure memory table is calculated. If the Euclidean distance between the starting point of the current path segment and the starting point of any path segment in the failure memory table is less than a preset tolerance threshold, and the Euclidean distance between the ending point of the current path segment and the ending point of the failure memory table is also less than a preset tolerance threshold, then the path segment is determined to have a failure risk, triggering the dynamic memory-guided negative incentive reversal mechanism. The dynamic memory-guided negative incentive reversal mechanism reverses the direction vector of the path segment with failure risk, adds a set disturbance angle offset to the direction vector, generates an updated path segment, and then stitches the updated path segment into the original path planning graph to output the updated path planning graph. 9.The method of claim 1, wherein, Step eight specifically involves: The direction vectors between adjacent path segments in the updated path planning graph are calculated to obtain the direction angle. When the direction angle between a path segment and the previous path segment is greater than a preset angle threshold, the direction vectors of the two path segments are weighted and averaged to obtain a smooth direction vector. Starting from the end coordinates of the previous path segment based on the smooth direction vector, the end coordinates of the path segment are repositioned to replace the original path segment, forming a path sequence with continuous directional changes less than or equal to a set angle threshold, thus constituting a logically consistent smooth path. Convert the coordinates of each pixel in the smooth path into actual 3D spatial coordinates; calculate the coordinate difference between two consecutive 3D spatial coordinates in turn. If the difference on the X-axis is positive, a forward command is generated; If the difference on the X-axis is negative, a backward command is generated; if the difference on the Y-axis is positive, a right shift command is generated; if the difference on the Y-axis is negative, a left shift command is generated. If the Z-axis difference is positive, an up command is generated; if the Z-axis difference is negative, a down command is generated. By connecting all the three-dimensional spatial coordinate points in the smooth path in sequence, the final obstacle avoidance path is obtained. The control commands generated in sequence are arranged in the path order to form the UAV flight control command sequence, which serves as the control signal output for the UAV to perform the final mission.
10. An artificial intelligence-based unmanned aerial vehicle intelligent obstacle avoidance system, which executes an artificial intelligence-based unmanned aerial vehicle intelligent obstacle avoidance method according to any one of claims 1 to 9. Includes the following modules: The multimodal image preprocessing module is used to acquire visual images, infrared images, and TOF images, and perform time synchronization and size unification processing to generate multimodal image tensors. The interference masking image generation module is used to calculate the pixel value difference between modes based on the multimodal image tensor and generate an interference masking image. Improved DCE-Net model modules, including a three-branch feature extraction network, a consensus convolution module, a decoupled stimulus feedback path, and a perturbation guidance subnetwork; The three-branch feature extraction network is used to receive multimodal image tensors and generate a set of modal feature maps; The consensus convolution module is used to obtain the intermodal consistency region and generate a consensus feature map; The decoupled excitation feedback path is used to construct a divergence feature map, generate a dynamic attention mask based on the divergence feature map, and use the dynamic attention mask to adjust the modal feature map to obtain an adjusted set of modal feature maps. The interference guidance subnetwork is used to fuse the interference masking map, consensus feature map, and adjusted modality feature map set to generate a multi-semantic label map; The path planning graph generation module is used to generate path planning graphs based on multi-semantic label graphs, construct a failure memory table, and introduce a dynamic memory-guided negative incentive reversal mechanism to generate updated path planning graphs. The instruction output module is used to perform logical consistency analysis on the updated path planning map, smooth the path segment direction, convert it into three-dimensional spatial coordinate points, generate forward, backward, left and right movement and up and down flight control instructions based on the coordinate difference, and output the final obstacle avoidance path and UAV flight control instruction sequence.