A detection and segmentation method, system, device and medium for low-altitude unmanned aerial vehicles

Abstract

The present application relates to the technical field of image analysis, in particular to a detection and segmentation method, system, device and medium for low-altitude unmanned aerial vehicles; the present application method extracts and fuses multi-level features of an infrared image through a backbone network of a target detection network, and outputs fused and enhanced features of multiple scales; then, a position quality estimation detection head is used to process the multi-scale fused and enhanced features by coupling the positioning quality information with the classification confidence; then, in the semantic segmentation network, the structural integrity of the target area is preserved through feature residual processing, and the multi-scale convolution pooling and feature attention mechanism are combined to respectively enhance the adaptive ability of the model to scale changes and the accurate perception ability of the model to spatial positions; finally, after the multi-scale fused features and the attention enhanced features are fused, the pixel-level segmentation result is output through layered feature fusion, convolution and up-sampling processing, and the collaborative optimization of detection and segmentation is realized.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, specifically to a detection and segmentation method, system, device, and medium for low-altitude unmanned aerial vehicles (UAVs). Background Technology

[0002] With the rapid development of drone technology, drones have been widely used in military reconnaissance, agricultural plant protection, and logistics transportation. However, the security threats posed by the misuse of drones are becoming increasingly serious. Unauthorized drones frequently intrude into no-fly zones, interfere with civil aviation operations, and conduct illegal reconnaissance, posing a serious challenge to public and national safety. In the field of visual detection for counter-drone operations, traditional visible light-based detection methods are highly sensitive to lighting and weather conditions. Their performance deteriorates significantly under complex weather conditions such as nighttime, low light, fog, rain, and snow, making it difficult to meet the needs of all-weather applications. Infrared thermal imaging technology, with its characteristics of not relying on external light sources and being able to operate in all weather conditions, has become an important technical means for visual monitoring of counter-drones.

[0003] However, existing technologies still have the following shortcomings and deficiencies. First, insufficient feature extraction capability: the feature extraction modules of existing target detection networks use standard convolution stacking, lacking the ability to explicitly model high-frequency edge information. Infrared images of drone targets exhibit weak texture, low contrast, and blurred edges. Standard convolutions aggregate features within the local receptive field with uniform weights, making it difficult to effectively capture the subtle gradient changes between the target and the background in infrared images, resulting in insufficient detection accuracy. Second, low efficiency in global context modeling: the computational complexity of the standard multi-head self-attention mechanism in existing networks increases quadratically with the spatial resolution of the input feature map, resulting in a significant computational burden when processing high-resolution infrared images. Furthermore, it lacks the ability to effectively aggregate multi-scale contextual information, making it difficult to cope with the drastic scale changes of drone targets due to variations in flight altitude and distance. Third, inconsistency between classification and localization: existing detection heads struggle to effectively establish the intrinsic correlation between classification confidence and bounding box localization accuracy in low-contrast infrared scenes, easily leading to inconsistencies between high-confidence predictions and low localization accuracy. In addition, the semantic segmentation accuracy is insufficient: existing networks only perform channel dimensionality reduction through simple convolution in the low-level feature processing stage, which cannot effectively model the dependencies between different spatial locations in the feature map and is difficult to adaptively process target features of varying scales, resulting in blurred segmentation boundaries. Summary of the Invention

[0004] The purpose of this invention is to provide a detection and segmentation method, system, device, and medium for low-altitude unmanned aerial vehicles (UAVs).

[0005] The technical solution of this invention is as follows: A detection and segmentation method for low-altitude unmanned aerial vehicles (UAVs) includes the following operations: S1. Acquire an infrared image containing the UAV target, process it through a target detection network to obtain the target detection result, including the bounding box coordinates and detection confidence of the UAV target; the target detection network includes a backbone network and a position quality estimation detection head. The backbone network performs multi-level feature extraction and fusion on the infrared image and outputs fused enhancement features at multiple scales; the position quality estimation detection head processes the fused enhancement features at multiple scales and outputs the target detection result. S2. Based on the target detection results, the infrared image is cropped to obtain candidate target region images. The candidate target region images are then processed by a semantic segmentation network to achieve pixel-level semantic segmentation, resulting in semantic segmentation results, including the segmentation mask for the UAV target. During the semantic segmentation network processing, the candidate target region images undergo feature residual processing to obtain target region residual feature maps. After multi-scale convolutional pooling, the target region residual feature maps are concatenated, convolved, and upsampled to obtain multi-scale fusion features. The target region residual feature maps undergo feature attention processing and convolution processing to obtain attention-enhanced features. After fusing the multi-scale fusion features and attention-enhanced features, the resulting semantic segmentation results are obtained through hierarchical feature fusion, convolution, and upsampling.

[0006] The backbone network in S1 performs multi-level feature extraction and fusion on infrared images as follows: convolution processing is applied to the infrared image, and edge-aware feature enhancement at multiple levels is performed to obtain edge-aware enhanced features at multiple scales; spatial pyramid pooling and low-resolution self-attention mechanism are applied to the highest-level edge-aware enhanced features at multiple scales to obtain the enhanced highest-level features; the enhanced highest-level features are upsampled level by level and fused with the lower-level edge-aware enhanced features to obtain fused enhanced features at multiple scales.

[0007] The edge-aware enhancement processing adopts a parallel dual-branch architecture. The first branch is the SobelConv edge extraction branch, which uses the parameter-frozen Sobel operator to perform 3D grouped convolutions in the horizontal and vertical directions on the input feature map to extract the horizontal gradient response map and the vertical gradient response map. After superposition, the edge feature response map is generated. The second branch is the PConv partial convolution branch, which divides the input feature map into a representative channel subset and a redundant channel subset along the channel dimension. Spatial convolution is performed on the representative channel subset, and identity transfer is performed on the redundant channel subset. After concatenation, point-by-point convolution is performed to complete channel fusion and obtain the edge convolution feature map. The edge feature response map is concatenated and convolved with the edge convolution feature map to obtain the initial fused feature. The initial fused feature and the edge convolution feature map are concatenated and convolved and then output.

[0008] In S1, the location quality estimation detection head processes the fused enhanced features at multiple scales as follows: Channel adaptation is performed on the fused enhanced features at multiple scales, mapping them to a unified feature space. The channel-adapted features at each scale are then input into a shared convolutional layer for processing, followed by classification and regression to obtain the target classification score and bounding box prediction results. Based on the discretized probability distribution of the bounding box prediction results, a location quality estimation factor is generated. This factor is then fused with the target classification score to obtain the classification confidence score. Based on the classification confidence score and the bounding box prediction results, non-maximum suppression is performed to obtain the target detection result.

[0009] The feature residual processing operation in S2 is implemented through the MobileNetV2 network.

[0010] In S2, the hierarchical feature fusion operation is as follows: the input is mapped to a preset dimension feature space, and the feature map is uniformly divided into n parallel sub-feature groups along the channel dimension; convolution kernels with different specific sizes are assigned to the n sub-feature groups to form n processing branches. For the i-th branch, the convolution operation is the element-wise sum of the output features of the current i-th sub-feature group and the (i-1)-th branch. After all n branches have completed the convolution processing, the outputs of these n branches are re-stitched along the channel dimension, and the number of feature channels after stitching is compressed to the target dimension by output projection 1×1 convolution.

[0011] A detection and segmentation system for low-altitude unmanned aerial vehicles (UAVs), used to implement the aforementioned detection and segmentation method for low-altitude UAVs, includes: Infrared image acquisition module, used to acquire infrared images containing UAV targets; The target detection module processes infrared images using a target detection network to obtain target detection results, including the bounding box coordinates and detection confidence of the UAV target. The target detection network consists of a backbone network and a position quality estimation detection head. The backbone network performs multi-level feature extraction and fusion on the infrared image, outputting fused and enhanced features at multiple scales. The position quality estimation detection head processes the fused and enhanced features at multiple scales and outputs the target detection results. The semantic segmentation module is used to crop regions from the infrared image based on the target detection results to obtain candidate target region images. These candidate target region images are then processed by a semantic segmentation network to achieve pixel-level semantic segmentation, yielding the semantic segmentation result, including a segmentation mask for the UAV target. During the semantic segmentation network processing, the candidate target region images undergo feature residual processing to obtain target region residual feature maps. These residual feature maps are then subjected to multi-scale convolutional pooling, followed by concatenation, convolution, and upsampling to obtain multi-scale fusion features. The target region residual feature maps are then subjected to feature attention processing and convolution to obtain attention-enhanced features. Finally, the multi-scale fusion features and attention-enhanced features are fused, followed by hierarchical feature fusion, convolution, and upsampling to obtain the semantic segmentation result.

[0012] A detection and segmentation device for low-altitude unmanned aerial vehicles (UAVs) includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the aforementioned detection and segmentation method for low-altitude UAVs.

[0013] A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned detection and segmentation method for low-altitude unmanned aerial vehicles.

[0014] The beneficial effects of this invention are as follows: This invention provides a detection and segmentation method for low-altitude unmanned aerial vehicles (UAVs). First, a target detection network performs multi-level feature extraction and fusion on infrared images, outputting multi-scale fused enhanced features. This enables the network to simultaneously capture global semantics and local details within different receptive fields, effectively improving the feature representation capability of UAV targets with weak texture and low contrast in infrared scenes, thereby significantly improving detection recall and positioning accuracy. Then, a position quality estimation detection head processes the multi-scale fused enhanced features. By coupling positioning quality information with classification confidence, the problem of low positioning accuracy corresponding to high-confidence predictions in low-contrast infrared scenes is effectively alleviated. The inconsistency issue was addressed, improving the reliability and stability of the detection results. Next, in the semantic segmentation network, feature residual processing preserved the structural integrity of the target region. Combined with multi-scale convolutional pooling and feature attention mechanisms, the model's adaptability to scale changes and its accurate perception of spatial location were enhanced, effectively resolving the ambiguity of segmentation boundaries and improving the integrity and precision of the segmentation mask. Finally, after fusing multi-scale fusion features and attention-enhanced features, pixel-level segmentation results were output through hierarchical feature fusion, convolution, and upsampling. This achieved collaborative optimization of detection and segmentation, providing high-quality detection, localization, and contour information for applications such as target recognition. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the exemplary embodiments of this application clearer, the technical solutions in the exemplary embodiments of this application are described clearly and completely below. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0016] This embodiment provides a detection and segmentation method for low-altitude unmanned aerial vehicles (UAVs), including the following operations: S1. Acquire an infrared image containing the UAV target, process it through a target detection network to obtain the target detection result, including the bounding box coordinates and detection confidence of the UAV target; the target detection network includes a backbone network and a position quality estimation detection head. The backbone network performs multi-level feature extraction and fusion on the infrared image and outputs fused enhancement features at multiple scales; the position quality estimation detection head processes the fused enhancement features at multiple scales and outputs the target detection result. S2. Based on the target detection results, the infrared image is cropped to obtain candidate target region images. These candidate target region images are then processed by a semantic segmentation network to achieve pixel-level semantic segmentation, yielding semantic segmentation results, including a segmentation mask for the UAV target. During the semantic segmentation network processing, the candidate target region images undergo feature residual processing to obtain target region residual feature maps. These residual feature maps are then subjected to multi-scale convolutional pooling, followed by concatenation, convolution, and upsampling to obtain multi-scale fusion features. The target region residual feature maps are then processed by feature attention and convolution to obtain attention-enhanced features. Finally, the multi-scale fusion features and attention-enhanced features are fused, followed by hierarchical feature fusion, convolution, and upsampling to obtain the semantic segmentation results. The specific steps are detailed below.

[0017] S1. Acquire an infrared image containing the UAV target, process it through a target detection network, and obtain the target detection result, including the detection box coordinates and detection confidence of the UAV target.

[0018] The target detection network consists of a backbone network and a location quality estimation detection head. The backbone network performs multi-level feature extraction and fusion on the infrared image and outputs fused enhanced features at multiple scales. The location quality estimation detection head processes the fused enhanced features at multiple scales and outputs the target detection result.

[0019] The backbone network performs multi-level feature extraction and fusion on infrared images as follows: The backbone network performs convolution processing on the infrared images and enhances edge perception features at multiple levels to obtain edge perception enhancement features at multiple scales; among the edge perception enhancement features at multiple scales, the highest level edge perception enhancement features are processed by spatial pyramid pooling and low-resolution self-attention mechanism to obtain the enhanced highest level features. The enhanced highest-level features are upsampled level by level and fused with the lower-level edge-aware enhancement features to obtain multi-scale fused enhancement features as the output.

[0020] Specifically, the infrared image undergoes multiple convolutional processes to obtain convolutional features. These convolutional features are then subjected to first, second, and third edge-aware feature enhancement processes to obtain first-level, second-level, and third-level edge-aware enhancement features. The third-level edge-aware enhancement features are processed by spatial pyramid pooling and a low-resolution self-attention mechanism to obtain third-level edge self-attention enhancement features. The third-level edge self-attention enhancement features are then upsampled and fused with the second-level edge-aware enhancement features (achieved through concatenation) to obtain the first fused feature. The first fused feature undergoes feature extraction to obtain the first fused enhancement feature. The first fused enhancement feature is then upsampled, fused with the first-level edge-aware enhancement feature, and then feature extracted to obtain the second fused enhancement feature. The second fused enhancement feature is then convolved, fused with the first fused enhancement feature, and then feature extracted to obtain the third fused enhancement feature. The third fused enhancement feature is then convolved, fused with the third-level edge self-attention enhancement feature, and then feature extracted to obtain the fourth fused enhancement feature. The second, third, and fourth fused enhancement features are used as the outputs of the backbone network, i.e., as multi-scale fused enhancement features.

[0021] The aforementioned feature extraction operations include, but are not limited to, implementation using the C3k2 module of the YOLO series models.

[0022] The edge-aware feature enhancement process is achieved by sequentially executing edge-aware enhancement and convolution processing. The edge-aware enhancement process employs a parallel dual-branch architecture.

[0023] The first branch is the SobelConv edge extraction branch, which uses the parameter-frozen Sobel operator to perform 3D grouped convolutions in the horizontal and vertical directions on the input feature map to extract the horizontal gradient response map and the vertical gradient response map. The two maps are then superimposed to generate the edge feature response map. In this branch, the weight parameters of the Sobel operator are kept completely frozen during network training and do not participate in gradient updates. This design ensures the physical consistency and numerical stability of the edge prior features and avoids the network losing its sensitivity to edge direction due to parameter drift during training.

[0024] The horizontal gradient response plot is calculated using the following formula: , This is the horizontal gradient response map, representing the gradient strength of the input feature map in the x-direction. The horizontal Sobel kernel matrix. , p , q These represent the local spatial index displacements of the convolution kernel in the vertical and horizontal directions, respectively. In the horizontal Sobel kernel matrix, the relative positions of the coordinates are... The weight value at that location, In the input feature map, the local neighborhood feature pixel values ​​corresponding to the current convolution calculation center position (h,w) are offset by p in the height direction and q in the width direction, where h and w are the height coordinates and width coordinates of the current convolution calculation center position, respectively.

[0025] The vertical gradient response plot is calculated using the following formula: , This is the vertical gradient response map, representing the gradient strength of the input feature map in the y-direction. The vertical Sobel kernel matrix. , In the vertical Sobel kernel matrix, the relative positions of the coordinates are... The weight value at that location.

[0026] The second branch is the PConv partial convolution branch, which divides the input feature map into a representative channel subset and a redundant channel subset along the channel dimension according to a scaling factor. A 3×3 spatial convolution is performed on the representative channel subset to capture local spatial semantic and texture features, while the redundant channel subset is identity-passed to avoid wasting computational power by performing convolution operations on all channels. After concatenation, a 1×1 pointwise convolution is performed to complete channel fusion, completing the redistribution and global fusion of information across the entire channel range, compressing it to the original channel dimension to obtain the edge convolution feature map. The edge feature response map is concatenated and convolved with the edge convolution feature map to obtain the initial fused feature. The initial fused feature and the edge convolution feature map are concatenated and convolved with 1×1 for output.

[0027] The aforementioned scaling factor is a hyperparameter that controls channel sparsity. Its value determines the proportion of channels participating in spatial feature extraction. It is obtained based on the redundancy characteristics of infrared images and aims to significantly reduce the computational overhead caused by infrared background redundancy.

[0028] The edge perception enhancement processing in this embodiment combines edge gradient priors based on a fixed Sobel kernel with semantic texture features from partial convolutions in a dual-branch collaborative fusion. This effectively compensates for the lack of perception capability of standard convolutions in infrared weak edge scenarios, enabling the network to extract effective target discrimination information from weak gray-level gradients and significantly improving the detection accuracy of infrared weak UAV targets.

[0029] The operation of the low-resolution self-attention is as follows: the input is processed by adaptive average pooling in the Query generation branch, downsampled to a preset fixed low-resolution size, and low-resolution query features are obtained, thereby reducing the computational complexity of attention from quadratic to approximately linear; at the same time, the input is processed by multi-scale pyramid pooling in the Key and Value generation branches respectively, and pooling features of multiple scales are obtained; each scale pooling feature is processed by depthwise separable convolution, injecting local spatial priors and geometric structures, and after concatenating all scale features along the spatial dimension, it is processed by layer normalization and linear projection to obtain low-resolution key features and low-resolution value features; based on the low-resolution query features, low-resolution key features, and low-resolution value features, scaling dot product attention is performed in the low-resolution space to obtain low-resolution attention features; the low-resolution attention features are processed by bilinear interpolation upsampling to restore the original feature map spatial resolution, and self-attention enhancement features are obtained, that is, the third-level edge self-attention enhancement features are obtained.

[0030] This embodiment transforms the quadratic computational complexity into an approximately linear one by using a low-resolution self-attention mechanism. Combined with a pyramid pooling strategy, it simultaneously captures contextual semantic information within multiple receptive fields, which can significantly reduce computational complexity while enhancing the adaptability to multi-scale UAV targets.

[0031] The location quality estimation detection head processes the fused enhanced features at multiple scales as follows: Channel adaptation is performed on the fused enhanced features at multiple scales, mapping them to a unified feature space. The channel-adapted features at each scale are then input into a shared convolutional layer for processing, followed by classification and regression, outputting the target classification score and bounding box prediction results. Based on the discretized probability distribution of the bounding box prediction results, a location quality estimation factor is generated. This factor is then fused with the target classification score to obtain the classification confidence score. Based on the classification confidence score and the bounding box prediction results, non-maximum suppression is performed, outputting the target detection result.

[0032] Specifically, scale-specific channel-aligned convolutions are performed on the second, third, and fourth fusion enhancement features to map each layer's features to a unified hidden dimension space. The mapped features are then input into a depthwise separable convolution with shared parameters to extract features, followed by classification and regression to obtain target classification scores and bounding box predictions. The bounding box predictions include bounding box coordinates and their discretized probability distributions. The top k maximum probability values ​​and their statistical mean are extracted from the discretized probability distributions to construct a location quality descriptor. This descriptor is mapped to a quality adjustment factor using a multilayer perceptron and weighted and fused with the classification score to effectively transfer location quality information to classification confidence, resulting in classification confidence. Non-maximum suppression post-processing is applied to the bounding box coordinates, and combined with the classification confidence, the target detection result is obtained. Notably, the weights and biases of the last layer of the multilayer perceptron are initialized to zero to ensure an identity mapping during the initial training phase. This embodiment mines the statistical characteristics of bounding box distribution through the location quality estimation detection head, establishes an explicit correlation between classification confidence and positioning accuracy, effectively alleviates the inconsistency between high confidence and low positioning accuracy, and the shared convolution strategy significantly reduces parameter redundancy.

[0033] In this embodiment, the target detection network reduces the number of parameters to 2.39M and the computational cost to 5.7 GFLOPs, while achieving mAP50 of 92.0% and mAP50-95 of 53.1%, which are 1.9 and 1.4 percentage points higher than the existing advanced method YOLO11n model, respectively. Compared with YOLOv5s, the number of parameters is reduced by 66.8% and the computational cost is reduced by 65.5%.

[0034] S2. Based on the target detection results, the infrared image is cropped to obtain candidate target region images; the candidate target region images are processed by a semantic segmentation network to achieve pixel-level semantic segmentation and obtain semantic segmentation results, including the segmentation mask of the UAV target.

[0035] In the semantic segmentation network processing, the candidate target region image is processed by feature residual processing to obtain the target region residual feature map; the target region residual feature map is then processed by multi-scale convolution pooling, followed by concatenation, convolution and upsampling to obtain multi-scale fusion features; the target region residual feature map is then processed by feature attention and convolution to obtain attention-enhanced features; after fusing the multi-scale fusion features and attention-enhanced features, the semantic segmentation result is obtained through hierarchical feature fusion, convolution and upsampling.

[0036] The above-mentioned feature residual processing operations can be implemented using the MobileNetV2 network.

[0037] Multi-scale convolutional pooling operations are achieved through parallel convolution and pooling processes at different scales.

[0038] The hierarchical feature fusion operation is as follows: the input is mapped to a feature space with a wider number of channels in a preset dimension. The feature map is uniformly divided into n parallel sub-feature groups along the channel dimension, and each sub-feature group contains partial channel information. Convolutional kernels with different specific sizes are assigned to the n sub-feature groups to form n processing branches. An innovative progressive cross-branch fusion strategy is adopted: for the i-th branch, the actual input of its convolution operation is no longer just the current i-th sub-feature group, but the element-wise sum of the current i-th sub-feature group and the output features of the (i-1)-th branch. This mechanism enables the features of each subsequent branch to implicitly fuse the receptive field of the previous branch, constructing a hierarchical feature transfer path from fine-grained details to coarse-grained semantics. After all n branches have completed convolution processing, the outputs of these n branches are re-stitched along the channel dimension. The number of feature channels of the stitched feature is compressed to the target dimension through output projection 1×1 convolution, completing the unified representation of multi-scale features.

[0039] The operational steps for feature attention processing are detailed below.

[0040] Step 1: Perform one-dimensional global average pooling along the horizontal and vertical directions on the residual feature map of the target region. This operation compresses the two-dimensional spatial features, outputting two orientation-aware feature maps that aggregate horizontal and vertical positional information, respectively. Horizontal features at height h. Vertical features at width w The calculation formula is: , , These represent global statistics in the height (h) and width (w) directions, respectively. : Input the pixel value of the feature map at coordinate (h,i) in the c-th channel.

[0041] Step 2: Concatenate the orientation-aware feature maps of horizontal and vertical position information in the spatial dimension. Input the concatenated features into a shared 1×1 convolutional transform for channel dimensionality reduction. The dimensionality-reduced features are then processed by batch normalization layers to stabilize the data distribution and activated using the h_swish nonlinear activation function to obtain an intermediate feature representation containing rich receptive fields and spatial dependencies.

[0042] Step 3: Re-split the intermediate feature representation along the spatial dimension into two independent branch features. For these two branches, restore their channel number to the original channel number of the input feature map by two independent 1×1 convolution transformations. Map the feature values ​​to the interval of 0 to 1 by the Sigmoid activation function to generate attention weight matrices corresponding to the horizontal and vertical directions, respectively.

[0043] Step 4: Using element-wise multiplication, the residual feature map of the target region is multiplied with the generated horizontal attention weights and vertical attention weights. This operation enables the feature map to obtain higher response weights at key target locations, resulting in initial attention-enhanced features with enhanced position awareness, which are then used to perform convolution operations.

[0044] The semantic segmentation network in this embodiment endows low-level features with spatial localization capabilities through feature attention processing and progressive cross-branch fusion of hierarchical features to achieve adaptive receptive field coverage. The mIOU reaches 90.65%, the precision reaches 94.15%, and the recall reaches 95.71%, which are 2.36%, 0.73%, and 2.16% higher than the existing advanced method DeepLabV3+ model, respectively.

[0045] This embodiment also provides a detection and segmentation system for low-altitude unmanned aerial vehicles (UAVs), used to implement the above-mentioned detection and segmentation method for low-altitude UAVs, including: Infrared image acquisition module, used to acquire infrared images containing UAV targets; The target detection module processes infrared images using a target detection network to obtain target detection results, including the bounding box coordinates and detection confidence of the UAV target. The target detection network consists of a backbone network and a position quality estimation detection head. The backbone network performs multi-level feature extraction and fusion on the infrared image, outputting fused and enhanced features at multiple scales. The position quality estimation detection head processes the fused and enhanced features at multiple scales and outputs the target detection results. The semantic segmentation module is used to crop regions from infrared images based on target detection results to obtain candidate target region images. These candidate target region images are then processed by a semantic segmentation network to achieve pixel-level semantic segmentation, yielding semantic segmentation results, including a segmentation mask for UAV targets. During the semantic segmentation network processing, the candidate target region images undergo feature residual processing to obtain target region residual feature maps. These residual feature maps are then subjected to multi-scale convolutional pooling, followed by concatenation, convolution, and upsampling to obtain multi-scale fusion features. The target region residual feature maps are then processed by feature attention and convolution to obtain attention-enhanced features. Finally, the multi-scale fusion features and attention-enhanced features are fused, followed by hierarchical feature fusion, convolution, and upsampling to obtain the semantic segmentation result. The embedded computing platform, which houses the target detection module and semantic segmentation module, adopts a heterogeneous computing development board integrating GPU and CPU. It includes an inference acceleration unit and uses the TensorRT inference optimization engine for acceleration. Deployed on the Jetson Xavier NX embedded platform, it consumes only 10-15W. After TensorRT acceleration, the target detection module achieves an inference speed of 54 FPS, meeting real-time detection requirements and is suitable for field monitoring sites and mobile platforms. The gimbal control module is used to generate gimbal control commands based on the coordinates of the detection frame and adjust the orientation of the infrared image acquisition module. The gimbal control module calculates the normalized offset of the detection frame center relative to the image center. When the absolute value of the normalized offset exceeds the preset dead zone threshold, it generates yaw angle increment and pitch angle increment control commands according to the linear mapping relationship to drive the gimbal to rotate and keep the target in the center area of ​​the field of view. The visualization display terminal is used to display detection and segmentation results in real time. It's a desktop application developed based on the PyQt5 graphical interface framework, which communicates asynchronously with various functional units within the embedded computing platform via a signal-slot mechanism. The visualization display terminal includes a detection result display area and a segmentation result display area, simultaneously displaying the target detection bounding box annotation map and the semantic segmentation mask overlay map. It also includes a gimbal trajectory display area, an indicator monitoring panel, and a status bar, used to display information such as the target motion trajectory, real-time confidence level, and inference time, respectively. The data storage unit is used to store data. It adopts an embedded SQLite relational database to persistently store data such as detection result images, segmentation result images, detection confidence, segmentation accuracy indicators and timestamps that meet the storage threshold conditions in the local file system for subsequent query, statistical analysis and system performance evaluation. The inference acceleration unit uses the TensorRT inference optimization engine to accelerate the processing of the object detection module and the semantic segmentation module. By converting the PyTorch model weight file (.pt format) into the ONNX intermediate format, and then using TensorRT to parse and generate the optimized inference engine, the unit achieves operator fusion of convolutional layers, batch normalization layers and activation layers, reducing the number of GPU memory accesses and improving GPU computing utilization.

[0046] The system operates as follows: After system startup, the visualization terminal sequentially loads the deep learning model weights of the target detection module and semantic segmentation module in a background thread, while the inference acceleration unit simultaneously constructs the TensorRT engine. An infrared thermal imaging camera acquires infrared video streams of low-altitude scenes. The infrared image acquisition module normalizes the video frames to 640×640 pixels using the OpenCV library and performs format conversion. The preprocessed infrared image is fed into the target detection module, where inference is performed in the TensorRT engine of the inference acceleration unit, outputting the detection box coordinates and confidence scores of all candidate UAV targets. The gimbal control module selects the target with the highest confidence score, calculates the normalized offset of its detection box center relative to the image center, and generates gimbal control commands after dead-zone filtering and linear mapping. This commands drive the gimbal to adjust the orientation of the infrared thermal imaging camera, keeping the target in the center of the field of view. The target detection module performs region cropping on the original infrared image based on the detection boxes, sends the cropped target region image to the semantic module, where inference is performed in the inference acceleration unit, outputting a pixel-level semantic segmentation mask. When the detection confidence score of the target detection module and the segmentation result of the semantic segmentation module simultaneously reach a preset threshold, the data storage unit automatically writes the detection result, segmentation result, and related indicator data of the current frame into the SQLite database. The visualization display terminal receives and displays the detection result image and segmentation result image in real time, and synchronously refreshes monitoring information such as gimbal trajectory, confidence score, and inference time. By repeating the above steps, continuous real-time detection, accurate segmentation, and adaptive tracking of low-altitude intruding UAV targets can be achieved.

[0047] This embodiment also provides a detection and segmentation device for low-altitude unmanned aerial vehicles (UAVs), including a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the above-described detection and segmentation method for low-altitude UAVs.

[0048] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described detection and segmentation method for low-altitude unmanned aerial vehicles.

[0049] This embodiment provides a detection and segmentation method for low-altitude unmanned aerial vehicles (UAVs). First, the backbone network of a target detection network performs multi-level feature extraction and fusion on infrared images, outputting multi-scale fused enhanced features. This enables the network to simultaneously capture global semantics and local details within different receptive fields, effectively improving the feature representation capability of UAV targets with weak texture and low contrast in infrared scenes, thereby significantly improving detection recall and positioning accuracy. Then, a position quality estimation detection head processes the multi-scale fused enhanced features. By coupling positioning quality information with classification confidence, the problem of low positioning accuracy corresponding to high-confidence predictions in low-contrast infrared scenes is effectively alleviated. The inconsistency issue was addressed, improving the reliability and stability of the detection results. Next, in the semantic segmentation network, feature residual processing preserved the structural integrity of the target region. Combined with multi-scale convolutional pooling and feature attention mechanisms, the model's adaptability to scale changes and its accurate perception of spatial location were enhanced, effectively solving the problem of blurred segmentation boundaries and improving the integrity and precision of the segmentation mask. Finally, after fusing multi-scale fusion features and attention-enhanced features, pixel-level segmentation results were output through hierarchical feature fusion, convolution, and upsampling. This achieved collaborative optimization of detection and segmentation, providing high-quality detection, localization, and contour information for applications such as target recognition.

[0050] While exemplary embodiments of the invention have been described herein, many other variations or modifications conforming to the principles of the invention can be directly determined or derived from the disclosure of the invention without departing from its spirit and scope. Therefore, the scope of the invention should be understood and recognized to cover all such other variations or modifications.

Claims

1. A detection and segmentation method for low-altitude unmanned aerial vehicles (UAVs), characterized in that, This includes the following operations: S1. Acquire an infrared image containing the UAV target, process it through a target detection network, and obtain the target detection result, including the detection box coordinates and detection confidence of the UAV target; The target detection network consists of a backbone network and a location quality estimation detection head. The backbone network performs multi-level feature extraction and fusion on infrared images and outputs fused enhanced features at multiple scales. The location quality estimation detection head processes fused enhancement features at multiple scales and outputs target detection results; S2. Based on the target detection results, the infrared image is cropped to obtain candidate target region images; the candidate target region images are processed by a semantic segmentation network to achieve pixel-level semantic segmentation and obtain semantic segmentation results, including the segmentation mask of the UAV target; In the semantic segmentation network process, the candidate target region image is processed by feature residual processing to obtain the target region residual feature map; after the target region residual feature map is processed by multi-scale convolution pooling, it is concatenated, convolved and upsampled to obtain multi-scale fusion features; The residual feature map of the target region is processed by feature attention and convolution to obtain attention-enhanced features. After fusing the multi-scale fusion features with the attention-enhanced features, the semantic segmentation result is obtained by hierarchical feature fusion, convolution and upsampling.

2. The detection and segmentation method for low-altitude unmanned aerial vehicles according to claim 1, characterized in that, In S1, the backbone network performs multi-level feature extraction and fusion on the infrared image as follows: By performing convolution processing on infrared images and enhancing edge perception features at multiple levels, edge perception enhancement features at multiple scales are obtained. Among the edge perception enhancement features at multiple scales, the highest-level edge perception enhancement feature is processed by spatial pyramid pooling and low-resolution self-attention mechanism to obtain the enhanced highest-level feature. The enhanced highest-level features are upsampled level by level and fused with the lower-level edge-aware enhancement features to obtain multi-scale fused enhancement features.

3. The detection and segmentation method for low-altitude unmanned aerial vehicles according to claim 2, characterized in that, The edge-aware enhancement processing adopts a parallel dual-branch architecture. The first branch is the SobelConv edge extraction branch, which uses the parameter-frozen Sobel operator to perform three-dimensional grouped convolutions in the horizontal and vertical directions on the input feature map to extract the horizontal gradient response map and the vertical gradient response map. The two are then superimposed to generate the edge feature response map. The second branch is the PConv partial convolution branch, which divides the input feature map along the channel dimension into a representative channel subset and a redundant channel subset. Spatial convolution is performed on the representative channel subset, and identity transfer is performed on the redundant channel subset. After concatenation, pointwise convolution is performed to complete channel fusion, resulting in an edge convolution feature map. The edge feature response map is concatenated and convolved with the edge convolution feature map to obtain the initial fused feature. The initial fused feature and the edge convolution feature map are concatenated and convolved, and then output.

4. The detection and segmentation method for low-altitude unmanned aerial vehicles according to claim 1, characterized in that, In S1, the location quality estimation detection head processes the fused enhancement features across multiple scales as follows: The fused and enhanced features at multiple scales are channel-adapted and mapped to a unified feature space. The channel-adapted features at each scale are then input into a shared convolutional layer for processing, followed by classification and regression to obtain target classification scores and bounding box prediction results. Based on the discretized probability distribution of the bounding box prediction results, a location quality estimation factor is generated. This factor is then fused with the target classification score to obtain classification confidence. Based on the classification confidence and bounding box prediction results, non-maximum suppression is applied to obtain the target detection results.

5. The detection and segmentation method for low-altitude unmanned aerial vehicles according to claim 1, characterized in that, The feature residual processing operation in S2 is implemented through the MobileNetV2 network.

6. The detection and segmentation method for low-altitude unmanned aerial vehicles according to claim 1, characterized in that, In S2, the hierarchical feature fusion operation is as follows: the input is mapped to a preset dimension feature space, and the feature map is uniformly divided into n parallel sub-feature groups along the channel dimension; convolution kernels with different specific sizes are assigned to the n sub-feature groups to form n processing branches. For the i-th branch, the convolution operation is the element-wise sum of the output features of the current i-th sub-feature group and the (i-1)-th branch. After all n branches have completed the convolution processing, the outputs of these n branches are re-stitched along the channel dimension, and the number of feature channels after stitching is compressed to the target dimension by output projection 1×1 convolution.

7. A detection and segmentation system for low-altitude unmanned aerial vehicles (UAVs), used to implement the detection and segmentation method for low-altitude UAVs as described in claim 1, characterized in that, include: Infrared image acquisition module, used to acquire infrared images containing UAV targets; The target detection module is used to process infrared images through a target detection network to obtain target detection results, including the bounding box coordinates and detection confidence of UAV targets. The target detection network includes a backbone network and a position quality estimation detection head. The backbone network performs multi-level feature extraction and fusion on the infrared images and outputs fused enhanced features at multiple scales. The location quality estimation detection head processes fused enhancement features at multiple scales and outputs target detection results; The semantic segmentation module is used to crop regions from infrared images based on target detection results to obtain candidate target region images. These candidate target region images are then processed by a semantic segmentation network to achieve pixel-level semantic segmentation, yielding semantic segmentation results, including a segmentation mask for UAV targets. During the semantic segmentation network processing, the candidate target region images undergo feature residual processing to obtain target region residual feature maps. These residual feature maps are then subjected to multi-scale convolutional pooling, followed by concatenation, convolution, and upsampling to obtain multi-scale fusion features. The residual feature map of the target region is processed by feature attention and convolution to obtain attention-enhanced features. After fusing the multi-scale fusion features with the attention-enhanced features, the semantic segmentation result is obtained by hierarchical feature fusion, convolution and upsampling.

8. A detection and segmentation device for low-altitude unmanned aerial vehicles (UAVs), characterized in that, It includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the detection and segmentation method for low-altitude unmanned aerial vehicles as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the detection and segmentation method for low-altitude unmanned aerial vehicles as described in any one of claims 1-6.