Improved YOLOv11-based unmanned aerial vehicle aerial photography high-speed scene element segmentation method

Abstract

The invention discloses an unmanned aerial vehicle aerial high-speed scene element segmentation method based on improved YOLOv11, and the method comprises the steps: obtaining a to-be-segmented unmanned aerial vehicle aerial image, inputting the to-be-segmented unmanned aerial vehicle aerial image to a trained improved YOLOv11 model, and outputting a segmentation result of a high-speed scene element in the unmanned aerial vehicle aerial image; according to the method, a YOLOv11 model is improved, one branch of a C2DA module keeps original features so as to reserve underlying semantic information, and the other branch adaptively focuses on a key space region through a DAtention attention mechanism so as to extract significant features, so that feature perception and target segmentation performance of the model in a complex scene can be significantly improved; the scale feature map output by the first stage of the first backbone network is also input to the first neck network, and the resolution of the scale feature map output by the first stage is higher, so that the fine structure of the tiny target can be captured more effectively. By using the feature map with higher resolution, the network is reserved, and finer spatial texture and boundary information are utilized.

Description

Technical Field

[0001] This invention belongs to the field of instance segmentation technology, specifically relating to a method for segmenting high-speed scene elements in UAV aerial photography based on an improved YOLOv11. Background Technology

[0002] The deep integration of unmanned aerial vehicles (UAVs) and deep learning technology is driving unprecedented transformation in the fields of intelligent highway inspection and traffic management. This combination not only provides innovative technological means to improve the safety monitoring level and operational efficiency of highways, but also lays a solid foundation for building future intelligent transportation systems.

[0003] Deep learning technology, especially convolutional neural networks (CNNs), has demonstrated enormous potential in the transportation field due to its powerful automated feature extraction and analysis capabilities. Against this backdrop, traffic scene element recognition technology has evolved from object detection to instance segmentation. Object detection algorithms, such as the classic YOLO series, achieve excellent real-time performance by quickly predicting the bounding boxes and categories of objects (such as vehicles and cones). However, for many sophisticated highway management tasks, simply identifying the location of objects is far from sufficient. Instance segmentation, as a more advanced vision task, not only needs to identify objects but also accurately delineate the contours of each object at the pixel level. For traffic elements with irregular and elongated shapes, such as emergency lanes and guide lines on highways, traditional bounding box detection methods cannot accurately describe their geometric extent, easily including a large number of normal driving lane areas incorrectly, thus seriously interfering with subsequent logical judgments. In contrast, instance segmentation can generate pixel-level precise masks, accurately defining the actual contours of these special functional areas. This refined spatial information is a necessary prerequisite and key foundation for realizing advanced application logic such as automatic identification of illegal occupation of emergency lanes and intelligent alarm for illegal parking of vehicles in the guide line area.

[0004] Traditional highway monitoring relies primarily on fixed cameras deployed along the route. These cameras have fixed viewing angles, limited monitoring ranges, numerous blind spots, and difficulty in responding quickly and flexibly to emergencies. Drones, with their unparalleled maneuverability and wide aerial perspective, can effectively compensate for the shortcomings of traditional monitoring, enabling dynamic inspections and emergency reconnaissance of the entire highway route from all angles.

[0005] Despite the numerous advantages of drone technology in highway inspection, it still faces many challenges in practical applications. First, the uncertainty of drone shooting angle and flight altitude leads to significant variations in target size in the image, placing extremely high demands on the scale adaptability of the model. Second, occlusion and background interference in complex environments severely interfere with the accurate segmentation of target contours. Finally, the high-speed movement of drones or vehicles introduces motion blur, reducing image clarity and affecting the final segmentation results. Summary of the Invention

[0006] The purpose of this invention is to address the problems mentioned in the background art by proposing a method for high-speed scene element segmentation in UAV aerial photography based on an improved YOLOv11.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] The present invention proposes a method for high-speed scene element segmentation in UAV aerial photography based on an improved YOLOv11, comprising: acquiring a UAV aerial image to be segmented, inputting it into a trained improved YOLOv11 model, and outputting the segmentation result of high-speed scene elements in the UAV aerial image;

[0009] The improved YOLOv11 model includes a first backbone network, a first neck network, and a first head network. The drone aerial image to be segmented serves as the input to the first backbone network. The first backbone network outputs feature maps at different scales in its four stages. All four feature maps at different scales are input to the first neck network, which performs cross-layer fusion enhancement to obtain four enhanced feature maps. These four enhanced feature maps are then used as input to the first head network to obtain feature maps at four different scales. Figure 1 The first head network outputs a prototype mask for each of the four corresponding detection scales. Each detection result includes the target's classification category, bounding box, and mask coefficients. The detection results at the four scales are concatenated in the dimension, and non-maximum suppression is performed on all bounding boxes to obtain a unique final bounding box for each target. For each final bounding box, the mask coefficients corresponding to the final bounding box are linearly combined with the prototype mask to generate an instance segmentation mask for the corresponding target, thereby obtaining the segmentation result of high-speed scene elements in the UAV aerial image.

[0010] Preferably, the C2PSA module in the fourth stage of the backbone network in the YOLOv11 model is replaced with a C2DA module to obtain the first backbone network;

[0011] The C2DA module includes a first convolutional layer, a first segmentation operation, two cascaded DAttention modules, and a second convolutional layer connected in sequence. The first segmentation operation evenly divides the number of channels of the output of the first convolutional layer into two parts. One part is used as the input of the two cascaded DAttention modules, and the other part is concatenated with the outputs of the two cascaded DAttention modules to obtain a first concatenation result. The first concatenation result is then used as the input of the second convolutional layer, and the output of the second convolutional layer is used as the output of the C2DA module.

[0012] Preferably, the DAttention module includes a convolutional structure, a first module, a convolutional layer, a second module, a convolutional layer, a third module, a convolutional layer, and a fourth module connected in sequence. The first module and the second module both include local attention and shift window attention connected in sequence. The third module and the fourth module both include local attention and deformable attention connected in sequence. The convolutional structure includes two convolutional layers connected in series.

[0013] Preferably, in the first neck network, the feature maps of different scales output from the four stages of the first backbone network are sequentially passed through the third, fourth, fifth, and sixth convolutional layers. The output of the sixth convolutional layer, after upsampling, is fused with the output of the fifth convolutional layer to obtain a first fused feature. This first fused feature then passes through the first C3k2_CFBlock module and the upsampling module, and is fused with the output of the fourth convolutional layer to obtain a second fused feature. The second fused feature then passes through the second C3k2_CFBlock module and the upsampling module, and is fused with the output of the third convolutional layer to obtain a third fused feature. This third fused feature then passes through the third C3k2_CFBlock module and the convolutional layer, and is fused with the outputs of the fourth and third convolutional layers. The fourth fusion feature is obtained. This fourth fusion feature then passes through the fourth C3k2_CFBlock module and a convolutional layer, and is then fused with the output of the fifth convolutional layer and the output of the first C3k2_CFBlock module to obtain the fifth fusion feature. This fifth fusion feature then passes through the fifth C3k2_CFBlock module and a convolutional layer, and is then fused with the output of the sixth convolutional layer to obtain the sixth fusion feature. This sixth fusion feature then passes through the sixth C3k2_CFBlock module. The outputs of the third, fourth, fifth, and sixth C3k2_CFBlock modules are the first enhanced feature map, the second enhanced feature map, the third enhanced feature map, and the fourth enhanced feature map, respectively. All four enhanced feature maps are used as inputs to the first head network.

[0014] The feature fusion operation is a weighted fusion.

[0015] Preferably, each of the C3k2_CFBlock modules includes a seventh convolutional layer, a second splitting operation, a branching structure, and an eighth convolutional layer connected in sequence. The branching structure includes multiple cascaded C3k_CFBlock modules or multiple cascaded CFBlock modules. When the parameter C3k of the C3k2_CFBlock module is true, the branching structure includes multiple cascaded C3k_CFBlock modules. When the parameter C3k of the C3k2_CFBlock module is false, the branching structure includes multiple cascaded CFBlock modules. The second splitting operation evenly divides the number of channels of the output of the seventh convolutional layer into two parts. One part is used as the input of the branching structure, and the other part is concatenated with the output of the branching structure to obtain a second concatenation result. The second concatenation result is then used as the input of the eighth convolutional layer to obtain the output of the C3k2_CFBlock module.

[0016] Preferably, the C3k_CFBlock module includes a ninth convolutional layer, a third splitting operation, multiple cascaded CFBlock modules, and a tenth convolutional layer connected in sequence. The third splitting operation evenly divides the number of channels of the output of the ninth convolutional layer into two parts. One part is used as the input of the multiple cascaded CFBlock modules, and the other part is concatenated with the output of the multiple cascaded CFBlock modules to obtain a third concatenation result. The third concatenation result is then used as the input of the tenth convolutional layer, and the output of the tenth convolutional layer is used as the output of the C3k_CFBlock module.

[0017] Preferably, the CFBlock module includes a Convolutional Attention module, a first normalization layer, a feedforward neural network, and a second normalization layer connected in sequence. The input of the Convolutional Attention module is added to the output of the Convolutional Attention module, and the result of the addition is used as the input of the first normalization layer. The output of the first normalization layer is added to the output of the feedforward neural network, and the result of the addition is used as the input of the second normalization layer. The output of the second normalization layer is used as the output of the CFBlock module.

[0018] Preferably, in the Convolutional Attention module, the input of the Convolutional Attention module is linearly transformed to obtain a first key vector and a second key vector. The input of the Convolutional Attention module is then convolved with the first key vector and the second key vector, respectively. The results of the two convolution operations are then grouped and double-normalized. The results of the two grouped double-normalized operations are then convolved with the transposes of the corresponding first key vector and the transposes of the second key vector, respectively, to obtain a first efficient feature and a second efficient feature. The first efficient feature and the second efficient feature are then added together to obtain the output of the Convolutional Attention module. The formulas for obtaining each efficient feature are as follows:

[0019] ;

[0020] ;

[0021] in, and These are the first and second high-efficiency features, respectively. This serves as the input to the Convolutional Attention module. The first key vector, This is the second key vector. For the channel, For height, For width, The number of learnable parameters. The kernel size for learnable parameters, For transpose, For convolution operations, This is a grouped double normalization operation, and the grouped double normalization operation is in × The Softmax function is executed on the dimension of , in Perform grouped L2 norm on the dimension, that is, on The dimension is divided into a preset number of groups, and the L2 norm is applied to each group.

[0022] Preferably, the first head network is an improvement on the TADDH module. The first head network includes a prototype branch, a first detection head, a second detection head, a third detection head, a fourth detection head, and a shared module. Each detection head includes a mask branch, a bounding box branch, and a classification branch. The prototype branch includes multiple concatenated convolutional layers. The mask branch includes a shared convolution, a depthwise convolution, a pointwise convolution, and a mask coefficient convolution connected in sequence. The first enhanced feature map is passed through the prototype branch to obtain a prototype mask. The shared module, the bounding box branches of each detection head, and the classification branches constitute the TADDH module.

[0023] The outputs of the mask branch, bounding box branch, and classification branch are the mask coefficient, bounding box, and classification category, respectively.

[0024] Among them, the first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map correspond one-to-one with the first detection head, the second detection head, the third detection head and the fourth detection head, respectively. After passing through the shared module, each enhanced feature map passes in parallel through the bounding box branch and the classification branch of the corresponding detection head. Each enhanced feature map also passes through the mask branch of the corresponding detection head. The outputs of the mask branch, bounding box branch and classification branch of each detection head constitute the target detection result corresponding to the corresponding enhanced feature map.

[0025] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0026] This method for high-speed scene segmentation in UAV aerial photography, based on an improved YOLOv11 model, enhances the YOLOv11 model by modifying one branch of the C2DA module to preserve the original features and retain underlying semantic information, while the other branch adaptively focuses on key spatial regions through the DAttention mechanism to extract salient features. This significantly improves the model's feature perception and target segmentation performance in complex scenes. The scale feature map output from the first stage of the first backbone network is also input into the first neck network. Because the scale feature map output from the first stage has a higher resolution, it can more effectively capture the fine structure of small targets. Using this higher-resolution feature map, the network can preserve and utilize more detailed spatial texture and boundary information. This allows the model to not only more accurately detect small targets, but more importantly, to depict their pixel-level contours with higher precision, thus significantly improving the segmentation quality and boundary clarity of small targets. Furthermore, the first head network not only significantly reduces the number of model parameters but also further improves segmentation accuracy. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the improved YOLOv11 in the UAV aerial photography high-speed scene element segmentation method based on improved YOLOv11 of the present invention;

[0028] Figure 2 This is a schematic diagram of the C2DA module in this invention;

[0029] Figure 3 This is a schematic diagram of the DAttention module in this invention;

[0030] Figure 4 This is a schematic diagram of the structure of the C3k2_CFBlock module in this invention;

[0031] Figure 5 This is a schematic diagram of the structure of the C3k_CFBlock module in this invention;

[0032] Figure 6 This is a schematic diagram of the CFBlock module in this invention;

[0033] Figure 7 This is a schematic diagram of the structure of the first head network in this invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0036] like Figures 1-7 As shown, a method for high-speed scene feature segmentation in UAV aerial photography based on an improved YOLOv11 is provided, including:

[0037] The drone aerial image to be segmented is obtained, input into the trained improved YOLOv11 model, and the output is an image masked for elements belonging to high-speed scenes in the drone aerial image;

[0038] The improved YOLOv11 model includes a first backbone network, a first neck network, and a first head network. The drone aerial image to be segmented serves as the input to the first backbone network. The four stages of the first backbone network output feature maps at different scales. All four feature maps at different scales are input to the first neck network, which performs cross-layer fusion enhancement to obtain four enhanced feature maps. These four enhanced feature maps are then used as input to the first head network, resulting in feature maps at four different scales. Figure 1 The first head network outputs a prototype mask for each of the four corresponding detection scales. Each detection result includes the target's classification category, bounding box, and mask coefficients. The detection results at the four scales are concatenated in the dimension, and non-maximum suppression is performed on all bounding boxes to obtain a unique final bounding box for each target. For each final bounding box, the mask coefficients corresponding to the final bounding box are linearly combined with the prototype mask to generate an instance segmentation mask for the corresponding target, thereby obtaining the segmentation result of high-speed scene elements in the UAV aerial image.

[0039] It should be noted that the first backbone network consists of four sequentially connected stages (the first stage includes two sequentially connected convolutional layers and a C3k2 module, the second stage includes a sequentially connected convolutional layer and a C3k2 module, the third stage includes a sequentially connected convolutional layer and a C3k2 module, and the fourth stage includes a sequentially connected convolutional layer, a C3k2 module, a fast spatial pyramid pooling (SPPF) module, and a C2DA module). When the size of the input UAV aerial image is 640×640×3, the scale feature map output by the first stage is 160×160×128, the scale feature map output by the second stage is 80×80×256, the scale feature map output by the third stage is 40×40×512, and the scale feature map output by the fourth stage is 20×20×1024. The first backbone network is obtained by replacing the C2PSA module in the fourth stage of the backbone network in the YOLOv11 model with a C2DA module (while keeping the rest of the structure unchanged).

[0040] like Figure 2 As shown, the C2DA module includes a first convolutional layer, a first segmentation operation, two cascaded DAttention modules, and a second convolutional layer connected sequentially (from data input to output). The first segmentation operation evenly divides the number of channels of the output of the first convolutional layer into two parts. One part is used as the input of the two cascaded DAttention modules, and the other part is concatenated with the outputs of the two cascaded DAttention modules to obtain the first concatenation result. The first concatenation result is then used as the input of the second convolutional layer, and the output of the second convolutional layer is used as the output of the C2DA module. In the C2DA module, one branch maintains the original features to preserve the underlying semantic information, while the other branch adaptively focuses on key spatial regions to extract salient features through the DAttention attention mechanism, which can significantly improve the feature perception and target segmentation performance of the model in complex scenes.

[0041] like Figure 3As shown, the DAttention module includes a convolutional structure (connected sequentially from data input to output), a first module, a convolutional layer (using a stride greater than 1 to reduce the feature map space size while extracting features through convolution), a second module, a convolutional layer (using a stride greater than 1 to reduce the feature map space size while extracting features through convolution), a third module, a convolutional layer (using a stride greater than 1 to reduce the feature map space size while extracting features through convolution), and a fourth module. The first and second modules both include Local Attention and Shift-Window Attention (connected sequentially from data input to output), while the third and fourth modules both include Local Attention and Deformable Attention (connected sequentially from data input to output). The convolutional structure includes two concatenated convolutional layers (3×3).

[0042] like Figure 1As shown, for the first neck network (the scale feature map output from the first stage of the first backbone network is also input into the first neck network, and because the scale feature map output from the first stage has a higher resolution, it can more effectively capture the fine structure of small targets. Using this higher resolution feature map, the network can retain and utilize more detailed spatial texture and boundary information. This allows the model to not only detect small targets more sensitively, but more importantly, to depict their pixel-level contours with higher accuracy, thereby significantly improving the segmentation quality and boundary clarity of small targets), in the first neck network, the different scale feature maps output from the four stages of the first backbone network are sequentially processed one by one. The third, fourth, fifth, and sixth convolutional layers (convolutional layers unify the number of channels in the feature maps, optimizing the fusion of subsequent feature maps) are used. The output of the sixth convolutional layer, after upsampling, is fused with the output of the fifth convolutional layer to obtain the first fused feature. This first fused feature then passes through the first C3k2_CFBlock module and the upsampling module, before being fused with the output of the fourth convolutional layer to obtain the second fused feature. This second fused feature then passes through the second C3k2_CFBlock module and the upsampling module, before being fused with the output of the third convolutional layer to obtain the third fused feature. This third fused feature then passes through the... After passing through the third C3k2_CFBlock module and convolutional layer, the features are fused with the outputs of the fourth convolutional layer and the third C3k2_CFBlock module to obtain the fourth fused feature. This fourth fused feature then passes through the fourth C3k2_CFBlock module and convolutional layer, and is fused with the outputs of the fifth convolutional layer and the first C3k2_CFBlock module to obtain the fifth fused feature. This fifth fused feature then passes through the fifth C3k2_CFBlock module and convolutional layer, and is fused with the output of the sixth convolutional layer to obtain the sixth fused feature. This sixth fused feature then passes through the sixth C3k2_CFBlock module... The first head network has four enhanced feature maps, and the outputs of the third, fourth, fifth, and sixth C3k2_CFBlock modules are the first enhanced feature map, the second enhanced feature map, the third enhanced feature map, and the fourth enhanced feature map, respectively. All four enhanced feature maps are used as inputs to the first head network (this first neck network performs efficient bidirectional cross-scale information interaction to fully integrate feature information from different levels). The spatial modeling and semantic aggregation capabilities of the feature maps are significantly enhanced through each C3k2_CFBlock module, C3k_CFBlock module, and CFBlock module.

[0043] The feature fusion operation is a weighted fusion.

[0044] Among them, such as Figure 4As shown, each C3k2_CFBlock module includes a seventh convolutional layer, a second splitting operation, a branching structure, and an eighth convolutional layer connected sequentially (from data input to output). The branching structure includes multiple cascaded C3k_CFBlock modules (one C3k_CFBlock module is used in this embodiment) or multiple cascaded CFBlock modules (one C3k_CFBlock module is used in this embodiment). When the parameter C3k of the C3k2_CFBlock module is true, the branching structure includes multiple cascaded C3k_CFBlock modules. When the parameter C3k of the C3k2_CFBlock module is false, the branching structure includes multiple cascaded CFBlock modules. The second splitting operation evenly divides the number of channels of the output of the seventh convolutional layer into two parts. One part is used as the input of the branching structure, and the other part is concatenated with the output of the branching structure to obtain a second concatenation result. The second concatenation result is then used as the input of the eighth convolutional layer to obtain the output of the C3k2_CFBlock module.

[0045] Among them, such as Figure 5 As shown, the C3k_CFBlock module includes a ninth convolutional layer, a third splitting operation, multiple cascaded CFBlock modules (one CFBlock module is used in this embodiment), and a tenth convolutional layer, which are connected in sequence (from data input to output). The third splitting operation divides the number of channels of the output of the ninth convolutional layer into two parts. One part is used as the input of multiple cascaded CFBlock modules, and the other part is concatenated with the output of multiple cascaded CFBlock modules to obtain the third concatenation result. The third concatenation result is then used as the input of the tenth convolutional layer, and the output of the tenth convolutional layer is used as the output of the C3k_CFBlock module.

[0046] Among them, such as Figure 6 As shown, the CFBlock module includes a Convolutional Attention module, a first normalization layer, a feedforward neural network, and a second normalization layer connected sequentially (from data input to output direction). The input of the Convolutional Attention module is added to the output of the Convolutional Attention module, and the result of this addition is used as the input of the first normalization layer. The output of the first normalization layer is added to the output of the feedforward neural network, and the result of this addition is used as the input of the second normalization layer. The output of the second normalization layer is used as the output of the CFBlock module.

[0047] In the Convolutional Attention module, the input is linearly transformed (through convolutional linear projection) to obtain a first key vector and a second key vector. The input to the Convolutional Attention module is then convolved with both the first and second key vectors. The results of these two convolutions are then grouped and double-normalized. The results of these two double-normalized groups are then convolved with the transposes of the corresponding first and second key vectors to obtain a first efficient feature and a second efficient feature, respectively. The first and second efficient features are then added together to obtain the output of the Convolutional Attention module. The formulas for obtaining each efficient feature are as follows:

[0048] ;

[0049] ;

[0050] in, and These are the first and second high-efficiency features, respectively. This serves as the input to the Convolutional Attention module. The first key vector, This is the second key vector. For the channel, For height, For width, The number of learnable parameters. The kernel size for learnable parameters, For transpose, For convolution operations, This is a grouped double normalization operation, and the grouped double normalization operation is in × The Softmax function is executed on the dimension of , in Perform grouped L2 norm on the dimension, that is, on The dimension is divided into a preset number of groups, and the L2 norm is applied to each group.

[0051] like Figure 7As shown, the first head network is an improvement on the TADDH module. The first head network includes a prototype branch, a first detection head, a second detection head, a third detection head, a fourth detection head, and a shared module. Each detection head includes a mask branch, a bounding box branch, and a classification branch. The prototype branch includes multiple concatenated convolutional layers. The mask branch includes a shared convolution (Conv-GN, 1×1), a depthwise convolution (3×3), a pointwise convolution (1×1, where the depthwise convolution and the pointwise convolution constitute a depthwise separable convolution), and a mask coefficient convolution (Conv-Mask) connected sequentially (from data input to output direction). The first enhanced feature map is passed through the prototype branch to obtain the prototype mask. The shared module, the bounding box branches of each detection head, and the classification branch constitute the TADDH module (the structure of the TADDH module belongs to the prior art).

[0052] The outputs of the mask branch, bounding box branch, and classification branch are mask coefficients, bounding boxes, and classification categories, respectively; the bounding box branch includes convolutional layers and scaling operations (introducing a learnable scaling factor for scaling), and the classification branch is a convolutional layer;

[0053] The first enhanced feature map passes through the sharing module and then passes through the bounding box branch and classification branch of the first detection head in parallel. The first enhanced feature map also passes through the mask branch of the first detection head. The outputs of the mask branch, bounding box branch and classification branch of the first detection head constitute the target detection result corresponding to the first enhanced feature map.

[0054] After the second enhanced feature map passes through the sharing module, it then passes through the bounding box branch and classification branch of the second detection head in parallel. The second enhanced feature map also passes through the mask branch of the second detection head. The outputs of the mask branch, bounding box branch, and classification branch of the second detection head constitute the target detection result corresponding to the second enhanced feature map.

[0055] After the third enhanced feature map passes through the shared module, it then passes through the bounding box branch and classification branch of the third detection head in parallel. The third enhanced feature map also passes through the mask branch of the third detection head. The outputs of the mask branch, bounding box branch, and classification branch of the third detection head constitute the target detection result corresponding to the third enhanced feature map.

[0056] After passing through the shared module, the fourth enhanced feature map is then passed in parallel through the bounding box branch and classification branch of the fourth detection head. The fourth enhanced feature map also passes through the mask branch of the fourth detection head. The outputs of the mask branch, bounding box branch, and classification branch of the fourth detection head constitute the target detection result corresponding to the fourth enhanced feature map.

[0057] In the shared module, the input sequentially passes through a first shared convolution (Conv-GN) and a second shared convolution. The outputs of the first and second shared convolutions are concatenated. The concatenated output then passes through a first Task Decomposition module, a second Task Decomposition module, a mask and offset generation module, and a gated activation structure (Conv ReLU ConvSigmoid, specifically using ReLU to generate the primary response and then using Sigmoid to generate a weight mask between 0 and 1, used to guide feature alignment and recalibration). The output of the first Task Decomposition module, along with the outputs of the mask and offset generation module, is input into a deformable convolution (DCNV2) to obtain deformable features. The second Task... The output of the Decomposition module is multiplied by the output of the gated activation structure to obtain the multiplication result. The deformation features are used as inputs to the bounding box branches of the four detectors (i.e., the first detector, the second detector, the third detector, and the fourth detector), and the multiplication result is used as inputs to the classification branches of the four detectors.

[0058] It should be noted that, firstly, TADDH's original multi-scale detection structure is extended into a segmentation structure, adding a mask coefficient prediction branch at each level, so that it no longer only outputs detection boxes, but also outputs mask coefficients at the corresponding scale. Simultaneously, to generate globally shared base mask material, an independent proto branch is introduced, extracting a set of proto masks from the high-resolution feature map as a universal base representation of the shape of all objects across the entire image.

[0059] At each scale, in addition to the masking branch, the original classification and regression branches are retained, used for instance filtering and precise localization, respectively. The classification branch undergoes supervised training by calculating the classification loss (Lcls), providing class confidence for NMS (Non-maximum Suppression) during the inference phase. The regression branch calculates the bounding box regression loss (Lbbox), improving target localization accuracy in a strongly supervised manner and helping to eliminate background and redundant candidate boxes. The masking branch is responsible for learning a unique set of linear combination coefficients for each potential target. These coefficients are linearly combined (i.e., matrix multiplication) with the global prototype mask generated by the Proto Branch during the inference phase to reconstruct a high-quality instance segmentation mask for that target.

[0060] In layman's terms, the four detection heads are responsible for detecting targets across the entire scale, from extremely small objects to very large objects. They do not directly output image segmentation masks, but rather output mask coefficients corresponding to their respective levels. The prototype mask generated by the prototype branch can be regarded as a basic "palette" for characterizing the shape of the object. During the inference phase, the outputs of all detection heads are concatenated in dimensions, and then the final valid targets are selected through NMS. For each retained target, the model uses its specific mask coefficients to linearly combine with the global prototype mask (i.e., matrix multiplication) to dynamically synthesize its accurate instance segmentation result.

[0061] Meanwhile, to further reduce computational cost, we replaced some of the standard convolutions in the detection head with depthwise separable convolutions, namely depthwise convolutions and pointwise convolutions shown in the figure, which significantly reduced the number of parameters and computational cost while maintaining feature representation capabilities.

[0062] Compared to the original head network, the first head network not only significantly reduces the number of model parameters but also further improves segmentation accuracy. This indicates that improvements in the detection head mask branch and prototype branch can more effectively utilize multi-scale features, thereby achieving more efficient and accurate instance segmentation.

[0063] Training and improving the YOLOv11 model:

[0064] A high-quality dataset was collected by drones on multiple highway sections and manually constructed. This dataset contains 248 images and five target classes, such as thin "solid lines" and complex "guide lines," and covers different lighting conditions, traffic densities, and shooting angles. This provides a solid data foundation for verifying the robustness and accuracy of the algorithm when handling small and irregular targets with drastic scale changes and complex backgrounds in the real world.

[0065] The training parameters are configured as follows: initial learning rate is 0.01, cosine annealing parameter is 0.01, image size is 640, optimizer is SGD, batch size is 8, and training iterations are 1500.

[0066] Finally, the training set is fed into the improved YOLOv11 model for training, resulting in a trained improved YOLOv11 model.

[0067] This method for high-speed scene segmentation in UAV aerial photography, based on an improved YOLOv11 model, enhances the YOLOv11 model by modifying one branch of the C2DA module to preserve the original features and retain underlying semantic information, while the other branch adaptively focuses on key spatial regions through the DAttention mechanism to extract salient features. This significantly improves the model's feature perception and target segmentation performance in complex scenes. The scale feature map output from the first stage of the first backbone network is also input into the first neck network. Because the scale feature map output from the first stage has a higher resolution, it can more effectively capture the fine structure of small targets. Using this higher-resolution feature map, the network can preserve and utilize more detailed spatial texture and boundary information. This allows the model to not only more accurately detect small targets, but more importantly, to depict their pixel-level contours with higher precision, thus significantly improving the segmentation quality and boundary clarity of small targets. Furthermore, the first head network not only significantly reduces the number of model parameters but also further improves segmentation accuracy.

[0068] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0069] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for high-speed scene element segmentation in UAV aerial photography based on an improved YOLOv11, characterized in that: The improved YOLOv11-based method for high-speed scene segmentation in UAV aerial photography includes: The drone aerial image to be segmented is obtained, input into the trained improved YOLOv11 model, and the segmentation result of high-speed scene elements in the drone aerial image is output. The improved YOLOv11 model includes a first backbone network, a first neck network, and a first head network. The UAV aerial image to be segmented is used as the input to the first backbone network. The four stages of the first backbone network output feature maps of different scales. All four feature maps of different scales are input to the first neck network, which performs cross-layer fusion enhancement to obtain four enhanced feature maps. All four enhanced feature maps are used as the input to the first head network to obtain four detection results corresponding one-to-one with the four feature maps of different scales. The first head network also outputs a prototype mask. Each detection result includes the target's classification category, bounding box, and mask coefficients. The detection results of the four scales are concatenated in dimension, and non-maximum suppression is performed on all bounding boxes to obtain a unique final bounding box for each target. For each final bounding box, the mask coefficients corresponding to the final bounding box are linearly combined with the prototype mask to generate an instance segmentation mask for the corresponding target, thereby obtaining the segmentation result of high-speed scene elements in the UAV aerial image.

2. The method for high-speed scene element segmentation in UAV aerial photography based on improved YOLOv11 as described in claim 1, characterized in that: The first backbone network is obtained by replacing the C2PSA module in the fourth stage of the backbone network in the YOLOv11 model with the C2DA module. The C2DA module includes a first convolutional layer, a first segmentation operation, two cascaded DAttention modules, and a second convolutional layer connected in sequence. The first segmentation operation evenly divides the number of channels of the output of the first convolutional layer into two parts. One part is used as the input of the two cascaded DAttention modules, and the other part is concatenated with the outputs of the two cascaded DAttention modules to obtain a first concatenation result. The first concatenation result is then used as the input of the second convolutional layer, and the output of the second convolutional layer is used as the output of the C2DA module.

3. The method for high-speed scene element segmentation in UAV aerial photography based on improved YOLOv11 as described in claim 1, characterized in that: The DAttention module includes a convolutional structure, a first module, a convolutional layer, a second module, a convolutional layer, a third module, a convolutional layer, and a fourth module connected in sequence. The first and second modules each include local attention and shift window attention connected in sequence. The third and fourth modules each include local attention and deformable attention connected in sequence. The convolutional structure includes two convolutional layers connected in series.

4. The method for high-speed scene element segmentation in UAV aerial photography based on improved YOLOv11 as described in claim 1, characterized in that: In the first neck network, the feature maps of different scales output from the four stages of the first backbone network are sequentially passed through the third, fourth, fifth, and sixth convolutional layers. The output of the sixth convolutional layer, after upsampling, is fused with the output of the fifth convolutional layer to obtain the first fused feature. This first fused feature then passes through the first C3k2_CFBlock module and the upsampling module, and is fused with the output of the fourth convolutional layer to obtain the second fused feature. The second fused feature then passes through the second C3k2_CFBlock module and the upsampling module, and is fused with the output of the third convolutional layer to obtain the third fused feature. This third fused feature then passes through the third C3k2_CFBlock module and the convolutional layer, and is fused with the outputs of the fourth and third convolutional layers to obtain the final fused feature. The fourth fusion feature is then passed through the fourth C3k2_CFBlock module and a convolutional layer, and then fused with the output of the fifth convolutional layer and the output of the first C3k2_CFBlock module to obtain the fifth fusion feature. The fifth fusion feature is then passed through the fifth C3k2_CFBlock module and a convolutional layer, and then fused with the output of the sixth convolutional layer to obtain the sixth fusion feature. The sixth fusion feature is then passed through the sixth C3k2_CFBlock module, and the outputs of the third, fourth, fifth, and sixth C3k2_CFBlock modules are the first enhanced feature map, the second enhanced feature map, the third enhanced feature map, and the fourth enhanced feature map, respectively. All four enhanced feature maps are used as inputs to the first head network. The feature fusion operation is a weighted fusion.

5. The method for high-speed scene element segmentation in UAV aerial photography based on improved YOLOv11 as described in claim 4, characterized in that: Each C3k2_CFBlock module includes a seventh convolutional layer, a second splitting operation, a branching structure, and an eighth convolutional layer connected in sequence. The branching structure includes multiple cascaded C3k_CFBlock modules or multiple cascaded CFBlock modules. When the parameter C3k of the C3k2_CFBlock module is true, the branching structure includes multiple cascaded C3k_CFBlock modules. When the parameter C3k of the C3k2_CFBlock module is false, the branching structure includes multiple cascaded CFBlock modules. The second splitting operation evenly divides the number of channels of the output of the seventh convolutional layer into two parts. One part is used as the input of the branching structure, and the other part is concatenated with the output of the branching structure to obtain a second concatenation result. The second concatenation result is then used as the input of the eighth convolutional layer to obtain the output of the C3k2_CFBlock module.

6. The method for high-speed scene element segmentation in UAV aerial photography based on improved YOLOv11 as described in claim 5, characterized in that: The C3k_CFBlock module includes a ninth convolutional layer, a third splitting operation, multiple cascaded CFBlock modules, and a tenth convolutional layer connected in sequence. The third splitting operation evenly divides the number of channels of the output of the ninth convolutional layer into two parts. One part is used as the input of the multiple cascaded CFBlock modules, and the other part is concatenated with the output of the multiple cascaded CFBlock modules to obtain a third concatenation result. The third concatenation result is then used as the input of the tenth convolutional layer, and the output of the tenth convolutional layer is used as the output of the C3k_CFBlock module.

7. The method for high-speed scene element segmentation in UAV aerial photography based on improved YOLOv11 as described in claim 6, characterized in that: The CFBlock module includes a Convolutional Attention module, a first normalization layer, a feedforward neural network, and a second normalization layer connected in sequence. The input and output of the Convolutional Attention module are added together, and the result is used as the input of the first normalization layer. The output of the first normalization layer is added together with the output of the feedforward neural network, and the result is used as the input of the second normalization layer. The output of the second normalization layer is used as the output of the CFBlock module.

8. The method for high-speed scene element segmentation in UAV aerial photography based on improved YOLOv11 as described in claim 7, characterized in that: In the Convolutional Attention module, the input is linearly transformed to obtain a first key vector and a second key vector. The input is then convolved with both the first and second key vectors. The results of these two convolutions are then grouped and double-normalized. Finally, the results of these two double-normalized operations are convolved with the transposes of the corresponding first and second key vectors to obtain a first efficient feature and a second efficient feature. The first and second efficient features are then added together to obtain the output of the Convolutional Attention module. The formulas for obtaining each efficient feature are as follows: ； ； in, and These are the first and second high-efficiency features, respectively. This serves as the input to the ConvolutionalAttention module. The first key vector, This is the second key vector. For the channel, For height, For width, The number of learnable parameters. The kernel size for learnable parameters, For transpose, For convolution operations, This is a grouped double normalization operation, and the grouped double normalization operation is in × The Softmax function is executed on the dimension of , in Perform grouped L2 norm on the dimension, that is, on The dimension is divided into a preset number of groups, and the L2 norm is applied to each group.

9. The method for high-speed scene element segmentation in UAV aerial photography based on improved YOLOv11 as described in claim 4, characterized in that: The first head network is an improvement on the TADDH module. The first head network includes a prototype branch, a first detection head, a second detection head, a third detection head, a fourth detection head, and a shared module. Each detection head includes a mask branch, a bounding box branch, and a classification branch. The prototype branch includes multiple concatenated convolutional layers. The mask branch includes a shared convolution, a depthwise convolution, a pointwise convolution, and a mask coefficient convolution connected in sequence. The first enhanced feature map is passed through the prototype branch to obtain a prototype mask. The shared module, the bounding box branches of each detection head, and the classification branches constitute the TADDH module. The outputs of the mask branch, bounding box branch, and classification branch are the mask coefficient, bounding box, and classification category, respectively. Among them, the first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map correspond one-to-one with the first detection head, the second detection head, the third detection head and the fourth detection head, respectively. After passing through the shared module, each enhanced feature map passes in parallel through the bounding box branch and the classification branch of the corresponding detection head. Each enhanced feature map also passes through the mask branch of the corresponding detection head. The outputs of the mask branch, bounding box branch and classification branch of each detection head constitute the target detection result corresponding to the corresponding enhanced feature map.

Images