Lightweight multi-task small object detection algorithm with adaptive pyramid and multi-stage path aggregation

Abstract

The application relates to the technical field of computer vision, and particularly discloses a lightweight multi-task small target detection algorithm based on adaptive pyramid and multi-stage path aggregation, which comprises an adaptive feature pyramid network, a multi-stage path aggregation module, a lightweight Transformer module, an optimized channel attention mechanism, a multi-task learning head and an algorithm training method. The lightweight multi-task small target detection algorithm based on adaptive pyramid and multi-stage path aggregation is adopted, the dynamic weight adjustment of different hierarchical features is realized by combining an Adaptive FPN, and the expression capability of multi-scale features is enhanced. The MPAM module introduces a lightweight Transformer module on the basis of the original module, utilizes an axial attention (Axia Attention) mechanism to model global features, improves the perception capability of the model for small targets, and optimizes the channel attention mechanism to reduce the calculation cost.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a lightweight multi-task small object detection algorithm based on adaptive pyramid and multi-stage path aggregation. Background Technology

[0002] With the rapid development of computer vision technology, deep learning-based object detection has made significant progress in various application areas such as intelligent surveillance, remote sensing image analysis, and medical image processing. However, small object detection remains a challenging task, mainly because small objects occupy a limited number of pixels in an image and are unevenly distributed, easily leading to high false alarm and false negative rates in existing algorithms. Therefore, researchers are constantly exploring new model architectures and algorithm optimization methods to improve the accuracy and efficiency of small object detection.

[0003] Existing Feature Pyramid Networks (FPNs) improve object detection performance through multi-scale feature fusion, but they still suffer from the problem of non-adaptive feature fusion weights when dealing with small targets. Furthermore, while the Transformer module excels in global feature modeling, its computational cost is high, hindering real-time detection requirements. Multi-task learning improves the model's generalization ability by sharing feature extraction modules, but current methods lack robust feature sharing mechanisms between tasks, making it difficult to fully utilize multi-scale feature information. Summary of the Invention

[0004] The purpose of this invention is to provide a lightweight multi-task small object detection algorithm and its training method based on adaptive pyramid and multi-stage path aggregation, aiming to solve several key problems existing in the current small object detection technology, including the lack of adaptability of feature fusion weights, insufficient small object feature modeling ability, high computational complexity, and insufficient feature sharing in multi-task learning.

[0005] To achieve the above objectives, this invention provides a lightweight multi-task small target detection algorithm based on adaptive pyramid and multi-stage path aggregation, including an Adaptive Feature Pyramid Network (AdaptiveFPN), a Multi-stage Path Aggregation Module (MPAM), a lightweight Transformer module, an optimized channel attention mechanism, a multi-task learning head, and an algorithm training method.

[0006] The adaptive feature pyramid network uses learnable weight parameters to automatically optimize the fusion ratio of features at different levels;

[0007] The Multi-Stage Path Aggregation Module (MPAM) is connected after the Adaptive Feature Pyramid Network. It achieves the fusion of multi-scale information through path aggregation in multiple stages. Each stage embeds a lightweight Transformer module for global feature modeling.

[0008] The optimized channel attention mechanism employs axial attention, performing attention calculations in both the height and width directions.

[0009] The multi-task learning head includes an object detection head, a bounding box regression head, and a semantic segmentation head. While performing object detection, it incorporates bounding box regression and semantic segmentation tasks, sharing the multi-scale features extracted by MPAM.

[0010] Preferably, the architecture of the adaptive feature pyramid network includes a low-level feature extraction layer, a feature fusion layer, and a multi-scale feature output layer. The low-level feature extraction uses the backbone network to extract multi-level feature maps, including low-level high-resolution features and high-level low-resolution features.

[0011] In the feature fusion layer, learnable weight parameters are introduced between each feature layer. Features from different layers are fused through weighted summation. For each feature layer, AdaptiveFPN uses a small fully connected network to generate fusion weights. These weights are dynamically adjusted based on the characteristics of the input features. The feature fusion process is represented as follows:

[0012]

[0013] Where F1, F2, ..., F n Feature maps extracted from the backbone network, F′ i w represents the feature map of the i-th layer. i，j The learnable weight parameters are dynamically generated through a small fully connected network, satisfying...

[0014] The multi-scale feature output is an adaptively fused feature map, which is then passed to the MPAM module as input for the multi-scale features.

[0015] The adaptive feature pyramid network adaptively optimizes the fusion weights of features at different levels through a dynamic weight adjustment mechanism combined with a self-supervised learning method.

[0016] Preferably, the MPAM module mainly consists of multi-stage path aggregation, a lightweight Transformer module, feature merging, and feature fusion F in the multi-stage path aggregation part. agg Represented as:

[0017] F agg =Concat(F stage1 F stage2 F stagr3 );

[0018] Among them, F stage1 F stage2 F stage3These represent the feature maps after path aggregation at each stage;

[0019] Global feature modeling F of the lightweight Transformer module global Represented as:

[0020] F global =AxialAttention(F agg );

[0021] Final feature merging F final Represented as:

[0022] F final =Merge(F agg F global );

[0023] The MPAM module employs model compression methods in its design, which include pruning, quantization, and knowledge distillation.

[0024] Among them, pruning technology removes redundant neural network connections or channels, reducing model complexity; quantization technology converts model parameters and activation 32-bit floating-point numbers into a lower bit representation; knowledge distillation technology trains a lightweight student model to mimic the behavior of a larger and higher-performing teacher model.

[0025] The MPAM module is integrated into the pyramid residual feature mapping submodule PRFM and the dilated convolution path aggregation submodule DCPAM, and combines convolution kernels of different sizes and pooling operations. The PRFM module is a submodule used for multi-scale feature fusion; the DCPAM module uses dilated convolution to aggregate feature information from different scales.

[0026] The MPAM module employs a multi-branch structure and cross-layer feature fusion technology to aggregate multi-scale information from different levels.

[0027] Preferably, the lightweight Transformer module is embedded in the MPAM module and employs a decomposed axial attention mechanism, the axial attention expression of which is:

[0028]

[0029] Where Q, K, and V are the query, key, and value matrices, respectively, and d k For the dimension of the key, attention is calculated in the height axis to capture vertical contextual information; attention is calculated in the width axis to capture horizontal contextual information, and attention is calculated separately in the height and width directions.

[0030] Preferably, the multi-task learning head includes an object detection head, a bounding box regression head, and a semantic segmentation head, sharing multi-scale features extracted by MPAM. The object detection head is responsible for predicting the category and location of the object and outputting the detection results; the bounding box regression head accurately locates the bounding box of the object and optimizes the bounding box regression; and the semantic segmentation head performs pixel-level classification of the image.

[0031] The multi-task learning head employs an ensemble learning approach, combining multiple detection models optimized by MPAM with different structures or training strategies, while simultaneously performing weighted averaging or using a voting mechanism.

[0032] Preferably, the loss function of the multi-task learning head is composed of the loss functions of each task, which are weighted by weight coefficients to form a joint loss function. This joint loss function includes Focal Loss and GIoU Loss, and the expression of the joint loss function is as follows:

[0033]

[0034] in, Focal Loss is used for target detection loss. For bounding box regression loss, GIoU Loss is adopted. For semantic segmentation loss, the cross-entropy loss function is adopted, where λ and μ are weight coefficients;

[0035] The formula for calculating Focal Loss is as follows:

[0036] FL(p t )=-α t (1-p t )γlog(p t );

[0037] Where, p t To predict the probability, α t γ represents the category weights, and γ is the adjustment factor.

[0038] The formula for calculating GIoU Loss is as follows:

[0039]

[0040] Where C is the smallest closed matrix containing the predicted box A and the ground truth box B; |(A∩B)| represents the intersection area of ​​the predicted bounding box A and the ground truth bounding box B, |A∪B| represents the union area of ​​the predicted bounding box A and the ground truth bounding box B, IoU represents the degree of overlap between the predicted bounding box and the ground truth bounding box, which is a key indicator for measuring the accuracy of object detection; \ represents the area of ​​the region in the smallest closed rectangle that is not occupied by the predicted bounding box and the ground truth bounding box.

[0041] Preferably, the algorithm training method employs a cosine annealing learning rate scheduling strategy, data augmentation techniques, and mixed precision training techniques.

[0042] Preferably, the cosine annealing learning rate scheduling strategy optimizes the training process by combining the learning rate warm-up phase of the first three epochs. Here, an epoch refers to the process of the entire training dataset being processed and learned once in the neural network; cosine annealing involves the learning rate gradually decreasing according to a cosine curve as the number of training epochs increases; and the learning rate warm-up involves the learning rate linearly increasing to the initial learning rate within the first three epochs of training, with the learning rate lr(t) at any given time being:

[0043]

[0044] Where t is the current training round number, T is the total number of training rounds, and T warmup Here, lr is the number of rounds in the warm-up phase, lr0 is the initial learning rate, and lr... final This is the final learning rate.

[0045] Preferably, data augmentation techniques include random cropping, rotation, and scaling;

[0046] Among them, random cropping involves randomly cropping different regions of the image to ensure that the model can focus on all parts of the image, especially the location of small targets; rotation involves randomly rotating the image by a certain angle; and scaling involves randomly scaling the image size to simulate target detection scenarios at different resolutions.

[0047] Preferably, a mixed-precision training technique is used, employing PyTorch's torch.cuda.amp to perform calculations using both 16-bit and 32-bit floating-point numbers.

[0048] Therefore, the present invention employs the aforementioned lightweight multi-task small object detection algorithm and training method based on adaptive pyramid and multi-stage path aggregation, with the following beneficial effects:

[0049] (1) This invention combines Adaptive FPN to achieve dynamic fusion weight adjustment of features at different levels, thereby enhancing the expressive power of multi-scale features.

[0050] (2) In this invention, the MPAM module introduces a lightweight Transformer module on the basis of the original, uses the axial attention mechanism to model global features, improves the model’s ability to perceive small targets, and optimizes the channel attention mechanism to reduce computational overhead.

[0051] (3) The present invention adopts a multi-task learning strategy, simultaneously performing object detection, bounding box regression and semantic segmentation, fully sharing multi-scale features and improving overall detection performance.

[0052] (4) This invention combines a cosine annealing learning rate scheduling strategy with a warm-up phase, applies data augmentation techniques such as random pruning, rotation and scaling, and adopts mixed precision training techniques to further improve training efficiency and model performance. Attached Figure Description

[0053] Figure 1 This is an overall model architecture diagram of an embodiment of the lightweight multi-task small target detection algorithm and its training method based on adaptive pyramid and multi-stage path aggregation of the present invention;

[0054] Figure 2 This is a detailed structural diagram of the MPAM module in an embodiment of the lightweight multi-task small target detection algorithm and training method of the present invention, which combines adaptive pyramid and multi-stage path aggregation.

[0055] Figure 3 This is the learning rate scheduling curve during the training process of an embodiment of the lightweight multi-task small target detection algorithm and training method of the present invention, which combines adaptive pyramid and multi-stage path aggregation. Detailed Implementation

[0056] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0057] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0058] Example

[0059] like Figure 1 As shown, a lightweight multi-task small object detection algorithm and its training method based on adaptive pyramid and multi-stage path aggregation are presented. This model significantly improves the accuracy and efficiency of small object detection by dynamically adjusting the fusion weights of features at different levels, introducing a lightweight Transformer module for global feature modeling, adopting an optimized axial attention mechanism, and achieving feature sharing for multi-task learning (object detection, bounding box regression, and semantic segmentation).

[0060] like Figure 2 As shown, the detailed structure of the Multi-stage Path Aggregation Module (MPAM) includes the implementation details of Multi-stage Path Aggregation and Axial Attention. Figure 2 This intuitively describes the data flow within the MPAM and the interactions between its components. The processing flow based on the Multi-Stage Path Aggregation Module (MPAM) is as follows:

[0061] S1. Input Feature Maps: These are multi-scale feature maps from the Adaptive Feature Pyramid Network (APN), typically consisting of feature layers at different resolutions (e.g., low-level high-resolution and high-level low-resolution feature maps). The input multi-stage path aggregation module contains multiple parallel paths, each employing convolutional kernels of different sizes (e.g., 3×3, 5×5) and pooling operations to extract multi-scale features. This step indicates that the input multi-scale feature maps enter the multi-stage path aggregation module for processing and fusion.

[0062] S2 is similar to S1, but may employ different kernel sizes or pooling strategies to further enrich feature representation.

[0063] S3. Further extract higher-level multi-scale features to enhance the diversity and expressive power of the feature map. The outputs of each path point to the Feature Fusion Node. Multi-scale features from each stage are fused through concatenation or weighted summation to form a comprehensive multi-scale feature representation. This step indicates that multi-scale features extracted from different stage paths are integrated and fused via arrows pointing to the Feature Fusion Node.

[0064] S4. The fused feature map is output to the convolutional layer of the Lightweight Transformer Module. Utilizing the decomposed axial attention mechanism, attention is calculated in both the height and width directions to achieve global feature modeling. This step indicates that the fused feature map enters the Lightweight Transformer Module for global feature modeling.

[0065] S5. Output the feature map to the query (Q), key (K), and value (V) convolutional layers. Use a 1×1 convolutional layer to convert the fused feature map into a query (Q), key (K), and value (V) matrix.

[0066] S6. Point the Q, K, and V convolutional layers to the height-axis attention calculation module to calculate attention along the height axis and capture contextual information in the vertical direction. Point the Q, K, and V convolutional layers to the width-axis attention calculation module to calculate attention along the width axis and capture contextual information in the horizontal direction. This step means that after the Q, K, and V matrices are generated by the convolutional layers, they are respectively fed into the height and width-axis attention calculation modules.

[0067] S7. Feature Reconstruction is performed based on the results of the height and width attention calculation modules. The attention calculation results in the height and width directions are superimposed to reconstruct the enhanced feature map. The global feature map processed by the axial attention mechanism further enhances the expressive power of the features. This step indicates that the feature maps after height and width attention calculations are reconstructed and superimposed.

[0068] S8. Input the feature map to the Transformer module. From the feature reconstruction node to the output feature map, this step represents the output of the feature map after processing by the axial attention mechanism, which is used for feature merging.

[0069] S9. Input the multi-stage path aggregation module and the lightweight Transformer module into the feature merging node. This step means that the global feature map output by the Transformer module is fused with the feature map output by the multi-stage path aggregation module.

[0070] S10. From the feature merging node to the multi-task learning heads, including the detection head, bounding box regression head, and semantic segmentation head, feature maps are fused through addition or concatenation operations to form the final high-quality feature representation. The fused final enhanced feature map is used for subsequent object detection, bounding box regression, and semantic segmentation tasks. This step indicates that the final fused high-quality feature map is passed to the multi-task learning heads to complete tasks such as object detection, bounding box regression, and semantic segmentation.

[0071] The Adaptive Feature Pyramid Network (FPN) dynamically adjusts the fusion weights of features at different levels, enhancing the expressive power of multi-scale features. It adaptively optimizes the fusion ratio of features at different levels through learnable weight parameters. Multi-stage path aggregation achieves deep fusion of multi-scale information; each stage contains multiple parallel paths, employing convolutional kernels of different sizes and pooling operations to extract rich multi-scale features. A lightweight Transformer Module is used for global feature modeling, improving the model's ability to perceive small objects. An axial attention mechanism is embedded to decompose attention computation in the height and width directions, reducing computational complexity. While maintaining global feature modeling capabilities, the axial attention mechanism significantly reduces computational overhead. A query, key, and value matrix is ​​generated through 1×1 convolutions; attention is calculated along the height axis to capture vertical contextual information; attention is calculated along the width axis to capture horizontal contextual information; and the attention results in the height and width directions are superimposed to reconstruct the enhanced feature map. The feature merge node fuses feature maps from multi-stage path aggregation and the Transformer module to form a final high-quality feature representation. Addition or concatenation operations are used to fuse feature maps from two sources. Multi-task learning heads simultaneously perform object detection, bounding box regression, and semantic segmentation tasks. Implementation details: Each task has its own independent branch, using shared multi-scale features extracted by MPAM to produce task-specific outputs.

[0072] By combining an adaptive feature pyramid network and an axial attention mechanism, the MPAM module effectively improves the model's ability to detect small targets while maintaining its computational efficiency advantage. Figure 2 The data processing within the module is demonstrated, clearly showing the core role and innovative design of the Multi-Stage Path Aggregation Module (MPAM) in this invention, and highlighting the technical advantages of this invention in multi-scale feature fusion, global feature modeling, and multi-task learning.

[0073] 1. Adaptive FPN (Adaptive Pyramid Network)

[0074] The Adaptive Feature Pyramid Network (APN) aims to enhance the expressive power of multi-scale features by dynamically adjusting the fusion weights of features at different levels. Traditional FPNs fuse multi-scale features using a fixed topology, failing to adaptively adjust the feature fusion strategy according to the specific task and dataset requirements. Adaptive FPN introduces learnable fusion weights, enabling the model to adaptively optimize the fusion ratio of features at different levels based on the characteristics of the input data, thereby improving the expressive effect of multi-scale features, especially when dealing with small objects.

[0075] The architecture of Adaptive FPN is based on traditional FPN and mainly consists of the following parts:

[0076] Low-level feature extraction: The backbone network is used to extract multi-level feature maps, which typically include low-level high-resolution features and high-level low-resolution features.

[0077] Feature fusion layer: Learnable weight parameters are introduced between each feature layer, and features from different layers are fused through weighted summation. Specifically, for each feature layer, Adaptive FPN uses a small fully connected network (such as a 1×1 convolution) to generate fusion weights, which are dynamically adjusted according to the characteristics of the input features.

[0078] Multi-scale feature output: The feature map after adaptive fusion is passed to the subsequent MPAM module as input for multi-scale features.

[0079] The feature fusion process of Adaptive FPN can be represented as:

[0080]

[0081] Where F1, F2, ..., F n Feature maps extracted from the backbone network, F′ i w represents the feature map of the i-th layer. i，j The learnable weight parameters are dynamically generated through a small fully connected network, satisfying...

[0082] 2. Multi-stage path aggregation module (MPAM)

[0083] The Multi-Stage Path Aggregation Module (MPAM) enhances the expressive power of feature maps through multi-level and multi-scale information fusion. In this invention, MPAM further introduces a lightweight Transformer module for global feature modeling, improving the model's ability to perceive small targets. Furthermore, MPAM optimizes cross-layer feature fusion and global modeling capabilities through multi-stage path aggregation and axial attention mechanisms.

[0084] The MPAM module mainly consists of multi-stage path aggregation, a lightweight transformer module, and feature merging.

[0085] Multi-stage path aggregation:

[0086] (1) Stage 1, Stage 2, Stage 3: Each stage contains multiple parallel paths, and each path extracts multi-scale features through convolution kernels of different sizes and pooling operations.

[0087] (2) Feature fusion: The outputs of each path are fused by splicing or weighted summation to form a comprehensive expression of multi-scale features.

[0088] The MPAM module is integrated within the pyramid residual feature mapping submodule PRFM and the dilated convolution path aggregation submodule DCPAM, and incorporates convolution kernels of different sizes and pooling operations.

[0089] The PRFM module is a submodule for multi-scale feature fusion, particularly used in pyramid networks (such as FPN). PRFM works by using a pyramid structure to process feature maps of different resolutions, then fusing these feature maps at different scales using residual connections. Finally, it optimizes the flow of information across scales, reducing information loss and enhancing feature representation. Therefore, the main task of this submodule is to process features from different scales, enhance the flow of multi-scale information through residual connections, prevent information loss, and improve the model's stability and convergence speed.

[0090] The DCPAM module first uses dilated convolution to allow the convolutional kernel to sample features in a "jumping" manner within the original image, thereby expanding the receptive field. Then, through a multi-scale aggregation strategy, it combines features from different resolutions to further improve the model's ability to detect targets at multiple scales, enhancing the diversity of the feature maps and enabling the model to better handle small targets of varying sizes and against complex backgrounds. Therefore, the design of the DCPAM submodule may utilize dilated convolution to aggregate feature information from different scales and enhance the model's ability to perceive small targets.

[0091] MPAM module pseudocode definition:

[0092]

[0093]

[0094] 3. Lightweight transformer module:

[0095] (1) Axial Attention Mechanism: A decomposed axial attention mechanism is adopted, which performs attention calculations in the height and width directions respectively, significantly reducing computational complexity.

[0096] (2) Global feature modeling: The lightweight transformer module is used to perform global modeling of the fused multi-scale features, capture the long-distance context information in the image, and enhance the perception of small targets.

[0097] (3) Feature Merge:

[0098] The feature maps, after multi-stage path aggregation and global feature modeling, are further fused to form the final high-quality feature representation for use by the multi-task learning head.

[0099] Feature fusion F in multi-stage path aggregation agg It can be represented as:

[0100] F agg =Concat(F stage1 ,F stage2 ,F stage3 );

[0101] Among them, F stage1 F stage2 F stage3 These represent the feature maps after path aggregation at each stage. The global feature modeling F of the lightweight transformer module... global It can be represented as:

[0102] F global =AxialAttention(F agg );

[0103] Final feature merging F final Represented as:

[0104] F final =Merge(F agg ,F global ).

[0105] pseudocode for defining a lightweight Transformer module:

[0106] classdef LightweightTransformer <handle

[0107] properties

[0108] axial Attention

[0109] end

[0110] methods

[0111] function obj=LightweightTransformer(channels,heads)

[0112] obj.axialAttention=AxialAttention(channels,heads);

[0113] end

[0114] function out = forward(obj, x)

[0115] out=obj.axialAttention(x);

[0116] end

[0117] end

[0118] end

[0119] 4. Optimized Axial Attention Mechanism

[0120] Channel attention mechanisms aim to enhance the expressive power of important features by assigning different weights to different channels. Axial attention, on the other hand, significantly reduces computational overhead and improves overall model performance while maintaining global feature modeling capabilities.

[0121] The Axial Attention mechanism decomposes global attention into axial attention, performing attention calculations in the height and width directions respectively, thereby reducing computational complexity. The specific design is as follows:

[0122] Query, key, and value computation: The input feature map is converted into queries, keys, and values ​​using a 1×1 convolutional layer.

[0123] Axial attention calculation: Attention is calculated along the height axis to capture vertical contextual information; attention is calculated along the width axis to capture horizontal contextual information. The results of the attention calculations in both directions are superimposed to obtain the final attention-enhanced feature map. Let the input feature map be X∈R. H×W×C The calculation process is as follows:

[0124] (1) Query, key-value calculation:

[0125] Q=Conv1×1(X), K=Conv1×1(X), V=Conv1×1(X);

[0126] (2) Calculation of attention in the height direction:

[0127]

[0128] (3) Attention calculation in the width direction:

[0129]

[0130] (4) Feature reconstruction:

[0131] X out =Attention H (Q, K, V) + Attention W (Q, K, V).

[0132] Define the pseudocode for the Axial Attention module:

[0133]

[0134]

[0135] 5. Multi-task Learning

[0136] Multi-task learning improves a model's generalization ability and robustness by simultaneously optimizing multiple related tasks. In this invention, in addition to object detection, bounding box regression and semantic segmentation tasks are introduced, sharing multi-scale features extracted by MPAM to enhance overall detection performance. Specifically, multi-task learning promotes information sharing between different tasks, enhancing the expressive power of features. Especially when dealing with small targets, sharing features allows for better capture of fine-grained feature information.

[0137] The multi-task learning component includes the following parts:

[0138] Shared feature extraction: The multi-scale features extracted by the MPAM module are shared for multiple tasks, including object detection, bounding box regression, and semantic segmentation.

[0139] Mission-specific branch:

[0140] Detection Head: Responsible for predicting the category and location of the target and outputting the detection results.

[0141] Bounding Box Regression Head: Accurately locates the bounding box of the target and optimizes the regression of the bounding box.

[0142] Semantic Segmentation Head: Performs pixel-level classification on images, improving the model's fine-grained understanding of target regions.

[0143] Let the shared feature be F. final The output of each task can be represented as:

[0144] Target detection output: Detection = DetectionHead(F final );

[0145] Bounding box regression output: BBoxRegression = BBoxRegressionHead(F final );

[0146] Semantic segmentation output: Segmentation = SegmentationHead(F final ).

[0147] The loss function for multi-task learning is composed of the loss functions of each task, which are weighted by weight coefficients to form a joint loss function. This joint loss function includes Focal Loss and GIoU Loss, and its expression is as follows:

[0148]

[0149] in, Focal Loss is used for target detection loss. For bounding box regression loss, GIoU Loss is adopted. For semantic segmentation loss, the cross-entropy loss function is adopted, with λ and μ as weight coefficients to balance the influence of various losses;

[0150] The formula for calculating Focal Loss is as follows:

[0151] FL(p t)=-α t (1-p t )γlog(p t );

[0152] Where, p t To predict the probability, α t γ represents the category weights, and γ is the adjustment factor.

[0153] The formula for calculating GIoU Loss is as follows:

[0154]

[0155] Where C is the smallest closed matrix containing the predicted box A and the ground truth box B; |(A∩B)| represents the intersection area of ​​the predicted bounding box A and the ground truth bounding box B, |A∪B| represents the union area of ​​the predicted bounding box A and the ground truth bounding box B, and IoU represents the degree of overlap between the predicted bounding box and the ground truth bounding box, which is a key indicator for measuring the accuracy of object detection; \ represents the area of ​​the region in the minimum closed rectangle that is not occupied by the predicted bounding box and the ground truth bounding box. This helps GIoU to take into account the spatial relationship between the bounding boxes, thereby providing more accurate loss function optimization.

[0156] The multi-task learning head is trained using a joint loss function to optimize class balance and bounding box regression, thereby improving the accuracy and recall of small object detection.

[0157] Define the pseudocode for the multi-task learning module:

[0158]

[0159]

[0160] Pseudocode for defining the joint loss function:

[0161]

[0162]

[0163] 6. Algorithm Training Methods

[0164] To fully leverage the model's performance, this invention employs a series of optimized training methods, including learning rate scheduling strategies, data augmentation techniques, mixed precision training, and model compression methods.

[0165] A cosine annealing learning scheduling strategy is employed, combined with a learning rate warm-up phase in the first three epochs, to optimize the training process and improve the final model performance. An epoch refers to the process of the entire training dataset being processed and learned once in the neural network; that is, in one epoch, the model performs one forward and one backward propagation on all training samples, and then updates the model parameters based on the feedback from the loss function. Therefore, the number of epochs represents the number of times the dataset is reused during training. Cosine annealing involves gradually decreasing the learning rate according to a cosine curve as the training epochs progress, which helps the model achieve better convergence in later stages. The learning rate warm-up phase, occurring in the first three epochs, means that in the initial stage of training, the learning rate is gradually increased from a small value until it reaches the set initial learning rate. This avoids excessively large learning rates in the early stages of training, which could lead to overly drastic gradient updates and negatively impact model convergence. The cosine annealing learning rate scheduling strategy means that the learning rate will gradually decrease in the later stages of training to help the model find a more accurate optimal solution. The learning rate lr(t) at any given time is:

[0166]

[0167] Where t is the current training round number, T is the total number of training rounds, and T warmup The number of rounds in the warm-up phase is T in this embodiment of the invention. warmup =3, lr0 is the initial learning rate, lr final This is the final learning rate.

[0168] Data augmentation techniques include random cropping, rotation, and scaling. Random cropping involves randomly cropping different regions of an image to ensure the model can focus on all parts of the image, especially the location of small targets. Rotation involves randomly rotating the image by a certain angle to enhance the model's robustness to changes in target rotation. Scaling involves randomly scaling the image size to simulate target detection scenarios at different resolutions. Data augmentation techniques are particularly effective for small targets, enhancing the diversity of training data and improving the model's generalization ability.

[0169] Mixed-precision training technology uses both 16-bit and 32-bit floating-point numbers for computation, improving training efficiency and reducing memory usage, making it suitable for training large models and high-resolution images.

[0170] Model compression methods include pruning, quantization, and knowledge distillation, which optimize the deployment efficiency of models on edge devices and are suitable for application scenarios with real-time detection requirements.

[0171] Among them, pruning technology reduces model complexity by removing redundant neural network connections or channels, adapting to the deployment needs of resource-constrained edge devices; quantization technology converts model parameters and activations from 32-bit floating-point numbers to a lower bit representation, reducing the model's storage requirements and computational overhead; knowledge distillation technology trains a lightweight student model to mimic the behavior of a larger and higher-performing teacher model, maintaining model performance while reducing model complexity.

[0172] Training results

[0173] Experimental results on the VisDrone2019 and MS COCO2017 datasets demonstrate that the proposed model exhibits significant performance improvements in small object detection tasks.

[0174] VisDrone 2019

[0175] The mAP50 improved by 3.5%, from 42.3% to 45.8%.

[0176] The accuracy of small object detection (APS) improved by 2.9%, from 14.8% to 17.7%.

[0177] Inference speed remains at real-time detection standards (>30 FPS), suitable for edge device deployment.

[0178] MS COCO2017

[0179] The mAP improved by 3.2%, from 0.646 to 0.678.

[0180] The accuracy of small target detection improved by 3.0%, from 0.288 to 0.318.

[0181] The number of model parameters increased by only about 15%, which improved detection performance while maintaining low computational overhead.

[0182] These results demonstrate that by introducing Adaptive FPN, an optimized Axial Attention mechanism, multi-task learning, and a series of training method optimizations, the model of this invention has significant advantages in small object detection tasks and is suitable for various practical application scenarios.

[0183] like Figure 3As shown, the parameter settings are: total_epochs: total number of training epochs (default is 50). armup_epochs: number of warm-up epochs (default is 3). initial_lr: initial learning rate (default is 1e-4). final_lr: final learning rate (default is 1e-6). Learning rate calculation: Warm-up phase (first 3 epochs): the learning rate increases linearly from 0 to initial_lr. Cosine annealing phase (subsequent epochs): the learning rate gradually decreases according to a cosine curve, from initial_lr to final_lr. Comparison curve: the red dashed line represents the learning rate change when only the cosine annealing strategy is used. Figure 3 This demonstrates intuitively the advantages of a learning rate scheduling strategy that combines cosine annealing with the warm-up phase during training. By linearly increasing the learning rate during the warm-up phase, the model can quickly adapt in the early stages of training, avoiding instability caused by an excessively high learning rate. Subsequently, the cosine annealing strategy gradually decreases the learning rate, helping the model achieve more stable convergence in later stages and improving the final detection performance.

[0184] Therefore, this invention employs the aforementioned lightweight multi-task small object detection algorithm and training method based on adaptive pyramid and multi-stage path aggregation. By combining an adaptive feature pyramid network and a multi-stage path aggregation module, the fusion weights of multi-scale features are dynamically adjusted. A lightweight Transformer module is introduced for global feature modeling, an optimized Axial Attention mechanism is used, and multi-task learning is achieved, effectively improving the accuracy and efficiency of small object detection. Furthermore, by combining a cosine annealing learning rate scheduling strategy, data augmentation techniques, mixed-precision training, and model compression methods, the training process and deployment efficiency of the model are further optimized. Experimental results on multiple public datasets verify the superiority of this invention, demonstrating broad application prospects and commercial value.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A lightweight multi-task small target detection algorithm of adaptive pyramid and multi-stage path aggregation, characterized in that, The adaptive feature pyramid network AdaptiveFPN, the multi-stage path aggregation module MPAM, the lightweight Transformer module, the optimized channel attention mechanism, the multi-task learning head and the algorithm training method are included. The adaptive feature pyramid network adopts a learnable weight parameter to automatically optimize the fusion ratio of different levels of features. The multi-stage path aggregation module MPAM is connected after the adaptive feature pyramid network, the multi-stage path aggregation includes multiple parallel paths, each path adopts a convolution kernel and a pooling operation of different sizes, and the fusion of multi-scale information is realized through multi-stage path aggregation, a lightweight Transformer module is embedded in parallel between the multi-stage path aggregation and the feature merging, and is used for global feature modeling. The optimized channel attention mechanism adopts axial attention, and attention calculation is performed in the height and width directions respectively. The multi-task learning head includes a target detection head, a bounding box regression head and a semantic segmentation head, the multi-task learning head performs target detection while adding the bounding box regression and semantic segmentation tasks, and shares the multi-scale features extracted by the MPAM. The target detection head is responsible for predicting the category and position of the target, and outputs the detection result; the bounding box regression head accurately locates the bounding box of the target and optimizes the regression of the bounding box; and the semantic segmentation head classifies the image at the pixel level. The multi-task learning head adopts an ensemble learning method, and combines multiple detection models optimized by the MPAM of different structures or different training strategies, and performs weighted averaging or uses a voting mechanism.

2. The lightweight multi-task small target detection algorithm of adaptive pyramid and multi-stage path aggregation according to claim 1, characterized in that, The adaptive feature pyramid architecture includes bottom feature extraction, feature fusion layer and multi-scale feature output, the bottom feature extraction extracts multi-level feature maps by using a backbone network, including low-level high-resolution features and high-level low-resolution features; In the feature fusion layer, a learnable weight parameter is introduced between each feature layer, different levels of features are fused by weighted summation, for each feature layer, the AdaptiveFPN uses a small fully connected network to generate fusion weights, these weights are dynamically adjusted according to the characteristics of the input features, and the feature fusion process is represented as: ； wherein, feature maps extracted for the backbone network, represent the feature maps of the layer, are learnable weight parameters, dynamically generated by a small fully connected network, satisfying =1; The multi-scale feature output is the adaptive fused feature map, and the multi-scale feature output is subsequently transmitted to the MPAM module as the input of the multi-scale feature; The adaptive feature pyramid network optimizes the fusion weights of different levels of features through a dynamic weight adjustment mechanism combined with a self-supervised learning method.

3. The lightweight multi-task small target detection algorithm of adaptive pyramid and multi-stage path aggregation according to claim 2, characterized in that, The MPAM module mainly comprises multi-stage path aggregation, a lightweight Transformer module, and feature merging, and the feature fusion of the multi-stage path aggregation part is represented as: ； wherein, , , respectively represent the feature maps after path aggregation of each stage. Global feature modeling of lightweight transformer module is represented as: ； Final feature merge is represented as: ； The MPAM module adopts a model compression method in the design, and the model compression method includes pruning, quantization and knowledge distillation; Among them, the pruning technology removes redundant neural network connections or channels to reduce the complexity of the model; the quantization technology converts the model parameters and activations from 32-bit floating-point numbers to lower-bit representations; the knowledge distillation technology trains a lightweight student model to imitate the behavior of a larger and superior teacher model; The MPAM module is in a pyramid residual feature mapping submodule PRFM and a dilated convolution path aggregation submodule DCPAM, and is combined with convolution kernels of different sizes and pooling operations, wherein the PRFM module is a submodule for multi-scale feature fusion; the DCPAM module aggregates feature information from different scales by using dilated convolution. The MPAM module adopts a multi-branch structure and a cross-layer feature fusion technology to aggregate multi-scale information of different levels.

4. The lightweight multi-task small target detection algorithm of adaptive pyramid and multi-stage path aggregation according to claim 3, characterized in that, The lightweight Transformer module is embedded in the MPAM module, and adopts a decomposed axial attention mechanism, and an axial attention expression is as follows: ； wherein, , , are query, key, value matrices respectively, is the dimension of the key, the attention is calculated in the height axis direction, and the context information in the vertical direction is captured; the attention is calculated in the width axis direction, and the context information in the horizontal direction is captured, and the attention is calculated in the height and width directions respectively.

5. The lightweight multi-task small target detection algorithm of adaptive pyramid and multi-stage path aggregation according to claim 4, characterized in that, The loss function of the multi-task learning head is composed of the loss functions of each task, is weighted by a weight coefficient, and forms a joint loss function, the joint loss function includes a Focal Loss and a GIoU Loss, and a joint loss function expression is as follows: ； wherein, is a target detection loss, Focal Loss is adopted, is a bounding box regression loss, GIoU Loss is adopted, is a semantic segmentation loss, a cross-entropy loss function is adopted, and is a weight coefficient; A Focal Loss calculation formula is as follows: ； wherein, is a predicted probability, is a class weight, is a tuning factor; A GIoU Loss calculation formula is as follows: ； Wherein, C is the minimum closed matrix containing the prediction frame A and the real frame B; IoU= , represents the intersection area of the prediction frame A and the real frame B, represents the union area of the prediction frame A and the real frame B, represents the overlap degree of the prediction frame and the real frame, which is a key indicator for measuring the accuracy of target detection; \ represents the area of the region in the minimum closed rectangle which is not occupied by the prediction frame and the real frame.

6. The lightweight multi-task small target detection algorithm of adaptive pyramid and multi-stage path aggregation according to claim 5, characterized in that: The algorithm training method adopts a cosine annealing learning rate scheduling strategy, a data enhancement technology and a mixed precision training technology.

7. The lightweight multi-task small target detection algorithm of adaptive pyramid and multi-stage path aggregation according to claim 6, characterized in that, The cosine annealing learning rate scheduling strategy combines the learning rate warm-up phase of the first three epochs to optimize the training process, wherein an epoch refers to the process of processing and learning the entire training dataset in a neural network once; cosine annealing is to gradually reduce the learning rate according to the cosine curve with the training round number in the training process; the learning rate warm-up is to linearly increase the learning rate to the initial learning rate within the first three epochs of training, and the learning rate at any time is : ； wherein, is the current training epoch, is the total number of training epochs, is the number of warm-up epochs, is the initial learning rate, is the final learning rate.

8. The lightweight multi-task small target detection algorithm of adaptive pyramid and multi-stage path aggregation according to claim 7, characterized in that: The data enhancement technology includes random cropping, rotation and scaling; The random cropping is random cropping of different regions of an image, ensures that the model can pay attention to each part of the image, especially a small target position; the rotation is random rotation of the image by a certain angle; and the scaling is random scaling of the image size, simulates a target detection scene under different resolutions. 9.The lightweight multi-task small object detection algorithm with adaptive pyramid and multi-stage path aggregation and the training method thereof according to claim 8, characterized in that: The mixed precision training technology uses torch.cuda.amp of PyTorch to simultaneously use 16-bit and 32-bit floating point numbers for calculation.

Images