A severe weather target detection method based on geometric perception aggregation and heterogeneous multi-scale perception
By employing a lightweight detection method combining geometric perception aggregation and heterogeneous multi-scale perception, the computational delay and noise suppression issues in target detection under adverse weather conditions are resolved, achieving efficient and real-time multi-scale target detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NORTHEAST FORESTRY UNIV
- Filing Date
- 2026-03-24
- Publication Date
- 2026-06-09
AI Technical Summary
Existing technologies suffer from high computational overhead, high latency, and insufficient noise suppression capabilities in target detection models under adverse weather conditions, making it difficult to meet the requirements for real-time and multi-scale perception.
A lightweight detection method based on geometric perception aggregation and heterogeneous multi-scale perception is adopted. The robustness of the model is enhanced by CSP-Darknet backbone network, multi-scale module (MS-Block), lightweight aggregation strategy (LAS) and geometric perception dynamic aggregation module (GDAM), so as to achieve efficient feature extraction and geometric distortion correction.
Without increasing computational overhead, the model's noise suppression and multi-scale perception capabilities were improved, achieving high-precision real-time detection with a 1.36-fold increase in signal-to-noise ratio, a 4.41% and a 4.91% increase in detection accuracy, respectively, while maintaining an inference speed of 118.5 FPS.
Smart Images
Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and autonomous driving perception technology, and in particular to a real-time target detection method applicable to adverse weather conditions such as rain, snow, and low visibility, specifically an AWR-YOLO detection method based on geometric perception aggregation and heterogeneous multi-scale perception. Background Technology
[0002] With the rapid development of autonomous driving and intelligent transportation systems, target detection technology has become a core perception module for ensuring driving safety. It needs to accurately classify and locate targets such as vehicles, pedestrians, and traffic facilities in complex environments. However, severe weather conditions such as rain, snow, glare, and low visibility that frequently occur in real driving scenarios generate high-frequency visual noise and geometric distortion, which seriously disrupt the distribution of image features and lead to a significant decline in the performance of traditional detection models.
[0003] Existing technologies for target detection in severe weather mainly suffer from two types of problems: one type uses a two-stage framework of "recovery before detection" to eliminate weather artifacts through image preprocessing, but this method introduces huge computational overhead and has serious serial latency, which cannot meet the real-time requirements of autonomous driving systems; the other type is a lightweight detection model, which relies on a static weight feature aggregation scheme and a single-scale receptive field design, ignores the semantic correlation between channels, resulting in insufficient noise suppression capabilities, and is difficult to cope with extreme scale changes of targets in driving scenarios. It is also prone to feature collapse and missed detection problems under high-frequency noise interference.
[0004] Furthermore, while visual Transformer-like architectures can provide a global receptive field, they suffer from irregular memory access and quadratic complexity, resulting in excessive latency on edge devices. Traditional lightweight CNNs, although highly hardware-compatible, have limited effective receptive fields due to their homogeneous convolutional kernel design, making them unable to simultaneously handle the detection needs of both distant, small targets and close-range, large targets. Therefore, improving the model's noise suppression, geometric distortion correction, and multi-scale perception capabilities under limited computational resources has become a key technical bottleneck in the field of target detection in severe weather. Summary of the Invention
[0005] To address the aforementioned shortcomings of existing technologies, the present invention aims to provide a target detection method for severe weather based on geometric perception aggregation and heterogeneous multi-scale perception. Through an innovative lightweight aggregation strategy and multi-scale perception mechanism, the method enhances the robustness of the model to environmental disturbances under severe weather conditions without sacrificing computational efficiency, thereby achieving high-precision real-time detection.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for detecting targets in severe weather based on geometric perception aggregation and heterogeneous multi-scale perception, characterized by comprising the following steps:
[0007] S1 and CSP-Darknet serve as the backbone network, performing initial feature extraction on the input severe weather image to obtain multi-scale basic feature maps;
[0008] S2. The intermediate feature stage embeds a multi-scale module (MS-Block), which dynamically expands the effective receptive field through hierarchical multi-branch feature cascading and heterogeneous kernel selection protocol to achieve robust multi-scale feature representation.
[0009] S3. Deploy a lightweight aggregation strategy (LAS) in the network Neck section. This strategy consists of a GSConv hybrid convolutional module and a geometry-aware dynamic aggregation module (GDAM). GSConv achieves efficient channel compression and feature transfer, while GDAM corrects the geometric distortion caused by weather through two-dimensional attention filtering and adaptive deformable alignment.
[0010] S4. Input the feature map after multi-scale perception and geometric perception aggregation into the detection head, and combine the EIoU loss function to perform target category discrimination and bounding box position regression.
[0011] S5. Output the final target detection results to achieve real-time and accurate detection under severe weather conditions.
[0012] Furthermore, the specific implementation of the multi-scale module (MS-Block) in S2 includes:
[0013] Global Query Learning (GQL): The input feature map undergoes global average pooling and channel attention calibration to generate a channel-level attention guidance map. Feature channel response weights are dynamically adjusted using element-wise multiplication, as shown in the following formula:
[0014]
[0015] in, For the input feature map, This represents the Sigmoid activation function. For learnable weights, This is a global average pooling operation. This is element-wise multiplication;
[0016] Hierarchical multi-branch feature cascade: The feature maps calibrated by GQL are reduced in channel dimension through convolution and divided into 3 heterogeneous feature groups. Feature fusion is performed using a recursive cascaded topology structure, as shown in the following formula:
[0017]
[0018] Heterogeneous kernel selection protocol: Differentiated convolutional kernels are used for different feature groups, with intermediate branches ( )use Compact core to preserve high-frequency texture details, final branch ( Large kernels are used to expand the effective receptive field and achieve global semantic aggregation;
[0019] Feature fusion output: The output features of the three branches are concatenated and then processed... Convolutional processing is used to fuse the components, resulting in the final output of the multi-scale modules:
[0020]
[0021] in, This indicates a splicing operation. This indicates a convolution operation.
[0022] Furthermore, the specific implementation of the Lightweight Aggregation Strategy (LAS) in step three includes:
[0023] The GSConv hybrid convolution module employs a parallel branching structure to achieve efficient feature transfer. The specific process is as follows:
[0024] Standard Convolutional (SC) Branch: For the input feature map Perform standard convolution operations to generate Dense feature map of the channel:
[0025]
[0026] Depthwise Separable Convolution (DSC) Branch: Performs a depthwise separable convolution operation on the same input feature map, generating the remaining... Channel feature map:
[0027]
[0028] Feature fusion: Features from two branches are merged through a splicing operation, and information flow is facilitated by channel shuffling.
[0029]
[0030]
[0031] in, This indicates a channel shuffling operation;
[0032] The Geometric Aware Dynamic Aggregation Module (GDAM) comprises two sub-units: two-dimensional attention-guided and adaptive deformable alignment.
[0033] Two-dimensional attention guidance:
[0034] Channel calibration: for input features Perform global average pooling and MLP processing to generate channel weight masks. Filtering out redundant channel noise through element-wise multiplication:
[0035]
[0036] Spatial Focusing: Channel-level mean pooling and max pooling are performed, followed by large-kernel convolution to generate spatial weight masks, thereby refining spatial features.
[0037]
[0038] Adaptive deformable alignment: for Each pixel position Regression 2D offset Geometric correction is achieved through feature resampling via bilinear interpolation.
[0039]
[0040] in, For standard sampling grids, To output integer coordinates on the feature map, The weights are bilinear interpolation weights; the resampled features are... After fine extraction by the convolution and FFN modules, it is concatenated with the identity mapping input and then passed through... Convolutional fusion output.
[0041] Furthermore, in S4, the EIoU loss function is used for bounding box regression. This loss function considers the overlap of the bounding boxes, the distance between the center points, and the aspect ratio, thereby improving the regression accuracy.
[0042] The beneficial effects of this invention are:
[0043] (1) A lightweight aggregation strategy (LAS) is proposed, which achieves a balance between computational efficiency and representational capability through GSConv hybrid convolution. Combined with the GDAM module, the geometric distortion caused by weather is explicitly corrected. While significantly reducing the computational overhead, the signal-to-noise ratio is improved by 1.36 times, effectively suppressing high-frequency weather noise.
[0044] (2) Design a heterogeneous multi-scale module (MS-Block), which dynamically expands the effective receptive field through hierarchical cascaded topology and heterogeneous kernel selection protocol, breaking through the homogeneous convolution limitation of traditional CNN, and can simultaneously and accurately capture small targets at a distance and large targets at a close distance, thus solving the scale mismatch problem in driving scenarios;
[0045] (3) By adopting a differentiated computing resource allocation strategy, efficient convolution is used in the feature transfer layer to reduce redundancy, and the saved computing resources are reallocated to the key perception layer, achieving the optimal trade-off between detection accuracy and real-time performance. While maintaining an inference speed of 118.5 FPS, the model achieves an mAP50 of 68.89% and an mAP50-95 of 55.69%, which are 4.41% and 4.91% better than the benchmark model, respectively.
[0046] (4) The model is based on an end-to-end architecture design, which does not require additional image restoration preprocessing steps, avoids serial delay, is suitable for autonomous driving edge devices with limited computing resources, and has broad engineering application value. Attached Figure Description
[0047] Figure 1 is an overall architecture diagram of the AWR-YOLO framework of the present invention;
[0048] Figure 2 shows a detailed microstructure diagram of the Geometrically Aware Dynamic Aggregation Module (GDAM);
[0049] Figure 3 shows a visual comparison of Grad-CAM under five driving scenarios;
[0050] Figure 4 shows the quantitative signal-to-noise ratio analysis, where (a) is the comparison of the average activation intensity of the target region and the background region, and (b) is the comparison of the signal-to-noise ratio.
[0051] Figure 5 shows the evolution trajectory of the signal-to-noise ratio between layers P3 and P4, where (a) represents the signal strength, (b) represents the background noise, and (c) represents the final signal-to-noise ratio.
[0052] Figure 6 shows the visualization results of the effective receptive field (ERF) of layer P4, where (a) is the baseline model and (b) is the model of this invention.
[0053] Figure 7 shows a comparison of qualitative detection results in extreme real-world scenarios, where (a) is the original input image, (b) is the detection result of the benchmark model, and (c) is the detection result of the method of the present invention. Detailed Implementation
[0054] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0055] like Figure 1 As shown, the severe weather target detection method based on geometric perception aggregation and heterogeneous multi-scale perception provided by this invention is built on an enhanced CSP-Darknet architecture, mainly consisting of a backbone network, a multi-scale module (MS-Block), a lightweight aggregation strategy (LAS), and a detection head. The specific implementation steps are as follows:
[0056] Experimental environment and data preparation:
[0057] The experiment was conducted on a workstation equipped with an NVIDIA GeForce RTX 3080 graphics card. The software environment was built on the Python 3.8 and PyTorch 2.1.0 framework, with CUDA 11.8 used for computational acceleration and the Ultralytics library used to implement the model.
[0058] The dataset is based on a customized subset of the BDD100K dataset, featuring 8,000 carefully selected low-visibility scene images (including rain, snow, nighttime, and dawn scenes). The categories have been reorganized: traffic lights and signs are merged into the "Traffic Equipment" category; bicycles and cyclists are grouped into the "Cyclist" category; motorcycles and trains are grouped into the "Rare Vehicles" category, while the remaining categories remain unchanged. The dataset is divided into training and validation sets in a 6:2 ratio, and the input image size is uniformly adjusted to 640×640.
[0059] Model training parameter settings:
[0060] The AdamW optimizer was used for training, with an initial learning rate of 0.001, a weight decay coefficient of 0.0005, a batch size of 16, and 400 training cycles. The loss function was a weighted sum of classification loss (cross-entropy loss) and regression loss (EIoU loss), with weight coefficients of 1.0 and 5.0, respectively.
[0061] The AdamW optimizer was used during training, with an initial learning rate of 0.001, a weight decay coefficient of 0.0005, a batch size of 16, and 400 training cycles. The loss function was a weighted sum of classification loss (cross-entropy loss) and regression loss (EIoU loss), with weight coefficients of 1.0 and 5.0, respectively.
[0062] Module implementation details:
[0063] Backbone network and multi-scale module deployment:
[0064] The backbone network employs an enhanced CSP-Darknet, generating basic feature maps at three scales (P3, P4, and P5) through five downsampling iterations. In the intermediate feature stage of the backbone network (corresponding to the second C2f module position before the P4 scale), a selective replacement strategy is used to deploy MS-Block, as implemented below:
[0065] Global Query Learning (GQL): Learning from input features Global average pooling is performed to obtain channel descriptors, which are then processed by a linear layer and Sigmoid activation to generate an attention guidance map. This map is then multiplied with the original features to achieve channel calibration.
[0066] Feature grouping and cascading: calibrated features are then... Convolution reduces the number of channels to half of the original number, dividing them into 3 groups of features. ,in Direct transmission, and After fusion Convolution processing, and After fusion, it is processed by convolution;
[0067] Feature fusion: , , After splicing, through Convolution restores the number of channels to the original dimensions, outputting multi-scale enhanced features.
[0068] Deployment of lightweight aggregation strategies:
[0069] Deploy the LAS strategy in the network neck part (feature pyramid and path aggregation network) to replace the traditional convolutional module:
[0070] GSConv module: As the core transmission unit of the Neck section, it performs standard convolution (output channels are half the total number of channels) and depthwise separable convolution (output channels are half the total number of channels) in parallel. After concatenation, it promotes information fusion through channel shuffling operation, reducing the number of parameters by 2.16% compared to traditional convolution.
[0071] GDAM module: GDAM is inserted into the feature aggregation node of the Neck to process features at scales P3, P4, and P5 respectively. The MLP hidden layer dimension of the two-dimensional attention module is 1 / 4 of the number of channels, and the offset of the adaptive deformable alignment unit is determined by... Convolutional layer prediction: The output features are concatenated with the identity mapping in a 1:1 ratio, and then... Convolutional Fusion
[0072] Detection head and loss function:
[0073] The detection head employs an anchorless frame structure, performing object classification and bounding box regression on the P3, P4, and P5 feature maps output by the Neck. The classification branch uses the Sigmoid activation function, while the regression branch outputs the center coordinates, width, and height information of the bounding box. The loss function used is EIoU loss.
[0074] Experimental verification and result analysis:
[0075] Ablation experiments verified:
[0076] To verify the effectiveness of each module, a stepwise ablation experiment was conducted, and the results are shown in Table 1:
[0077] Table 1. Results of Lightweight Aggregation Strategy (LAS) Ablation Experiments
[0078]
[0079] As shown in Table 1, the GSConv module achieved a 14.3% inference speedup, and the GDAM module improved the recall rate by 1.1%, verifying the effectiveness of the collaboration among the components.
[0080] For the deployment strategy of MS-Block, a comparative experiment was conducted on overall replacement and selective replacement. The results are shown in Tables 2 and 3:
[0081] Table 2 Performance Comparison of Overall Replacement Strategy (Backbone Network: All Replaced with MS-Block) under Different Detector Head Configurations
[0082]
[0083] Table 3 Performance comparison of selective replacement strategy (backbone network: only the second C2f is replaced with MS-Block) under different detection head configurations
[0084]
[0085] Experimental results show that the selective replacement strategy reduces the number of parameters by 34.5% while maintaining detection accuracy and increases the inference speed to 118.5 FPS, which is more in line with the deployment requirements of edge devices. Therefore, it was selected as the final deployment solution.
[0086] Comparison experiment with mainstream models:
[0087] The method of this invention is compared with heavyweight models of the Transformer class and lightweight models of the YOLO series. The results are shown in Table 4:
[0088] Table 4. Quantitative comparison results with advanced methods
[0089]
[0090] As shown in Table 4, the method of this invention outperforms existing mainstream models in both precision (80.51%) and recall (60.07%), with mAP50 and mAP50-95 reaching 68.89% and 55.69% respectively, representing improvements of 4.41% and 4.91% over the benchmark model YOLOv8s. Meanwhile, with only 11.44M model parameters, it maintains a real-time inference speed of 118.5 FPS, achieving an optimal balance between accuracy and efficiency.
[0091] Interpretability verification:
[0092] Attention distribution was visualized using Grad-CAM technology (Figure 3). The results show that the model of this invention can effectively suppress background noise in rainy and snowy environments, and the activated region closely matches the target contour. Quantitative analysis of signal-to-noise ratio (SNR) (Figures 4 and 5) confirms that the model attenuates the background noise intensity from 0.16 to 0.11, improving the SNR by 1.36 times. Visualization of the effective receptive field (ERF) (Figure 6) shows that the model achieves a significant expansion of the receptive field through MS-Block, enabling the establishment of long-range dependencies. Qualitative comparison with real-world scenarios (Figure 7) verifies the excellent detection performance of the model in extreme scenarios such as nighttime glare, snowy roads, and raindrop occlusion.
[0093] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.
Claims
1. A method for severe weather target detection based on geometrically aware aggregation with heterogeneous multi-scale aware, characterized in that, Includes the following steps: S1. Construct a task-aligned heterogeneous architecture based on the enhanced CSP-Darknet, which integrates a lightweight aggregation strategy (LAS) and a multi-scale module (MS-Block) to form an end-to-end detection framework; S2. Input the image of the severe weather scene into the architecture and perform initial feature extraction through the enhanced CSP-Darknet; S3. Deploy LAS in the network Neck part to perform channel compression and geometric distortion correction on the extracted initial features to obtain robust features; S4. Embed MS-Block in the middle layer of the backbone network to dynamically expand the receptive field and aggregate multi-scale information for robust features, and generate feature representations that are adapted to targets of different scales. S5. Input the aggregated features into the detection head and output the target detection result, which includes the target category, location and bounding box information.
2. The adverse weather target detection method according to claim 1, characterized in that, In step S3, LAS includes a GSConv hybrid convolution module and a geometry-aware dynamic aggregation module (GDAM). The GSConv module is used to reduce computational overhead and preserve semantic information, while the GDAM module is used to filter high-frequency meteorological noise and correct geometric misalignments.
3. The method for detecting targets in severe weather according to claim 2, characterized in that, The implementation process of the GSConv hybrid convolution module includes: For the input feature map Generate through standard convolution (SC) branch The dense feature map of the channel, the formula is: ; The remaining part is generated by the depthwise separable convolution (DSC) branch. The feature map of the channel, the formula is: ; The feature maps of the two branches are fused through a concatenation operation, as shown in the formula: ; The channel shuffling operation is performed on the fused feature map, using the following formula: ,in Disrupt the channel operation.
4. The method for detecting targets in severe weather according to claim 2, characterized in that, The implementation process of the GDAM includes: Two-dimensional attention guidance: for input features Perform channel calibration and generate channel weight masks. Redundant noise is filtered out through element-level broadcast multiplication, as shown in the formula. ;right Perform spatial focusing to generate a spatial weight mask. Refined features are obtained through spatial weighting. ; Adaptive deformable alignment: for Each pixel position Regression 2D offset Feature resampling is performed using bilinear interpolation, as shown in the formula below. ,in For standard sampling grids, To output integer coordinates of the feature map, These are the weights for bilinear interpolation; The resampled features are processed The convolutional layers and feedforward network are finely extracted and concatenated with the identity mapping input before being processed. Convolutional fusion.
5. The method for detecting targets in severe weather according to claim 1, characterized in that, The implementation process of MS-Block in step S4 includes: Global Query Learning (GQL): Learning from input tensors Global average pooling (GAP) is performed, and a channel-level attention guidance map is generated through a linear layer with a sigmoid activation function, as shown in the formula. ,in For the Sigmoid function, For learnable weights, This is element-wise multiplication; Hierarchical multi-branch feature cascade: through After convolution to reduce the channel dimension, the features are divided into 3 groups of heterogeneous features. Cross-scale information flow is achieved through recursive cascading, as shown in the formula: ,in Reverse bottleneck operation for a specific core size; Heterogeneous nuclear polymerization: employing A compact kernel performs local context aggregation on the intermediate branches, a large-scale kernel performs global topology reconstruction on the final branch, and the outputs of the three branches are concatenated before... Convolutional fusion, the formula is as follows .
6. The method for detecting targets in severe weather according to claim 1, characterized in that, In step S1, a selective replacement strategy is used to deploy MS-Block, replacing the second C2f module in the middle layer of the backbone network with MS-Block; in step S3, GDAM modules are deployed in the p3 and p4 layers of the detection head.
7. The method for detecting targets in severe weather according to claim 1, characterized in that, The training process of the method includes: The training environment was built based on Python 3.8 and PyTorch 2.1.0 frameworks, and CUDA 11.8 was used to accelerate computation. The AdamW optimizer was used, with an initial learning rate of 0.001, a weight decay factor of 0.0005, a batch size of 16, and 400 training cycles. The input image size is uniformly set to 640×640 during both the training and validation phases; The training dataset was constructed based on the BDD100K dataset, with 8000 carefully selected low-visibility scene images, divided into training and validation sets in a 6:2 ratio. The original categories were reorganized into traffic equipment, cyclists, rare vehicles, and other retained categories.
8. The method for detecting targets in severe weather according to claim 1, characterized in that, The evaluation metrics for the method include mean precision (mAP), number of parameters, number of floating-point operations (FLOPs), and frames per second (FPS), where mAP is calculated based on... and , For the precision-recall function, The target number of categories is reported, along with the mAP50 and mAP50-95 metrics.
9. The severe weather target detection model based on geometric perception aggregation and heterogeneous multi-scale perception according to claim 1, characterized in that, include: Backbone network module: built on an enhanced CSP-Darknet, used for initial feature extraction of the input image; Lightweight aggregation module: Deployed in the network Neck part, including GSConv hybrid convolution submodule and GDAM geometry-aware dynamic aggregation submodule, used to implement channel compression, noise filtering and geometric distortion correction; Multi-scale perception module: Embedded in the middle layer of the backbone network, including the GQL global query learning submodule, the hierarchical multi-branch feature cascade submodule and the heterogeneous kernel aggregation submodule, which are used to dynamically expand the effective receptive field and aggregate multi-scale information; Detection head module: Used to receive aggregated features and output target category, location, and bounding box information.
10. The severe weather target detection model according to claim 9, characterized in that, The model has an inference speed of 118.5 FPS, an mAP50 of 68.89% on the severe weather dataset, an mAP50-95 of 55.69%, 11.44M parameters, and 34.71 GFLOPs of floating-point operations.