Complex weather small target detection system based on multi-scale image enhancement and dynamic domain adaptation
The complex weather small target detection system, which combines multi-scale image enhancement and dynamic domain adaptation, solves the problems of target detection accuracy and stability under complex weather conditions. It achieves adaptive capabilities in image quality enhancement and feature extraction, and improves the cross-weather domain robustness and detection accuracy of small target detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGDONG LEIYON INTELLIGENCE TECH CORP
- Filing Date
- 2026-01-23
- Publication Date
- 2026-05-01
AI Technical Summary
Existing target detection models exhibit significant performance degradation under complex weather conditions (such as rain, fog, snow, haze, or low light), primarily due to image degradation leading to reduced contrast, increased noise, and the submergence or weakening of small target features. Furthermore, the detection network lacks sufficient generalization ability across weather domains.
A complex weather small target detection system employing multi-scale image enhancement and dynamic domain adaptation includes an LMIENet image enhancement module, an SC2f deformable convolutional feature extraction module, and a DDA dynamic domain adaptation module. Through image brightness adaptive correction, contrast enhancement, detail restoration, spatial adaptive sampling, and dynamic domain feature distribution alignment, it achieves feature stability and detection accuracy improvement across weather domains.
It significantly improves the accuracy and robustness of small target detection under complex weather conditions, enhances image perception quality and feature extraction capabilities, solves the detection stability problem across weather domains, and achieves end-to-end collaborative optimization effects.
Smart Images
Figure CN121963142A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of target detection in autonomous driving, and in particular to a complex weather small target detection system based on multi-scale image enhancement and dynamic domain adaptation. Background Technology
[0002] With the rapid development of computer vision and deep learning technologies, object detection plays a crucial role in the field of autonomous driving. However, under complex weather conditions (such as rain, fog, snow, haze, or low light), existing mainstream detection models (including the YOLO series, Faster R-CNN, etc.) often experience significant performance degradation. The main reasons for this can be attributed to the following two aspects:
[0003] Reason 1: Image degradation caused by severe weather can lead to reduced contrast, increased noise, and blurred details, which seriously interferes with the feature extraction process;
[0004] For example, in autonomous driving scenarios, complex weather conditions can significantly reduce image quality, leading to problems such as blurred target edges, uneven brightness, and missing details, thereby interfering with the feature extraction and discrimination of the detection model. Although the existing YOLOv8 detection model shows high performance in multi-scale feature fusion and detection speed, it is quite sensitive to the quality of the input image. When the image degrades, the features of small targets are easily submerged or weakened, and the detection accuracy decreases significantly as a result.
[0005] Reason 2: Small targets have a limited pixel ratio in the image, weak feature information, and are easily masked by weather noise; in addition, the receptive field of the deep layers of the detection network is large, which may cause the features of small targets to be further lost in the multi-scale fusion process, affecting the detection accuracy.
[0006] To address the aforementioned problems, existing technologies are mainly improved through the following two methods, specifically:
[0007] Method 1, Image Enhancement: Preprocessing the input image using traditional image processing methods (such as histogram equalization) or deep learning networks (such as DehazeNet and PReNet) to improve image quality;
[0008] Method 2: Detection network structure optimization: Introduce attention mechanisms (such as SE, CBAM), feature pyramid structure improvements (such as BiFPN, ASFF), or deformable convolution (DCN) into the network to enhance feature representation capabilities.
[0009] However, for the image enhancement methods mentioned above, since these methods are usually general-purpose designs and lack deep coupling with the target detection task, they are prone to artifacts or disruption of the original feature distribution, thus offering limited improvement in detection performance. As for the aforementioned detection network structure optimization method, although it performs well on the standard dataset COCO, it still suffers from insufficient generalization ability in complex weather environments.
[0010] Therefore, it is necessary to design a target detection system that is synergistically optimized in multi-scale feature fusion, image quality enhancement, and domain adaptation to improve the accuracy and stability of small target detection under complex weather conditions. Summary of the Invention
[0011] The purpose of this invention is to provide a small target detection system for complex weather conditions based on multi-scale image enhancement and dynamic domain adaptation, which addresses the shortcomings of existing technologies. This system integrates multi-scale feature fusion, complex weather preprocessing, and dynamic domain adaptation, and can effectively improve the detection accuracy and robustness of small targets under complex weather conditions.
[0012] To achieve the above objectives, the present invention is implemented through the following technical solutions.
[0013] A complex weather target detection system based on multi-scale image enhancement and dynamic domain adaptation includes:
[0014] LMIENet image enhancement module: An image enhancement network structure based on local illumination perception and multi-scale feature fusion, used to achieve adaptive brightness correction, contrast enhancement and detail restoration of input images;
[0015] SC2f Deformable Convolution Feature Extraction Module: Deformable convolution is introduced in the YOLOv8 backbone and neck structure to replace standard convolution, in order to achieve spatial adaptive sampling;
[0016] The DDA dynamic domain adaptation module is a cross-weather domain feature distribution alignment mechanism based on adversarial learning and dynamic weight adjustment. It is used to achieve adaptive fusion of the feature spaces of the source domain and the target domain to alleviate the detection error caused by domain bias.
[0017] This complex weather-based small target detection system, based on multi-scale image enhancement and dynamic domain adaptation, employs the following detection method to achieve small target detection under complex weather conditions, specifically including the following steps:
[0018] Step a) Collect images of various weather scenes and construct the original image dataset; collect image data under different meteorological conditions, including rain, fog, snow, low light at night, and mixed weather, in typical complex environments. Data collection should cover various road types, including highways, urban areas, and suburbs, to ensure the diversity of the samples across multiple scenes; the collected original images should retain pixel details of small targets in the distance for subsequent small target detection performance optimization.
[0019] The output results form the original image dataset D_raw;
[0020] Step b, Data cleaning and fine-grained annotation: The original image dataset D_raw is cleaned using a two-stage method that combines automation and manual work to remove invalid, blurry, or overexposed images;
[0021] After the cleaning is completed, typical traffic targets in the image are manually bounding boxes labeled. These typical traffic targets include pedestrians, cyclists, and motor vehicles. Cross-validation is set up in the labeling process to ensure labeling accuracy and target integrity.
[0022] Output: A clean and labeled dataset D_clean is obtained;
[0023] Step c, Image Enhancement Preprocessing: The LMIENet image enhancement module is used to enhance the D_clean dataset to improve the visual quality under low light and haze conditions;
[0024] The enhancement process achieves adaptive image brightness, contrast enhancement, and texture detail restoration under unsupervised conditions, and the enhanced image obtained maintains a one-to-one correspondence with the original annotation.
[0025] Output: Enhanced model M_enhance and enhanced dataset D_enhanced;
[0026] Step d, Structured dataset partitioning: The dataset is divided into training, validation and test sets using a stratified sampling strategy;
[0027] The test set is further subdivided into a known weather test subset and an unknown challenge subset to evaluate the model’s generalization performance across weather domains;
[0028] Output: Structured data subsets Train, Val, and Test;
[0029] Step e, Training and calibration of the LMIENet image enhancement module: Using the original images in the Train set as input, the enhancement effect is optimized and calibrated using a no-reference image quality evaluation index;
[0030] After training, the image quality before and after enhancement is compared on the validation set to ensure that the enhancement results meet the expected visual standards in terms of brightness consistency and spatial contrast.
[0031] Output: The calibrated enhancement model M_enhance_final;
[0032] Step f: Construction of the detection network integrating deformable convolution: The SC2f deformable convolution feature extraction module achieves adaptive modeling of non-uniform degradation features by dynamically learning the convolution sampling offset;
[0033] The SC2f deformable convolutional feature extraction module is trained using the enhanced dataset D_enhanced to progressively optimize detection accuracy, especially for the feature representation ability of small-scale targets.
[0034] Output: Improved detection model M_detect_sc;
[0035] Step g: Introduce joint training with dynamic domain adaptation: Embed the DDA dynamic domain adaptation module into the multi-layer feature output of the M_detect_sc network, and achieve adaptive alignment of feature distribution between the source domain and the target domain through dynamic weight adjustment;
[0036] During training, a strategy of alternating warm-up and adversarial training is adopted to gradually improve cross-domain robustness and feature consistency.
[0037] Output: M_detect_full, a detection model with weather domain adaptability;
[0038] Step h, pipeline construction and end-to-end optimization:
[0039] The calibrated enhancement model M_enhance_final from step e is concatenated with the weather-domain adaptive detection model M_detect_full from step g to form a complete detection system S_full;
[0040] The system input is a real-time image of complex weather, and the output is the corresponding target category and location;
[0041] Output: The final detection system S_full_final can be directly used for multi-weather small target detection in autonomous driving scenarios;
[0042] Step i, System performance verification: Use the structured test suite Test to perform functional verification of the system.
[0043] The LMIENet image enhancement module consists of a local illumination estimation module, a multi-scale detail restoration module, and a channel attention fusion module.
[0044] The steps in the SC2f deformable convolution feature extraction module for spatial adaptive feature extraction using deformable convolution include:
[0045] Step 1.1: Input feature map X undergoes initial 1×1 convolution dimensionality reduction;
[0046] Step 1.2: Divide into two paths: one path enters the DCN submodule, and the other path is directly transmitted as a residual branch;
[0047] Step 1.3: Several layers of DCN and standard convolution are stacked alternately;
[0048] Step 1.4: After channel fusion and 1×1 convolution integration, output feature Y.
[0049] Specifically, the SC2f deformable convolution feature extraction module introduces a learnable spatial offset in deformable convolution to adapt the sampling position of the convolution kernel to changes in the target shape. The convolution formula is:
[0050] ;
[0051] in, For input features, For output features, For convolution kernel weights, sampling points are combined A standard rule grid is defined, and ; It is learned by independent small convolutional layers, and its learning objective is to make the kernel sampling position dynamically follow the salient structural regions in the feature map;
[0052] Since the sampling position may not fall on integer coordinates, bilinear interpolation is used to obtain the pixel value. In the formula, It is a bilinear interpolation kernel; through this mechanism, the convolution kernel can automatically focus on regions with clear texture or semantic significance in the image, significantly improving the ability to capture key features under complex weather conditions;
[0053] To enhance information interaction between features at different scales, the SC2f deformable convolutional feature extraction module adopts the partial channel fusion idea of C2f; assuming the input features are... Some of the features Enhanced features were obtained through DCN calculation. Another part of the features By preserving the original semantic information through identity mapping, the final output fusion can be represented as: The fusion strategy enables the morphological adaptation features captured by DCN to complement the original feature space, maintaining high-level semantic consistency while enhancing the expressive power of local structures.
[0054] The DDA dynamic domain adaptation module mainly consists of a domain discriminator, a dynamic weight adjuster, and a feature map alignment layer.
[0055] The workflow of the DDA dynamic domain adaptation module is as follows:
[0056] Step 2.1: Extract the features F of the trunk or neck layer;
[0057] Step 2.2: The dynamic weight adjuster estimates the distribution adaptation coefficient based on the current weather conditions of the sample to control the inter-domain alignment strength;
[0058] Step 2.3: The domain discriminator uses a lightweight adversarial network to align the distribution of features between the source and target domains.
[0059] Step 2.4, the feature mapping alignment layer, is based on dynamic loss weights to impose class-level consistency constraints on the detection-related features, ensuring that small targets are stably represented during the alignment process.
[0060] In the dynamic weight adjustment mechanism, inter-domain differences are sample-dependent, and the degree of degradation varies greatly under severe weather conditions, making fixed weights unsuitable. Therefore, a dynamic adjustment factor is introduced: ;in, , These are the learnable parameters. For ReLU function, For the Sigmoid function, Indicates features Perform global average pooling;
[0061] In step h, short-term joint fine-tuning is performed in end-to-end mode to make the enhanced output features correspond optimally with the input distribution of the detection model, so as to improve the overall matching and the efficiency of the collaboration between the front and rear modules.
[0062] In step i, the verification includes the small target detection accuracy under multiple weather conditions, cross-domain stability, and overall real-time performance.
[0063] Compared with the prior art, the present invention has the following beneficial effects, specifically:
[0064] 1. Comprehensive Model Improvement Strategy: Existing solutions often optimize only a single aspect, such as simple image dehazing, contrast enhancement, or feature layer structure adjustment, lacking systematic coordination. This invention organically integrates image enhancement, feature extraction improvement, and domain adaptation mechanisms to form a three-layer collaborative optimization structure from the input to the semantic layer. This enables the model to simultaneously cope with multiple complex factors such as image degradation, feature distortion, and domain distribution differences, thereby comprehensively improving detection stability under adverse weather conditions.
[0065] 2. Significantly improved image perception quality: The LMIENet image enhancement module enables the system to adaptively restore image brightness and details in low-light, foggy, and rainy / snowy environments. Compared with traditional dehazing or illumination correction methods based on fixed parameters, the LMIENet image enhancement module can dynamically adjust the enhancement intensity according to scene characteristics, effectively maintaining visual consistency and feature integrity, and providing higher quality input data for the detection network.
[0066] 3. Enhanced spatial adaptability of feature extraction: Deformable convolution is introduced into the YOLOv8 framework, and an improved module SC2f is constructed, which enables the convolution sampling position to be adaptively shifted according to the image content, thereby capturing the features of small targets with irregular shapes, partial occlusion, or blurred edges more accurately. This design significantly improves the feature representation ability of the model in complex scenes, and the small target detection accuracy is significantly improved compared with the original YOLOv8 model.
[0067] 4. Improved dynamic robustness across weather domains: By introducing the DDA dynamic domain adaptation module, the domain alignment weights can be dynamically adjusted according to the degradation degree of the input image, effectively reducing the distribution differences between sunny and complex weather samples; the DDA dynamic domain adaptation module utilizes adversarial learning and class consistency constraints in the feature layer to maintain semantic stability of small targets during cross-weather domain migration, solving the problems of unstable training and severe feature drift of small targets in existing domain adaptation methods.
[0068] 5. System Collaboration and End-to-End Optimization: The three modules of this invention are jointly optimized end-to-end within the same detection framework, forming an information complementarity mechanism between the enhancement process, feature extraction, and domain adaptation, thus avoiding the feature distribution mismatch problem between independent modules; the optimization of the overall system realizes a closed loop of image quality enhancement, robust feature extraction, and cross-domain calibration, improving the consistency, convergence speed, and overall performance of the model.
[0069] 6. The detection accuracy and practicality have been significantly improved. Attached Figure Description
[0070] The present invention will be further described below with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention.
[0071] Figure 1 This is a system flowchart of the present invention.
[0072] Figure 2 This is a schematic diagram of the SC2f deformable convolution feature extraction module of the present invention.
[0073] Figure 3 This is a schematic diagram of the DDA dynamic domain adaptation module of the present invention. Detailed Implementation
[0074] The present invention will now be described in conjunction with specific embodiments.
[0075] Example 1, as Figure 1 As shown, a complex weather small target detection system based on multi-scale image enhancement and dynamic domain adaptation includes:
[0076] LMIENet image enhancement module: An image enhancement network structure based on local illumination perception and multi-scale feature fusion, used to achieve adaptive brightness correction, contrast enhancement and detail restoration of input images;
[0077] SC2f Deformable Convolution Feature Extraction Module: Deformable convolution is introduced in the YOLOv8 backbone and neck structure to replace standard convolution, in order to achieve spatial adaptive sampling;
[0078] The DDA dynamic domain adaptation module is a cross-weather domain feature distribution alignment mechanism based on adversarial learning and dynamic weight adjustment. It is used to achieve adaptive fusion of the feature spaces of the source domain and the target domain to alleviate the detection error caused by domain bias.
[0079] The complex weather small target detection system based on multi-scale image enhancement and dynamic domain adaptation uses the following detection method to achieve small target detection under complex weather conditions, specifically including the following steps:
[0080] Step a) Collect images of various weather scenes and construct the original image dataset; collect image data under different meteorological conditions, including rain, fog, snow, low light at night, and mixed weather, in typical complex environments. Data collection should cover various road types, including highways, urban areas, and suburbs, to ensure the diversity of the samples across multiple scenes; the collected original images should retain pixel details of small targets in the distance for subsequent small target detection performance optimization.
[0081] The output results form the original image dataset D_raw;
[0082] Step b, Data cleaning and fine-grained annotation: The original image dataset D_raw is cleaned using a two-stage method that combines automation and manual work to remove invalid, blurry, or overexposed images;
[0083] After the cleaning is completed, typical traffic targets in the image are manually bounding boxes labeled. These typical traffic targets include pedestrians, cyclists, and motor vehicles. Cross-validation is set up in the labeling process to ensure labeling accuracy and target integrity.
[0084] Output: A clean and labeled dataset D_clean is obtained;
[0085] Step c, Image Enhancement Preprocessing: The LMIENet image enhancement module is used to enhance the D_clean dataset to improve the visual quality under low light and haze conditions;
[0086] The enhancement process achieves adaptive image brightness, contrast enhancement, and texture detail restoration under unsupervised conditions, and the enhanced image obtained maintains a one-to-one correspondence with the original annotation.
[0087] Output: Enhanced model M_enhance and enhanced dataset D_enhanced;
[0088] Step d, Structured dataset partitioning: The dataset is divided into training, validation and test sets using a stratified sampling strategy;
[0089] The test set is further subdivided into a known weather test subset and an unknown challenge subset to evaluate the model’s generalization performance across weather domains;
[0090] Output: Structured data subsets Train, Val, and Test;
[0091] Step e, Training and calibration of the LMIENet image enhancement module: Using the original images in the Train set as input, the enhancement effect is optimized and calibrated using a no-reference image quality evaluation index;
[0092] After training, the image quality before and after enhancement is compared on the validation set to ensure that the enhancement results meet the expected visual standards in terms of brightness consistency and spatial contrast.
[0093] Output: The calibrated enhancement model M_enhance_final;
[0094] Step f: Construction of the detection network integrating deformable convolution: The SC2f deformable convolution feature extraction module achieves adaptive modeling of non-uniform degradation features by dynamically learning the convolution sampling offset;
[0095] The SC2f deformable convolutional feature extraction module is trained using the enhanced dataset D_enhanced to progressively optimize detection accuracy, especially for the feature representation ability of small-scale targets.
[0096] Output: Improved detection model M_detect_sc;
[0097] Step g: Introduce joint training with dynamic domain adaptation: Embed the DDA dynamic domain adaptation module into the multi-layer feature output of the M_detect_sc network, and achieve adaptive alignment of feature distribution between the source domain and the target domain through dynamic weight adjustment;
[0098] During training, a strategy of alternating warm-up and adversarial training is adopted to gradually improve cross-domain robustness and feature consistency.
[0099] Output: M_detect_full, a detection model with weather domain adaptability;
[0100] Step h, pipeline construction and end-to-end optimization:
[0101] The calibrated enhancement model M_enhance_final from step e is concatenated with the weather-domain adaptive detection model M_detect_full from step g to form a complete detection system S_full;
[0102] The system input is a real-time image of complex weather, and the output is the corresponding target category and location;
[0103] Output: The final detection system S_full_final can be directly used for multi-weather small target detection in autonomous driving scenarios;
[0104] Step i, System performance verification: Use the structured test suite Test to perform functional verification of the system.
[0105] It should be noted that the LMIENet image enhancement module in this embodiment exhibits good task coupling and real-time performance, and can adaptively restore image brightness and detail information while maintaining a lightweight structure. Compared with traditional general image restoration models based on dehazing, deraining, or low-light enhancement, the LMIENet image enhancement module's enhancement process is co-optimized with the target detection task, effectively avoiding artifacts and feature distortion caused by over-enhancement. Furthermore, this LMIENet image enhancement module has a strong ability to preserve details of small targets and can enhance local textures at different scales, thereby improving the discriminability and stability of downstream detection models.
[0106] For the SC2f deformable convolutional feature extraction module in this embodiment, YOLOv8 continues the high-efficiency concept of previous YOLO series in its structural design. Its backbone adopts the C2f module to achieve feature reuse and information flow optimization. The C2f (Cross Stage Partial Fusion) structure effectively improves the gradient propagation and feature diversity of the network through cross-stage feature connections and partial channel fusion. However, this module still uses standard convolution operations, and its receptive field shape and sampling position are fixed. For targets with diverse shapes, significant scale changes, or weather degradation, especially small targets and targets with blurred edges, its feature capture capability is limited. In complex weather scenes, targets are often affected by factors such as light attenuation, scattering blur, or occlusion, resulting in irregular distribution of effective features. Standard convolution is insufficient in dealing with such spatially deformable features. Therefore, this invention introduces deformable convolution (DCN) into the convolutional unit of the original C2f structure to construct a new SC2f (Spatially-adaptive C2f) deformable convolutional feature extraction module. The SC2f deformable convolutional feature extraction module utilizes learnable spatial offsets to adaptively adjust the sampling position of the convolution kernel, thereby enhancing the ability to model the deformation of degraded targets and complex weather textures. In terms of overall structure, the SC2f deformable convolutional feature extraction module maintains the basic framework of C2f, namely the idea of "partial channel residual + serial-parallel fusion". At the same time, it replaces the original standard convolutional unit with DCN. Compared with the original C2f, the SC2f deformable convolutional feature extraction module can adaptively adjust the convolution sampling position, so that the convolution kernel focuses on the effective target area rather than background noise, thereby improving the spatial adaptability of feature representation.
[0107] For the DDA (Dynamic Domain Adaptation) module in this embodiment, under complex weather conditions, there is a significant domain deviation between the distribution of the input image and the standard dataset COCO. This leads to inconsistency in the detection model at the feature level and a decrease in generalization performance. Even with the application of LMIENet image enhancement and SC2f structure improvement, it is still difficult to completely solve the feature drift problem across meteorological domains. The enhanced image still differs from the clear sky domain in the feature space, and small target features are more prone to instability in the transform domain. Therefore, this invention further introduces the DDA (Dynamic Domain Adaptation) module into the improved YOLOv8 network to achieve cross-domain calibration at the feature distribution level. The DDA module maintains semantic consistency and detection stability across different weather domains through dynamic weight adjustment and adversarial feature alignment mechanisms, thereby significantly improving the detection accuracy of small targets under complex weather conditions.
[0108] In summary, the complex weather small target detection system based on multi-scale image enhancement and dynamic domain adaptation in this embodiment has the following advantages, specifically:
[0109] 1. Comprehensive Model Improvement Strategy: Existing solutions often optimize only a single aspect, such as simple image dehazing, contrast enhancement, or feature layer structure adjustment, lacking systematic coordination. This invention organically integrates image enhancement, feature extraction improvement, and domain adaptation mechanisms to form a three-layer collaborative optimization structure from the input to the semantic layer. This enables the model to simultaneously cope with multiple complex factors such as image degradation, feature distortion, and domain distribution differences, thereby comprehensively improving detection stability under adverse weather conditions.
[0110] 2. Significantly improved image perception quality: The LMIENet image enhancement module enables the system to adaptively restore image brightness and details in low-light, foggy, and rainy / snowy environments. Compared with traditional dehazing or illumination correction methods based on fixed parameters, the LMIENet image enhancement module can dynamically adjust the enhancement intensity according to scene characteristics, effectively maintaining visual consistency and feature integrity, and providing higher quality input data for the detection network.
[0111] 3. Enhanced spatial adaptability of feature extraction: Deformable convolution is introduced into the YOLOv8 framework, and an improved module SC2f is constructed, which enables the convolution sampling position to be adaptively shifted according to the image content, thereby capturing the features of small targets with irregular shapes, partial occlusion, or blurred edges more accurately. This design significantly improves the feature representation ability of the model in complex scenes, and the small target detection accuracy is significantly improved compared with the original YOLOv8 model.
[0112] 4. Improved dynamic robustness across weather domains: By introducing the DDA dynamic domain adaptation module, the domain alignment weights can be dynamically adjusted according to the degradation degree of the input image, effectively reducing the distribution differences between sunny and complex weather samples; the DDA dynamic domain adaptation module utilizes adversarial learning and class consistency constraints in the feature layer to maintain semantic stability of small targets during cross-weather domain migration, solving the problems of unstable training and severe feature drift of small targets in existing domain adaptation methods.
[0113] 5. System Collaboration and End-to-End Optimization: The three modules of this invention are jointly optimized end-to-end within the same detection framework, forming an information complementarity mechanism between the enhancement process, feature extraction, and domain adaptation, thus avoiding the feature distribution mismatch problem between independent modules; the optimization of the overall system realizes a closed loop of image quality enhancement, robust feature extraction, and cross-domain calibration, improving the consistency, convergence speed, and overall performance of the model.
[0114] 6. The detection accuracy and practicality have been significantly improved.
[0115] Example 2 differs from Example 1 in that the LMIENet image enhancement module consists of a local illumination estimation module, a multi-scale detail restoration module, and a channel attention fusion module.
[0116] Specifically, such as Figure 2 As shown, the steps in the SC2f deformable convolution feature extraction module for spatial adaptive feature extraction using deformable convolution include:
[0117] Step 1.1: Input feature map X undergoes initial 1×1 convolution dimensionality reduction;
[0118] Step 1.2: Divide into two paths: one path enters the DCN submodule, and the other path is directly transmitted as a residual branch;
[0119] Step 1.3: Several layers of DCN and standard convolution are stacked alternately;
[0120] Step 1.4: After channel fusion and 1×1 convolution integration, output feature Y.
[0121] Furthermore, the SC2f deformable convolution feature extraction module introduces a learnable spatial offset in deformable convolution to adapt the sampling position of the convolution kernel to changes in the target shape. The convolution formula is:
[0122] ;
[0123] in, For input features, For output features, For convolution kernel weights, sampling points are combined A standard rule grid is defined, and ; It is learned by independent small convolutional layers, and its learning objective is to make the kernel sampling position dynamically follow the salient structural regions in the feature map;
[0124] Since the sampling position may not fall on integer coordinates, bilinear interpolation is used to obtain the pixel value. In the formula, It is a bilinear interpolation kernel; through this mechanism, the convolution kernel can automatically focus on regions with clear texture or semantic significance in the image, significantly improving the ability to capture key features under complex weather conditions;
[0125] To enhance information interaction between features at different scales, the SC2f deformable convolutional feature extraction module adopts the partial channel fusion idea of C2f; assuming the input features are... Some of the features Enhanced features were obtained through DCN calculation. Another part of the features By preserving the original semantic information through identity mapping, the final output fusion can be represented as: The fusion strategy enables the morphological adaptation features captured by DCN to complement the original feature space, maintaining high-level semantic consistency while enhancing the expressive power of local structures.
[0126] Example 3 differs from Example 1 in that the DDA dynamic domain adaptation module mainly consists of a domain discriminator, a dynamic weight adjuster, and a feature mapping alignment layer.
[0127] Specifically, such as Figure 3 As shown, the workflow of the DDA dynamic domain adaptation module is as follows:
[0128] Step 2.1: Extract the features F of the trunk or neck layer;
[0129] Step 2.2: The dynamic weight adjuster estimates the distribution adaptation coefficient based on the current weather conditions of the sample to control the inter-domain alignment strength;
[0130] Step 2.3: The domain discriminator uses a lightweight adversarial network to align the distribution of features between the source and target domains.
[0131] Step 2.4, the feature mapping alignment layer, is based on dynamic loss weights to impose class-level consistency constraints on the detection-related features, ensuring that small targets are stably represented during the alignment process.
[0132] Furthermore, in the dynamic weight adjustment mechanism, inter-domain differences are sample-dependent, and the degree of degradation varies greatly under severe weather conditions, making fixed weights unsuitable. Therefore, a dynamic adjustment factor is introduced: ;in, , These are the learnable parameters. For ReLU function, For the Sigmoid function, Indicates features Perform global average pooling.
[0133] It should be noted that by introducing a dynamic adjustment factor, this dynamic weight control mechanism can adaptively adjust the domain alignment strength according to the current image feature state, so that the model reduces adversarial alignment loss when the model is slightly degraded, and enhances the domain calibration strength when the model is severely degraded, thereby achieving cross-domain balance robustness.
[0134] Example 4 differs from Example 1 in that: in step h, short-term joint fine-tuning is performed in end-to-end mode to make the enhanced output features correspond optimally with the input distribution of the detection model, thereby improving the overall matching and the collaborative efficiency of the front and rear modules.
[0135] Example 5 differs from Example 1 in that, in step i, the verification content includes the small target detection accuracy, cross-domain stability, and overall real-time performance under multiple weather conditions.
[0136] The above description is only a preferred embodiment of the present invention. For those skilled in the art, there will be changes in the specific implementation and application scope based on the ideas of the present invention. The content of this specification should not be construed as a limitation of the present invention.
Claims
1. A complex weather small target detection system based on multi-scale image enhancement and dynamic domain adaptation, characterized in that, Including: LMIENet image enhancement module: An image enhancement network structure based on local illumination perception and multi-scale feature fusion, used to achieve adaptive brightness correction, contrast enhancement and detail restoration of input images; SC2f Deformable Convolution Feature Extraction Module: Deformable convolution is introduced in the YOLOv8 backbone and neck structure to replace standard convolution, in order to achieve spatial adaptive sampling; The DDA dynamic domain adaptation module is a cross-weather domain feature distribution alignment mechanism based on adversarial learning and dynamic weight adjustment. It is used to achieve adaptive fusion of the feature spaces of the source domain and the target domain to alleviate the detection error caused by domain bias. This complex weather-based small target detection system, based on multi-scale image enhancement and dynamic domain adaptation, employs the following detection method to achieve small target detection under complex weather conditions, specifically including the following steps: Step a) Collect images of various weather scenes and construct the original image dataset; collect image data under different meteorological conditions, including rain, fog, snow, low light at night, and mixed weather, in typical complex environments. Data collection should cover various road types, including highways, urban areas, and suburbs, to ensure the diversity of the samples across multiple scenes; the collected original images should retain pixel details of small targets in the distance for subsequent small target detection performance optimization. The output results form the original image dataset D_raw; Step b, Data cleaning and fine-grained annotation: The original image dataset D_raw is cleaned using a two-stage method that combines automation and manual work to remove invalid, blurry, or overexposed images; After the cleaning is completed, typical traffic targets in the image are manually bounding boxes labeled. These typical traffic targets include pedestrians, cyclists, and motor vehicles. Cross-validation is set up in the labeling process to ensure labeling accuracy and target integrity. Output: A clean and labeled dataset D_clean is obtained; Step c, Image Enhancement Preprocessing: The LMIENet image enhancement module is used to enhance the D_clean dataset to improve the visual quality under low light and haze conditions; The enhancement process achieves adaptive image brightness, contrast enhancement, and texture detail restoration under unsupervised conditions, and the enhanced image obtained maintains a one-to-one correspondence with the original annotation. Output: Enhanced model M_enhance and enhanced dataset D_enhanced; Step d, Structured dataset partitioning: The dataset is divided into training, validation and test sets using a stratified sampling strategy; The test set is further subdivided into a known weather test subset and an unknown challenge subset to evaluate the model’s generalization performance across weather domains; Output: Structured data subsets Train, Val, and Test; Step e, Training and calibration of the LMIENet image enhancement module: Using the original images in the Train set as input, the enhancement effect is optimized and calibrated using a no-reference image quality evaluation index; After training, the image quality before and after enhancement is compared on the validation set to ensure that the enhancement results meet the expected visual standards in terms of brightness consistency and spatial contrast. Output: The calibrated enhancement model M_enhance_final; Step f: Construction of the detection network integrating deformable convolution: The SC2f deformable convolution feature extraction module achieves adaptive modeling of non-uniform degradation features by dynamically learning the convolution sampling offset; The SC2f deformable convolutional feature extraction module is trained using the enhanced dataset D_enhanced to progressively optimize detection accuracy, especially for the feature representation ability of small-scale targets. Output: Improved detection model M_detect_sc; Step g: Introduce joint training with dynamic domain adaptation: Embed the DDA dynamic domain adaptation module into the multi-layer feature output of the M_detect_sc network, and achieve adaptive alignment of feature distribution between the source domain and the target domain through dynamic weight adjustment; During training, a strategy of alternating warm-up and adversarial training is adopted to gradually improve cross-domain robustness and feature consistency. Output: M_detect_full, a detection model with weather domain adaptability; Step h, pipeline construction and end-to-end optimization: The calibrated enhancement model M_enhance_final from step e is concatenated with the weather-domain adaptive detection model M_detect_full from step g to form a complete detection system S_full; The system input is a real-time image of complex weather, and the output is the corresponding target category and location; Output: The final detection system S_full_final can be directly used for multi-weather small target detection in autonomous driving scenarios; Step i, System performance verification: Use the structured test suite Test to perform functional verification of the system.
2. The complex weather small target detection system based on multi-scale image enhancement and dynamic domain adaptation according to claim 1, characterized in that: The LMIENet image enhancement module consists of a local illumination estimation module, a multi-scale detail restoration module, and a channel attention fusion module.
3. The complex weather small target detection system based on multi-scale image enhancement and dynamic domain adaptation according to claim 1, characterized in that, In the process of spatial adaptive feature extraction using deformable convolution in the SC2f deformable convolution feature extraction module, the steps include: Step 1.1: Input feature map X undergoes initial 1×1 convolution dimensionality reduction; Step 1.2: Divide into two paths: one path enters the DCN submodule, and the other path is directly transmitted as a residual branch; Step 1.3: Several layers of DCN and standard convolution are stacked alternately; Step 1.4: After channel fusion and 1×1 convolution integration, output feature Y.
4. The complex weather small target detection system based on multi-scale image enhancement and dynamic domain adaptation according to claim 3, characterized in that: The SC2f deformable convolution feature extraction module introduces a learnable spatial offset in deformable convolution to adapt the sampling position of the convolution kernel to changes in the target shape. The convolution formula is: ； in, For input features, For output features, For convolution kernel weights, sampling points are combined A standard rule grid is defined, and ; It is learned by independent small convolutional layers, and its learning objective is to make the kernel sampling position dynamically follow the salient structural regions in the feature map; Since the sampling position may not fall on integer coordinates, bilinear interpolation is used to obtain the pixel value. In the formula, It is a bilinear interpolation kernel; through this mechanism, the convolution kernel can automatically focus on regions with clear texture or semantic significance in the image, significantly improving the ability to capture key features under complex weather conditions; To enhance information interaction between features at different scales, the SC2f deformable convolutional feature extraction module adopts the partial channel fusion idea of C2f; assuming the input features are... Some of the features Enhanced features were obtained through DCN calculation. Another part of the features By preserving the original semantic information through identity mapping, the final output fusion can be represented as: The fusion strategy enables the morphological adaptation features captured by DCN to complement the original feature space, maintaining high-level semantic consistency while enhancing the expressive power of local structures.
5. The complex weather small target detection system based on multi-scale image enhancement and dynamic domain adaptation according to claim 1, characterized in that: The DDA dynamic domain adaptation module mainly consists of a domain discriminator, a dynamic weight adjuster, and a feature mapping alignment layer.
6. The complex weather small target detection system based on multi-scale image enhancement and dynamic domain adaptation according to claim 5, characterized in that, The workflow of the DDA dynamic domain adaptation module is as follows: Step 2.1: Extract the features F of the trunk or neck layer; Step 2.2: The dynamic weight adjuster estimates the distribution adaptation coefficient based on the current weather conditions of the sample to control the inter-domain alignment strength; Step 2.3: The domain discriminator uses a lightweight adversarial network to align the distribution of features between the source and target domains. Step 2.4, the feature mapping alignment layer, is based on dynamic loss weights to impose class-level consistency constraints on the detection-related features, ensuring that small targets are stably represented during the alignment process.
7. A complex weather small target detection system based on multi-scale image enhancement and dynamic domain adaptation according to claim 6, characterized in that: In the dynamic weight adjustment mechanism, inter-domain differences are sample-dependent, and the degree of degradation varies greatly under severe weather conditions, making fixed weights unsuitable. Therefore, a dynamic adjustment factor is introduced: ;in, , These are the learnable parameters. For ReLU function, For the Sigmoid function, Indicates features Perform global average pooling.
8. The complex weather small target detection system based on multi-scale image enhancement and dynamic domain adaptation according to claim 1, characterized in that: In step h, short-term joint fine-tuning is performed in end-to-end mode to make the enhanced output features correspond optimally with the input distribution of the detection model, so as to improve the overall matching and the collaborative efficiency of the front and rear modules.
9. A complex weather small target detection system based on multi-scale image enhancement and dynamic domain adaptation according to claim 1, characterized in that: In step i, the verification includes the small target detection accuracy under multiple weather conditions, cross-domain stability, and overall real-time performance.