Self-driving automobile single-domain target detection method based on visual language diffusion model
By optimizing the multi-scale progressive enhancement module and domain offset library, and combining domain-invariant representation learning of the diffusion process, the single-domain generalization problem of autonomous driving models under different weather conditions is solved, improving detection robustness and cross-domain adaptability.
Patent Information
- Application Number
- CN202511807283.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-03
- Publication Date
- 2026-02-10
AI Technical Summary
Existing deep learning models suffer from decreased detection accuracy in complex and dynamic traffic scenarios in autonomous driving due to domain offset issues in training and testing data. They also perform poorly under different weather conditions, and collecting multi-domain datasets is costly and makes it difficult to achieve effective single-domain generalization.
We employ a method based on data augmentation, visual-linguistic information alignment, and domain-invariant representation learning. We generate diverse data representations through multi-scale progressive augmentation modules, construct a domain offset library, and utilize the diffusion process to learn domain-invariant representations, thereby enhancing the model's cross-domain adaptability.
It improves the model's detection robustness under different weather conditions and its ability to understand cross-domain scene changes, enhances detection performance under adverse conditions, and achieves effective generalization in unseen domains after training in a single source domain.
Smart Images
Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of machine learning and computer vision technology. Background Technology
[0002] Object detection is one of the core tasks in machine learning and computer vision, playing a crucial role in autonomous driving systems. In recent years, deep learning has made significant progress in object detection, greatly promoting the development and application of computer vision in autonomous driving perception. However, traditional deep learning models typically rely on several strong assumptions, one of which is that training and testing data are independent and identically distributed. Unfortunately, this assumption is largely invalid in real and complex traffic scenarios. Due to various factors such as weather variations, lighting changes, and sensor noise, there is often a significant domain shift between the source domain (training data) and the target domain (test data). Existing research has fully demonstrated that even slight distribution shifts can significantly affect detection accuracy. Traditional methods often require data from multiple different source domains during training to learn domain-invariant discriminative features. However, collecting and labeling multi-domain datasets is usually costly, and real-world data is difficult to obtain in many weather-related or extreme scenarios. Therefore, single-domain generalization, a more challenging but practically significant research paradigm, has attracted widespread attention. The goal of this task is to enable a model trained on a single source domain to achieve generalization performance in a completely unseen target domain. For complex and dynamic real-world traffic scenarios, single-domain generalization research is of great significance for autonomous driving perception.
[0003] Single-domain generalization problems encompass two main categories of methods: data augmentation and domain-invariant representation learning. The primary challenge lies in the limited diversity of training data. Because only single-domain samples are available, models tend to learn spurious correlations, leading to performance degradation in unseen domains. Existing research indicates that diverse data augmentation can significantly mitigate this problem. Furthermore, some methods further learn domain-invariant representations based on augmented data to enhance cross-domain generalization capabilities. Summary of the Invention
[0004] The purpose of this invention is to design a domain generalization network based on data augmentation, visual-language information alignment, and domain-invariant representation learning, so as to realize a single-domain target detection method for autonomous vehicles based on a visual-language diffusion model for cross-domain target detection tasks under different weather conditions.
[0005] The steps of this invention are: S1. Design a multi-scale progressive enhancement module: Atmospheric scattering model: (1) in, Indicates a fog-free image. Indicates atmospheric light intensity. It is the transmission function; General perturbation model: (2) (3) in, Indicates the intensity of incident light. Indicates atmospheric light. Represents a set of non-transparent particles, transmission function , and Corresponding to the reflected images Particle set and atmospheric component , This indicates the global illumination change caused by scattering and attenuation; Enhanced image The calculation method is as follows: (4) (5) in, Represents a multi-scale mapping function. It is to satisfy Normalized scale weights, ; S2, Domain Offset Library Optimization: Construct a domain offset distribution library consisting of multiple Gaussian components. : (6) Each Gaussian component This represents a typical domain offset pattern, where and These represent the mean and variance parameters, respectively. A text suggestion library was built. : (7) in Indicates source domain hint, This represents a set of auxiliary hints describing the conditions of the possible target domain; Semantic Adjusted Embedding : (8) The features after perturbation are obtained through By further projecting into the embedding space, we obtain the offset representation. : (9) in, From Gaussian components Mid-sampling; The sampling probability is determined by the prior Gaussian mixture model: (10) in: The prior probability on the Gaussian component in the domain offset distribution library; Optimize the domain offset distribution library: (11) in: Alignment between forced offset injection features and semantically guided features; Ensure consistency under disturbances; Encourage diversity among different Gaussian components; The learned domain offset distribution is regularized to the prior mixture model; Domain offset distribution library Optimize by applying gradient descent to the total loss: (12) in Indicates the learning rate; S3. Domain-invariant representation learning based on diffusion process: S31, Forward Diffusion At time step Perturbation features after applying domain offset : (13) in, Indicates the time-dependent decay coefficient; These are the initial source domain features; It is sampled from the domain offset library and obeys The One offset; S32, Reverse offset removal Design a diffusion-guided loss function (14) in This represents the actual domain offset sampled from the domain offset library. It is a suppression threshold; The detector parameters are optimized using gradient descent: (15) in This represents the learning rate.
[0006] The positive effects of this invention are: 1. Enhancing the diversity of data distribution: This invention proposes a multi-scale progressive enhancement module (MPEM), which adjusts illumination, scattering and texture in layers based on atmospheric imaging mechanisms to generate physically consistent and diverse scene representations. This effectively expands the data distribution of single-domain training and improves the detection robustness of the model under adverse conditions such as night and rain. 2. Enhance the model's adaptability to domain changes: This invention utilizes a visual language model to establish a domain offset library, explicitly describing various weather and imaging degradation distributions with text prompts, achieving effective alignment between visual and semantic aspects, and improving the detection model's understanding and stability of cross-domain scene changes; 3. Enhance the model's ability to represent invariant features under different environments: This invention embeds the domain offset distribution into the diffusion generation trajectory and gradually removes environmental interference through the offset-reverse inference process, thereby obtaining more stable domain invariant features and improving the model's detection performance in scenarios such as insufficient lighting and occlusion. Attached Figure Description
[0007] Figure 1 This is the overall framework of the ViLD model; Figure 2 It is the framework of the MPEM module; Figure 3 These are the detection results from the DWD dataset; Figure 4 It is a comparison of feature maps at different stages. Detailed Implementation
[0008] Specifically, this invention relates to a target detection system for autonomous vehicles that can achieve single-domain generalization under various weather conditions.
[0009] This invention primarily addresses the single-domain generalization target detection problem for autonomous vehicles under varying weather conditions. It designs a domain generalization network based on data augmentation, visual-language information alignment, and domain-invariant representation learning to enable cross-domain target detection tasks for autonomous vehicles under diverse weather conditions. A multi-scale progressive augmentation method based on an atmospheric optical model is employed to generate physically consistent and diverse representations, thereby expanding the coverage of the single-domain distribution. A visual-language-guided domain offset library is designed, where predefined text prompts represent various domain offset distributions, enabling targets to adapt to domain-specific changes. A diffusion-based domain-invariant representation learning paradigm is established, embedding the domain offset distribution into the generated trajectory and representing it as an explicit stochastic process that can be inferred and eliminated during offset removal, thus enhancing domain-invariant representation learning. This invention effectively solves the single-domain generalization problem of target detection for autonomous vehicles under different weather conditions, enhancing the model's adaptability to cross-domain scenarios.
[0010] This invention proposes a vision-language-guided diffusion-based object detection (ViLD) algorithm. This method expands the data distribution through multi-scale progressive data augmentation; it optimizes the domain offset library using the alignment of visual language information, and then incorporates the domain offset distribution into the diffusion process to achieve domain-invariant representation learning. The structure of this invention includes the following parts: a multi-scale progressive augmentation module, domain offset library optimization, and domain-invariant representation learning based on the diffusion process.
[0011] Multi-scale Progressive Enhancement Module: This invention designs a multi-scale progressive enhancement module (MPEM), which, based on atmospheric optical imaging principles, adjusts illumination, scattering, and texture in layers to reconstruct changes in the real world. It generates physically consistent and diverse representations, thereby expanding the coverage of single-domain distributions.
[0012] Domain Shift Bank Optimization: This invention designs a visual language-guided domain shift bank, in which predefined text prompts represent various domain shift distributions, enabling the target to adapt to domain-specific changes.
[0013] Domain-Invariant Representation Learning Based on Diffusion Process: This invention establishes a domain-invariant representation learning paradigm based on diffusion process. By embedding the domain offset distribution into the generated trajectory and representing it as an explicit stochastic process, this process can be inferred and eliminated during denoising, thereby enhancing domain-invariant representation learning.
[0014] The implementation method of this invention includes the following parts: The object detection algorithm based on the visual language diffusion model of this invention was implemented on a server equipped with two NVIDIA GeForce RTX 4090 GPUs, each with 24GB of VRAM. The network was constructed using the deep learning platform PyTorch 2.3.0. The network training consisted of three phases: the domain offset optimization phase had 2000 iterations, the diffusion process had 50000 iterations, and the object detection training phase had 100000 iterations.
[0015] The dataset is then divided into a training set and a test set. The training set is used to train the model parameters, and the test set is used to verify the algorithm performance after the model is trained.
[0016] Finally, a series of comparative experiments were conducted to compare the method with mainstream object detection algorithms and discuss the impact of different modules on the algorithm, thus explaining the effectiveness of the method proposed in this invention.
[0017] The overall framework of the target detection algorithm based on the visual language diffusion model described in this invention is as follows: Figure 1 As shown, firstly, the input image is enhanced using the proposed multi-scale progressive enhancement module to enrich the diversity of visual representations. Subsequently, the enhanced image is fed into the CLIP image encoder to extract semantic features. Next, a domain shift detector trained during the diffusion process is used to eliminate domain shifts. Finally, the refined features are passed to a Faster R-CNN detection network for training and object detection. The domain shift detector is trained based on denoising diffusion probabilistic models (DDPM), which allows the model to explicitly model and correct inter-domain differences. Specifically, we construct a domain shift bank guided by visual-language alignment to represent the distribution of domain shifts, such as... Figure 1 As shown in (B) of Figure 1, during the diffusion process, domain offset features are sampled from the library and injected into the intermediate feature map in a similar diffusion manner, thereby simulating perturbations of specific domains in the visual representation. Through a progressive de-offset process, the detector gradually learns to identify and eliminate these domain offsets, resulting in a more stable and domain-invariant feature representation. The domain offset library consists of a series of Gaussian distributions, which together form a Gaussian mixture model representing various domain offset patterns. As shown in (B) of Figure 1, the library is constructed under the guidance of visual-language alignment, where textual cues are used to semantically model and encode different types of domain offsets.
[0018] The ultimate goal of this invention is to solve the single-domain generalization problem of target detection for autonomous vehicles under different weather conditions by training only on a single source domain, namely, a sunny daytime scene.
[0019] The specific implementation steps of this invention are as follows: 1. Dataset Preparation To comprehensively evaluate the robustness of the proposed model against domain shifts caused by different weather conditions in urban traffic scenarios, this invention uses a Diverse-Weather Dataset (DWD). This dataset integrates samples from multiple public sources, covering five representative weather domains: Daytime Sunny, Night Clear, Dusk Rainy, Night Rainy, and Daytime Foggy. The Daytime Sunny domain contains 27,708 images collected from the BDD100K dataset, of which 19,395 were used for training and 8,313 were used for testing on clear daytime conditions. The Night Clear domain contains 26,158 nighttime images under clear weather conditions, also sampled from BDD100K. The Dusk Rainy and Night Rainy domains were collected under rainy weather conditions, containing 3,501 and 2,494 images, respectively. The daytime haze dataset contains 3,775 images captured in foggy weather, sourced from the FoggyCityscapes and Adverse-Weather datasets. All datasets share seven common urban object categories: bus, bike, car, motorcycle, pedestrian, rider, and truck. The model was trained and validated on the daytime clear daytime dataset and tested on the remaining four datasets to verify its generalization ability.
[0020] 2. Multi-scale progressive data augmentation module Physical imaging models have been widely used in the field of computer vision. Among them, the atmospheric scattering model is a representative physical model used to describe the imaging process under hazy conditions, and its formula is as follows: (1) in, Indicates a fog-free image. Indicates atmospheric light intensity. It is the transmission function, used to characterize the attenuation and scattering of light during propagation.
[0021] To model image degradation caused by the interaction of light with particles during propagation, a general perturbation model has been proposed: (2) (3) in, Indicates the intensity of incident light. Indicates atmospheric light. This represents a set of non-transparent particles (such as dust or water droplets). Transmission function. , and Corresponding to the reflected images Particle set and atmospheric component . This represents the global illumination change caused by scattering and attenuation, while Localized occlusions from particle reflection and absorption were captured.
[0022] To achieve physically consistent and perceptually scaled perturbations, we propose a multi-scale mapping function. This is integrated into a progressive update mechanism, as shown in Figure 2. Enhanced image. The calculation method is as follows: (4) (5) In formula (4), Modeling of illumination, scattering, and particle effects at multiple scales. Control the overall brightness. It is to satisfy The normalized scale weights. In formula (5) Control the intensity of progressive enhancement. The model starts from a clear image. Initially, iterative perturbations conforming to physical laws are applied, and then... The final enhanced image is obtained after the steps. Specifically, given an input image Light mapping Modulate global illumination conditions to adjust the overall scene brightness to simulate lighting variations between domains. Particle mask. Introducing spatially varying occlusion to mimic atmospheric particles such as dust, raindrops, or fog. Spatial-frequency transformation. and its inverse transformation Through spectral components Perform frequency domain modulation to maintain a coherent relationship between low-frequency lighting and high-frequency texture details. Atmospheric lighting adjustment. The global tone and color distribution are further refined to enhance visual consistency. Finally, a residual connection fuses the enhanced output with the original input to obtain a progressively refined image. and the final enhancement results .
[0023] 3. Optimization of the domain offset library In single-domain generalization, the model can only access training data from a single source domain. To characterize potential distributional differences, this invention explicitly models domain shifts and constructs a library of domain shift distributions composed of multiple Gaussian components. : (6) Each Gaussian component This represents a typical domain offset pattern, where and These represent the mean and variance parameters, respectively. Specifically, Capture deviations in space and channel, and This distribution library controls the intensity of feature perturbations in each dimension. It provides explicit offset guidance for the diffusion process, enabling the model to better capture and eliminate domain offsets during generation and feature alignment, thus promoting robust domain-invariant representation learning. Next, we will elaborate on the optimization methods of this domain offset distribution library and how its parameters are progressively updated.
[0024] This invention utilizes CLIP's image encoder and text encoder To extract multimodal features. Specifically, the image encoder. It is broken down into two parts: and .in Visual features injected with domain offset were extracted, and These features are then projected into the embedding space for semantic alignment. Previous research has shown that algebraic operations in the text embedding space can effectively capture semantically related concepts. To leverage this semantic vector difference, we constructed a text hint library. The definition is as follows: (7) in This indicates a source domain cue (e.g., "an image taken during the day"), while This represents a set of auxiliary cues describing possible target domain conditions (e.g., "an image taken on a foggy night"). We expect the Gaussian distribution in the domain offset distribution library to characterize... and The feature offsets corresponding to the semantic differences between them are used to achieve cross-semantic domain offset modeling.
[0025] Within the ViLD framework, the processing is divided into an offset injection branch and a semantic guidance branch. In the semantic branch, the input image... It is first fed into an image encoder to obtain its embedding representation in the latent space. Meanwhile, the source domain prompts... and virtual target domain prompts via text encoder Process them to generate their corresponding semantic prototypes. and By leveraging the semantic relationship between these two cues, we obtained the semantically adjusted embedding. The formula is as follows: (8).
[0026] In the offset injection branch, the sampled domain offset is injected into the visual feature space. Specifically, a perturbation is sampled from the domain offset library and added to it. The output is then processed by... By further projecting into the embedding space, we obtain the offset representation. : (9) in, From Gaussian components Sampled from the middle.
[0027] Its sampling probability is determined by the prior Gaussian mixture model: (10) here, The prior probability on the Gaussian component in the domain offset distribution library.
[0028] Subsequently, the overall objective function is minimized by integrating multiple complementary constraints. Optimize the domain offset distribution library: (11) in: Alignment between forced offset injection features and semantically guided features; Ensure consistency under disturbances; Encourage diversity among different Gaussian components; The learned domain offset distribution is regularized to the prior mixture model. Meanwhile, and The strengths of consistency, diversity, and prior regularization are controlled separately. This joint optimization enables the domain offset library to simultaneously maintain semantic alignment, perturbation stability, and distribution diversity.
[0029] Finally, the domain offset distribution library (Depend on (Parameterization) is optimized by applying gradient descent to the total loss: (12) in This represents the learning rate. This update allows... The Gaussian components in the representation are iteratively refined through backpropagation to achieve adaptive alignment between the semantic representation and the distributed representation.
[0030] 4. Domain-invariant representation learning based on diffusion process After optimizing the domain offset library, we obtain a set of Gaussian distributions aligned with semantic embedding differences, which effectively characterize various types of domain offsets. To explicitly model and eliminate these offsets in the feature space, we integrate them into a diffusion-based process to achieve domain-invariant representation learning.
[0031] (1) Forward diffusion This invention uses Represents the initial source domain features. Sampled from the domain offset library and following a Gaussian distribution. Domain offset Applying a time-dependent scaling factor This is used to simulate the gradual evolution of the domain variance and the asymptotically increasing perturbation. At time step... The disturbance characteristics at time are represented as The formula for this process is as follows: (13) in, Indicates the time-dependent decay coefficient; These are the initial source domain features; Indicates at time step Perturbation characteristics after applying domain offset; It is sampled from the domain offset library and obeys The One offset. Time step. from Uniform sampling is used to gradually control the diffusion intensity.
[0032] (2) Reverse offset In the reverse process, we train a domain offset detector. This is used to estimate potential domain perturbations from noisy representations. Specifically, it is used for perturbation features obtained from forward diffusion. The detector predicts the corresponding domain offset as At the same time, when inputting clean features At that time, the detector should output a response close to zero. This ensures It can effectively distinguish between perturbed and unperturbed representations. This dual-objective design forms the core of our inverse diffusion training. To achieve this goal, we designed a diffusion-guided loss function to jointly optimize detection accuracy and response suppression: (14) in This represents the actual domain offset sampled from the domain offset library. It is a suppression threshold used to control the tolerance for zero response. The first term... Force accurate recovery of the true domain offset on perturbed samples; second term Penalize incorrect or overstated predictions on clean samples; The strength of the zero-response suppression. Together they make... It can learn discriminative and stable mappings between features and their corresponding domain offsets.
[0033] The detector parameters are optimized using gradient descent: (15) in The learning rate is represented by this parameter. Through iterative optimization, the detector gradually learns to locate and quantize domain perturbations in a fine-grained manner, providing reliable supervision for clean representation reconstruction in the subsequent denoising process, and ultimately promoting the learning of domain-invariant features.
[0034] Experimental results (1) Evaluation indicators To verify the effectiveness of the method of this invention, the following key evaluation indicators were used in the single-domain generalized target detection experiment to quantitatively evaluate the detection accuracy and computational efficiency of the model: Precision (P) measures the proportion of actual positive samples among those predicted as positive by the model, reflecting the accuracy of the model's positive class predictions. It is defined as follows:
[0035] in, It is the number of true positive samples detected. This is the number of false positive samples detected. It represents the number of real positive samples that were missed.
[0036] Recall (R) measures a model's ability to cover actual positive samples, i.e., the proportion of correctly detected positive samples. It is defined as follows: .
[0037] Mean Average Precision (mAP) is a commonly used metric in object detection, reflecting both classification and localization performance. The Average Precision (AP) for a single class is an approximation of the area under the PR curve, and the specific formula is as follows:
[0038] in, At the threshold Precision of time, At the threshold Recall rate at that time It is the recall rate of the previous threshold. It means dividing the PR curve into several rectangular regions, calculating the area of each region, and adding them together to obtain an approximate value of AP.
[0039] Finally, calculate the mAP for all categories. mAP is the average of the APs for all categories, and the formula is as follows: in, It is the total number of categories. It is a category The average accuracy was calculated. mAP@0.5 and mAP@0.5:0.95 were used as the final evaluation metrics, where mAP@0.5 refers to the average detection performance across all categories when IoU is greater than 0.5.
[0040] (2) Quantitative analysis of category-level detection As shown in Figure 3, extreme weather conditions can significantly obscure or blur the appearance of targets, posing a significant challenge to accurate detection. This figure illustrates a visual comparison between Ground Truth, Faster R-CNN, and the ViLD framework proposed in this invention.
[0041] To further validate these observations, the quantitative results summarized in Tables 1 through 4 provide a comprehensive comparison between our proposed ViLD framework and representative baseline models, including Faster R-CNN, CLIP-GAP, and S-DGOD. Evaluations were conducted for four unseen target domains (clear night, rainy dusk, rainy night, and hazy daytime), using metrics including class mean precision (AP) and mean mean precision (mAP) for seven target categories (bus, bicycle, car, motorcycle, pedestrian, rider, and truck). The reported averages are calculated over these four unseen target domains, excluding the source domain "clear daytime." In the following analysis, we will discuss the detection performance in detail for each weather domain and different target categories.
[0042] Table 1. Detection results of Daytime Clear generalization to Day Foggy (mAP@50 / %)
[0043] Table 2. Detection results of Daytime Clear generalized to Dusk Rainy (mAP@50 / %)
[0044] Table 3. Detection results of Daytime Clear generalization to Night Clear (mAP@50 / %)
[0045] Table 4. Detection results of Daytime Clear generalization to Night Rainy (mAP@50 / %)
[0046] In the Day Foggy domain, targets are severely obscured by dense fog. As shown in Table 1, our method achieved a 40.4% mAP, outperforming Faster R-CNN and CLIP-GAP by 8.4% and 1.9%, respectively. Compared to CLIP-GAP, our method achieved improvements of over 5% in the Car, Pedestrian, and Truck categories, demonstrating its effectiveness in handling blurred and low-visibility conditions under foggy conditions. Moderate declines were observed in the Bike, Motor, and Rider categories because these categories typically contain small or low-contrast targets whose features easily blend into the fog background, making them susceptible to being partially treated as offset-induced noise during correction. Despite these challenging cases, ViLD achieved the highest overall mAP, confirming its robustness in dense fog.
[0047] In the Dusk Rainy domain, the combination of low light and raindrop interference leads to severe visual degradation. As shown in Table 2, our method achieves 35.3% mAP, consistently outperforming S-DGOD and CLIP-GAP across all categories. Significant improvements were observed, particularly in the Bike, Car, and Truck categories, highlighting the robustness of our method under challenging rainy and low-light conditions.
[0048] In the Night Clear domain, the sharp drop in illumination and the presence of noise make detection particularly challenging. As shown in Table 3, our method achieved a mAP of 39.6%, outperforming S-DGOD and CLIP-GAP by 3.0% and 2.7%, respectively. ViLD demonstrated superior performance across most categories, particularly in Bus, Car, Pedestrian, and Truck categories, with improvements exceeding 4%. These results indicate that our model effectively maintains semantic consistency and robust feature representations under low-light conditions. For the same reasons explained in the Day Fog domain, a moderate decline was observed in Motor and Rider categories.
[0049] In the Night Rainy domain, object detection becomes extremely challenging due to the combined effects of low light, heavy rain, and reflective surfaces. As shown in Table 4, our method achieved a mAP of 20.8%, which is 8.4% and 2.1% higher than Faster R-CNN and CLIP-GAP, respectively. ViLD showed consistent improvements in most categories, especially in the Bus, Bike, Car, Rider, and Truck categories, confirming its robustness in handling severe visual degradation under complex nighttime rainy conditions. For the same reasons explained in the "Daytime Haze" domain, a moderate decline was observed in the Motor category.
[0050] (3) Ablation test Table 5 presents the ablation study conducted on the Diverse-Weather Dataset to evaluate the contributions of the Multi-Scale Progressive Augmentation Module (MPEM) and the Diffusion process. All results are reported as mean average accuracy (mAP, %) across the five unseen weather domains. The mean performance here is calculated as the average across the four unseen target domains (excluding the source domain "Sunny Day"). When neither module is applied, the baseline detector (Faster R-CNN) achieves a mean mAP of 26.1%, showing a significant performance drop when generalizing to unseen domains. Introducing the Diffusion process alone significantly improves the mean mAP to 32.3%, a performance improvement of 6.2%. This demonstrates that the diffusion-guided de-offset process is highly effective in enhancing the model's generalization ability, especially under challenging nighttime and rainy conditions (e.g., a +7.0% improvement in the "Rainy Night" domain). In contrast, applying MPEM alone improved the average mAP to 29.6%, demonstrating that multi-scale progressive enhancement effectively enriches the diversity of source domain images and improves robustness to illumination and scattering variations, especially in the "sunny daytime" and "hazy daytime" domains. When both modules are integrated, the proposed framework achieves the highest overall mAP (34.0%), 7.9% higher than the baseline. The combined use of MPEM and the diffusion module provides complementary advantages: MPEM generates physically consistent and diverse representations, while the diffusion module explicitly corrects for domain offset features during training. Notably, the greatest improvements occur in the twilight rain (+9.3% mAP) and nighttime rain (+8.4% mAP) domains, confirming that this combination effectively handles illumination degradation and severe weather scattering, resulting in superior cross-domain generalization performance.
[0051] Table 5 Ablation test results (mAP@50 / %)
[0052] (4) Qualitative analysis of the offset removal process To examine how the proposed framework performs domain offset correction during object detection training, we visualized intermediate results from the correction pipeline, such as... Figure 4 As shown. From left to right in the image: the original input image. Enhanced images generated by the Multi-Scale Progressive Enhancement Module (MPEM) ; by image encoder Extracted coding feature map ; by detector Estimated prediction domain offset ; and the corrected features obtained after compensating for the predicted offset. The visualization results show that MPEM enhancement... Controlled domain perturbations, such as illumination fluctuations, contrast distortion, and rain-like streaks, are introduced to mimic cross-domain appearance variations encountered in real-world scenes. These perturbations alter global brightness and local texture, effectively expanding the diversity of source domain samples and encouraging the model to learn domain-invariant representations. Corresponding encoded features... These perturbation modes are manifested as high-frequency activation and biased color responses. The resulting prediction offset These domain-specific biases were captured, exhibiting a spatially structured response, and aligned with the augmentation region. (Subtracting...) Then, correct the features Visually, the image becomes smoother and more consistent, preserving target contours and scene semantics while removing noise caused by perturbations. This indicates that the diffusion-guided correction mechanism effectively isolates and eliminates enhancement-related biases introduced by MPEM, enabling the model to retain task-relevant features and obtain stable, domain-consistent representations during detection training.
[0053] (5) Comparison with other single-domain generalization methods Table 6 presents the comparative results of various single-domain generalization methods under different weather conditions on the DWD dataset. The average performance was calculated on four unseen target domains, excluding the source domain Daytime Sunny. The evaluated methods include feature normalization methods such as IBN-Net, IterNorm, SW, and ISW, as well as SDG-specific methods such as SHADE, S-DGOD, SRCD, and CLIP-GAP. The results show that normalization-based methods offer limited performance improvements, while SDG-specific methods, although showing improvements in robustness, still perform poorly in harsh environments. In contrast, the proposed ViLD achieves a state-of-the-art performance of 34.0% on average mAP and significantly improves performance in the Night Sunny, Dusk Rainy, and Night Rainy domains. These results validate the effectiveness of our proposed vision-language guided diffusion process and MPEM in enhancing semantic consistency and improving robustness to changes in light and weather.
[0054] Table 6. Detection results of different methods on the DWD dataset (mAP@50 / %)
[0055] (6) Comparison with other data augmentation methods As shown in Table 7, we compared different data augmentation strategies on four unseen target domains, and the results are reported as mAP (%). Average performance was calculated on the four unseen target domains, excluding the source domain DaytimeSunny. Traditional data augmentation techniques such as Cutout, Mixup, and Cutmix only brought limited or inconsistent performance improvements compared to the baseline method Faster R-CNN, indicating their limited ability to model domain shifts under diverse weather conditions. The TBSA method achieved a moderate performance improvement, increasing the average mAP by 1.3% compared to the baseline, demonstrating that texture- and brightness-sensitive augmentation methods can enhance cross-domain robustness to some extent. In contrast, our proposed MPEM module achieved the best overall performance across all target domains, with an average mAP of 29.6%, surpassing TBSA and other augmentation baselines. The most significant improvements were observed in the Dusk Rainy and Night Rainy domains, which suffer from the most severe illumination degradation and visual occlusion. This indicates that MPEM can effectively generate physically consistent representations with multi-scale augmentation properties, thereby enabling the detector to generalize better under adverse weather conditions.
[0056] Table 7. Detection results of different data augmentation operations on the DWD dataset (mAP@50 / %)
[0057] In summary, this invention proposes a novel single-domain generalized target detection algorithm called ViLD to address the challenge of target detection generalization in autonomous vehicles under various complex weather conditions. This method integrates a multi-scale progressive augmentation module, domain offset library generalization, and domain-invariant representation learning based on a diffusion model to construct a detection network framework with strong single-domain generalization capabilities. First, the source domain data distribution is expanded through a multi-scale progressive data augmentation module. Second, a domain offset distribution library is constructed using visual-linguistic information alignment, enabling targets to adapt to domain-specific changes. Finally, a domain-invariant representation learning paradigm based on a diffusion process is established. By embedding the domain offset distribution into the generated trajectory and representing it as an explicit stochastic process, this process can be inferred and eliminated during denoising, thereby enhancing domain-invariant representation learning. System experiments on the diverse weather dataset DWD show that ViLD significantly outperforms existing mainstream methods in complex scenarios, including rainy evenings and rainy nights, demonstrating good generalization capabilities and practical application value.
Claims
1. A single-domain target detection method for autonomous vehicles based on a visual language diffusion model, characterized in that: The steps are as follows: S1. Design a multi-scale progressive enhancement module: Atmospheric scattering model: （1） in, Indicates a fog-free image. Indicates atmospheric light intensity. It is the transmission function; General perturbation model: （2） （3） in, Indicates the intensity of incident light. Indicates atmospheric light. Represents a set of non-transparent particles, transmission function , and Corresponding to the reflected images Particle set and atmospheric component , This indicates the global illumination change caused by scattering and attenuation; Enhanced image The calculation method is as follows: （4） （5） in, Represents a multi-scale mapping function. It is to satisfy Normalized scale weights, ; S2, Domain Offset Library Optimization: Construct a domain offset distribution library consisting of multiple Gaussian components. : （6） Each Gaussian component This represents a typical domain offset pattern, where and These represent the mean and variance parameters, respectively. A text suggestion library was built. : （7） in Indicates source domain hint, This represents a set of auxiliary hints describing the conditions of the possible target domain; Semantic Adjusted Embedding : （8） The perturbated features are obtained through By further projecting into the embedding space, we obtain the offset representation. : （9） in, From Gaussian components Mid-sampling; The sampling probability is determined by the prior Gaussian mixture model: （10） in: The prior probability on the Gaussian component in the domain offset distribution library; Optimize the domain offset distribution library: （11） in: Alignment between forced offset injection features and semantically guided features; Ensure consistency under disturbances; Encourage diversity among different Gaussian components; The learned domain offset distribution is regularized to the prior mixture model; Domain offset distribution library Optimize by applying gradient descent to the total loss: （12） in Indicates the learning rate; S3. Domain-invariant representation learning based on diffusion process: S31, Forward Diffusion At time step Perturbation features after applying domain offset : （13） in, Indicates the time-dependent decay coefficient; These are the initial source domain features; It is sampled from the domain offset library and obeys The One offset; S32, Reverse Offset Design a diffusion-guided loss function （14） in This represents the actual domain offset sampled from the domain offset library. It is a suppression threshold; The detector parameters are optimized using gradient descent: （15） in This represents the learning rate.