A visual enhancement method for all-weather environment perception of urban roads

Abstract

The application discloses a kind of visual enhancement methods for urban road all-weather environment perception, it is related to computer vision and intelligent transportation technical field, steps include: acquisition includes the data set of multiple typical weather and illumination condition;Build physical real cross-domain visual simulation environment, based on training set data, carry out night simulation, dynamic simulation, rain fog simulation, generate the synthetic image with target domain feature;Optimization is carried out to feature, including: the feature fusion based on CutMix and the consistent feature representation based on CORAL Loss;Model adaptation is carried out, from three levels of basic model reconstruction, backbone network light weight, feature fusion structure optimization, systematic adaptation is carried out, improve the perception ability under rain, fog, night complex weather conditions.The method of the application can provide technical support for intelligent perception of urban road under complex weather through real-time deployment and application on edge device.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent transportation technology, and in particular to a visual enhancement method for all-weather environmental perception of urban roads. Background Technology

[0002] With the rapid development of intelligent transportation and autonomous driving technologies, target detection in urban road environments, as a core perception capability, directly determines the safety and operational efficiency of the system. However, in practical applications, while existing mainstream target detection algorithms (such as the YOLO series and Faster R-CNN) perform excellently under ideal conditions (such as sunny days), their detection performance significantly decreases in adverse weather scenarios such as rain, fog, and nighttime, severely limiting their generalization ability in complex urban scenarios.

[0003] The current technology still faces three major challenges: (1) poor adaptability to severe weather: traditional image processing methods (such as random cropping and color jitter) can only improve the data diversity under normal lighting and clarity, and it is difficult to truly simulate typical severe weather effects such as raindrop occlusion, fog scattering, and nighttime glare, resulting in high false negative rates and frequent false positives in rainy nights, dense fog, or low-light environments at dusk; (2) insufficient robustness of cross-domain features: existing methods mostly focus on feature learning in the source domain (sunny daytime) and lack modeling of image degradation characteristics under severe weather (such as reduced contrast, blurred edges, and uneven lighting), causing key target features to be distorted or lost during cross-domain migration; (3) difficulty in balancing real-time performance and accuracy: for edge deployment scenarios such as intersection monitoring and vehicle perception, complex models are difficult to meet real-time requirements due to high computational overhead, while lightweight designs that ignore the preservation of semantic information under severe weather conditions are prone to a cliff-like drop in accuracy.

[0004] In summary, existing technologies lack accurate modeling of physical degradation processes such as rain lines, fog layers, and low light at the data level; at the feature level, they fail to effectively separate content from weather-related interference factors, resulting in severe inter-domain distribution shifts; and the model structure design fails to achieve synergistic optimization between generalization ability and inference efficiency.

[0005] Therefore, developing a single-domain generalized detection method with strong adaptability to severe weather, by integrating physically guided image data simulation, domain-invariant feature learning, and lightweight network architecture, can improve the accuracy and stability of target recognition under complex conditions such as rain, fog, and night. This is of great significance for core applications of intelligent transportation such as intelligent traffic control, vehicle active safety, and advanced autonomous driving. Summary of the Invention

[0006] The problem to be solved by this invention is to provide a visual enhancement method for all-weather environmental perception of urban roads. Through physical mechanism-driven visual simulation, implicit feature decoupling and lightweight robust model design, it can achieve single-domain generalization capability in urban road scenarios that only rely on training data of a single ideal weather (such as a sunny day).

[0007] This invention adopts the following technical solution: a visual enhancement method for all-weather environmental perception of urban roads, comprising the following steps: Step 1: Data Collection: Collect a dataset containing various typical weather and lighting conditions, divide it into training and test sets, and perform data preprocessing. Step 2: Construct a physically realistic cross-domain visual simulation environment: Based on the training set data, perform dynamic simulation, night simulation, and rain and fog simulation. Model the complex visual effects of low light environment, dynamic interference, and rain and fog occlusion through the optical degradation mechanism under typical meteorological conditions, and generate synthetic images with target domain features from source domain data. Step 3, Feature Optimization, includes: Feature fusion based on CutMix, which uses a region cropping and replacement mechanism to spatially fuse local regions of the image under different enhancement conditions with their corresponding labels to construct virtual scene samples with local semantic stitching; Consistent feature representation based on CORAL Loss, which uses an unsupervised domain alignment mechanism to autonomously learn domain-invariant features and extract essential semantic features that are robust to environmental interference by minimizing the statistical difference in feature distribution between the source domain and the implicit target domain. Step 4, Model Adaptation: Construct a lightweight and robust collaborative optimization architecture, and systematically adapt it from three levels: basic model reconstruction, backbone network lightweighting, and feature fusion structure optimization. This will improve the perception capability under complex weather conditions such as rain, fog, and night, and enable the deployment and application of edge devices.

[0008] Preferably, in step 1, various typical weather and lighting conditions data are obtained based on public datasets, including: sunny daytime, sunny nighttime, rainy evening, rainy nighttime, and foggy daytime, to characterize various complex visual degradation scenarios in the urban traffic environment. The training set contains only image data of daytime sunny scenes, which is used to simulate the limited conditions in actual deployment where only a single ideal environment labeled sample can be obtained. The test set covers image data of all weather scenarios to verify adaptability and robustness in unseen environments; The data preprocessing converts the original labeled data into a standard format supported by the model framework, and normalizes and uniformly allocates the size of the images.

[0009] Preferably, in step 2, the dynamic simulation achieves a blurring effect on traffic participants by simulating the continuous trajectory afterimages left by moving objects on the imaging plane. The specific steps are as follows: Step 211, Setting the length of the afterimage trajectory: Define the spatial span parameter of the afterimage trajectory. Determine the number of trajectory sampling points The sampling points are evenly distributed along the trajectory, among which, To round up; Step 212, Constructing the trajectory direction vector: Let the angle between the direction of motion and the horizontal axis be... Construct the unit direction vector Then the first sampling points The relative coordinates are: ; Step 213, Image Afterimage Weight Allocation: Assign weights to each sampling point using a quadratic function decay method; Step 214, Afterimage Synthesis Calculation: Original Image The simulated image is obtained by weighting and superimposing the offset images of each sampling point. .

[0010] Preferably, in step 2, the night simulation is achieved by darkening the overall brightness, using fixed noise to simulate nighttime noise, and simulating the ambient color of the nighttime environment to generate a nighttime simulation scene. The specific steps are as follows: Step 221: Use a darkening factor to reduce the overall brightness of the image, simulating insufficient lighting at night, to obtain a darkened image. ; Step 222: Simulate Nighttime Noise: Add Gaussian noise of fixed intensity to obtain a noisy image. ; Step 223: Simulate the ambient color of nighttime: By adjusting the RGB channel ratio, simulate the color tone deviation of nighttime to obtain a nighttime color-biased image. ; Step 224: Adjust the night-colored image By truncating outliers to avoid simulating overly dark images, the final simulated nighttime image is produced. .

[0011] Preferably, in step 2, the rain and fog simulation refers to the dynamic simulation and night simulation process. It simulates the characteristics of raindrops by generating linear trails with consistent direction on the noisy image and generates a uniform cloud layer by adding low-frequency Gaussian noise, thereby simulating the hazy and dynamic coexistence effect of rain and fog.

[0012] Preferably, in step 3, the feature fusion based on CutMix involves the following specific steps: Step 311: Randomly select two samples from the training dataset. , and their corresponding tags , ; Step 312: Randomly generate the cropping area: Determine the width of the cropping area. ,high and center coordinates Among them, the width and height of the area are determined by... Distribution-generated scaling factor Decide; Step 313: Perform sample fusion: Combine the samples Replace the content of the cropped area with the sample The corresponding region yields a new sample. ;according to Percentage of the area of ​​the reserved area , merge tags to obtain In this way, new training samples are constructed.

[0013] Preferably, in step 3, the consistent feature representation based on CORAL Loss uses CORAL Loss as an unsupervised domain alignment mechanism for sequence features. and Perform pairwise alignment for domain adaptation tasks. and This represents the covariance between the source domain and the target domain, for multi-source domain generalization tasks. and This represents the covariance from data with different distributions.

[0014] Preferably, in step 4, the basic model reconstruction uses RT-DETR as the basic perception model and constructs an end-to-end detection process based on the Transformer architecture;

[0015] The Transformer architecture consists of an encoder and a decoder. It uses a self-attention mechanism to ensure that each element in the sequence pays attention to other elements, thereby enhancing global dependencies. The original image is processed by the backbone network to obtain initial features. The input feature map is then sequence-transformed and passed through the encoder of the Transformer architecture to obtain the encoded global feature sequence. The decoder then obtains the feature vector of the corresponding prediction target and processes it through the prediction head to obtain the adaptation result.

[0016] Preferably, in step 4, the backbone network is lightweighted by replacing the multi-level feature extraction backbone network composed of HGStem, HGBlock and DWConv in the original RT-DETR with ResNet18. Through the residual connection mechanism, the transmission of shallow detail features, including raindrop edges and weak light contours, is ensured with only a few parameters, balancing computing power consumption and feature fidelity, and adapting to low computing power edge devices.

[0017] Preferably, in step 4, the feature fusion structure is optimized by using the GSConv module to construct a lightweight feature fusion neck to replace the original FPN structure, and by using grouped convolution and channel shuffling strategies to enhance cross-scale feature information interaction and optimize the multi-scale feature fusion path. In the GSConv module, the number of channels is The number of channels in the feature map becomes after passing through the convolutional layer. , will contain The number of channels is obtained by passing the first feature map of the channel number through a depthwise separable convolutional layer. The second feature map is obtained by concatenating the first and second feature maps. The third feature map is then randomly arranged to obtain the number of channels after processing by GSConv. The output feature map; A lightweight feature fusion is constructed based on the GSConv module. The three feature maps of different scales after lightweight processing of the backbone network are processed by the GSConv module respectively. The feature maps of different scales are combined through upsampling and concatenation operations. The combined feature map is processed by the GSConv module again, and then the decoder extracts and fuses features. Finally, the detection head predicts the final result.

[0018] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects: 1. High adaptability and reliability in complex environments: The visual simulation method of this invention addresses the single-domain generalization bottleneck faced by existing urban road recognition technologies in adverse weather scenarios. Through a closed loop of technology including adverse weather image simulation, cross-domain feature alignment, and efficient and robust detection, it comprehensively improves the adaptability and reliability of the model in complex environments such as rain, fog, and night.

[0019] 2. Enhance the ability to simulate severe weather images and improve the cross-domain representation of training data: This invention proposes an image simulation strategy based on physical mechanisms, which can simulate severe weather such as raindrop obstruction, light glare, fog scattering, and low-light noise. It can also simulate synthetic data of various target domain features such as rainy nights, clear nights, rainy dusks, and foggy days, enabling models trained on a single source domain to have strong generalization ability for unseen severe weather conditions, thus alleviating the problems of sharp drop in accuracy and increase in false alarm rate.

[0020] 3. Enhance feature robustness and achieve cross-domain consistent representation: Existing models tend to fit source domain features and ignore inter-domain distribution shifts. This invention improves the ability to represent key targets under multi-view and multi-interference conditions through a feature extraction and alignment mechanism with domain invariance properties, effectively aligns the feature statistical distributions of the source domain and the unknown target domain, and improves the stability of the model in challenging scenarios such as rainy nights, dense fog, and low light.

[0021] 4. Collaborative optimization of model efficiency and accuracy, supporting real-time deployment at the edge: This invention constructs a lightweight and robust collaborative perception architecture, which reduces the number of parameters and computational overhead while retaining the ability to respond to targets and ambiguous targets under severe weather conditions. This enables efficient, accurate, and real-time operation on resource-constrained edge devices. Ultimately, it provides highly robust and deployable technical support for business scenarios such as intelligent traffic monitoring, autonomous driving environmental perception, and urban traffic flow analysis, improving the safety, reliability, and operational efficiency of traffic systems under complex weather conditions. Attached Figure Description

[0022] Figure 1 This is an overall flowchart of the visual enhancement method of the present invention; Figure 2 This is an example image of a weather scene from an embodiment of the present invention; Figure 3 This is a flowchart illustrating the physically realistic cross-domain visual simulation processing of an embodiment of the present invention. Figure 4 This is a schematic diagram illustrating the entire process of adapting the GSConv lightweight architecture to an embodiment of the present invention. Figure 5 This is a flowchart illustrating the overall implementation of the GSConv lightweight architecture in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the application will be further described in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments involved in this invention. All non-innovative embodiments based on these embodiments by other researchers in the art are within the protection scope of this invention. Furthermore, the step numbers in the embodiments of this invention are only set for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0024] This invention presents a visual enhancement method for all-weather environmental perception of urban roads. It is systematically constructed around three technical modules: physical-guided visual simulation, implicit feature decoupling, and efficient perception architecture design. It fully combines the actual deployment requirements of urban road edge monitoring equipment, clarifies the experimental environment configuration, dataset processing flow, implementation details of each module, and key parameter settings, and ensures that the technical solution has complete feasibility and engineering implementation capability.

[0025] The overall process of this invention is as follows: Figure 1 As shown, it includes the following steps: Step 1: Data Collection: Collect a dataset containing various typical weather and lighting conditions, divide it into training and test sets, and perform data preprocessing. Step 2: Construct a physically realistic cross-domain visual simulation environment: Based on the training set data, perform dynamic simulation, night simulation, and rain and fog simulation. Model the complex visual effects of low light environment, dynamic interference, and rain and fog occlusion through the optical degradation mechanism under typical meteorological conditions, and generate synthetic images with target domain features from source domain data. Step 3, Feature Optimization, includes: Feature fusion based on CutMix, which uses a region cropping and replacement mechanism to spatially fuse local regions of the image under different enhancement conditions with their corresponding labels to construct virtual scene samples with local semantic stitching; Consistent feature representation based on CORAL Loss, which uses an unsupervised domain alignment mechanism to autonomously learn domain-invariant features and extract essential semantic features that are robust to environmental interference by minimizing the statistical difference in feature distribution between the source domain and the implicit target domain. Step 4, Model Adaptation: Construct a lightweight and robust collaborative optimization architecture, and systematically adapt it from three levels: basic model reconstruction, backbone network lightweighting, and feature fusion structure optimization. This will improve the perception capability under complex weather conditions such as rain, fog, and night, and enable the deployment and application of edge devices.

[0026] In one embodiment of the present invention, the experimental environment is set up as follows: Hardware environment: Equipped with an NVIDIA RTX 3090 GPU (24GB VRAM, supporting large batch training) and an Intel(R) Core(TM) i9-12900K CPU (16 cores and 24 threads) to meet the computing power requirements for complex visual enhancement and Transformer architecture training.

[0027] Software environment: The operating system used is Ubuntu 22.04 LTS, which has good compatibility with the open source ecosystem.

[0028] Core toolchain: The deep learning framework uses PyTorch 2.1.2 (supporting dynamic graph mechanism and adapted to RT-DETR end-to-end architecture), the programming language is Python 3.8, and the CUDA version is 12.1, ensuring seamless integration of GPU acceleration and mainstream libraries.

[0029] The specific implementation process is as follows: 1. Dataset construction and preprocessing.

[0030] The dataset used in this embodiment is based on a public dataset and comprehensively covers various typical weather and lighting conditions, including but not limited to: sunny daytime, sunny nighttime, rainy evening, rainy nighttime, and foggy daytime. It can fully characterize the complex visual degradation situations commonly seen in urban traffic environments.

[0031] To scientifically evaluate the model's single-domain generalization ability in unknown target domains such as severe weather, this embodiment follows the evaluation paradigm of single-source domain training and multi-target domain testing. The dataset is divided into two subsets: the training set contains only daytime clear scene images to simulate the limited conditions in actual deployment where only ideal environment labeled data is available; the test set covers all other weather scenarios to comprehensively verify the model's adaptability and robustness in unseen environments (such as rainy nights, low-light twilight, fog, etc.).

[0032] In addition, both the training and test sets cover seven core perception target categories for urban roads, including pedestrians, cars, riders, trucks, motorcycles, bicycles, and buses. Each category has a sufficient and balanced sample distribution under different weather conditions. This data configuration can provide reliable support for the training optimization and performance evaluation of all-weather environmental perception methods, ensuring that the model has a good generalization foundation in the task of identifying diverse traffic participants.

[0033] Typical image examples for each weather scenario in this embodiment are as follows: Figure 2 As shown, the visual characteristics and degradation patterns under various environmental conditions are presented intuitively.

[0034] During the data preprocessing stage, the original annotation format is uniformly converted to a standard format (such as YOLO format) supported by the object detection framework (such as RT-DETR), and the images are normalized and their sizes are uniformly allocated to ensure the compatibility of the input data and the stability of training.

[0035] 2. Construct a physically realistic cross-domain visual simulation environment.

[0036] This module operates on a daytime, sunny training set and aims to expand the semantic boundaries of the source domain data and improve the model's adaptability to unknown environments by simulating the optical degradation effects of typical severe weather conditions such as rain, fog, and night.

[0037] In this embodiment, automated operation is achieved using Python scripts. Random activation and dynamic parameter adjustment strategies are adopted to generate diverse visual samples without compromising the target semantics, and to perform dynamic simulation, night simulation, and rain / fog simulation.

[0038] Step 2-1) Dynamic simulation: The core idea of ​​dynamic simulation is to achieve a blurring effect for traffic participants by simulating the continuous trajectory afterimages left by moving objects on the imaging plane. The specific steps are as follows: (1) Setting the length of the afterimage trajectory: Define the spatial span parameter of the afterimage trajectory. (Unit: pixels), based on which the number of trajectory sampling points is determined. ( (To round up), the sampling points are evenly distributed along the trajectory.

[0039] (2) Construction of trajectory direction vector: Let the angle between the direction of motion and the horizontal axis be θ. Construct the unit direction vector : ; Then the i-th sampling point sampling points The relative coordinates are .

[0040] (3) Afterimage weight allocation: A weight is assigned to each sampling point, using a quadratic function decay form: ; in, Indicates the number of trajectory sampling points. This represents the allocation result.

[0041] (4) Calculation of afterimage synthesis: original image The simulation results are obtained by weighted superposition of the offset images of each sampling point: ; in, This indicates that the original image is offset along the trajectory direction. The image after.

[0042] Step 2-2) Nighttime Simulation: For night scene simulation, this embodiment darkens the overall brightness, uses fixed noise to simulate nighttime noise, and simulates the ambient color of nighttime to generate a nighttime simulation scene. The specific steps are as follows:

[0043] (1) Use a darkening factor to reduce the overall brightness of the image to simulate insufficient lighting at night and obtain a darkened image. Represented as: ; Among them, the darkening coefficient , The smaller it is, the darker it is.

[0044] (2) Simulating noise in the dark: Images are prone to noise in the dark. Gaussian noise of fixed intensity is added directly to obtain a noisy image. : ; in, It follows a mean of 0 and a variance of . Gaussian noise.

[0045] (3) Simulate the ambient color of night: By adjusting the ratio of RGB channels, simulate the color deviation of nighttime to obtain a nighttime color-biased image. : ; in, Used to adjust different channels , , The proportions simulate the colors of a dark night environment.

[0046] (4) The image processed above is truncated to remove outliers and avoid overly dark image simulation, thus producing the final night simulation image. : ;

[0047] It should be noted that, in order to construct a physically realistic cross-domain visual simulation environment, this invention employs three simulation methods: nighttime simulation, dynamic simulation, and rain / fog simulation. These methods are based on the optical degradation mechanism under typical meteorological conditions of urban roads, accurately modeling the complex visual effects of low-light environments, dynamic interference, and rain / fog obstruction. The aim is to generate synthetic images with characteristics of the target domain (such as rainy nights, rainy dusks, and foggy days) starting from source domain (sunny daytime) data, thereby expanding the environmental adaptation boundary of the model.

[0048] This embodiment uses dynamic simulation and night simulation as examples to illustrate its implementation mechanism. The design concept and implementation method of the rain and fog simulation module are similar. The overall construction process of the cross-domain visual simulation environment and its integration path in the all-weather perception system are explained below. Figure 3 As shown, the specific implementation process is as follows:

[0049] Set the simulated trigger probability to During each training iteration, the system generates random values. ,like Then, the simulation construction process is initiated, and a combination of three simulation construction technologies is randomly selected and applied to ensure that the degradation characteristics closely resemble real road scenarios.

[0050] In this embodiment, all simulation operations only change the visual attributes of the image, while the original target bounding box remains unchanged, thus avoiding the introduction of annotation errors.

[0051] Specifically, the above functions are encapsulated into a unified `WeatherAugment` class, which is called in real time within the RT-DETR data loader. Each time an image is loaded, it checks whether enhancement is triggered. If triggered, the selected enhancement operation is executed, and a degraded image is generated. This enhanced image, along with the original annotations, is then input into the model for training. Through real-time generation and dynamic injection, cross-domain visual variation information is continuously injected into the training process, guiding the model to learn robust general feature representations for weather and lighting changes. This lays a high-quality data foundation for subsequent feature decoupling and generalization capability improvement.

[0052] 3. Strengthen the consistent expression of cross-domain features.

[0053] This module is integrated into the model training process. By reconstructing the loss function and forward propagation logic, it improves the feature stability of the model under different visual conditions. Its core lies in building a dual optimization mechanism of CutMix implicit fusion and CORAL Loss distribution alignment, which enables the model to learn domain-invariant features even without explicit domain labels.

[0054] Step 3-1) Feature fusion based on CutMix

[0055] In this embodiment, the CutMix method plays a key role through a cross-weather region fusion mechanism. CutMix randomly selects a rectangular region for images with different weather conditions in the source domain and the virtual target domain, and replaces a local area of ​​one weather image with the corresponding position of another weather image to form a mixed sample of the main weather and other local weather conditions.

[0056] Its core value lies in breaking through the limitations of features in a single weather scenario and constructing training samples that are closer to real, complex environments. The basic operation is represented by the following formula: ; ; Two samples were randomly selected. , and their corresponding tags , , For the sample width and height, , To randomly crop the region boundaries, the area percentage of the region is retained. This controls the size of the random cropping.

[0057] In the visual enhancement framework of this embodiment, CutMix enhances the local discriminativeness of the model by cropping and replacing spatial regions, while relying on the stitching of different weather features to achieve feature fusion of different scene elements; by directly fusing the local features of one weather image (such as the low light features of night weather) with the main features of another weather image (such as the clear edge features of a sunny day) in the spatial dimension, the model can learn invariant information across weather features.

[0058] Therefore, when faced with poor visual effects due to complex occlusion, local degradation, or extreme weather, the model can improve its generalization ability and robustness by fusing features to supplement the information missing under a single weather condition.

[0059] In each training iteration, two images of different weather conditions and their corresponding labels (including categories and bounding boxes) are randomly selected from the training set. A local region of one image (carrying features unique to that weather, such as the effect of night) is cropped and pasted into the corresponding position of the other image to fuse the different weather features of the two images. At the same time, the labels of the two images are proportionally weighted and fused to ensure the consistency between feature fusion and label supervision.

[0060] In this embodiment, the feature fusion based on CutMix effectively simulates local visual degradation (such as rain and fog obstruction, uneven lighting) and scene recombination under multiple weather conditions. At the same time, by directly fusing cross-weather features, the model is exposed to richer feature combination patterns during training, which enhances the cross-domain feature integration capability and improves robustness in complex all-weather environments.

[0061] Step 3-2) Consistency feature representation based on CORAL Loss:

[0062] It is particularly important to note that in the cross-domain visual simulation environment constructed in this embodiment, to overcome the generalization bottleneck under single-source domain training, cross-domain visual simulation is first achieved through image-level enhancement technology to generate synthetic data covering various severe weather degradation features, thereby simulating a multi-domain coexistence training environment at the input level. Furthermore, the CutMix spatial feature fusion mechanism is introduced to perform regional cropping-replacement mixing of weather images under different visual simulation conditions, constructing virtual scene samples of mixed weather. This effectively transforms the traditional single-source domain learning problem into an implicit cross-domain feature integration learning problem, strengthening the model's understanding of local weather features and the overall weather scene. However, since the pseudo-target domain data generated by the above enhancement and mixing processes lacks explicit domain labels (such as category labels like "nighttime rain" or "dense fog"), traditional supervised domain adaptation methods are difficult to apply.

[0063] To this end, this embodiment further uses CORAL Loss as an unsupervised domain alignment mechanism to achieve autonomous learning of domain-invariant features. This method does not rely on any explicit domain labels, but instead guides the model to extract essential semantic features that are robust to environmental disturbances by minimizing the statistical difference in feature distribution between the source domain and the implicit target domain.

[0064] Specifically, CORAL Loss is based on the assumption that when the feature covariance matrices of two domains tend to be consistent, their overall distribution will also tend to be similar. By aligning the second-order statistics (i.e., covariance matrices) of the features of the source and target domains, the model learns to maintain consistent feature representations under different visual conditions.

[0065] Compared to first-order methods that only align the means, this embodiment constructs a complete single-domain generalization loop using the CORAL Loss method that does not require target domain labeling and does not rely on explicit domain classification. This can more comprehensively characterize the feature distribution patterns and correlations between variables in the high-dimensional feature space, which are the most significant dimensions of change when severe weather such as rain, fog, and night causes image degradation.

[0066] In this embodiment, CORAL Loss enables the model to remain highly adaptable to unknown and complex environments such as rainy nights, low-light twilight, and foggy daytime conditions, even when trained solely on labeled daytime data. This truly achieves robust visual enhancement for all-weather perception of urban roads, as shown in the following formula: ; in, Denotes the Frobenius norm. Representing feature dimensions; for generalization tasks targeting "implicit" multi-source domains, and The covariance representing different data distributions is calculated as follows: ; in, and These represent feature vectors with different data distributions, where 1 represents a vector of all 1s of the same size, and the superscript T represents the transpose. and The number of feature vectors representing the source and target domains.

[0067] In this embodiment, CORAL Loss is introduced into the loss calculation module of the RT-DETR detection head and jointly optimized with classification loss and regression loss. In each iteration, two sets of features are extracted: one set is the deep features of the original sunny image (simulating the source domain), and the other set is the features of the degraded image after visual enhancement (simulating the target domain). The covariance matrix of the two sets of features is calculated respectively, and their Frobenius norm difference is calculated as CORAL Loss.

[0068] Furthermore, the total loss function is defined as the sum of classification loss, regression loss, and contrastive loss (CORAL Loss). By minimizing the second-order statistical difference between the features of the source domain and the implicit target domain, the model is constrained to ignore domain-specific interference (such as glare and blur) and focus on a consistent semantic structure across scenes, which significantly improves the perceptual stability in unknown environments such as nighttime rain and dense fog.

[0069] 4. Construct a lightweight, robust, and collaborative high-efficiency perception architecture.

[0070] To enable the efficient deployment of all-weather environmental perception capabilities for urban roads on resource-constrained edge devices (such as embedded cameras at intersections and vehicle-mounted computing units), this embodiment addresses the issue of traditional models' inability to balance accuracy and real-time performance. It systematically reconstructs the RT-DETR model with lightweight features, constructing a lightweight and robust collaborative optimization architecture for efficient inference. The architecture is adapted across three levels: basic model selection, backbone network simplification, and feature fusion structure improvement. This reduces computational overhead while ensuring stable perception capabilities under complex weather conditions such as rain, fog, and night, ensuring that all-weather visual enhancement effects can be effectively translated into high-performance inference in actual deployments.

[0071] Step 4-1) Basic model reconstruction: Select an end-to-end high-efficiency architecture.

[0072] Traditional two-stage detection models (such as Faster R-CNN) suffer from problems such as process redundancy, high inference latency, and weak cross-domain generalization ability. Therefore, this embodiment adopts RT-DETR (Real-Time Detection Transformer) as the basic perception architecture, constructing an end-to-end object detection process based on the Transformer. It abandons the anchor box mechanism and non-maximum suppression (NMS) post-processing stage, reducing the computational bottleneck in the inference process. At the same time, its global attention mechanism has a stronger ability to recognize complex backgrounds and low-contrast objects, and can maintain stable localization and recognition performance under adverse weather conditions such as rain, fog, and nighttime glare, providing a highly robust foundation for subsequent lightweight design.

[0073] Specifically, the infrastructure selection and training configuration in this embodiment are as follows: RT-DETR-L (a lightweight version) was chosen as the basic framework. It implements end-to-end detection based on Transformer, reducing inference latency by eliminating redundant steps such as candidate box and NMS post-processing. The Transformer architecture mainly consists of an encoder and a decoder, and uses a self-attention mechanism to ensure that each element in the sequence pays attention to other elements, enhancing global dependencies.

[0074] In this embodiment, the input image size is set to 640*640, the training batch size is set to 16, the optimizer is AdamW, and the training period is set to 50 epochs.

[0075] The original input image is processed by the backbone network to obtain initial features. The feature map is then sequence-transformed to match the input of the Transformer architecture. The transformed feature sequence is then passed through the encoder of the Transformer architecture to obtain the encoded global feature sequence. After that, the decoder obtains the feature vector of the corresponding prediction target. Finally, the prediction head processes the data to obtain our final result.

[0076] Step 4-2) Lightweighting the backbone network: Balancing computational power consumption and feature fidelity.

[0077] This embodiment addresses the limited computing resources of edge devices by simplifying the backbone network structure, replacing the computationally intensive backbone (a multi-level feature extraction network consisting of HGStem, HGBlock, and DWConv) in the original RT-DETR with ResNet18.

[0078] While the computationally intensive backbone used by the original RT-DETR has powerful feature extraction capabilities, it has significant adaptation defects in all-weather urban road scenarios: its high number of parameters and high FLOPs (floating-point operations) will cause inference latency of edge devices (such as roadside embedded terminals and low-cost in-vehicle GPUs), which cannot meet the requirements of real-time environmental perception; at the same time, the dense stacking of convolutional layers can easily cause shallow detail features (such as the edges of raindrops in rainy weather, and the weak light contours at night and dusk) to be attenuated during transmission, affecting the target detection accuracy under adverse weather conditions.

[0079] In this embodiment, the ResNet18 network uses a residual connection mechanism to ensure the effective transmission of shallow texture and edge information, avoiding the loss of key features such as raindrops and low-light contours due to lightweighting. It effectively alleviates the gradient vanishing problem with only a few parameters, ensuring the efficient transmission of shallow detail features (such as raindrop edges and low-light contours).

[0080] Even after parameter compression, ResNet18 still possesses excellent basic feature extraction capabilities. It can maintain a sensitive response to key targets such as pedestrians and non-motorized vehicles while reducing the computational load of forward inference, meeting the stringent real-time requirements of embedded platforms, and better adapting to edge devices with low computing power.

[0081] Step 4-3) Feature fusion structure optimization: Achieve synergistic gains in lightweight and enhanced performance.

[0082] This embodiment further optimizes the multi-scale feature fusion path based on the lightweight backbone: a lightweight feature fusion neck is constructed using the GSConv module to replace the original high-complexity FPN structure.

[0083] Specifically, the entire adaptation process of the GSConv module in this embodiment is as follows: Figure 4 As shown, the number of channels is The number of channels in the feature map becomes after passing through the convolutional layer. , will contain The first feature map is processed through a depthwise separable convolutional layer to obtain a channel number of The second feature map is obtained by concatenating the first and second feature maps to obtain the third feature map, which is then randomly arranged to obtain the number of channels after processing by GSConv. The output feature map.

[0084] This embodiment uses GSConv to construct a lightweight feature fusion implementation process as follows: Figure 5 As shown, the three feature maps P3, P4, and P5 at different scales after being processed by the lightweight backbone network are processed by the GSConv module. Then, the feature maps at different scales are combined through upsampling and concatenation operations. The combined feature maps are processed again by the GSConv module. Finally, the decoder further extracts and fuses features and predicts the final result through the detection head.

[0085] Finally, deploy edge-end all-weather scene recognition applications.

[0086] To enable the technological achievements to be applied in real traffic scenarios, this embodiment further promotes the lightweight deployment of the model into urban roadside monitoring equipment, as detailed below:

[0087] (1) Model compression and format conversion.

[0088] After training, in order to adapt to edge platforms, the optimized RT-DETR model is processed as follows: quantization is performed to convert the model weights from FP32 precision to INT8 format, reducing memory usage and computational power consumption while keeping the accuracy loss under control; then the format is exported to convert the PyTorch model to ONNX intermediate format, and finally compiled into a dedicated format supported by edge inference engines such as TensorRT to improve running efficiency.

[0089] (2) Edge device integration and inference acceleration.

[0090] The optimized model is deployed on an embedded camera at an intersection or an in-vehicle edge computing device. The input is a real-time video stream of urban roads, with the frame rate matching the device's processing power (typically 15–25 FPS). Forward inference is performed using a hardware acceleration module to meet real-time requirements. The final output is the identification results of traffic participants, including category labels, confidence scores, and bounding box coordinates.

[0091] (3) All-weather scene recognition function is realized.

[0092] The system deployed based on the method of this embodiment runs continuously at the edge and has the following core capabilities: Identifying severe weather: Even under poor visual conditions such as rainy nights, obscured lights, dense fog, and twilight backlight, it can still accurately identify key traffic participants, effectively reducing false detections and missed reports; Adaptable to dynamic environments: It does not require separate model training for each type of weather. Relying on the generalization ability built in the early stage, it can automatically adapt to changing scenarios such as sudden changes in lighting and weather switching. Low local decision-making latency: All calculations are performed locally, avoiding transmission delays and bandwidth dependence, making it suitable for weak network environments such as tunnels and suburbs; Linked business support: The test results can be directly connected to application systems such as intelligent traffic light control and traffic accident early warning to support the closed-loop operation of intelligent transportation.

[0093] This embodiment of the solution is applicable to typical traffic nodes such as urban main roads, highway entrances and exits, tunnel entrances, and overpasses, improving the availability and safety of the perception system under adverse weather conditions such as rain, snow, fog, and night. In the future, it can be extended to key application scenarios such as intelligent traffic control systems, autonomous driving domain controllers, and automatic traffic incident alarm platforms, providing technical support for building a safe and efficient next-generation intelligent transportation infrastructure.

[0094] In summary, the method of this invention achieves a complete technological innovation across the entire process, from data simulation and feature optimization to model lightweighting, realizing a complete closed loop from laboratory training to edge device deployment. The constructed perception system can operate stably on edge devices based on training using only a single clear-day data set, and can achieve all-weather, all-time, and all-scenario road traffic participant identification, realizing the technical goals of single-source domain training, multi-target domain availability, lightweight deployment, and stable and reliable operation.

[0095] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A visual enhancement method for all-weather environmental perception of urban roads, characterized in that, Includes the following steps: Step 1: Data Collection: Collect a dataset containing various typical weather and lighting conditions, divide it into training and test sets, and perform data preprocessing. Step 2: Construct a physically realistic cross-domain visual simulation environment: Based on the training set data, perform dynamic simulation, night simulation, and rain and fog simulation. Model the complex visual effects of low light environment, dynamic interference, and rain and fog occlusion through the optical degradation mechanism under typical meteorological conditions, and generate synthetic images with target domain features from source domain data. Step 3, Feature Optimization, includes: Feature fusion based on CutMix, which uses a region cropping and replacement mechanism to spatially fuse local regions of the image under different enhancement conditions with their corresponding labels to construct virtual scene samples with local semantic stitching; Consistent feature representation based on CORAL Loss, which uses an unsupervised domain alignment mechanism to autonomously learn domain-invariant features and extract essential semantic features that are robust to environmental interference by minimizing the statistical difference in feature distribution between the source domain and the implicit target domain. Step 4, Model Adaptation: Construct a lightweight and robust collaborative optimization architecture, and systematically adapt it from three levels: basic model reconstruction, backbone network lightweighting, and feature fusion structure optimization. This will improve the perception capability under complex weather conditions such as rain, fog, and night, and enable the deployment and application of edge devices.

2. The visual enhancement method for all-weather environmental perception of urban roads according to claim 1, characterized in that, In step 1, the various typical weather and lighting conditions data, including: sunny daytime, sunny nighttime, rainy evening, rainy nighttime, and foggy daytime, are used to characterize various complex visual degradation situations in the urban traffic environment. The training set contains only image data of daytime sunny scenes, which is used to simulate the limited conditions in actual deployment where only a single ideal environment labeled sample can be obtained. The test set covers image data of all weather scenarios to verify adaptability and robustness in unseen environments; The data preprocessing converts the original labeled data into a standard format supported by the model framework, and normalizes and uniformly allocates the size of the images.

3. The visual enhancement method for all-weather environmental perception of urban roads according to claim 1, characterized in that, The dynamic simulation described in step 2 achieves a blurring effect on traffic participants by simulating the continuous trajectory afterimages left by moving objects on the imaging plane. The specific steps are as follows: Step 211, Setting the length of the afterimage trajectory: Define the spatial span parameter of the afterimage trajectory. Determine the number of trajectory sampling points The sampling points are evenly distributed along the trajectory, among which, To round up; Step 212, Constructing the trajectory direction vector: Let the angle between the direction of motion and the horizontal axis be... Construct the unit direction vector Then the first sampling points The relative coordinates are: ; Step 213, Afterimage Weight Allocation: Assign weights to each sampling point using a quadratic function attenuation method: ； in, Indicates the number of trajectory sampling points. For the allocation result; Step 214, Afterimage Synthesis Calculation: Original Image The simulation results are obtained by weighted superposition of the offset images of each sampling point: ； in, This indicates that the original image coordinates Offset along the trajectory direction The image after.

4. The visual enhancement method for all-weather environmental perception of urban roads according to claim 3, characterized in that, Step 2, the night simulation, involves darkening the overall brightness, using fixed noise to simulate nighttime noise, and simulating the ambient color of the nighttime environment to generate a nighttime simulation scene. The specific steps are as follows: Step 221: Use a darkening factor to reduce the overall brightness of the image, simulating insufficient lighting at night, to obtain a darkened image. : ； Among them, the darkening coefficient , The smaller the size, the darker it is; Step 222: Simulate Nighttime Noise: Add Gaussian noise of fixed intensity to obtain a noisy image. : ； in, Follows a mean of 0 and a variance of Gaussian noise; Step 223: Simulate the ambient color of nighttime: By adjusting the RGB channel ratio, simulate the color tone deviation of nighttime to obtain a nighttime color-biased image. : ； in, Used to adjust the channel , , The proportions of the colors simulate the colors of a dark night environment; Step 224: Remove the nighttime color cast from the image. By truncating outliers to avoid simulating overly dark images, the final simulated nighttime image is produced. : 。 5. The visual enhancement method for all-weather environmental perception of urban roads according to claim 4, characterized in that, The rain and fog simulation described in step 2, referring to the dynamic simulation and night simulation process, simulates the characteristics of raindrops by generating linear trails with consistent direction on the noisy image, and generates a uniform cloud layer by adding low-frequency Gaussian distributed noise, thus simulating the hazy and dynamic coexistence effect of rain and fog intertwining.

6. The visual enhancement method for all-weather environmental perception of urban roads according to claim 1, characterized in that, In step 3, the feature fusion based on CutMix involves the following specific steps: Step 311: Randomly select two samples from the training dataset. , and their corresponding tags , ; Step 312: Randomly generate the cropping area: Determine the width of the cropping area. ,high and center coordinates Among them, the width and height of the area are determined by... Distribution-generated scaling factor Decide: ， ； in, The sample width and height are defined, and the center coordinates are randomly selected within the sample range. Step 313: Perform sample fusion: Combine the samples Replace the content of the cropped area with the sample The corresponding region yields a new sample. ;according to Percentage of the area of ​​the reserved area , merge tags to obtain To construct new training samples, the formula is: ； ； in, , To trim the boundaries of the area.

7. The visual enhancement method for all-weather environmental perception of urban roads according to claim 6, characterized in that, In step 3, the consistent feature representation based on CORAL Loss uses CORAL Loss as an unsupervised domain alignment mechanism to align sequence features pairwise, as shown in the formula: ； in, Denotes the Frobenius norm. Indicates the feature dimension; For domain adaptation tasks, and This represents the covariance between the source domain and the target domain, for multi-source domain generalization tasks. and The covariance, representing data from different distributions, is calculated as follows: ； in, and Feature vectors representing different data distributions This represents a vector of all ones of the same size, with the superscript T indicating transpose. and This represents the number of eigenvectors in the source and target domains.

8. The visual enhancement method for all-weather environmental perception of urban roads according to claim 1, characterized in that, In step 4, the basic model reconstruction adopts RT-DETR as the basic perception model and constructs an end-to-end detection process based on the Transformer architecture. The Transformer architecture consists of an encoder and a decoder. It uses a self-attention mechanism to ensure that each element in the sequence pays attention to other elements, thereby enhancing global dependencies. The original image is processed by the backbone network to obtain initial features. The input feature map is then sequence-transformed and passed through the encoder of the Transformer architecture to obtain the encoded global feature sequence. The decoder then obtains the feature vector of the corresponding prediction target, and the adaptation result is obtained through the prediction head.

9. The visual enhancement method for all-weather environmental perception of urban roads according to claim 8, characterized in that, In step 4, the backbone network is lightweighted by replacing the multi-level feature extraction backbone network composed of HGStem, HGBlock and DWConv in the original RT-DETR with ResNet18. The residual connection mechanism ensures the transmission of shallow detailed features, including raindrop edges and low-light contours, balancing computing power consumption and feature fidelity, and adapting to low-computing-power edge devices.

10. The visual enhancement method for all-weather environmental perception of urban roads according to claim 9, characterized in that, In step 4, the feature fusion structure is optimized by using the GSConv module to construct a lightweight feature fusion neck to replace the original FPN structure. The cross-scale feature information interaction is enhanced by grouped convolution and channel shuffling strategy, and the multi-scale feature fusion path is optimized. In the GSConv module, the number of channels is The number of channels in the feature map becomes after passing through the convolutional layer. , will contain The number of channels is obtained by passing the first feature map of the channel number through a depthwise separable convolutional layer. The second feature map is obtained by concatenating the first and second feature maps. The third feature map is then randomly arranged to obtain the number of channels after processing by GSConv. The output feature map; A lightweight feature fusion is constructed based on the GSConv module. The three feature maps of different scales after lightweight processing of the backbone network are processed by the GSConv module respectively. The feature maps of different scales are combined through upsampling and concatenation operations. The combined feature map is processed by the GSConv module again, and then the decoder extracts and fuses features. Finally, the detection head predicts the final result.

Images