A cognitive compensation visual perception method and system for complex environment single-source domain generalization

Abstract

The application discloses a kind of cognitive compensation target detection methods for complex weather single-source domain generalization, to solve the problem of insufficient weather experience during training period, difficult compensation during test period environment shift and dependence on target domain labeling.The method takes imitating human cognition as the main line, constructs a unified trainable and inferable detection framework, integrates experience accumulation during training period and environment compensation during inference period in the same shared detector.In the training stage, set experience generalization module, generate diversified weather samples through frequency domain disturbance and physical degradation in series, and use the same scene source image to construct weather invariant object constraint, expand distribution coverage and reduce artifact dependence.In the inference stage, set environment compensation module, without target domain labeling, form complementary observation by fusing environment reference, scale transformation and flip view, retain cross-view class consistent and spatial position stable prediction, suppress performance decline caused by single enhancement inadaptation and compensate residual domain shift.The application is suitable for automatic driving, roadside perception, mobile robot and outdoor monitoring scenes, and significantly improves detection generalization and robustness under unknown complex weather.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, deep learning, and intelligent transportation perception technology, specifically to a cognitive compensation target detection method and system that utilizes only single-source domain labeled data for training and is designed for scenarios with fog, rain, low illumination at night, and combined degradation due to night and rain. Background Technology

[0002] Autonomous driving, roadside perception, and outdoor mobile robots require continuous and stable detection of vehicles, pedestrians, cyclists, and road obstacles in open road environments. Fog, rain, low light at night, and combined night-rain weather conditions in real roads can simultaneously alter image brightness, contrast, spectral statistics, edge texture, and local occlusion, causing target detectors to miss detections, make false detections, and cause bounding box localization shifts, thereby affecting the safety and reliability of intelligent transportation systems.

[0003] Existing solutions for improving the performance of complex weather detection mainly include collecting multi-weather labeled data, performing domain adaptation on the target domain, using defogging, deraining, or low-light enhancement preprocessing, and introducing semantic priors from visual language models. While these methods can achieve certain results on specific datasets, they typically require target domain images, additional weather annotations, large-scale multi-domain training samples, or manual parameter tuning for specific weather conditions. These methods are difficult to adapt to real-world deployment scenarios where complex weather occurs randomly, its intensity changes continuously, and multiple degradations coexist.

[0004] Single-source-domain generalized object detection uses only labeled data from one source domain for training, requiring the model to maintain detection capability in target weather domains not seen during training, thus meeting the engineering requirement of "train once, deploy across multiple domains." However, simple data augmentation can easily produce artifacts inconsistent with real weather, while simple domain-invariant learning struggles to cover fine-grained degradations with physical imaging origins, such as fog, rain, low light, and nighttime rain reflections. Existing object-centric dynamic detection methods can select target-related features through visual and linguistic cues, but the collaborative modeling of weather experience accumulation during training and sample-level environmental compensation during inference remains insufficient.

[0005] When the human visual system identifies targets in complex weather conditions, it typically first relies on long-term experience to form stable concepts of objects such as vehicles and pedestrians. Then, it adjusts its observation methods based on current conditions such as fog, rain streaks, or low light, and confirms the actual existence of the target through multiple observations. Based on this cognitive process, a target detection method is needed that can build complex weather experience during the training phase, compensate for the current environment during the inference phase, and verify the prediction results. Summary of the Invention

[0006] The purpose of this invention is to provide a cognitive compensation visual perception method and system for generalizing from a single source domain in complex environments, which solves the problems of insufficient weather experience coverage during the training period, difficulty in stably compensating for environmental offset of a single image during the testing period, and strong dependence on target domain annotation in the prior art.

[0007] A cognitive compensation visual perception method for generalization to a single source domain in complex environments includes a training phase and an inference phase: The training phase includes: acquiring a single clear source domain image and its detection annotation; inputting the source domain image into the empirical generalization module to generate a degraded training view containing frequency domain perturbations and physical weather degradation; The source image and the degraded training view are input into a shared detector to extract source image features and degraded image features, respectively. The parameters of the shared detector are jointly optimized based on detection loss and weather-invariant object constraints, whereby the weather-invariant object constraints cause the degraded image features to converge with the source image features of the same scene. The inference stage includes: for the weather image of the target to be detected, the environment compensation module performs weather routing to determine the weather type and generates an environment reference view; constructs a multi-view set containing the original image, the environment reference view and its geometric transformation view; inputs each view in the multi-view set into the trained shared detector to obtain candidate boxes for each view and maps them back to the original image coordinate system; filters reliable candidate boxes based on cross-view category consistency and spatial location support, and outputs the final detection result after non-maximum suppression.

[0008] Preferably, the specific steps for the experience generalization module to generate the degenerate training view include: Perform a two-dimensional discrete Fourier transform on the source domain image to obtain the amplitude spectrum A and the phase spectrum φ: Constructing a low-frequency mask and high-frequency mask The low-frequency gain is determined based on the sampled weather type w and weather intensity s. High-frequency gain and phase noise The amplitude spectrum and phase spectrum after perturbation are expressed as follows:

[0009]

[0010] In the formula, ⊙ represents element-wise multiplication; The standard deviation of phase noise; Represents a normal distribution; low-frequency mask and high-frequency mask Determined by the distance from a frequency point to the center of the spectrum: After centering the spectrum, let d be the distance from any frequency point to the center of the spectrum, and let R be the reference radius. When d is not greater than The position of ×R is set to 1, and the other positions are set to 0. When d is not less than The position of ×R is set to 1, and the other positions are set to 0; =1+ (w)·s =1+ (w)·s = · (w)·s in, (w) and (w) represents the low-frequency gain coefficient and high-frequency gain coefficient corresponding to weather type w, respectively. (w) is the phase noise modulation coefficient, and the phase noise... , The standard deviation of phase noise is the reference value. The frequency domain perturbation image is obtained by performing an inverse Fourier transform on the perturbed spectrum and then fused with the original image. Physical weather degradation simulations are performed on the fused images based on the sampled weather types, including fog, rain, night, and night rain.

[0011] Preferably, the weather-invariant object constraint is as follows: in the high-level feature layer of the shared detector, after normalizing the source image features and the degraded image features, a cosine consistency loss is calculated. The branch containing the source image features stops gradient backpropagation when calculating this loss. The loss function is: =mean(1 cos( , )) In the formula, cos(·,·) represents the cosine similarity calculated after L2 normalization of the source image feature Fsrc and the degraded image feature Fdeg along the channel dimension, and mean(·) represents the average over the spatial location and batch dimension.

[0012] Preferably, the specific method by which the environmental compensation module performs weather routing is as follows: the weather type is determined based on the image's average brightness, brightness quantiles, contrast, dark channel response, and local high-frequency energy.

[0013] Preferably, the environmental compensation module generates the environmental reference view in the following ways: for foggy images, limited dehazing is performed; for nighttime images, limited brightening is performed; for rainy images, slight contrast enhancement is performed; for nighttime rainy images, limited brightening is performed first, followed by slight contrast enhancement.

[0014] Preferably, the multi-view set includes at least the original image, a horizontally flipped view, a scale-transformed view, the environment reference view, and a scale-transformed view of the environment reference view; wherein the scale factor of the scale transformation is selected from at least one of 0.88, 1.00, and 1.12.

[0015] Preferably, the specific method for filtering reliable candidate boxes based on cross-view category consistency and spatial location support is as follows: after mapping the candidate boxes of each view back to the original image coordinate system, for any candidate box, count the number of candidate boxes in other views that have the same category and an intersection-union ratio not less than the spatial support threshold, and obtain the spatial support number; only candidate boxes that simultaneously satisfy the confidence level not less than the confidence level threshold and the spatial support number not less than the support threshold are included in the reliable candidate box set.

[0016] Preferably, the environmental compensation module also sets a quality rollback mechanism: if the environmental reference view has at least one of the following conditions relative to the original image: excessively high brightness saturation ratio, excessive noise increase, abnormal contrast change, insufficient number of reliable candidates, or abnormal expansion of the number of candidates, then the relevant candidates of the reference view are abandoned, and the prediction result of the original image is rolled back or only the consensus result of the geometric transformation of the original image is used.

[0017] Preferably, the shared detector includes a visual backbone network, an object center gating system, a region proposal network, an ROIAlign layer, and a detection head; the visual backbone network adopts CLIP-RN101, a residual network, or a visual Transformer.

[0018] A cognitive compensation target detection system employing the above method includes: The data processing module is used to read source domain training images, detection annotations, and target images to be tested; The experience generalization module is used to generate a degraded training view that includes frequency domain perturbations and physical weather degradation, and to construct weather-invariant object constraints. A shared detection module is used to perform feature extraction, region proposal, ROI alignment, and detection head prediction; The environmental compensation module is used to perform weather routing, generate environmental reference views, construct multi-view sets, and perform cross-view consensus filtering during the inference phase. The detection output module is used to perform reliable prediction screening, nonmaximum suppression, and quality backoff.

[0019] Compared with the prior art, the present invention has at least the following beneficial effects.

[0020] (1) This invention takes imitating human cognition as the main technical line, constructs a single-source domain generalized target detection cognitive framework for complex weather scenarios, unifies the experience accumulation during the training period and the current environment compensation during the inference period into the same shared detector, forming a trainable and inferable detection mechanism, providing clear cognitive modeling basis for cross-domain target detection under complex weather conditions, and is applicable to application scenarios such as autonomous driving, roadside perception, mobile robots and outdoor video surveillance.

[0021] (2) The present invention proposes an empirical generalization module (EGM) that connects frequency domain weather disturbances and physical weather degradation to generate diverse weather samples and uses source images of the same scene to construct weather-invariant object constraints. While expanding the training weather distribution, it reduces the detector's dependence on synthetic artifacts and accidental weather textures, thereby improving the model's generalization ability under unknown and complex weather conditions.

[0022] (3) The present invention proposes an environmental compensation module (ECM), which does not require target domain labeling during inference. It forms complementary observations through environmental reference view, scale transformation view and flip view, and retains only the prediction results that are consistent across view categories and have stable spatial positions, so as to reduce the performance degradation caused by a single enhancement intensity being unsuitable for the current sample and to compensate for the residual domain offset at the test sample level. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the overall framework of the cognitive compensation target detection method of the present invention, showing the EGM experience generalization process in the training phase and the ECM environment compensation process in the inference phase.

[0024] Figure 2 This is a schematic diagram of the structure and data flow of the EGM of the present invention.

[0025] Figure 3 This is a schematic diagram of the structure and multi-view consensus filtering of the ECM of the present invention.

[0026] Figure 4 This is a schematic diagram illustrating the detection effect of the present invention in complex weather scenarios. Detailed Implementation

[0027] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0028] This invention proposes a cognitively compensated object detection framework (CCOD). Figure 1As shown, the framework simulates the "experience generalization-environmental compensation" process in human visual cognition, dividing complex weather target detection into two interconnected processes: experience generalization in the training phase and environmental compensation in the inference phase. It also sets up an experience generalization module (EGM) and an environmental compensation module (ECM).

[0029] The training process revolves around Figure 1 The upper part of the process is executed as follows: First, a single clear source domain image and its target category and bounding box annotations are acquired; then, the source domain image is input into the Empirical Generalization Module (EGM) to generate a degraded training view with weather physical meaning and frequency domain differences; next, the original source domain image and the degraded view are fed into the CLIP-RN101 backbone network, which shares a visual backbone network, to extract the original image features and degraded features; then, the target category and bounding box are output through object center gating, Region Proposal Network (RPN), ROI alignment layer, and detector head; finally, the detector is jointly optimized based on the detection loss and weather-invariant object constraints to make the object representation in the degraded view closer to the stable object representation in the source image of the same scene.

[0030] The reasoning process revolves around Figure 1 The second half of the process is executed. For unlabeled target weather images, the Environment Compensation (ECM) module first performs weather routing to determine whether the current image is in a foggy, rainy, nighttime, or rainy night state. Then, an environmental reference view is generated based on the weather type, and a multi-view set is formed by combining the original image, reference view, scale-transformed view, and horizontally flipped view. Subsequently, the shared detector obtained from training is reused to detect each view, and the candidate boxes are back-mapped to the target original image coordinate system. Finally, reliable predictions are selected based on cross-view category consistency and spatial location support, and the final target category, confidence score, and bounding box are output through non-maximum suppression.

[0031] This invention provides a cognitively compensated target detection system, comprising a data processing module, an experience generalization module (EGM), a shared detection module, an environment compensation module (ECM), and a detection output module. The data processing module reads source domain training images, detection annotations, and target images. The experience generalization module generates complex weather training experience and constructs weather-invariant constraints. The shared detection module performs feature extraction, region proposal, ROI alignment, and detection head prediction. The environment compensation module generates environmental reference views, multi-view candidates, and cross-view consensus. The detection output module performs reliable prediction screening, non-maximum suppression, and quality backoff.

[0032] The following embodiments are used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Unless otherwise specified, the technical features in the embodiments can be combined with each other. The source domain, as referred to herein, is a single image distribution accessible during the training phase and having detection annotations; the target domain is a set of images inaccessible during the training phase and having unknown weather distributions; the "environmental reference view" is a complementary observation view obtained by performing limited corrections based on the current weather state of the target image.

[0033] (1) Overall architecture and data definition like Figure 1 As shown, the method of the present invention includes a training phase and an inference phase. The training phase uses a single source domain dataset, wherein, This represents the i-th sharp image in the source domain. This represents the corresponding detection annotation. Detection annotations include at least the target category and bounding box, and can be represented as follows: The image tensor can be represented as 3×H×W, and during batch training, it is represented as B×3×H×W.

[0034] The target weather image is denoted as It does not participate in model optimization during the training phase, nor does it provide target domain annotations. The goal of this invention is to enable the detector to generalize to unknown weather images such as fog, rain, nighttime, and nighttime rain, using only source domain labeled data for training. Target categories can include buses, bicycles, cars, motorcycles, pedestrians, cyclists, and trucks, and can also be extended to categories such as traffic signs, cones, and obstacles depending on the actual scene.

[0035] The shared detector comprises a visual backbone network, an object center gating system, a Region Proposal Network (RPN), a ROI Align layer, and a detection head. The visual backbone can employ CLIP-RN101, a residual network, a visual Transformer, or other networks with equivalent feature extraction capabilities. The object center gating system enhances the response of target-related regions; the RPN generates candidate regions; ROIAlign aligns candidate regions to fixed-scale object features; and the detection head ψ outputs class scores, confidence scores, and bounding box regression values.

[0036] Training phase Figure 1 The data stream consists of: source domain clear images. On the one hand, it serves as a stable cognitive reference; on the other hand, it is used as input to EGM to generate weather degradation images. After entering the shared visual backbone, both images obtain the source image features respectively. and degraded image features .in, Used to provide a stable reference in the same scene for stopping gradients. This is used for detection training and object representation learning. Backbone features are input into the RPN to generate candidate regions, and also into object center gating to enhance target-related responses. Subsequently, ROI alignment is performed by combining the candidate regions, and the detection head ψ outputs candidate boxes, categories, and confidence scores. During training, the shared detector parameters are updated jointly by the detection loss and weather-invariant object constraints.

[0037] The data stream in Figure 1 during the inference phase is: weather image of the target to be detected. First, access ECM to complete the weather routing and obtain the weather type. ∈ {foggy day, rainy day, night, rainy night}; ECM accordingly Visibility compensation matching the weather type is applied—for foggy days, dehazing restoration based on dark channel priors is used, that is, atmospheric light is estimated from a small number of the brightest pixels in the image, and transmittance is estimated from the dark channel and truncated to a lower limit to solve the fog-free image. For nighttime, gamma correction with an exponent of less than 1 is used to enhance the response in dark areas. For rainy nights, contrast stretching centered on the channel mean is superimposed on the gamma correction, and only this contrast stretching is applied to rainy days—and the compensation result is linearly interpolated with the original image according to a configurable compensation intensity, thereby suppressing weather degradation while avoiding over-enhancement that introduces artifacts. Moreover, this compensation only applies to the input image and does not change any parameters of the detector. Then, ECM uses the original image as a reference and combines several environmental reference views, scale-transformed views, and horizontally flipped views with different compensation intensities to form a candidate view set. Each view is sequentially input into the same set of weight-shared backbone network and detector head to complete forward inference. Subsequently, the predicted boxes of each view are uniformly mapped back to the original image coordinate system through inverse flipping (mirroring back to the original direction along the view width) and inverse scaling (restoring according to the width and height ratio of the view and the original image and truncating to the image boundary). Under this unified coordinate system, the pairwise intersection-union ratio (IU) of the original image detection box and the detection boxes of each reference view is calculated. Only when the categories are consistent and the IU is not lower than the support threshold is it recorded as a valid support. The support is calculated according to the number of different views that support it. Detection boxes with a support score not lower than the preset minimum value are judged as reliable. For reliable boxes, weighted box voting is performed with confidence score as the weight, and the coordinates are corrected according to the fusion coefficient, while the confidence score is increased. For boxes that do not receive sufficient support and whose confidence score is lower than the preset upper limit, the weight is reduced by the attenuation coefficient. Finally, after non-maximum suppression and number truncation, the detection head and post-processing output the final target category, confidence score and bounding box.

[0038] From a cognitive perspective, Figure 1 The upper part corresponds to the process by which humans form stable object concepts through various weather experiences. Figure 1The lower part corresponds to the process by which humans change their observation methods, perform local compensation, and verify the target when faced with damaged observations. This training-inference closed loop is a key technical feature that distinguishes this invention from ordinary data augmentation or simple testing-based augmentation.

[0039] (2) Experience Generalization Module (EGM) like Figure 2 As shown, the Experience Generalization (EGM) module is used to build complex weather experiences for the detector and learn stable object representations without using target domain data. This module is not a simple random data augmentation, but is executed in the order of "frequency domain weather perturbation - physical weather degradation simulation - shared detector feature extraction - weather invariant constraints", so that the model can form generalizable cognitive priors for complex weather such as fog, rain, night, and night rain during the training phase.

[0040] Specifically, a source-domain sharp image is denoted as... EGM firstly Normalization is performed, and a two-dimensional discrete Fourier transform is calculated on its image tensor I to obtain the amplitude spectrum A and the phase spectrum φ: Î=F(I)=A⊙e^(jφ) The low-frequency components mainly correspond to image brightness, color, and large-scale contrast, while the high-frequency components mainly correspond to structural information such as target edges, texture details, rain streaks, and local occlusions. By adjusting the low-frequency and high-frequency components separately in the frequency domain, EGM can reduce the model's dependence on the intrinsic spectral statistics of clear daytime images and provide a richer appearance basis for subsequent physical weather degradation.

[0041] Subsequently, EGM constructs a low-frequency mask. and high-frequency mask The low-frequency gain is determined based on the sampled weather type and intensity. High-frequency gain and phase noise standard deviation The perturbations in the amplitude and phase spectra can be expressed as:

[0042]

[0043] in, For image The frequency domain perturbation image is obtained by perturbing the amplitude spectrum A′ and the phase spectrum φ′ and then performing an inverse Fourier transform.

[0044] In the formula, ⊙ represents element-wise multiplication; low-frequency mask and high-frequency mask Determined by the distance from a frequency point to the center of the spectrum: After centering the spectrum, let d be the distance from any frequency point to the center of the spectrum, and take the reference radius R as 0.5 × min(H, W). When d is not greater than The position of ×R is set to 1, and the other positions are set to 0. When d is not less than The position of ×R is set to 1, and the rest are set to 0. For low-frequency radius ratio, It is the high-frequency radius ratio, and Greater than Phase noise Only through It acts on the high-frequency region, thereby perturbing the phase relationship of edges and textures without destroying the low-frequency brightness structure.

[0045] Low frequency gain High-frequency gain and phase noise standard deviation The weather intensity is not learned through a network, but is determined by the sampled weather type w and weather intensity s. EGM first samples the weather type w from clear, foggy, nighttime, rain, and nighttime rain according to a preset probability. Then, it uniformly samples the weather intensity s within the intensity range corresponding to that weather type, where s ranges from [0,1]. Each perturbation parameter is linearly determined by the following formula: =1+ (w)·s =1+ (w)·s = · (w)·s in, (w) and (w) represents the low-frequency gain coefficient and high-frequency gain coefficient corresponding to weather type w, respectively. (w) is the phase noise modulation coefficient. This is the standard deviation of the phase noise baseline. In one executable embodiment, it is taken as the standard deviation of the phase noise baseline. = 0.08 = 0.44 =1.0; taken at night = 0.18 = 0.16 =0.5; take on rainy days = 0.05 =+0.28、 =1.0; The value is set to 0.018; for nighttime rain, the above two perturbations are performed in series in the order of nighttime intensity min(1, 1.05s) and rainy day intensity min(1, 0.9s). When the sampled weather type is clear, s=0 is taken. = =1 and =0, the frequency domain perturbation degenerates into an identity transformation, and the training view is the clear image in the source domain. .

[0046] The sign and magnitude of the aforementioned gain coefficients correspond to the imaging mechanisms of various weather conditions: in foggy weather, high frequencies are significantly attenuated to simulate the weakening of distant edges and loss of texture caused by scattering, while low frequencies are only slightly downgraded, and the overall brightness increase is achieved by the subsequent atmospheric scattering model; at night, both low and high frequencies are slightly attenuated simultaneously to simulate the overall spectral energy decrease under low-light conditions, and phase noise is halved to avoid excessive distortion of dark area structures; in rainy weather, high frequencies are boosted to simulate directional high-frequency structures such as rain streaks, while low frequencies are slightly downgraded to reduce overall contrast. Therefore, the same set of formulas can cover different weather types and continuously changing weather intensities without the need for manual parameter tuning for specific weather conditions.

[0047] After frequency domain weather perturbation, EGM adjusts the image based on the sampled weather type. Physical weather degradation simulations were performed. For foggy weather, an atmospheric scattering model was used, utilizing air light and the transmittance t to simulate reduced contrast, weakened edges, and increased overall brightness in distant areas. Without depth annotations, an approximate depth was constructed based on the vertical position of the road image, and low-frequency smoothing noise was superimposed to simulate spatial variations in fog concentration. For nighttime weather, gamma transform, brightness scaling, desaturation, color shift, and brightness-related sensor noise were used to simulate low-light imaging. For rainy weather, a directional rain ripple layer with random length, width, tilt angle, and density was generated, motion-blurred, and superimposed on the image to simulate rain ripples, local occlusion, and reflection interference. For nighttime rain, low-light degradation and rain degradation were combined in sequence, while the intensity of individual degradation methods was appropriately reduced to avoid excessive degradation that would render targets completely unrecognizable.

[0048] After the aforementioned frequency domain perturbation and physical degradation, the EGM generates a degraded training view. : = ( ) in, (·) represents the combination of frequency domain perturbation and physical degradation transformation determined by weather type w. Since the degradation process does not change the geometric position of the target, the target category label and bounding box label of the source image can be directly inherited to the degraded image. If the brightness, visibility, or contrast of the degraded target bounding box region is lower than a preset threshold, the degradation intensity is reduced or the weather parameters are resampled to avoid generating training samples that are unlearnable for the target.

[0049] like Figure 2 As shown in the detector and feature extraction section, the original source domain image and degraded images The input is the same shared detector. The shared detector may include the CLIP-RN101 visual backbone network, the Object Center Gate (OC Gate), the Region Proposal Network (RPN), and the detection head ψ. The CLIP-RN101 backbone network extracts features from the original image at high-level feature layers, such as the res4 layer. and degradation characteristics Subsequently, the degraded view features are fed into the object center gating, RPN, and detection head, outputting the target category, confidence level, and bounding box regression results.

[0050] To prevent the detector from learning synthetic weather artifacts, EGM further constructs weather-invariant object constraints in the res4 layer. Specifically, The branch in question serves as a stable reference for the same scenario, and gradient backpropagation is stopped when calculating the weather-invariant loss. The branch in question participates in detection training and feature constraint learning. and After normalization, the cosine consistency loss between the two is calculated: =mean(1 cos( , )) In the formula, cos(·,·) represents the cosine similarity calculated after L2 normalization of Fsrc and Fdeg along the channel dimension, and mean(·) represents the average over the spatial location and batch dimension. Both are fixed operators without learnable parameters.

[0051] This loss constraint causes the object responses in the degraded view to converge toward stable object representations from the same scene source image, thereby reducing the model's dependence on accidental weather textures such as rain streaks, fogged edges, and low-light noise, and enhancing the consistency of object semantics under complex weather conditions.

[0052] During training, the detection loss is applied at least to the sampled training view. When the sampled weather type is sharp, the training view is considered a sharp image from the source domain. When the sampled weather type is fog, rain, night, or night rain, the training view is a degraded image. The joint training objective in the main embodiment can be expressed as:

[0053] in, This represents the training view obtained from the current sampling. This indicates a shared detector, where y represents the source domain detection label. This represents the loss from detection classification and bounding box regression. This indicates that the loss weight is constant regardless of the weather. In one executable embodiment, It can be set to 0.05.

[0054] In an optional embodiment that simultaneously retains the source image detection branch, the joint training objective can also be expressed as:

[0055] In one embodiment, the weather sampling probability can be set to 0.45 for clear conditions, 0.15 for fog, 0.15 for nighttime conditions, 0.10 for rain, and 0.15 for nighttime rain; the low-frequency radius ratio can be set to 0.10, and the high-frequency radius ratio can be set to 0.38; the frequency domain fusion coefficient α can be set between 0.35 and 0.65. The above parameters are only used to illustrate an executable implementation of the present invention and can be adjusted according to the dataset, camera imaging conditions, target category, and applied weather type, and do not constitute a limitation on the scope of protection.

[0056] Through the above process, EGM enables the detector to undergo various complex weather degradations with frequency domain differences and physical imaging significance during the training phase. Simultaneously, it uses object features from source images of the same scene as a stable reference, forcing object responses in degraded views to maintain semantic consistency. Thus, the detector can form complex weather cognitive priors similar to long-term human visual experience, providing a foundation for subsequent generalization of target detection in unknown weather images such as fog, rain, nighttime, and nighttime rain.

[0057] (3) Environmental Compensation Module (ECM) like Figure 3 As shown, the Environment Compensation Module (ECM) is used to perform sample-level compensation on the current target weather image during the inference phase. While the EGM can provide weather experience from the training period, local light sources, raindrop density, fog concentration, and sensor responses in real-world roads may still have residual offsets compared to the synthesized weather data from training. Therefore, the ECM does not update the complete detector parameters by default. Instead, it constructs an environmental reference view and multi-view consensus for the current image, simulating the process by which humans adjust their observation methods and verify targets based on the current environment.

[0058] The ECM first performs weather routing. Weather routing is based solely on the image to be detected. The pixel information is used as input, without relying on filenames, paths, acquisition metadata, or any upstream environmental sensing information: First, the first three color channels of the image are taken, illegal values ​​are set to zero and adaptively normalized to the [0,1] interval according to the value range, and then the longest side of the image is scaled to a preset upper limit (e.g., 128 pixels) using bilinear interpolation, which significantly reduces the computational cost while preserving global illumination and structural features; then, a luminance map and its statistics are calculated on the normalized image, including average luminance, luminance standard deviation (characterizing global contrast), average saturation, dark pixel ratio (the proportion of pixels with luminance below the preset dark pixel threshold), and the average gradient magnitude of the horizontal and vertical directions of the luminance map; then, three intermediate discrimination indicators are generated based on the above statistics—when the average luminance is not higher than the nighttime luminance threshold or the dark pixel ratio is not lower than the nighttime dark pixel ratio threshold, the nighttime indicator is established; when the average luminance is not lower than the foggy luminance lower limit, the global contrast is not higher than the foggy contrast upper limit, and the average saturation is lower than the foggy luminance lower limit, ... When the average saturation is not higher than the upper limit of foggy saturation, the foggy weather flag is established. When the average saturation is not higher than the upper limit of rainy weather saturation, the global contrast is not lower than the lower limit of rainy weather contrast, the horizontal gradient is not lower than the lower limit of rainy weather gradient, and the ratio of horizontal gradient to vertical gradient is not lower than the preset anisotropy ratio, the rainy weather flag is established. Then, the weather is merged according to priority: when the nighttime flag is established, if the rainy weather flag is also established, a rainy night is output; otherwise, nighttime is output. If the nighttime flag is not established but the foggy weather flag is established, foggy weather is output. If only the rainy weather flag is established, rainy weather is output. If none of the above conditions are met, daytime is output, which is considered a normal clear weather condition and does not trigger subsequent environmental compensation. This is how the weather type is obtained. , ∈ {foggy day, rainy day, night, rainy night}; when a batch contains multiple images, the result is only adopted if the judgment results of each image are completely consistent. Otherwise, it is judged as uncertain and environmental compensation is skipped to avoid performance loss caused by misrouting. Moreover, the above discrimination thresholds are all configurable parameters that can be uniformly adjusted according to the deployment scenario without retraining the model.

[0059] After determining the weather type, ECM generates an environmental reference view. For foggy images, the light and dark channel transmittance can be estimated, and a minimum transmittance can be set to perform limited dehazing; for nighttime images, limited brightening can be performed using a power transform with gamma less than 1; for rainy images, mild contrast enhancement can be performed around the channel mean; for nighttime rainy images, limited brightening can be performed first, followed by mild contrast enhancement. It is not considered a real, clear image, nor is it used as a source of pseudo-labels; its purpose is to provide a viewing perspective that complements the original target image.

[0060] Subsequently, ECM constructs a multi-view reference set V( The set includes at least the original image. Horizontal Flip View ), scale transformation view S ( ), Environmental Reference View And the scaled view S of the reference view ( The scale factor can take values ​​of 0.88, 1.00, and 1.12, and can be scaled up or down depending on the target size and computational resources. All views input the same shared detector. Candidate boxes were obtained respectively. ,category and confidence level .

[0061] For candidate boxes generated by scaling or horizontally flipping views, ECM maps them back to the original target image coordinate system through inverse scaling or inverse flipping. Let the merged candidate set be... For any candidate box Statistically, for other views, those with the same category and an Intersection over Union (IoU) not less than the spatial support threshold are counted. The number of candidate boxes is used to obtain the spatial support count. Only when both conditions are met Not less than the confidence threshold and Not less than the support threshold Only candidate boxes that meet the criteria are included in the reliable candidate set.

[0062] In one embodiment, It can be set to 0.35 It can be set to 0.62. It can be set to 3; for scenarios with small or sparse targets, the value can be reduced. It also incorporates a center distance constraint. For a reliable candidate set, ECM can perform weighted box voting, confidence recalibration, or nonmaximum suppression to obtain the final detection result B. c and p If isolated candidates appear only at a few low-confidence locations in different views, they are considered unstable predictions and suppressed.

[0063] ECM also includes a quality fallback mechanism. If the environmental reference view exhibits excessively high brightness saturation, excessive noise increase, abnormal contrast changes, insufficient reliable candidate count, or abnormally inflated candidate count compared to the original image, then the relevant candidates from the reference view will not be used, and the prediction results will fall back to the original image or only the geometric transformation consensus results of the original image will be used. This avoids new false detections caused by excessive dehazing, excessive brightening, or inappropriate enhancement.

[0064] In an optional embodiment, the ECM can be updated lightly online, but only minor changes are allowed to the normalized parameters of the visual language projection layer, object center gating, or detector head calibration parameters, with deviations limited by source parameter anchoring terms. Update objectives may include minimizing reliable ROI entropy, cross-view category consistency loss, bounding box consistency loss, and parameter anchoring loss. The online update is skipped when the number of reliable candidates is insufficient, the average confidence is insufficient, or the prediction entropy is too high. The main embodiment preferably employs gradient-free prediction-level environmental compensation to reduce the risk of false label drift.

[0065] By employing weather routing, environmental reference views, multi-view detection, spatial consensus, and quality backoff, ECM enables the detector to perform conservative compensation for current adverse weather images without accessing the target domain annotations, and outputs more stable target categories and bounding boxes.

[0066] (4) Training, inference configuration and effect verification In one feasible embodiment, the visual backbone uses CLIP ResNet-101, the detection architecture employs a two-stage detector in the style of Faster R-CNN, the region proposal network is RPN, and the ROI detection head uses object center gating and a detection classification regression head. The training source domain is Daytime Clear, the number of training iterations is 140,000, the base learning rate is 0.0015, and the weather-invariant loss weight lambda_inv is 0.05. The above parameters are only used to illustrate the feasibility of the invention and do not limit the selection of other backbone networks, training epochs, learning rates, or the number of categories.

[0067] When validating on the Diverse-Weather road target detection benchmark, no targets were found in the following weather domains: NightClear, Dusk Rainy, Night Rainy, and Daytime Foggy. All models were trained using only source domain annotations, and target domain annotations were used only for offline evaluation. The evaluation metric used was AP50 under the Pascal VOC 2007 protocol. The experimental results are shown in the table below.

[0068]

[0069] The results above show that, under the same single-source domain training protocol, adding either EGM or ECM alone can improve the average AP50 in the unseen weather domain. The proposed method, employing both EGM and ECM, achieves 35.82%, a 3.52 percentage point improvement over the PDOC baseline, and demonstrates higher detection performance across the four target weather domains: Night Clear, Dusk Rainy, Night Rainy, and Daytime Foggy. Particularly in the Night Rainy composite degradation scenario, the proposed method shows a significant improvement over the baseline, indicating that experience generalization during training and environmental compensation during inference have complementary effects.

[0070] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, or improvements made to the frequency domain mask shape, weather degradation function, environmental reference view generation method, multi-view combination, spatial consensus threshold, detector backbone network, or lightweight online update parameters within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A cognitive compensation visual perception method for generalization from a single source domain in complex environments, characterized in that, Includes a training phase and an inference phase: The training phase includes: acquiring a single clear source domain image and its detection annotation; inputting the source domain image into the empirical generalization module to generate a degraded training view containing frequency domain perturbations and physical weather degradation; The source image and the degraded training view are input into a shared detector to extract source image features and degraded image features, respectively. The parameters of the shared detector are jointly optimized based on detection loss and weather-invariant object constraints, whereby the weather-invariant object constraints cause the degraded image features to converge with the source image features of the same scene. The inference stage includes: for the weather image of the target to be detected, the environment compensation module performs weather routing to determine the weather type and generates an environment reference view; constructs a multi-view set containing the original image, the environment reference view and its geometric transformation view; inputs each view in the multi-view set into the trained shared detector to obtain candidate boxes for each view and maps them back to the original image coordinate system; filters reliable candidate boxes based on cross-view category consistency and spatial location support, and outputs the final detection result after non-maximum suppression.

2. The method according to claim 1, characterized in that, The specific steps by which the experience generalization module generates the degenerate training view include: Perform a two-dimensional discrete Fourier transform on the source domain image to obtain the amplitude spectrum A and the phase spectrum φ: Constructing a low-frequency mask and high-frequency mask The low-frequency gain is determined based on the sampled weather type w and weather intensity s. High-frequency gain and phase noise The amplitude spectrum and phase spectrum after perturbation are expressed as follows: In the formula, ⊙ represents element-wise multiplication; The standard deviation of phase noise; Represents a normal distribution; low-frequency mask and high-frequency mask Determined by the distance from a frequency point to the center of the spectrum: After centering the spectrum, let d be the distance from any frequency point to the center of the spectrum, and let R be the reference radius. When d is not greater than The position of ×R is set to 1, and the other positions are set to 0. When d is not less than The position of ×R is set to 1, and the other positions are set to 0; =1+ (w)·s =1+ (w)·s = · (w)·s in, (w) and (w) represents the low-frequency gain coefficient and high-frequency gain coefficient corresponding to weather type w, respectively. (w) is the phase noise modulation coefficient, and the phase noise... , The standard deviation of phase noise is the reference value. The frequency domain perturbation image is obtained by performing an inverse Fourier transform on the perturbed spectrum and then fused with the original image. Physical weather degradation simulations are performed on the fused images based on the sampled weather types, including fog, rain, night, and night rain.

3. The method according to claim 1, characterized in that, The weather-invariant object constraint is as follows: In the high-level feature layer of the shared detector, after normalizing the source image features and the degraded image features, a cosine consistency loss is calculated. The branch containing the source image features stops gradient backpropagation when calculating this loss. The loss function is: =mean(1 cos( , )) In the formula, cos(·,·) represents the cosine similarity calculated after L2 normalization of the source image feature Fsrc and the degraded image feature Fdeg along the channel dimension, and mean(·) represents the average over the spatial location and batch dimension.

4. The method according to claim 1, characterized in that, The specific method by which the environmental compensation module performs weather routing is as follows: the weather type is determined based on the image's average brightness, brightness quantiles, contrast, dark channel response, and local high-frequency energy.

5. The method according to claim 1, characterized in that, The specific method by which the environmental compensation module generates the environmental reference view is as follows: for foggy images, limited dehazing is performed; for nighttime images, limited brightening is performed; for rainy images, slight contrast enhancement is performed; for nighttime rainy images, limited brightening is performed first, followed by slight contrast enhancement.

6. The method according to claim 1, characterized in that, The multi-view set includes at least the original image, a horizontally flipped view, a scale-transformed view, the environment reference view, and a scale-transformed view of the environment reference view; The scaling factor of the scaling transformation is selected from at least one of 0.88, 1.00, and 1.

12.

7. The method according to claim 1, characterized in that, The specific method for filtering reliable candidate boxes based on cross-view category consistency and spatial location support is as follows: After mapping the candidate boxes of each view back to the original image coordinate system, for any candidate box, count the number of candidate boxes in other views that have the same category and whose intersection-union ratio is not less than the spatial support threshold, and obtain the spatial support number. Only candidate boxes that simultaneously satisfy both a confidence level not less than the confidence threshold and a spatial support number not less than the support threshold are included in the reliable candidate box set.

8. The method according to claim 1, characterized in that, The environmental compensation module also has a quality rollback mechanism: if the environmental reference view has at least one of the following conditions relative to the original image: excessively high brightness saturation ratio, excessive noise increase, abnormal contrast change, insufficient number of reliable candidates, or abnormal expansion of the number of candidates, then the relevant candidates of the reference view will be abandoned, and the prediction result of the original image will be rolled back or only the consensus result of the geometric transformation of the original image will be used.

9. The method according to claim 1, characterized in that, The shared detector includes a visual backbone network, an object-centered gating system, a region proposal network, an ROI Align layer, and a detection head; the visual backbone network adopts CLIP-RN101, a residual network, or a visual Transformer.

10. A cognitive compensation visual perception system employing the method according to any one of claims 1-9, characterized in that, include: The data processing module is used to read source domain training images, detection annotations, and target images to be tested; The experience generalization module is used to generate a degraded training view that includes frequency domain perturbations and physical weather degradation, and to construct weather-invariant object constraints. A shared detection module is used to perform feature extraction, region proposal, ROI alignment, and detection head prediction; The environmental compensation module is used to perform weather routing, generate environmental reference views, construct multi-view sets, and perform cross-view consensus filtering during the inference phase. The detection output module is used to perform reliable prediction screening, nonmaximum suppression, and quality backoff.

Images