A target detection method and device in rain and fog weather based on multi-dimensional feature joint learning

Abstract

This invention belongs to the field of object detection and discloses a method and apparatus for object detection in rainy and foggy weather based on multi-dimensional feature joint learning. The method includes: first, performing color channel separation on rainy and foggy images and extracting features using an edge perception module; simultaneously, performing high-frequency filtering on the image and extracting features using a residual density module. Then, using a joint learning module on the two types of features, a feature map with strong weather-independent representation is obtained. Finally, the feature map is fed into a first-stage object detection model, and the model is optimized and improved to achieve real-time object detection in rainy and foggy weather. This invention combines traditional image processing methods with deep learning methods to extract weather-independent features from object detection images in rainy and foggy conditions, effectively utilizing original image information and improving feature robustness.

Description

Technical Field

[0001] This invention belongs to, but is not limited to, the field of target detection technology, and particularly relates to a target detection method and device based on multi-dimensional feature joint learning in rainy and foggy weather. Background Technology

[0002] Object detection is a crucial research area in computer vision, involving the task of locating all objects in a given image and classifying each object. Object detection has wide applications in object tracking, autonomous driving, image retrieval, human-computer interaction, and medical image analysis. With the rise of deep learning, object detection has achieved significant progress. However, in adverse weather conditions such as rain and fog, low visibility, insufficient lighting, and line-of-sight obstruction alter the external features of the target object under the camera's view, greatly increasing the difficulty of detection.

[0003] Rain and fog are two common natural phenomena, but they differ. Fog is composed of opaque particles such as dust and smoke, which blur the images captured by a camera, reducing image quality. Rain, on the other hand, consists of small or large water droplets, which not only blur the images captured by the camera but also obscure them. Furthermore, when these water droplets disperse in the air and fall onto the camera, they refract light, causing distortion and partial obstruction of the target image. These two different natural phenomena cause different kinds of interference to image detection, but they often occur simultaneously.

[0004] To improve target detection accuracy in rain and fog, researchers typically employ the following three strategies:

[0005] The first method is transfer learning. The object detection model trained on large-scale general datasets such as MS-COCO and ImageNet will reduce the domain difference between the source domain (clear image) and the target domain (rainy and foggy image) through domain adaptation, thereby achieving object detection in rainy and foggy weather.

[0006] The second step is to first remove the fog or rain from the image containing rain and fog to obtain a clearer image without rain and fog, and then perform target detection.

[0007] Third, the processing of rain and fog is directly integrated into the deep target detection model to obtain weather-independent target features, achieving end-to-end detection.

[0008] Based on the above analysis, the urgent technical problems that need to be solved in the existing technology are:

[0009] Existing real-world rain and fog datasets are scarce, while artificially synthesized data deviates from real-world scenes in terms of physical parameters. Existing domain adaptation methods focus on feature alignment between two different domain distributions, while ignoring the potential information in the weather-based image restoration process.

[0010] Existing defogging and deraining preprocessing networks have complex structures and high computational overhead. As a preprocessing part of target detection, they affect detection speed and limit application scenarios.

[0011] Existing rain and fog target detection methods can only handle one type of rain or fog at a time, and a single model is difficult to handle different intensities of weather such as drizzle, rainstorm, light fog, and dense fog at the same time. Summary of the Invention

[0012] To address the problems existing in the prior art, this invention provides a target detection method and device for rainy and foggy weather based on multi-dimensional feature joint learning.

[0013] This invention is implemented as follows: a target detection method for rainy and foggy weather based on multi-dimensional feature joint learning, characterized in that the target detection method for rainy and foggy weather based on multi-dimensional feature joint learning specifically includes:

[0014] Step 1: Perform multi-channel color decomposition on the rain-fog image and extract channel features separately;

[0015] Step 2: Construct channel difference representations based on the global extreme value features of each color channel, and perform weighted suppression on the channel difference representations through learnable latent modulation variables to obtain a first feature representation that is weakly correlated with fog interference;

[0016] Step 3: Perform frequency domain enhancement processing on the rain-fog image to highlight high-frequency components, extract rain texture features from the high-frequency components, and eliminate the influence of rain texture from the original features by feature subtraction to obtain a second feature representation that is weakly correlated with rain interference.

[0017] Step 4: Perform joint constraint learning on the first feature representation and the second feature representation to obtain target discrimination features that are insensitive to rain and fog interference.

[0018] Furthermore, the process of constructing channel difference representations based on global extreme value features includes:

[0019] For each color channel feature, calculate the global maximum response value and the global minimum response value, and use the difference between the two to characterize the degree of illumination shift caused by fog in that channel.

[0020] Furthermore, the latent modulation variables are learned from the channel difference representation through convolutional mapping and modulated on the original channel features in an element-wise multiplication manner to suppress components related to fog distribution.

[0021] Another objective of this invention is to provide a target detection method under rainy and foggy weather, comprising the following steps:

[0022] A target detection method based on multi-dimensional feature joint learning in rainy and foggy weather is adopted to obtain rain and fog-independent target discrimination features;

[0023] Attention constraints based on channel importance allocation are applied to the target discrimination features to enhance the feature dimensions related to the target category and weaken the background dimension;

[0024] The attention-constrained features are fed into the target detection network to complete target localization and category recognition in rainy and foggy weather.

[0025] Furthermore, the attention constraint is achieved by assigning different weights to different feature channels, so that rain-independent features and fog-independent features form a complementary and reinforcing relationship in the channel dimension.

[0026] Furthermore, the target detection network employs the same rain and fog-independent feature generation mechanism in both the training and detection phases to ensure consistent feature distribution.

[0027] Another objective of this invention is to provide a target detection device for rainy and foggy weather based on multi-dimensional feature joint learning, comprising:

[0028] The fog decoupling feature generation module is used to generate a first feature representation that is weakly correlated with fog interference based on the color channel extreme value difference and potential modulation variables.

[0029] The rain decoupling feature generation module is used to generate a second feature representation that is weakly correlated with rain interference based on high-frequency feature extraction and feature subtraction mechanism;

[0030] The joint feature learning module is used to perform collaborative constraint learning on the first feature representation and the second feature representation to generate rain and fog-independent target discrimination features.

[0031] The target detection module is used to perform target detection in rainy and foggy weather based on the target discrimination features.

[0032] Furthermore, the rain decoupling feature generation module includes a frequency domain enhancement unit and a multi-scale feature learning unit. The frequency domain enhancement unit is used to highlight high-frequency information in the image, and the multi-scale feature learning unit is used to extract rain texture features from different scales.

[0033] Furthermore, the joint feature learning module adopts a global activation mechanism based on channel weight adjustment, so that the rain decoupling features and fog decoupling features form a consistent discriminative representation in the same feature space.

[0034] Furthermore, the target detection module is adapted to a one-stage target detection structure, enabling the rain and fog-independent features to be directly used in end-to-end target detection tasks.

[0035] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0036] Existing research on target detection in rainy and foggy weather mostly focuses on a single interference factor, typically designing separate rain removal or fog removal models for rainy or foggy weather. When rain and fog coexist, a single mechanism struggles to distinguish between different degradation sources, easily leading to target texture loss or background misjudgment. This invention, starting from the imaging mechanism, treats rain interference and fog interference as two heterogeneous degradation factors present in different feature spaces, constructing two independent yet co-constrained feature decoupling paths. This avoids the limitations of simply attributing complex degradation to a single factor.

[0037] Regarding fog interference suppression, this invention does not employ traditional image enhancement or explicit dehazing methods. Instead, it separates color channels and leverages the physical characteristic that different color channels exhibit varying degrees of brightness diffusion under fog conditions. It extracts the global extremum differences of each channel to characterize the impact of fog on image contrast and dynamic range. Furthermore, it introduces a learnable latent modulation mechanism to adaptively model these differences, achieving the extraction of fog-independent information at the feature level. This mechanism avoids the destruction of target edge and structural information during dehazing, ensuring that target discrimination features remain stable even under conditions of dense fog and non-uniform fog distribution.

[0038] In terms of rain interference suppression, this invention utilizes the characteristic that rain textures are mainly concentrated in the high-frequency space. It highlights high-frequency information through frequency domain enhancement and models the rain texture distribution in a multi-scale feature space. Then, it eliminates rain-related components from the original features through feature interpolation, thereby learning rain-independent features. This process is not a simple filtering, but rather combines multi-scale modeling and inversion elimination mechanisms to accurately separate rain textures while preserving the key structural and detailed information of the target.

[0039] More importantly, this invention employs a joint learning mechanism to collaboratively constrain rain-independent and fog-independent features, creating a complementary and enhancing relationship between the two types of features along the channel dimension. This results in a unified discriminative feature insensitive to both rain and fog interference. This collaboration is not a simple superposition of technical elements, but rather a complementary mechanism working together to address the same technical problem, significantly improving the stability and generalization ability of target detection under complex weather conditions. Extensive experimental results demonstrate that in mixed rain and fog scenarios, this invention achieves significant improvements in detection accuracy and robustness compared to existing technologies, exhibiting outstanding substantive features and remarkable progress.

[0040] (1) This invention solves the core pain point of outdoor visual perception under severe weather conditions, and has significant engineering practical value, industry economic value and social safety value. It can be applied to multiple fields with strict requirements for detection accuracy and reliability.

[0041] In the field of intelligent transportation, firstly, autonomous driving and assisted driving systems heavily rely on visual target detection, and rainy and foggy weather is a high-risk scenario for traffic accidents. This invention achieves high-precision, real-time detection of road targets in rainy and foggy weather, providing reliable environmental perception information for autonomous driving systems and improving the driving safety of autonomous vehicles in adverse weather conditions. It can also be applied to functions such as adaptive cruise control, emergency braking, and lane keeping, significantly reducing the traffic accident rate in rainy and foggy weather.

[0042] Secondly, traffic monitoring systems on highways, urban roads, bridges, and tunnels often experience problems such as malfunctioning monitoring images, inability to identify violations, and distorted traffic flow statistics in rainy or foggy weather. This invention can reliably detect violations such as driving in the wrong direction, speeding, illegal parking, and not following lane directions in rainy or foggy weather, as well as accurately calculate traffic flow and congestion levels. It helps traffic management departments achieve intelligent traffic control around the clock and in all weather conditions, improving road traffic efficiency.

[0043] In the field of security monitoring, scenarios such as communities, industrial parks, airports, train stations, and border areas require 24 / 7 uninterrupted target monitoring. Rain and fog can blur monitoring images, leading to missed or false alarms regarding intrusions, left-behind items, and abnormal behavior. This invention effectively suppresses rain and fog interference, accurately identifies targets such as people, vehicles, and prohibited items, and triggers alarms promptly. It ensures reliable security monitoring in critical areas under adverse weather conditions, reduces the cost and risk of manual inspections, and enhances public safety and security capabilities. Particularly important for outdoor work scenarios such as mines, ports, and forests, where personnel, machinery, and vehicle detection in rain and fog are crucial for safe production. This invention is adaptable to complex industrial outdoor environments, enabling real-time monitoring of unauthorized personnel entry, abnormal machinery operation, and vehicle collision risks, thus preventing safety accidents.

[0044] (2) The technical solution of the present invention can be transformed into standardized products and customized solutions, which can directly generate sales revenue.

[0045] Embedded hardware products: Lightweight algorithm models are ported to hardware such as automotive domain controllers, security cameras, drone flight control chips, ship vision terminals, and edge computing boxes, generating revenue through algorithm licensing, module sales, and complete system sales. Mass production is targeted at intelligent driving manufacturers, security equipment manufacturers, drone companies, and shipping equipment suppliers.

[0046] Software and Solutions: Providing rain and fog enhanced visual monitoring systems, autonomous driving adverse weather perception upgrade packages, and inspection solutions to traffic management departments, smart city operators, industrial parks, and port and mining areas. Revenue is generated through project contracts, annual service fees, and software licensing, along with ongoing operational and maintenance income.

[0047] Data Services and Value-Added Revenue: Leveraging deployed terminal equipment and detection systems, and under the premise of compliance and user authorization, we clean and analyze anonymized data such as traffic flow, security incidents, and equipment operating status during rainy and foggy weather. We provide weather-traffic linkage data analysis reports and safety early warning data services to traffic management, emergency management, and meteorological departments. We also provide paid data services such as scenario-based data annotation and model iteration for autonomous driving and security companies, extending the data value chain. Attached Figure Description

[0048] Figure 1 This is a flowchart of a target detection method based on multi-dimensional feature joint learning in rainy and foggy weather provided in an embodiment of the present invention;

[0049] Figure 2 This is a structural diagram of a target detection device based on multi-dimensional feature joint learning in rainy and foggy weather provided in an embodiment of the present invention;

[0050] Figure 3 This is a model training curve diagram provided in an embodiment of the present invention;

[0051] Figure 4 These are visualizations of target detection results from different models provided in embodiments of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0053] like Figure 1As shown, this embodiment of the invention provides a target detection method based on multi-dimensional feature joint learning in rainy and foggy weather. The method specifically includes:

[0054] S101, Perform multi-channel color decomposition on rain and fog images and extract channel features respectively;

[0055] S102, construct a channel difference representation based on the global extreme value features of each color channel, and perform weighted suppression on the channel difference representation through learnable latent modulation variables to obtain a first feature representation that is weakly correlated with fog interference;

[0056] S103, the rain-fog image is subjected to frequency domain enhancement processing to highlight high-frequency components, rain texture features are extracted from the high-frequency components, and the rain texture influence is eliminated from the original features by feature subtraction to obtain a second feature representation that is weakly correlated with rain interference.

[0057] S104, perform joint constraint learning on the first feature representation and the second feature representation to obtain target discrimination features that are insensitive to rain and fog interference.

[0058] This invention provides a process for constructing channel difference representations based on global extreme value features, including:

[0059] For each color channel feature, calculate the global maximum response value and the global minimum response value, and use the difference between the two to characterize the degree of illumination shift caused by fog in that channel.

[0060] This invention provides a method to learn potential modulation variables from channel difference representations through convolutional mapping and modulate the original channel features by element-wise multiplication to suppress components related to fog distribution.

[0061] This invention provides a target detection method under rainy and foggy weather, comprising the following steps:

[0062] A target detection method based on multi-dimensional feature joint learning in rainy and foggy weather is adopted to obtain rain and fog-independent target discrimination features;

[0063] Attention constraints based on channel importance allocation are applied to the target discrimination features to enhance the feature dimensions related to the target category and weaken the background dimension;

[0064] The attention-constrained features are fed into the target detection network to complete target localization and category recognition in rainy and foggy weather.

[0065] The present invention provides attention constraints by assigning different weights to different feature channels, so that rain-independent features and fog-independent features form a complementary and enhancing relationship in the channel dimension.

[0066] The embodiments of the present invention provide an object detection network that uses the same rain and fog-independent feature generation mechanism in both the training and detection phases to ensure the consistency of feature distribution.

[0067] This invention provides a target detection method based on multi-dimensional feature joint learning in rainy and foggy weather. The detailed method includes:

[0068] (1): Separate the three color channels of the rain and fog image, and extract features for each channel using multi-layer dilated convolution;

[0069] (2): The features obtained in (1) are fused together, and the global maximum and global minimum features of the three color channels are extracted respectively to obtain the feature difference; at the same time, a latent variable M is learned, and M is multiplied with the extracted features to obtain a feature map that is independent of fog;

[0070] (3): High-frequency images are obtained by applying a high-frequency filtering module to the rain-fog images. A residual dense network is designed to extract rain information features from the high-frequency images. The rain information features are subtracted from the rain-fog feature map to obtain a feature map that is unrelated to rain.

[0071] (4): Use an attention mechanism to perform joint learning of multiple features on the feature maps obtained in (2) and (3);

[0072] (5): Input the weather-independent feature map obtained in (4) into the first-stage target detection model to complete the training of the target detection model under rain and fog.

[0073] (6): Target detection in rainy and foggy weather based on the trained target detection model.

[0074] The (1) includes:

[0075] The RGB three color channels of the color image containing rain and fog are separated to obtain red, blue and green channel images respectively. Then, multiple convolutions are performed on each color channel, and a 3×3 convolution kernel with dilation is selected. After the convolution, group normalization (GN) and ReLU modules are followed. At the same time, a bypass is added after the two groups of convolutions to perform feature concatenation (concat) operation to obtain feature maps of the three color channels.

[0076] The (2) includes:

[0077] For the effective features of each color channel extracted in (1), the feature difference is obtained by extracting the global maximum and global minimum features of the three color channels. Based on the principle of fog formation, a 1×1 convolutional layer is used to learn the latent variable M, and M is multiplied with the extracted features to obtain a feature map that is independent of fog.

[0078] The (3) includes:

[0079] First, the rain-fog image is processed using a classic Gaussian high-pass filter to obtain a high-frequency map. Then, three convolutional layer branches of different sizes are used on the high-frequency map to learn rain features. Bypasses are added for feature concatenation, and an activation module is added to obtain the overall feature set. Next, the features learned from the three branches are concatenated and followed by a 1×1 convolution to obtain the rain features in the rain-fog image. Finally, the rain features are subtracted from the rain features to obtain a feature map unrelated to rain.

[0080] The (4) includes:

[0081] After obtaining feature maps that are independent of fog and independent of rain through (2) and (3) respectively, the two sets of feature maps are fused using a joint learning module. The SE (Squeeze-and-Excitation) compression-activation attention module is used to achieve multi-dimensional feature joint learning based on channel attention by assigning weights between different channels.

[0082] The SE module is a channel-attention-based global activation module that prioritizes channels by distributing weights across them. It consists of two steps: compression and activation. In the compression phase, global average pooling is used to generate channel-level statistics. :

[0083] (1)

[0084] The output for ,enter for , Indicates the first The first channel, the... The element in row j.

[0085] In the subsequent activation phase, two fully connected layers are used to capture the dependencies between channels, and a sigmoid activation function is used as a simple gating mechanism.

[0086] (2)

[0087] in, It is the sigmoid function. For ReLU function, and . This is a contraction factor that constructs a bottleneck structure in two fully connected layers, forming a gating mechanism. Finally, the output of the dense block... It is rescaled using an activation function. .

[0088] (3)

[0089] in This indicates bitwise multiplication. The weights output during the activation phase are considered as the importance of each feature channel. By multiplying, the features before each channel are weighted, thus completing the recalibration of the original features in the channel dimension.

[0090] like Figure 2 As shown in the figure, an embodiment of the present invention provides a target detection device for rainy and foggy weather based on multi-dimensional feature joint learning. The device specifically includes:

[0091] The fog decoupling feature generation module is used to generate a first feature representation that is weakly correlated with fog interference based on the color channel extreme value difference and potential modulation variables.

[0092] The rain decoupling feature generation module is used to generate a second feature representation that is weakly correlated with rain interference based on high-frequency feature extraction and feature subtraction mechanism;

[0093] The joint feature learning module is used to perform collaborative constraint learning on the first feature representation and the second feature representation to generate rain and fog-independent target discrimination features.

[0094] The target detection module is used to perform target detection in rainy and foggy weather based on the target discrimination features.

[0095] The present invention provides a rain decoupling feature generation module including a frequency domain enhancement unit and a multi-scale feature learning unit. The frequency domain enhancement unit is used to highlight high-frequency information in the image, and the multi-scale feature learning unit is used to extract rain texture features from different scales.

[0096] The embodiments of the present invention provide a joint feature learning module that employs a global activation mechanism based on channel weight adjustment, so that rain decoupling features and fog decoupling features form a consistent discriminative representation in the same feature space.

[0097] This invention provides a target detection module adapted to a one-stage target detection structure, enabling the rain and fog-independent features to be directly used in end-to-end target detection tasks.

[0098] I. Evidence related to the technical effects obtained by the embodiments of the present invention

[0099] To verify the effectiveness of the invention, the following experiment was conducted:

[0100] Validation was performed using the RTTS and RainyCityscapes datasets. The RTTS dataset contains 4322 real-world foggy images with diverse scenes and lighting conditions; the RainyCityscapes dataset contains 10620 synthetic rainy images and eight labeled object categories (cars, trains, trucks, motorcycles, buses, bicycles, cyclists, and pedestrians). Each clear image was paired with 36 variants. For uniform testing, only five categories were considered for both datasets: cars, bicycles, motorcycles, buses, and pedestrians. The training set consisted of 11000 images, with 3000 images from the RTTS dataset and 8000 images from the RainyCityscapes dataset. The test set consisted of 3942 images, with 1322 images from the RTTS dataset and 2620 images from the RainyCityscapes dataset.

[0101] Because the training and test sets of methods in other papers are inconsistent, making performance comparison impossible, we chose to compare the results with the YOLOv5s and YOLOv8s methods.

[0102] Table 1 Experimental Results

[0103]

[0104] As shown in Table 1, compared with YOLOv5s and YOLOv8s, the method of this invention achieves better performance. While maintaining processing speed, the method of this invention outperforms YOLOv5s and YOLOv8s in terms of precision, recall, and average precision (mAP) under both parameter settings.

[0105] Figure 3 The first row of columns 1 to 3 in the graph shows the three loss values ​​for the training set, and the second row shows the three loss values ​​for the validation set. It is easy to see that the model performs well after multiple rounds of training. The last two columns correspond to the model's precision, recall, mean precision of 0.5, and mean precision of 0.5:0.95, respectively.

[0106] Figure 4 A visual comparison of this model with YOLO5s and YOLO8s reveals that our algorithm is better at finding targets and achieves higher confidence scores.

[0107] Example 1: Implementation of Haze Interference Suppression Based on Color Channel Extrema Difference

[0108] In this embodiment, the input color image containing rain and fog is first decomposed into color channels to obtain single-channel images for red, green, and blue channels. For each channel, its global maximum and global minimum response values ​​are independently extracted, and the difference between them is calculated to characterize the degree of brightness diffusion and contrast compression caused by fog in that channel. Subsequently, this difference is used as input to learn a spatially correlated modulation factor through convolutional mapping. This modulation factor is multiplied element-wise with the original channel features, thereby adaptively suppressing fog-related components at the feature level and obtaining a fog-independent feature representation.

[0109] Example 2: Fog Decoupling Feature Generation Method Based on Latent Modulation Variables

[0110] Building upon Example 1, to avoid the problem of fixed rules being insufficiently adaptable to different scenarios, a latent modulation variable learning mechanism is introduced. This latent variable is obtained by convolutional mapping of channel difference features, and its value varies with spatial location, used to characterize the non-uniform distribution characteristics of fog in the image. By dynamically modulating the channel features through this latent variable, dense fog areas achieve stronger suppression while weak fog areas retain more original information. This achieves implicit decoupling of fog interference without explicit image dehazing, avoiding damage to target edges and textures during the dehazing process.

[0111] Example 3: Rain Texture Modeling and Elimination Based on High Frequency Enhancement

[0112] To address the characteristic of rain interference as primarily manifesting as a slender, high-frequency structure, this embodiment performs frequency domain enhancement processing on rain- and fog-containing images to highlight high-frequency components. Subsequently, a multi-scale feature extraction structure is used to model the high-frequency map, ensuring effective capture of rain textures at different scales. By fusing rain texture features at various scales, a rain feature representation with overall characterization of rain distribution is obtained. Finally, by subtracting this rain feature from the original image features, targeted elimination of rain interference components is achieved, resulting in a rain-independent feature map.

[0113] Example 4: Collaborative Constraint Learning Approach for Rain and Fog Decoupling Features

[0114] In this embodiment, fog decoupling features and rain decoupling features are simultaneously input into the joint feature learning module. This module assigns different importance weights to different feature channels, enabling the two types of features to form a complementary relationship within the same feature space, rather than simply being superimposed. Through this collaborative constraint method, fog decoupling features are used to stabilize the overall contour and structural information of the target, while rain decoupling features are used to enhance the target edges and local textures, thereby generating a unified feature representation that is insensitive to both rain and fog interference and highly sensitive to target discrimination.

[0115] Example 5: Training process for target detection based on joint features

[0116] In this embodiment, the aforementioned rain- and fog-independent features are used as input to the detection network for model training. During training, the network no longer directly learns appearance changes under rain and fog scenes, but instead performs target localization and category differentiation based on the decoupled discriminative features. Since rain and fog interference have been separated at the feature level, the model can maintain stable detection performance under different rain and fog intensities and different environmental conditions, significantly reducing sensitivity to changes in weather conditions and improving the model's generalization ability.

[0117] Example 6: Comprehensive Application Effects in Complex Weather Scenarios

[0118] In practical applications, the above method was deployed in scenarios with various complex weather conditions, including light fog, heavy fog, light rain, and heavy rain. Experiments show that through the synergistic effect of two orthogonal mechanisms—fog decoupling and rain decoupling—the target features maintain a consistent distribution under different weather conditions, eliminating the need for separate training of the detection model for different weather conditions. This implementation demonstrates that the proposed solution is not simply a rain or fog removal preprocessing, but rather achieves structural separation of weather factors at the feature level, thereby significantly improving the detection stability and reliability under severe weather conditions while ensuring the integrity of target information.

[0119] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0120] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A target detection method based on multi-dimensional feature joint learning in rainy and foggy weather, characterized in that, Includes the following steps: Step 1: Perform multi-channel color decomposition on the rain-fog image and extract channel features separately; Step 2: Construct channel difference representations based on the global extreme value features of each color channel, and perform weighted suppression on the channel difference representations through learnable latent modulation variables to obtain a first feature representation that is weakly correlated with fog interference; Step 3: Perform frequency domain enhancement processing on the rain-fog image to highlight high-frequency components, extract rain texture features from the high-frequency components, and eliminate the influence of rain texture from the original features by feature subtraction to obtain a second feature representation that is weakly correlated with rain interference. Step 4: Perform joint constraint learning on the first feature representation and the second feature representation to obtain target discrimination features that are insensitive to rain and fog interference.

2. The method as described in claim 1, characterized in that, The process of constructing channel difference representations based on global extreme value features includes: For each color channel feature, calculate the global maximum response value and the global minimum response value, and use the difference between the two to characterize the degree of illumination shift caused by fog in that channel.

3. The method as described in claim 1, characterized in that, The latent modulation variables are learned from the channel difference representation through convolutional mapping and modulated on the original channel features by element-wise multiplication to suppress components related to fog distribution.

4. A target detection method under rainy / foggy weather, implementing the target detection method as described in any one of claims 1-3, characterized in that, Includes the following steps: A target detection method based on multi-dimensional feature joint learning in rainy and foggy weather is adopted to obtain rain and fog-independent target discrimination features; Attention constraints based on channel importance allocation are applied to the target discrimination features to enhance the feature dimensions related to the target category and weaken the background dimension; The attention-constrained features are fed into the target detection network to complete target localization and category recognition in rainy and foggy weather.

5. The method as described in claim 4, characterized in that, The attention constraint is achieved by assigning different weights to different feature channels, so that rain-independent features and fog-independent features form a complementary and reinforcing relationship in the channel dimension.

6. The method as described in claim 4, characterized in that, The target detection network employs the same rain and fog-independent feature generation mechanism in both the training and detection phases to ensure consistent feature distribution.

7. A target detection device for rainy / foggy weather based on multi-dimensional feature joint learning, implementing the target detection method as described in any one of claims 1-3, characterized in that, include: The fog decoupling feature generation module is used to generate a first feature representation that is weakly correlated with fog interference based on the color channel extreme value difference and potential modulation variables. The rain decoupling feature generation module is used to generate a second feature representation that is weakly correlated with rain interference based on high-frequency feature extraction and feature subtraction mechanism; The joint feature learning module is used to perform collaborative constraint learning on the first feature representation and the second feature representation to generate rain and fog-independent target discrimination features. The target detection module is used to perform target detection in rainy and foggy weather based on the target discrimination features.

8. The target detection device for rainy and foggy weather based on multi-dimensional feature joint learning as described in claim 7, characterized in that, The rain decoupling feature generation module includes a frequency domain enhancement unit and a multi-scale feature learning unit. The frequency domain enhancement unit is used to highlight high-frequency information in the image, and the multi-scale feature learning unit is used to extract rain texture features from different scales.

9. The target detection device for rainy and foggy weather based on multi-dimensional feature joint learning as described in claim 7, characterized in that, The joint feature learning module adopts a global activation mechanism based on channel weight adjustment, so that rain decoupling features and fog decoupling features form a consistent discriminative representation in the same feature space.

10. The target detection device for rainy and foggy weather based on multi-dimensional feature joint learning as described in claim 7, characterized in that, The target detection module is adapted to a one-stage target detection structure, enabling the rain and fog-independent features to be directly used in end-to-end target detection tasks.

Images