Pairwise image data based rain day target detection method with knowledge distillation

Abstract

The application discloses a rain day target detection method based on paired image data generation and knowledge distillation, and belongs to the technical field of automatic driving target detection. The method comprises the following steps: acquiring a real image of a road scene and a semantic segmentation mask; inputting the semantic segmentation mask and weather condition text information into an improved diffusion model trained to obtain a generated image of the road scene under rain condition; introducing a CoT module and an ODConv module into a target detection model for optimization, and then improving the target detection model by using a dynamic weighted aggregation loss function to obtain an optimized target detection model; performing knowledge distillation to obtain a rain day target detection enhancement model; and performing target detection by using the rain day target detection enhancement model. The method provided by the application performs well in a rain day scene, has good generalization ability, can provide higher-precision target detection in various complex scenes, and effectively improves the detection accuracy and stability.

Description

Technical Field

[0001] The present application belongs to the technical field of autonomous driving target detection, and specifically relates to a rainy day target detection method based on paired image data generation and knowledge distillation. Background Art

[0002] ODC (Operational Design Conditions) refers to the specific conditions and limitations under which an autonomous driving system can operate safely and reliably. These conditions cover factors such as the vehicle's operating environment, weather conditions, road type, traffic conditions, vehicle status, and the driver or passenger status, and are an important basis for the design of an autonomous driving system. The core of ODC is to clarify the system's capability boundaries in different operating scenarios and optimize the system's stability and safety based on these boundaries. Therefore, the expansion of ODC boundaries generated based on unknown scenarios has become an important technical means to improve the robustness of the system. By generating a variety of severe weather scenario data and simulating complex driving environments, this technology can expand the diversity of the training data set, thereby improving the performance of the target detection model under different weather conditions and the adaptability of the system.

[0003] In practical applications, adverse weather conditions pose a major challenge in the field of autonomous driving perception. While traditional object detection algorithms such as YOLOv5 and YOLOv8 perform well under ideal conditions, image quality is severely impacted in adverse weather conditions such as rain, fog, and snow. Weather factors often cause image blur and reduced contrast. Rainy days are particularly severe, as raindrops or dense rain lines on glass surfaces interfere with vision, significantly degrading object detection performance. Traditional object detection algorithms lack adaptability to these complex scenarios, and existing methods are generally unable to effectively handle image degradation caused by weather changes in rainy weather. Furthermore, in real-time applications such as autonomous driving, while performing restoration operations such as image deraining prior to object detection can improve detection results, the high computational overhead makes it difficult to meet the requirements of real-time detection. Furthermore, the scarcity of labeled data in adverse weather conditions further limits the generalization ability of object detection models in such scenarios.

[0004] Prior art 1:

[0005] Existing technology 1 uses style transfer technology to enhance the performance of target detection models under dense fog weather conditions. Using the drone dataset Visdrone2019, combined with the style transfer model and the atmospheric scattering model, a dataset with multiple styles and different fog concentrations was constructed. By extracting the gradient features, dark channel features, and wavelet features of the image, and using the support vector machine (SVM) to classify the fog concentration of the drone images, the images are divided into three categories: no fog, light fog, and dense fog. Different processing strategies are adopted for different categories of images: fog-free and light fog images can be directly used for target detection or defogging before detection, while dense fog images are processed using style transfer to change the image style without changing the content, forcing the target detection model to change from relying on the texture and surface information of the target object to learning the contour information of the target object.

[0006] One shortcoming of existing technologies is the efficiency of their processing. The need to perform style transfer on foggy images before performing object detection adds an additional computational step. Style transfer itself is a computationally intensive process, requiring significant time and resources. This two-stage process significantly increases the overall object detection time. This delay becomes a significant obstacle to practical application, particularly in scenarios requiring real-time or near-real-time processing.

[0007] Prior art 2:

[0008] The second existing technology improves detection performance in hazy environments by introducing feature enhancement and adaptively guided feature fusion. Specifically, the framework consists of two main parts: a detection branch and a feature guidance branch. The detection branch adopts a common one-stage target detection architecture, consisting of a backbone network, a feature extraction module (DFEM), and a detection head. The downsampling feature enhancement module (DFEM) combines multi-branch downsampling operations and reparameterization techniques to prevent the loss of low-level feature information during image downsampling, and enhances feature extraction capabilities by introducing a reparameterized Squeeze-Excitation module. At the same time, the adaptive feature fusion module (SAFFM) uses a prior knowledge (PK) component to achieve dynamic adjustment and fusion of features. During the training phase, the DFEM module enhances the ability to capture potential information under hazy conditions through multi-scale feature extraction, while the SAFFM module uses a prior knowledge component to adaptively fuse and adjust the input features, allowing multi-scale features to self-adjust under different conditions, thereby improving the accuracy of target detection.

[0009] Disadvantages of Technique Two: Although it performs well in smoggy environments, it also has some obvious drawbacks. First, this model is mainly optimized for smoggy conditions and uses a prior knowledge component specific to foggy days to guide feature fusion, but its generalization ability in rainy scenes is weak, and the model needs to be re-adjusted or optimized to adapt to different environments. Second, due to the introduction of complex modules: Down-Sampling Feature Enhancement Module (DFEM) and Self-Adaptive Feature Fusion Module (SAFFM), these modules, although they improve detection accuracy, increase computational complexity and slow down inference speed. Especially in real-time processing application scenarios, Technique Two cannot meet the real-time detection requirements.

[0010] Existing Technique Three:

[0011] The Rectified Cross-Attention (RCA) module is introduced and embedded into the pre-trained text-to-image diffusion model to effectively combine the input layout and image generation process. Through this module, the spatial relationship between the layout and the text semantics is accurately captured, ensuring that the semantic content of the text description only affects the area specified by the layout, achieving spatial alignment. On this basis, the model can generate high-fidelity images based on the conditional input of layout and text, and support the addition of new objects, attributes and styles during generation. The RCA module performs in the cross-attention layer, which adjusts the attention distribution to tightly integrate the layout and text semantics, effectively ensuring the accurate matching of the generated results and the layout. This method utilizes the generation ability of the pre-trained large-scale language-image diffusion model to achieve more free and high-fidelity image generation, especially in unseen scene combinations, the model can also better follow the given layout constraints to generate images.

[0012] Disadvantages of Existing Technique Three: Although Technique Three performs well in semantic and layout alignment, it still has some shortcomings in style generation. The RCA module mainly focuses on semantic and spatial alignment, and the control of style details is weak, resulting in a lack of consistency in the generated images in terms of style, which cannot fully meet the requirements of specific styles. Summary of the Invention

[0013] The purpose of the embodiments of the present application is to provide a rainy day target detection method based on paired image data generation and knowledge distillation, which aims to effectively improve the detection accuracy of the model under rainy day conditions without sacrificing real-time performance, and at the same time optimize data utilization efficiency to cope with the challenges brought by data scarcity, thereby solving the problem that the existing YOLOv8 algorithm often leads to a decline in target recognition performance due to the decline in image quality under weather conditions such as rain, fog, and snow. In addition, existing solutions usually rely on performing image restoration operations such as image defogging and rain removal before performing target detection. Although this can improve the detection effect, the processing speed is slow and it is difficult to meet application scenarios with high real-time requirements such as autonomous driving. In addition, it is difficult to obtain labeled data in bad weather, and the scarcity of training data further limits the generalization ability of the model.

[0014] In order to solve the above technical problems, this application is implemented as follows:

[0015] The embodiment of the present application provides a method for rainy day target detection based on paired image data generation and knowledge distillation, which is characterized by comprising the following steps:

[0016] Step S1, obtaining real images of road scenes under sunny and rainy conditions and semantic segmentation masks corresponding to the real images of road scenes from a real dataset with semantic segmentation mask annotations;

[0017] Step S2, inputting the real image of the road scene, the semantic segmentation mask, and the weather condition text information into the improved diffusion model for training, and after the training is completed, obtaining a trained improved diffusion model;

[0018] Step S3, inputting the semantic segmentation mask and the weather condition text information into the trained improved diffusion model to obtain a generated image of the road scene under rainy conditions;

[0019] Step S4, integrating the real road scene image and the generated road scene image into a new data set;

[0020] Step S5: pre-training the optimized target detection model based on the new data set. The optimized target detection model is optimized using the following method:

[0021] The CoT module and ODConv module are introduced into the target detection model for optimization, and then the bounding box loss function of the target detection model is improved by using the dynamic weighted aggregation loss function;

[0022] Step S6: After pre-training, the optimized object detection model is combined with the new dataset to perform paired image knowledge distillation to obtain an enhanced model for rainy day object detection;

[0023] Step S7: Perform target detection using the rainy day target detection enhancement model.

[0024] Optionally, in step S1, the real data sets include Wilddash2, IDDAW, Cityscapes, ACDC, BDD100K and Wilddash2.

[0025] Optionally, step S1 further includes: unifying the format of the real images of the road scenes in the real data set.

[0026] Optionally, in step S5, the target detection model includes a backbone network, a feature fusion network and a task head, the CoT module is embedded in the low-level convolutional layer of the backbone network to capture detailed features in the image; the ODConv module is introduced into the high-level convolutional layer of the backbone network to process complex semantic features, thereby enhancing the representation capability of high-level features. In addition, the ODConv module is also integrated in the feature fusion network to optimize the fusion of multi-scale features.

[0027] Optionally, the dynamic weighted aggregation loss function improves the bounding box regression performance of the target detection model in complex scenarios by introducing a dynamic weighting mechanism of target scale and position information.

[0028] Optionally, in step S5, the target detection model is a YOLOv8 target detection model; and the optimized target detection model is a COTD-YOLOv8 target detection model.

[0029] Optionally, in step S6, the optimized object detection model is combined with the new dataset to perform paired image knowledge distillation, specifically including:

[0030] In the pre-training phase, Co-DETR is used as the teacher model and trained on real images of road scenes under sunny conditions from the new dataset to generate accurate category probability distributions and bounding box predictions. The optimized object detection model is used as the student model and pre-trained on real images of road scenes under sunny conditions and generated images of road scenes under rainy conditions from the new dataset.

[0031] During the knowledge distillation phase, the teacher model is trained using real images of road scenes under sunny conditions to generate high-quality soft labels, which include category probability distribution and bounding box regression. The student model is trained using images generated by road scenes under rainy conditions and generates its own soft and hard predictions. The soft labels generated by the teacher model are then used to guide the training of the student model.

[0032] Optionally, use the soft labels generated by the teacher model to guide the training of the student model, including:

[0033] The student model measures the difference between its soft predictions on the generated images of road scenes under rainy conditions and the soft labels generated by the teacher model on the real images of road scenes under sunny conditions through distillation loss and similarity-weighted distillation loss;

[0034] The student model completes training by comparing hard predictions with true hard labels and calculating detection loss.

[0035] The beneficial effects of the embodiments of the present application are:

[0036] 1. This invention realizes the function of generating rainy day images based on semantic masks and text descriptions, expands the training data of the model, and effectively solves the problem of insufficient annotated data. Secondly, the CoT module and ODConv module are introduced for model optimization, which significantly enhances the feature extraction capability of the target detection model (YOLOv8) in complex environments, especially in rainy conditions. Specifically, the CoT module enhances the context perception capability of the model. Through static context encoding and dynamic self-attention mechanism, the model can accurately capture image features under complex conditions such as low contrast and blur. The ODConv module improves the extraction capability of high-level semantic features through a multi-dimensional convolution adaptive mechanism, further enhancing the robustness and feature expression capability of the model.

[0037] 2. The dynamic weighted aggregation loss function proposed in this paper improves the bounding box regression performance of the object detection model in complex scenes by introducing a dynamic weighting mechanism for target scale and position information;

[0038] 3. The optimized target detection module (CTOD-YOLOv8) provided by the present invention retains the lightweight design of CSPNet. By optimizing the convolution operation, it ensures the high efficiency of the model and the controllability of computational overhead, making it suitable for scenarios such as autonomous driving with high real-time requirements.

[0039] 4. This paper optimizes the target detection capability of the student model (COTD-YOLOv8) by combining paired sunny and rainy images and using knowledge distillation. Due to the high-precision detection capability of the teacher model (Co-DETR) on sunny images, the student model can learn more detailed and accurate target features, thereby significantly improving the detection performance under adverse weather conditions such as rainy days.

[0040] 5. The method provided by the present invention performs well in rainy scenes, has good generalization ability, and can provide higher-precision target detection in various complex scenes, effectively improving the overall detection accuracy and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which:

[0042] Figure 1 Flowchart of the rainy day target detection method based on paired image data generation and knowledge distillation provided in this application;

[0043] Figure 2 Schematic diagram of the generation process of the new dataset provided for this application;

[0044] Figure 3 The network structure diagram of the optimized target detection model provided in this application;

[0045] Figure 4 Schematic diagram of the target detection module training process provided in this application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0047] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0048] See Figure 1 As shown, the embodiment of the present application provides a method for rainy day target detection based on paired image data generation and knowledge distillation, comprising the following steps:

[0049] Step S1, obtaining real images of road scenes under sunny and rainy conditions and semantic segmentation masks corresponding to the real images of road scenes from a real dataset with semantic segmentation mask annotations;

[0050] Step S2, inputting the real image of the road scene, the semantic segmentation mask, and the weather condition text information into the improved diffusion model for training, and after the training is completed, obtaining a trained improved diffusion model;

[0051] Step S3, inputting the semantic segmentation mask and the weather condition text information into the trained improved diffusion model to obtain a generated image of the road scene under rainy conditions;

[0052] Step S4, integrating the real road scene image and the generated road scene image into a new data set;

[0053] Step S5: pre-training the optimized target detection model based on the new data set. The optimized target detection model is optimized using the following method:

[0054] The CoT module and ODConv module are introduced into the target detection model for optimization, and then the bounding box loss function of the target detection model is improved by using the dynamic weighted aggregation loss function;

[0055] Step S6: After pre-training, the optimized object detection model is combined with the new dataset to perform paired image knowledge distillation to obtain an enhanced model for rainy day object detection;

[0056] Step S7: Perform target detection using the rainy day target detection enhancement model.

[0057] In step S1, the real datasets include Wilddash2, IDDAW, Cityscapes, ACDC, BDD100K, and Wilddash2. Specifically, Wilddash2 focuses on scenes collected under complex and harsh conditions, including inclement weather such as rain, fog, and snow. IDDAW includes scenes of fog and haze, rainy days, and night driving. Cityscapes only includes urban street scene data under good weather conditions. ACDC focuses on adverse driving conditions such as rainy days, snowy days, foggy days, and nighttime conditions. BDD100K covers a variety of scenes, weather conditions, and time conditions.

[0058] Step S1 also includes unifying the formats of the real images of the road scenes in the real dataset. The so-called format unification is to unify images of different types and sizes into a standard format to obtain a more standardized dataset for subsequent use.

[0059] In step S2, combined Figure 2As shown, the improved diffusion model is a Stable Diffusion text-to-image diffusion model based on large-scale pre-training, which introduces a multimodal input and output mechanism for weather conditions and semantic segmentation labels. In the model structure, a weather condition embedding module is added to convert weather descriptions into vectors. The RCA+ module adjusts the attention map by combining semantics and weather conditions. The RCA+ module enables the model to control style generation based on the weather style description of each image while maintaining semantic and layout alignment. It introduces style description word embedding and encodes style descriptions such as "sunny" and "rainy" into style vectors using a pre-trained word embedding model. The style vector is combined with the multi-head attention mechanism of the RCA+ module as the input for style control, making the generation process not only strictly based on semantic layout but also able to integrate global style information. In the Key (K) and Value (V) matrices, the style embedding is combined with semantic features, so that each generated region maintains a consistent style while meeting semantic requirements.

[0060] In step S5, the target detection model is the YOLOv8 target detection model; and the optimized target detection model is the COTD-YOLOv8 target detection model.

[0061] Recombination Figure 3 As shown, the object detection model includes a backbone network, a feature fusion network, and a task head. The CoT module is embedded in the low-level convolutional layers of the backbone network to capture detailed features in the image. The ODConv module is introduced into the high-level convolutional layers of the backbone network to process complex semantic features, thereby enhancing the representation capability of high-level features, especially improving the detection performance of complex objects in the high-level feature extraction stage. In addition, the ODConv module is also integrated into the feature fusion network to optimize the fusion of multi-scale features.

[0062] The dynamic weighted aggregation loss function improves the bounding box regression performance of the target detection model in complex scenarios by introducing a dynamic weighting mechanism for target scale and position information. It should be further explained that in machine learning and deep learning, the loss function is a function that measures the difference between the model's predicted value and the actual value. By optimizing the algorithm, the loss function can be reduced to reduce the error of the model. The original loss function of the target detection model (YOLOv8) includes three items: bounding box loss, confidence loss, and classification loss. The total loss function formula is as follows:

[0063] L detection =L bbox +L conf +Lclass

[0064] The main bounding box loss function used in YOLOv8 and its predecessors is usually IoU (Intersection over Union) and its variants (such as GIoU, DIoU, CIoU). These loss functions have shown superior performance in target detection tasks, but in rainy weather scenes, there are still limitations such as insufficient accuracy in detecting small targets and lack of adaptability. The dynamic weighted aggregation loss function proposed in this invention focuses on improving the bounding box loss. The dynamic weighted aggregation loss function improves the bounding box regression performance of the target detection model in complex scenarios by introducing a dynamic weighting mechanism for target scale and position information. Specifically, the loss function combines the scale and position features of the target, and solves the shortcomings of the existing loss function in insufficient detection of small targets and ignoring targets on the edge of the image by adaptively allocating loss weights. The scale weighting mechanism assigns higher loss weights to small targets to ensure that they are sufficiently optimized in the model regression process. The formula is:

[0065]

[0066] Where A is the target area. The position weighting mechanism improves the detection accuracy of edge targets by assigning a greater loss weight to targets farther from the image center. The weight calculation formula is:

[0067]

[0068] Where d is the distance from the center of the object to the center of the image. The dynamic aggregation loss function that combines scale weighting and position weighting makes the model perform better in complex environments, and enhances the robustness and wide applicability of the model by adaptively adjusting the optimization strategy. The final bounding box regression loss function is:

[0069] L bbox =α·w scale ·CIoU+β·w position CIoU

[0070] Among them, α and β are adjustable weight coefficients to ensure that the model can achieve the best detection performance in different tasks and scenarios.

[0071] In step S6, combined Figure 4 As shown in the figure, the optimized target detection model is combined with the new dataset to perform paired image knowledge distillation, specifically including:

[0072] In the pre-training phase, Co-DETR is used as the teacher model and trained on real images of road scenes under sunny conditions from the new dataset to generate accurate category probability distributions and bounding box predictions. The optimized object detection model is used as the student model and pre-trained on real images of road scenes under sunny conditions and generated images of road scenes under rainy conditions from the new dataset.

[0073] During the knowledge distillation phase, the teacher model is trained using real images of road scenes under sunny conditions to generate high-quality soft labels, which include category probability distribution and bounding box regression. The student model is trained using images generated by road scenes under rainy conditions and generates its own soft and hard predictions. The soft labels generated by the teacher model are then used to guide the training of the student model.

[0074] The soft labels generated by the teacher model are used to guide the training of the student model, including:

[0075] The student model learns target features under adverse weather conditions by minimizing the difference between its soft predictions on generated images of road scenes under rainy conditions and the soft labels generated by the teacher model on real images of road scenes under sunny conditions. However, due to the difference in object changes between sunny and rainy days, it is a huge challenge to extract instance feature changes directly from the sunny teacher to the rainy student. Through analysis, it is found that the changes in instances are mainly reflected in the differences in shape and texture. To overcome this challenge, the present invention introduces an Adaptive Weighted Distillation based on Shape and Texture module in the process of migrating from the teacher model to the student model, and uses the similarity between the corresponding sunny and rainy objects as the distillation weight of the instance features. The similarity includes texture similarity and shape similarity. This module not only improves the efficiency of knowledge distillation but also improves the robustness of the model on sunny and rainy days.

[0076] The difference between the student model and the teacher model during the learning process is calculated by the distillation loss L distillation and similarity weighted distillation loss L ins To measure, the student model also calculates the detection loss L by comparing its hard predictions with the true hard labels (Ground Truth) detection , thereby improving detection performance in harsh environments such as rainy days. In this way, the student model not only learns effective features under sunny conditions from the teacher model, but can also better apply these features in complex rainy environments, thereby improving the model's robustness and detection performance.

[0077] Further, the loss function adopts a weighted combination of the standard target detection loss and the knowledge distillation loss, ensuring that the student model can optimize its performance through supervised learning and learn the features of the teacher model through distillation learning under rainy conditions. The formula is as follows:

[0078] L total = aL detection + bL distillation + gL ins

[0079] wherein, L detection is the detection loss, L distillation is the distillation loss, and L ins is the similarity weighted distillation loss. The a, b, g in the loss function are weight coefficients, used to balance the weight of extracting instance features from the sunny teacher to the rainy student, and the relationship between the performance of the student model on the real rainy day data and the knowledge learned from the teacher model through the distillation process. Finally, this weighted loss design can improve the detection performance of the student model under rainy conditions while maximizing the knowledge transfer from the teacher model under sunny conditions, so that the student model maintains high detection accuracy and robustness in complex scenarios such as rainy days.

[0080] The standard target detection loss is the improved loss function L detection mentioned above, composed of classification loss, bounding box regression loss, and confidence loss, as shown in the formula below:

[0081] L detection = L bbox + L conf + L class

[0082] The distillation loss loss function is composed of classification loss and regression loss. The formula is as follows:

[0083] L distillation = L cls + L reg

[0084] The similarity weighted distillation loss is related to the difference in similarity and texture between sunny and rainy conditions. The formula is as follows:

[0085]

[0086] wherein, S texture and S shape represent texture similarity and shape similarity, respectively, and w texture and w shape are hyperparameters representing the weights of texture similarity and shape similarity in the loss function, respectively, and N represents the corresponding pixel point sets P c and Pr The number of pixels contained in the intersection, subscripts c and r represent clear and rainy days respectively, so that the loss function will not be affected by the number of instances. texture and S shape , suppose all pixels contained in the true value object frame under sunny and rainy conditions are P s and P r , then its texture similarity S texture and shape similarity S shape The formula is:

[0087]

[0088] Among them, I c and I r Represent the instance features extracted by the model, d chamfer Indicates the chamfer distance (ChamferDistance).

[0089] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0090] Furthermore, it should be noted that the scope of the methods and systems in the embodiments of the present application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in reverse order depending on the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

[0091] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A rainy day target detection method based on paired image data generation and knowledge distillation, characterized by: The following steps are involved: Step S1, obtaining real images of road scenes under sunny and rainy conditions and semantic segmentation masks corresponding to the real images of road scenes from a real dataset with semantic segmentation mask annotations; Step S2, inputting the real image of the road scene, the semantic segmentation mask, and the weather condition text information into the improved diffusion model for training, and after the training is completed, obtaining a trained improved diffusion model; Step S3, inputting the semantic segmentation mask and the weather condition text information into the trained improved diffusion model to obtain a generated image of the road scene under rainy conditions; Step S4, integrating the real road scene image and the generated road scene image into a new data set; Step S5: pre-training the optimized target detection model based on the new data set. The optimized target detection model is optimized using the following method: The CoT module and ODConv module are introduced into the target detection model for optimization, and then the bounding box loss function of the target detection model is improved by using the dynamic weighted aggregation loss function; Step S6: After pre-training, the optimized object detection model is combined with the new dataset to perform paired image knowledge distillation to obtain an enhanced model for rainy day object detection; Step S7: Perform target detection using the rainy day target detection enhancement model.

2. The method for rainy day target detection based on paired image data generation and knowledge distillation according to claim 1, characterized in that: In step S1, the real data sets include Wilddash2, IDDAW, Cityscapes, ACDC, BDD100K and Wilddash2.

3. The method for rainy day target detection based on paired image data generation and knowledge distillation according to claim 2, characterized in that: Step S1 also includes: unifying the format of the real images of the road scenes in the real data set.

4. The method for rainy day target detection based on paired image data generation and knowledge distillation according to claim 1, characterized in that: In step S5, the target detection model includes a backbone network, a feature fusion network and a task head. The CoT module is embedded in the low-level convolutional layer of the backbone network to capture the detailed features in the image; the ODConv module is introduced into the high-level convolutional layer of the backbone network to process complex semantic features, thereby enhancing the representation capability of high-level features. In addition, the ODConv module is also integrated in the feature fusion network to optimize the fusion of multi-scale features.

5. The method for rainy day target detection based on paired image data generation and knowledge distillation according to claim 1, characterized in that: The dynamic weighted aggregation loss function improves the bounding box regression performance of the target detection model in complex scenes by introducing a dynamic weighting mechanism of target scale and position information.

6. The method for rainy day target detection based on paired image data generation and knowledge distillation according to claim 1, characterized in that: In step S5, the target detection model is the YOLOv8 target detection model; and the optimized target detection model is the COTD-YOLOv8 target detection model.

7. The method for rainy day target detection based on paired image data generation and knowledge distillation according to claim 1, characterized in that: In step S6, the optimized object detection model is combined with the new dataset to perform paired image knowledge distillation, which specifically includes: In the pre-training phase, Co-DETR is used as the teacher model and trained on real images of road scenes under sunny conditions from the new dataset to generate accurate category probability distributions and bounding box predictions. The optimized object detection model is used as the student model and pre-trained on real images of road scenes under sunny conditions and generated images of road scenes under rainy conditions from the new dataset. During the knowledge distillation phase, the teacher model is trained using real images of road scenes under sunny conditions to generate high-quality soft labels, which include category probability distribution and bounding box regression. The student model is trained using images generated by road scenes under rainy conditions and generates its own soft and hard predictions. The soft labels generated by the teacher model are then used to guide the training of the student model.

8. The method for rainy day target detection based on paired image data generation and knowledge distillation according to claim 7, characterized in that: The soft labels generated by the teacher model are used to guide the training of the student model, including: The student model measures the difference between its soft predictions on the generated images of road scenes under rainy conditions and the soft labels generated by the teacher model on the real images of road scenes under sunny conditions through distillation loss and similarity-weighted distillation loss; The student model completes training by comparing hard predictions with true hard labels and calculating detection loss.

Images