Rainy day target detection method based on paired image data generation and knowledge distillation

Abstract

The invention discloses a rainy day target detection method based on paired image data generation and knowledge distillation, and belongs to the technical field of automatic driving target detection. The method comprises the following steps: acquiring a real image of a road scene and a semantic segmentation mask; inputting the semantic segmentation mask and the weather condition text information into a trained improved diffusion model to obtain a road scene generation image under a rainy day condition; introducing a CoT module and an ODConv module into the target detection model for optimization, and then improving the target detection model by adopting a dynamic weighted aggregation loss function to obtain an optimized target detection model; performing knowledge distillation to obtain a rainy day target detection enhancement model; and performing target detection by using the rainy day target detection enhancement model. The method provided by the invention is excellent in performance in a rainy day scene, has good generalization ability, can provide higher-precision target detection in various complex scenes, and effectively improves the detection accuracy and stability.

Description

Technical Field

[0001] The present application belongs to the technical field of autonomous driving target detection, and specifically relates to a rainy day target detection method based on paired image data generation and knowledge distillation. Background Art

[0002] ODC (Operational Design Conditions) refers to the specific conditions and restrictions under which the autonomous driving system can operate safely and reliably. These conditions cover factors such as the vehicle's operating environment, weather conditions, road types, traffic conditions, vehicle status, and driver or passenger status, and are an important basis for the design of autonomous driving systems. The core of ODC is to clarify the system's capability boundaries in different operating scenarios and optimize the system's stability and safety based on these boundaries. Therefore, ODC boundary expansion based on unknown scenario generation has become an important technical means to improve the robustness of the system. By generating a variety of severe weather scenario data and simulating complex driving environments, this technology can expand the diversity of training data sets, thereby improving the performance of target detection models under different weather conditions and the adaptability of the system.

[0003] In practical applications, severe weather conditions are a major challenge in the field of autonomous driving perception. Although traditional target detection algorithms such as YOLOv5 and YOLOv8 perform well in ideal environments, image quality is seriously affected in severe weather conditions such as rain, fog, and snow. Weather factors often cause problems such as image blur and reduced contrast. Especially on rainy days, raindrops or dense rain lines on the glass surface interfere with vision, which significantly reduces the performance of target detection. Traditional target detection algorithms lack adaptability to these complex scenes. Existing methods are usually unable to effectively handle image degradation caused by weather changes when dealing with rainy scenes. In addition, in application scenarios such as autonomous driving that require high real-time performance, performing image deraining and other restoration operations before target detection can improve the detection effect, but the computational overhead is high and it is difficult to meet the needs of real-time detection. In addition, the scarcity of labeled data under severe weather conditions further limits the generalization ability of target detection models in such scenarios.

[0004] Prior art 1:

[0005] Prior art 1 uses style transfer technology to enhance the performance of target detection models under dense fog weather conditions. Using the drone dataset Visdrone2019, combined with the style transfer model and the atmospheric scattering model, a dataset with multiple styles and different fog concentrations was constructed. By extracting the gradient features, dark channel features, and wavelet features of the image, and using the support vector machine (SVM) to classify the fog concentration of drone images, the images are divided into three categories: no fog, light fog, and dense fog. Different processing strategies are adopted for images of different categories: fog-free and light fog images can be directly used for target detection or defogging before detection, while dense fog images are processed by style transfer to change the image style without changing the content, forcing the target detection model to change from relying on the texture and surface information of the target to learning the contour information of the target.

[0006] The first disadvantage of the existing technology is the efficiency of its processing flow. Since the foggy image needs to be style-transferred before object detection, an additional calculation step is added, and style transfer itself is a computationally intensive process that requires a lot of time and resources. This two-stage processing flow will significantly increase the time of the overall object detection task, especially in application scenarios that require real-time or near-real-time processing. This delay becomes a major obstacle to practical application.

[0007] Prior art 2:

[0008] The second prior art improves the detection performance in haze environment by introducing feature enhancement and adaptively guided feature fusion. Specifically, the framework consists of two main parts: detection branch and feature guided branch. The detection branch adopts a common one-stage target detection architecture, including a backbone network, a feature extraction module (DFEM) and a detection head. The downsampling feature enhancement module (DFEM) combines multi-branch downsampling operations and reparameterization technology to prevent the loss of low-level feature information during image downsampling, and enhances the feature extraction capability by introducing a reparameterized Squeeze-Excitation module. At the same time, the adaptive feature fusion module (SAFFM) realizes dynamic adjustment and fusion of features through a priori knowledge (PK) component. In the training stage, the DFEM module enhances the ability to capture potential information under haze conditions through multi-scale feature extraction, while the SAFFM module uses a priori knowledge component to adaptively fuse the input features, so that the multi-scale features can self-adjust under different conditions, thereby improving the accuracy of target detection.

[0009] Disadvantages of Technology 2: Although it performs well in haze environments, it also has some obvious disadvantages. First, the model is mainly optimized for haze conditions, using a priori knowledge components specific to foggy days to guide feature fusion, but the generalization ability in rainy scenes is weak, and the model needs to be readjusted or optimized to adapt to different environments. Secondly, because the model introduces complex modules: down-sampled feature enhancement module (DFEM) and adaptive feature fusion module (SAFFM), these modules improve detection accuracy, but also increase the amount of calculation and complexity, resulting in slower inference speed. Especially in application scenarios that require real-time processing, Technology 2 cannot meet the requirements of real-time detection.

[0010] Existing technology three:

[0011] The Rectified Cross-Attention (RCA) module is introduced and embedded into the pre-trained text-to-image diffusion model to achieve an effective combination of the input layout and the image generation process. Through this module, the spatial relationship between the layout and the text semantics is accurately captured, ensuring that the semantic content described by the text only affects the area specified by the layout, achieving the purpose of spatial alignment. On this basis, the model can generate high-fidelity images based on the conditional input of the layout and text, and supports adding new objects, attributes and styles during the generation process. The RCA module is executed in the cross-attention layer. It adjusts the attention distribution to closely integrate the layout with the text semantics, effectively ensuring that the generation result accurately matches the layout. This method uses the generation capability of the pre-trained large-scale language-image diffusion model to achieve more free and high-fidelity image generation, especially in the case of unseen scene combinations, the model can also better follow the given layout constraints for image generation.

[0012] Disadvantages of the existing technology 3: Although technology 3 performs well in semantics and layout alignment, it still has some shortcomings in style generation. The RCA module mainly focuses on semantics and spatial alignment, but has weak control over style details, resulting in the lack of style consistency in the generated images and failing to fully meet specific style requirements. Summary of the invention

[0013] The purpose of the embodiment of the present application is to provide a rainy day target detection method based on paired image data generation and knowledge distillation, which aims to effectively improve the detection accuracy of the model under rainy day conditions without sacrificing real-time performance, while optimizing data utilization efficiency to cope with the challenges brought by data scarcity, thereby solving the problem that the existing YOLOv8 algorithm often leads to a decline in target recognition performance due to the decline in image quality under weather conditions such as rain, fog, and snow. In addition, existing solutions usually rely on image restoration operations such as image defogging and rain removal before performing target detection. Although this can improve the detection effect, the processing speed is slow and it is difficult to meet the application scenarios with high real-time requirements such as autonomous driving. In addition, it is difficult to obtain labeled data in bad weather, and the scarcity of training data further limits the generalization ability of the model.

[0014] In order to solve the above technical problems, this application is implemented as follows:

[0015] The embodiment of the present application provides a method for rainy day target detection based on paired image data generation and knowledge distillation, which is characterized by comprising the following steps:

[0016] Step S1, obtaining real images of road scenes under sunny and rainy conditions and semantic segmentation masks corresponding to the real images of road scenes from a real data set with semantic segmentation mask annotations;

[0017] Step S2, inputting the real image of the road scene, the semantic segmentation mask and the weather condition text information into the improved diffusion model for training, and after the training is completed, obtaining a trained improved diffusion model;

[0018] Step S3, inputting the semantic segmentation mask and the weather condition text information into the trained improved diffusion model to obtain a generated image of the road scene under rainy conditions;

[0019] Step S4, integrating the real road scene image and the generated road scene image into a new data set;

[0020] Step S5, pre-training the optimized target detection model based on the new data set, wherein the optimized target detection model is optimized using the following method:

[0021] The CoT module and ODConv module are introduced into the target detection model for optimization, and then the bounding box loss function of the target detection model is improved by using the dynamic weighted aggregation loss function;

[0022] Step S6, after pre-training, the optimized target detection model is combined with the new data set to perform paired image knowledge distillation to obtain a rainy day target detection enhanced model;

[0023] Step S7, using the rainy day target detection enhancement model to perform target detection.

[0024] Optionally, in step S1, the real data sets include Wilddash2, IDDAW, Cityscapes, ACDC, BDD100K and Wilddash2.

[0025] Optionally, step S1 further includes: unifying the format of the real images of the road scenes in the real data set.

[0026] Optionally, in step S5, the target detection model includes a backbone network, a feature fusion network and a task head, the CoT module is embedded in the low-level convolutional layer of the backbone network to capture detailed features in the image; the ODConv module is introduced into the high-level convolutional layer of the backbone network to process complex semantic features, thereby enhancing the representation capability of high-level features; in addition, the ODConv module is also integrated in the feature fusion network to optimize the fusion of multi-scale features.

[0027] Optionally, the dynamic weighted aggregation loss function improves the bounding box regression performance of the target detection model in complex scenes by introducing a dynamic weighting mechanism of target scale and position information.

[0028] Optionally, in step S5, the target detection model is a YOLOv8 target detection model; and the optimized target detection model is a COTD-YOLOv8 target detection model.

[0029] Optionally, in step S6, the optimized object detection model is combined with the new data set to perform paired image knowledge distillation, specifically including:

[0030] In the pre-training stage, Co-DETR is used as a teacher model and trained using real images of road scenes under sunny conditions in the new dataset to generate accurate category probability distribution and bounding box predictions; the optimized object detection model is used as a student model and pre-trained using real images of road scenes under sunny conditions and generated images of road scenes under rainy conditions in the new dataset;

[0031] In the knowledge distillation stage, the teacher model is trained using real images of road scenes under sunny conditions to generate high-quality soft labels, which include category probability distribution and bounding box regression; the student model is trained using images generated by road scenes under rainy conditions and generates its own soft predictions and hard predictions; the soft labels generated by the teacher model are then used to guide the training of the student model.

[0032] Optionally, the soft labels generated by the teacher model are used to guide the training of the student model, including:

[0033] The student model measures the difference between its soft predictions on the generated images of road scenes under rainy conditions and the soft labels generated by the teacher model on the real images of road scenes under sunny conditions through distillation loss and similarity-weighted distillation loss;

[0034] The student model is trained by comparing the hard predictions with the true hard labels and calculating the detection loss.

[0035] The beneficial effects of the embodiments of the present application are:

[0036] 1. The present invention realizes the function of generating rainy day images based on semantic masks and text descriptions, expands the training data of the model, and effectively solves the problem of insufficient annotated data. Secondly, the CoT module and ODConv module are introduced for model optimization, which significantly enhances the feature extraction capability of the target detection model (YOLOv8) in complex environments, especially in rainy conditions; specifically, the CoT module enhances the context perception capability of the model, and through static context encoding and dynamic self-attention mechanism, enables the model to accurately capture image features under complex conditions such as low contrast and blur; the ODConv module improves the extraction capability of high-level semantic features through a multi-dimensional convolution adaptive mechanism, further enhancing the robustness and feature expression capability of the model.

[0037] 2. The dynamic weighted aggregation loss function proposed in this invention improves the bounding box regression performance of the target detection model in complex scenes by introducing a dynamic weighting mechanism of target scale and position information;

[0038] 3. The optimized target detection module (CTOD-YOLOv8) provided by the present invention retains the lightweight design of CSPNet. By optimizing the convolution operation, it ensures the high efficiency of the model and the controllability of the computational overhead. It is suitable for scenarios such as autonomous driving with high real-time requirements.

[0039] 4. The present invention optimizes the target detection capability of the student model (COTD-YOLOv8) by combining paired sunny and rainy images and using the knowledge distillation method; due to the high-precision detection capability of the teacher model (Co-DETR) on sunny images, the student model can learn more detailed and accurate target features, thereby significantly improving the detection performance under adverse weather conditions such as rainy days.

[0040] 5. The method provided by the present invention performs well in rainy scenes, has good generalization ability, can provide higher-precision target detection in various complex scenes, and effectively improves the overall detection accuracy and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0042] Figure 1 A flowchart of the rainy day target detection method based on paired image data generation and knowledge distillation provided in this application;

[0043] Figure 2 Schematic diagram of the process of generating the new dataset provided for this application;

[0044] Figure 3 The optimized target detection model network structure diagram provided for this application;

[0045] Figure 4 Schematic diagram of the target detection module training process provided for this application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0048] See also Figure 1 As shown, the embodiment of the present application provides a rainy day target detection method based on paired image data generation and knowledge distillation, comprising the following steps:

[0049] Step S1, obtaining real images of road scenes under sunny and rainy conditions and semantic segmentation masks corresponding to the real images of road scenes from a real data set with semantic segmentation mask annotations;

[0050] Step S2, inputting the real image of the road scene, the semantic segmentation mask and the weather condition text information into the improved diffusion model for training, and after the training is completed, obtaining a trained improved diffusion model;

[0051] Step S3, inputting the semantic segmentation mask and the weather condition text information into the trained improved diffusion model to obtain a generated image of the road scene under rainy conditions;

[0052] Step S4, integrating the real road scene image and the generated road scene image into a new data set;

[0053] Step S5, pre-training the optimized target detection model based on the new data set, wherein the optimized target detection model is optimized using the following method:

[0054] The CoT module and ODConv module are introduced into the target detection model for optimization, and then the bounding box loss function of the target detection model is improved by using the dynamic weighted aggregation loss function;

[0055] Step S6, after pre-training, the optimized target detection model is combined with the new data set to perform paired image knowledge distillation to obtain a rainy day target detection enhanced model;

[0056] Step S7, using the rainy day target detection enhancement model to perform target detection.

[0057] In step S1, the real data sets include Wilddash2, IDDAW, Cityscapes, ACDC, BDD100K and Wilddash2. Specifically, Wilddash2 focuses on scenes collected under complex and harsh conditions, including bad weather such as rain, fog, and snow. IDDAW includes smog, rainy days, and night driving. Cityscapes only includes urban street view data under good weather conditions. ACDC focuses on adverse driving conditions such as rainy days, snowy days, foggy days, and nights. BDD100K covers a variety of scenes, weather, and time conditions.

[0058] Step S1 also includes unifying the format of the real images of the road scenes in the real data set. The so-called format unification is to unify images of different types and sizes into a standard format to obtain a more standardized data set for subsequent use.

[0059] In step S2, combined Figure 2As shown, the improved diffusion model is a Stable Diffusion text-to-image diffusion model based on large-scale pre-training that introduces a multimodal input and output mechanism for weather conditions and semantic segmentation labels. In terms of the model structure, a weather condition embedding module is added to convert the weather description into a vector, and the attention map is adjusted by combining semantics and weather conditions through the RCA+ module. The RCA+ module enables the model to control style generation based on the weather style description of each image while maintaining the semantic and layout alignment functions. The style description word embedding is introduced, and style descriptions such as "sunny" and "rainy" are encoded into style vectors through a pre-trained word embedding model. The style vector is combined with the multi-head attention mechanism of the RCA+ module as the input of style control, so that the generation process is not only strictly based on semantic layout, but also can integrate global style information. In the Key (K) and Value (V) matrices, the style embedding is combined with the semantic features, so that each generated area maintains a consistent style while meeting the semantic requirements.

[0060] In step S5, the target detection model is the YOLOv8 target detection model; and the optimized target detection model is the COTD-YOLOv8 target detection model.

[0061] Recombination Figure 3 As shown, the target detection model includes a backbone network, a feature fusion network and a task head. The CoT module is embedded in the low-level convolutional layer of the backbone network to capture the detailed features in the image; the ODConv module is introduced into the high-level convolutional layer of the backbone network to process complex semantic features, thereby enhancing the representation ability of high-level features, especially improving the detection performance of complex targets in the high-level feature extraction stage. In addition, the ODConv module is also integrated in the feature fusion network to optimize the fusion of multi-scale features.

[0062] The dynamic weighted aggregation loss function improves the bounding box regression performance of the target detection model in complex scenarios by introducing a dynamic weighting mechanism for target scale and position information. It should be further explained that in machine learning and deep learning, the loss function is a function that measures the difference between the model's predicted value and the actual value. By optimizing the algorithm, the loss function can be reduced to reduce the model's error. The original loss function of the target detection model (YOLOv8) includes three items: bounding box loss, confidence loss, and classification loss. The total loss function formula is as follows:

[0063] L detection =L bbox +L conf +Lclass

[0064] The main bounding box loss function used in YOLOv8 and its previous versions is usually IoU (Intersection over Union) and its variants (such as GIoU, DIoU, CIoU). These loss functions have shown superior performance in target detection tasks, but in rainy weather scenes, they still have limitations such as insufficient accuracy in detecting small targets and lack of adaptability. The dynamic weighted aggregation loss function proposed in the present invention focuses on improving the bounding box loss. The dynamic weighted aggregation loss function improves the bounding box regression performance of the target detection model in complex scenes by introducing a dynamic weighting mechanism for target scale and position information. Specifically, the loss function combines the scale and position characteristics of the target, and solves the shortcomings of the existing loss function of insufficient detection of small targets and ignoring targets on the edge of the image by adaptively allocating loss weights. The scale weighting mechanism assigns higher loss weights to small targets to ensure that they are sufficiently optimized in the model regression process. The formula is:

[0065]

[0066] Where A is the target area. The position weighting mechanism improves the detection accuracy of edge targets by assigning a greater loss weight to targets that are farther from the center of the image. The weight calculation formula is:

[0067]

[0068] Where d is the distance from the center of the target to the center of the image. The dynamic aggregation loss function combining scale weighting and position weighting makes the model perform better in complex environments, and enhances the robustness and wide applicability of the model by adaptively adjusting the optimization strategy. The final bounding box regression loss function is:

[0069] L bbox =α·w scale ·CIoU+β·w position ·CIoU

[0070] Among them, α and β are adjustable weight coefficients to ensure that the model can achieve the best detection performance in different tasks and scenarios.

[0071] In step S6, combining Figure 4 As shown in the figure, the optimized target detection model is combined with the new dataset to perform paired image knowledge distillation, including:

[0072] In the pre-training stage, Co-DETR is used as a teacher model and trained using real images of road scenes under sunny conditions in the new dataset to generate accurate category probability distribution and bounding box predictions; the optimized object detection model is used as a student model and pre-trained using real images of road scenes under sunny conditions and generated images of road scenes under rainy conditions in the new dataset;

[0073] In the knowledge distillation stage, the teacher model is trained using real images of road scenes under sunny conditions to generate high-quality soft labels, which include category probability distribution and bounding box regression; the student model is trained using images generated by road scenes under rainy conditions and generates its own soft predictions and hard predictions; the soft labels generated by the teacher model are then used to guide the training of the student model.

[0074] The soft labels generated by the teacher model are used to guide the training of the student model, including:

[0075] The student model learns the target features under bad weather conditions by minimizing the difference between its soft predictions on the generated images of road scenes under rainy conditions and the soft labels generated by the teacher model on the real images of road scenes under sunny conditions. However, due to the difference in object changes on sunny and rainy days, it is a huge challenge to extract instance feature changes directly from sunny teachers to rainy students. Through analysis, it is found that the changes in instances are mainly reflected in the differences in shape and texture. To overcome this challenge, the present invention introduces an adaptive weighted instance distillation based on shape and texture features (Adaptively Weighted Distillation based on Shape and Texture) module in the process of migrating from the teacher model to the student model, and uses the similarity between the corresponding sunny and rainy objects as the distillation weight of the instance features. The similarity includes texture similarity and shape similarity. This module not only improves the efficiency of knowledge distillation but also improves the robustness of the model on sunny and rainy days.

[0076] The difference between the student model and the teacher model during the learning process is calculated by the distillation loss L distillation and similarity weighted distillation loss L ins To measure, the student model also calculates the detection loss L by comparing its hard predictions with the true hard labels (Ground Truth) detection , thereby improving the detection performance in adverse environments such as rainy days. In this way, the student model not only learns effective features under sunny conditions from the teacher model, but also can better apply these features in complex rainy environments, thereby improving the robustness of the model and the detection performance.

[0077] Furthermore, the loss function uses a weighted combination of standard target detection loss and knowledge distillation loss to ensure that the student model can optimize its own performance through supervised learning under rainy conditions and learn the characteristics of the teacher model through distillation. The formula is as follows:

[0078] L total =αL detection +βL distillation +γL ins

[0079] Among them, L detection To detect the loss, L distillation is the distillation loss, L ins is the similarity weighted distillation loss. α, β, and γ in the loss function are weight coefficients used to balance the weights of instance features extracted from the sunny teacher to the rainy student and the relationship between the student model's performance on real rainy data and the knowledge learned from the teacher model through the distillation process. Ultimately, this weighted loss design can maximize the use of the teacher model's knowledge transfer under sunny conditions while improving the student model's detection performance under rainy conditions, so that the student model maintains high detection accuracy and robustness in complex scenes such as rainy days.

[0080] The standard target detection loss is the improved loss function L mentioned above detection , which consists of classification loss, bounding box regression loss, and confidence loss. The formula is as follows:

[0081] L detection =L bbox +L conf +L class

[0082] The distillation loss function consists of classification loss and regression loss. The formula is as follows:

[0083] L distillation =L cls +L reg

[0084] The similarity-weighted distillation loss is related to the difference in similarity and texture under sunny and rainy conditions. The formula is as follows:

[0085]

[0086] Among them, S texture and S shape Respectively represent texture similarity and shape similarity, ω texture and ω shape are hyperparameters representing the weights of texture similarity and shape similarity in the loss function, respectively. N represents the corresponding pixel point set P under sunny and rainy conditions. c and Pr The number of pixels contained in the intersection part, the subscripts c and r represent clear and rainy days respectively, so that the loss function will not be affected by the number of instances. texture and S shape , suppose all pixels contained in the true value object frame under sunny and rainy conditions are P s and P r , then its texture similarity S texture and shape similarity S shape The formula is:

[0087]

[0088] Among them, I c and I r They represent the instance features extracted by the model, d chamfer Indicates the chamfer distance (ChamferDistance).

[0089] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0090] In addition, it should be noted that the scope of the methods and systems in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

[0091] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A rainy day target detection method based on paired image data generation and knowledge distillation, characterized in that: The following steps are involved: Step S1, obtaining real images of road scenes under sunny and rainy conditions and semantic segmentation masks corresponding to the real images of road scenes from a real data set with semantic segmentation mask annotations; Step S2, inputting the real image of the road scene, the semantic segmentation mask and the weather condition text information into the improved diffusion model for training, and after the training is completed, obtaining a trained improved diffusion model; Step S3, inputting the semantic segmentation mask and the weather condition text information into the trained improved diffusion model to obtain a generated image of the road scene under rainy conditions; Step S4, integrating the real road scene image and the generated road scene image into a new data set; Step S5, pre-training the optimized target detection model based on the new data set, wherein the optimized target detection model is optimized using the following method: The CoT module and ODConv module are introduced into the target detection model for optimization, and then the bounding box loss function of the target detection model is improved by using the dynamic weighted aggregation loss function; Step S6, after pre-training, the optimized target detection model is combined with the new data set to perform paired image knowledge distillation to obtain a rainy day target detection enhanced model; Step S7, using the rainy day target detection enhancement model to perform target detection.

2. The rainy day target detection method based on paired image data generation and knowledge distillation according to claim 1, characterized in that: In step S1, the real data sets include Wilddash2, IDDAW, Cityscapes, ACDC, BDD100K and Wilddash2.

3. The method for rainy day target detection based on paired image data generation and knowledge distillation according to claim 2, characterized in that: Step S1 also includes: unifying the format of the real images of the road scenes in the real data set.

4. The method for rainy day target detection based on paired image data generation and knowledge distillation according to claim 1, characterized in that: In step S5, the target detection model includes a backbone network, a feature fusion network and a task head. The CoT module is embedded in the low-level convolutional layer of the backbone network to capture the detailed features in the image; the ODConv module is introduced into the high-level convolutional layer of the backbone network to process complex semantic features, thereby enhancing the representation capability of high-level features. In addition, the ODConv module is also integrated in the feature fusion network to optimize the fusion of multi-scale features.

5. The method for rainy day target detection based on paired image data generation and knowledge distillation according to claim 1, characterized in that: The dynamic weighted aggregation loss function improves the bounding box regression performance of the target detection model in complex scenes by introducing a dynamic weighting mechanism of target scale and position information.

6. The method for rainy day target detection based on paired image data generation and knowledge distillation according to claim 1, characterized in that: In step S5, the target detection model is the YOLOv8 target detection model; and the optimized target detection model is the COTD-YOLOv8 target detection model.

7. The method for rainy day target detection based on paired image data generation and knowledge distillation according to claim 1, characterized in that: In step S6, the optimized object detection model is combined with the new data set to perform paired image knowledge distillation, which specifically includes: In the pre-training stage, Co-DETR is used as a teacher model and trained using real images of road scenes under sunny conditions in the new dataset to generate accurate category probability distribution and bounding box predictions; the optimized object detection model is used as a student model and pre-trained using real images of road scenes under sunny conditions and generated images of road scenes under rainy conditions in the new dataset; In the knowledge distillation stage, the teacher model is trained using real images of road scenes under sunny conditions to generate high-quality soft labels, which include category probability distribution and bounding box regression; the student model is trained using images generated by road scenes under rainy conditions and generates its own soft predictions and hard predictions; the soft labels generated by the teacher model are then used to guide the training of the student model.

8. The method for rainy day target detection based on paired image data generation and knowledge distillation according to claim 7, characterized in that: The soft labels generated by the teacher model are used to guide the training of the student model, including: The student model measures the difference between its soft predictions on the generated images of road scenes under rainy conditions and the soft labels generated by the teacher model on the real images of road scenes under sunny conditions through distillation loss and similarity-weighted distillation loss; The student model is trained by comparing the hard predictions with the true hard labels and calculating the detection loss.

Images