Cross-domain target detection method and system based on semi-supervised learning in extreme weather

Abstract

The invention provides a cross-domain target detection method and system based on semi-supervised learning in extreme weather, and the method comprises the steps: enabling a student model to learn bounding box regression knowledge of a trained teacher model through distillation, so as to achieve the regression capability in extreme weather; performing instance-level feature adversarial training on a pre-training detection model which is obtained in a distillation stage and has a regression capability in extreme weather by using an intermediate-state image which is generated by a diffusion model and is closer to a target domain and has smaller noise, and determining a new teacher model and a new student model based on the detection model after the adversarial training; in the fine tuning stage, a dynamic category pseudo label screening strategy is utilized to screen pseudo labels generated by a new teacher model, and a screened target domain pseudo label is utilized to train a new student model to obtain a target detection model used for detecting a to-be-detected image in extreme weather. The phenomena of leak detection and error detection in extreme weather scenes in the prior art can be reduced, and the original reasoning speed is kept.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a cross-domain target detection method and system based on semi-supervised learning under extreme weather conditions. Background Art

[0002] At present, the research on object detection under extreme weather conditions is of great significance, mainly in the following aspects: (1) It can improve safety: The generalized object detection technology can help vehicles and other means of transportation better identify obstacles and pedestrians on the road under low visibility conditions, thereby significantly improving traffic safety; (2) It helps the development of autonomous driving technology: Autonomous driving vehicles need to accurately identify and locate surrounding objects in various weather conditions. Object detection in severe weather is an important challenge in the development of autonomous driving technology, and its research is crucial to the realization of all-weather autonomous driving systems; (3) It will help disaster emergency management: Extreme weather may cause traffic delays and accidents, especially at key transportation nodes such as airports and seaports.

[0003] However, compared with conventional object detection, there is less work on object detection under extreme weather conditions. There is a lack of sufficient annotated extreme weather datasets. In addition, due to the lack of corresponding datasets, images are usually derained or defogged (offset). line), and then pass the preprocessed image into the detection model, such as the IA-YOLO detection model, in which the image is adaptively repaired through a differentiable image processing module and a learnable small neural network, and then passed into the YOLOv3 detection network. Although this detection method can improve the overall quality of the image, it may also lose important semantic information for detection, resulting in a large number of missed detections. On this basis, a symbiotic model of repair and detection is built, and the feature information extracted from the unified backbone network is used for repair and detection. For example, Together-Net uses YOLOX as the detection model, and builds a symbiotic repair network on its backbone network to perform repair and detection tasks at the same time. However, it is limited by the complexity of the network and it is difficult to achieve a balance between the two tasks of repair and detection. For example, the average teacher model is a semi-supervised learning method that is widely used in cross-domain target detection tasks. It can fully train in the target domain in a targeted manner. Although it can improve the quality of pseudo-labels to a certain extent, the quality of pseudo-labels is still relatively low in scenes with heavy rain or fog. Using a single filtering strategy based on classification confidence, it is impossible to screen out high-quality pseudo-labels.

[0004] In view of this, providing a solution to the above-mentioned technical problems has become an urgent concern for those skilled in the art. Summary of the invention

[0005] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a cross-domain target detection method and system based on semi-supervised learning under extreme weather conditions are provided, aiming to solve the problem that the existing target detection technology under extreme weather conditions cannot realize targeted training due to the lack of sufficient labeled extreme weather data sets, and the problem that a single filtering strategy based on classification confidence is adopted in semi-supervised learning, which cannot screen out high-quality pseudo-labels.

[0006] The technical solution adopted by the present invention to solve the technical problem is as follows: A cross-domain target detection method based on semi-supervised learning in extreme weather, wherein the method comprises: Distilling the bounding box regression knowledge of the trained teacher model to the student model, and obtaining a pre-trained detection model with regression capability under extreme weather conditions; wherein the student model is a detection model with detection capability under normal weather conditions, and the trained teacher model is a model obtained by training a detection model with a bounding box regression capability superior to the student model using a preset number of extreme weather images with labels; The original source domain image under normal weather and the original target domain image under extreme weather are respectively input into the diffusion model, and the intermediate state image between the source domain and the target domain output by the diffusion model is input into the pre-trained detection model for instance-level feature adversarial training to obtain the detection model after adversarial training; Determine a new teacher model and a new student model based on the detection model after the adversarial training, and use a preset dynamic category pseudo-label screening method to screen the pseudo-labels generated by the new teacher model based on the original target domain image, and use the screened target domain pseudo-labels to fine-tune the new student model to obtain a trained target detection model; The trained target detection model is used to perform target detection on the image to be detected under extreme weather conditions to obtain corresponding detection results.

[0007] In one implementation, the bounding box regression knowledge of the trained teacher model is distilled and transferred to the student model to obtain a pre-trained detection model with regression capability under extreme weather conditions, including: The bounding box regression knowledge of the trained teacher model is distilled and transferred to the student model using a progressive knowledge distillation strategy, thus obtaining a pre-trained detection model with regression capabilities under extreme weather conditions. Among them, the progressive knowledge distillation strategy is a distillation strategy that gradually increases the weight of the regression distillation loss according to the preset loss weight adjustment strategy during the knowledge distillation process, so that the student model gradually has the regression ability under extreme weather conditions, and the initial weight of the regression distillation loss is not greater than the preset weight value.

[0008] In one implementation, the calculation formula for the regression distillation loss is: ; in, represents the regression distillation loss, β represents the weight of the regression distillation loss, N represents the total number of samples, represents the regression output of the trained teacher model for the i-th sample, represents the regression output of the student model for the i-th sample, It represents the mean square error between the regression output of the trained teacher model for the i-th sample and the regression output of the student model for the i-th sample.

[0009] In one implementation, the teacher model is an RT-DETR model, and the student model is a YOLOv11 model.

[0010] In one implementation, the diffusion model is a denoising probabilistic diffusion model, and the diffusion model generates the intermediate state image by adding noise and gradually denoising, and adds content loss and style loss in the denoising process of the diffusion model; The content loss is calculated by comparing the features of the intermediate image with the original target domain image or the original source domain image on the convolutional layer, and the style loss is calculated by comparing the Gram matrix of the intermediate image with the original source domain image or the original target domain image.

[0011] In one implementation, the cross-domain target detection method based on semi-supervised learning in extreme weather further includes: During the feature adversarial training, a target frame region is extracted from the intermediate state image, and instance-level feature alignment is performed based on a mask of the target frame region and in an adversarial manner between a generator and a discriminator; The discriminator is used to determine whether the features of the target frame area corresponding to the source domain and the target domain are distinguishable, and the generator is used to minimize the determination ability of the discriminator.

[0012] In one implementation, the process of fine-tuning the new student model using the screened target domain pseudo labels to obtain a trained target detection model further includes: Obtaining the parameters of the new student model in each training round in real time; Based on a preset adaptive double exponential moving average strategy, the parameters of the new teacher model are dynamically updated using the parameters of the new student model in each training round; wherein the decay rate in the preset adaptive double exponential moving average strategy is dynamically adjusted based on the training rounds to dynamically control the parameter update speed of the new teacher model; And, the calculation formula of the attenuation rate is: ; in, Indicates the current training round number, Indicates the decay rate under the current number of training rounds, represents the total number of training rounds, represents the minimum value of the attenuation rate, Indicates the maximum value of the decay rate.

[0013] In one implementation, the using a preset dynamic category pseudo-label screening method to screen the pseudo-labels generated by the new teacher model based on the original target domain image includes: Performing category statistics and distribution statistics on the original target domain image to obtain corresponding statistical results, and performing targeted data enhancement processing on the original target domain image based on the statistical results to obtain a new target domain image; The pseudo labels generated by the new teacher model based on the original target domain image and the new target domain image are screened using a preset dynamic category pseudo label screening method.

[0014] In one implementation, the preset dynamic category pseudo-label screening method is: Compare the classification confidence score corresponding to when the pseudo label indicates that the target to be detected belongs to the target category with the first threshold and the second threshold corresponding to the target category to obtain corresponding comparison results; wherein the first threshold is not less than the second threshold, and during the fine-tuning training, the first threshold and the second threshold are dynamically adjusted according to a preset threshold dynamic adjustment strategy; When the comparison result indicates that the classification confidence score is higher than the first threshold, the pseudo label is determined as a positive sample to obtain the target domain pseudo label; When the comparison result indicates that the classification confidence score is lower than the second threshold, the pseudo label is determined as a negative sample, and the negative sample is regularized to obtain a regularized negative sample for fine-tuning training of the new student model; When the comparison result shows that the classification confidence score is between the first threshold and the second threshold, determining whether the pseudo label is consistent with the pseudo label predicted by the preset large model; When the pseudo label is consistent with the pseudo label predicted by the preset large model, the pseudo label is determined as a positive sample to obtain the target domain pseudo label; When the pseudo label is inconsistent with the pseudo label predicted by the preset large model, the pseudo label is determined as a negative sample, and the negative sample is regularized to obtain a regularized negative sample for fine-tuning training of the new student model.

[0015] The present invention also discloses a cross-domain target detection system based on semi-supervised learning under extreme weather conditions, wherein the system comprises: A model distillation module, for distilling and transferring bounding box regression knowledge possessed by a trained teacher model to a student model, to obtain a pre-trained detection model with regression capability under extreme weather conditions; wherein the student model is a detection model with detection capability under normal weather conditions, and the trained teacher model is a model obtained by training a detection model with a bounding box regression capability superior to that of the student model using a preset number of extreme weather images with labels; An adversarial training module is used to input the original source domain images under normal weather conditions and the original target domain images under extreme weather conditions into the diffusion model respectively, and input the intermediate state images between the source domain and the target domain output by the diffusion model into the pre-trained detection model to perform instance-level feature adversarial training, so as to obtain a detection model after adversarial training; A model fine-tuning module is used to determine a new teacher model and a new student model based on the detection model after the adversarial training, and use a preset dynamic category pseudo-label screening method to screen the pseudo-labels generated by the new teacher model based on the original target domain image, and use the screened target domain pseudo-labels to fine-tune the new student model to obtain a trained target detection model; The target detection module is used to use the trained target detection model to perform target detection on the image to be detected under extreme weather conditions to obtain corresponding detection results.

[0016] The present invention also discloses a terminal, which includes: a memory, a processor, and a cross-domain target detection program based on semi-supervised learning under extreme weather, which is stored in the memory and can be run on the processor. When the cross-domain target detection program based on semi-supervised learning under extreme weather is executed by the processor, the steps of the cross-domain target detection method based on semi-supervised learning under extreme weather are implemented as described above.

[0017] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program can be executed to implement the steps of the cross-domain target detection method based on semi-supervised learning under extreme weather as described above.

[0018] The present invention provides a method and system for cross-domain target detection based on semi-supervised learning under extreme weather. The method for cross-domain target detection based on semi-supervised learning under extreme weather comprises: distilling and transferring bounding box regression knowledge possessed by a trained teacher model to a student model to obtain a pre-trained detection model with regression capability under extreme weather; wherein the student model is a detection model with detection capability under normal weather, and the trained teacher model is a model obtained by training a detection model with a bounding box regression capability superior to the student model using a preset number of extreme weather images carrying labels; separating the acquired original source domain images under normal weather and the acquired original target domain images under extreme weather into a pre-trained detection model with regression capability under extreme weather. The source domain and the target domain are respectively input into the diffusion model, and the intermediate state image between the source domain and the target domain output by the diffusion model is input into the pre-trained detection model for instance-level feature adversarial training to obtain the detection model after adversarial training; a new teacher model and a new student model are determined based on the detection model after adversarial training, and a preset dynamic category pseudo-label screening method is used to screen the pseudo-labels generated by the new teacher model based on the original target domain image, and the screened target domain pseudo-labels are used to fine-tune the new student model to obtain a trained target detection model; the trained target detection model is used to perform target detection on the images to be detected under extreme weather conditions to obtain corresponding detection results. It can be seen that the present invention enables the student model to have the regression ability of the trained teacher model under extreme weather conditions through distillation, and then generates an intermediate state image that is closer to the target domain and has less noise through a diffusion model, and uses the intermediate state image to perform instance-level feature adversarial training on the pre-trained detection model that has learned the bounding box regression knowledge of the trained teacher model and has the regression ability under extreme weather conditions to reduce the feature difference between the source domain and the target domain to obtain the detection model after adversarial training, and then determines a new teacher model and a new student model based on the detection model after adversarial training. When the new teacher model is used to generate pseudo labels in the subsequent process, the quality of the pseudo labels generated by the new teacher model can be improved. Then, in the fine-tuning stage, the pseudo labels generated by the new teacher model are screened using a dynamic category pseudo-label screening strategy, and then the high-quality target domain pseudo labels are screened out to train the new student model to obtain a trained target detection model for target detection of images to be detected under extreme weather conditions, thereby greatly reducing the missed detection and false detection phenomena of current technologies in extreme weather scenarios, and the inference speed remains basically unchanged. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of a preferred embodiment of a cross-domain target detection method based on semi-supervised learning under extreme weather conditions in the present invention; Figure 2 It is a schematic diagram of a specific knowledge distillation method disclosed in the present invention; Figure 3It is a schematic diagram of a specific model pre-training method disclosed in the present invention; Figure 4 It is a schematic diagram of a specific pseudo-label dynamic screening method disclosed in the present invention; Figure 5 It is a specific flow chart of the model fine-tuning method disclosed in the present invention; Figure 6 It is a functional principle block diagram of a preferred embodiment of a cross-domain target detection system based on semi-supervised learning under extreme weather conditions in the present invention; Figure 7 It is a functional principle block diagram of a preferred embodiment of the terminal in the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0021] See also Figure 1 , Figure 1 Flowchart of the cross-domain target detection method based on semi-supervised learning under extreme weather conditions in the present invention. Figure 1 As shown, the cross-domain target detection method based on semi-supervised learning in extreme weather according to an embodiment of the present invention includes: Step S11, distilling and transferring the bounding box regression knowledge of the trained teacher model to the student model to obtain a pre-trained detection model with regression capability under extreme weather conditions; wherein the student model is a detection model with detection capability under normal weather conditions, and the trained teacher model is a model obtained by training a detection model having a bounding box regression capability superior to that of the student model using a preset number of extreme weather images carrying labels.

[0022] In this embodiment, first, a student model and a teacher model are determined, wherein the teacher model is a detection model with a better bounding box regression capability than the student model, and the teacher model is more complex than the student model, but its complex structure can provide more comprehensive knowledge for the student model, and then a preset number of extreme weather images with labels are used to train the teacher model to obtain a trained teacher model, and then the bounding box regression knowledge of the trained teacher model is distilled and transferred to the student model to obtain a pre-trained detection model with regression capability under extreme weather conditions, that is, the student model has the regression capability of the trained teacher model under extreme weather conditions through knowledge distillation. Among them, the preset number of extreme weather images with labels can be 1,000, and extreme weather refers to abnormal meteorological phenomena with low probability of occurrence but strong destructiveness, which are usually caused by climate change, natural factors or human activities, such as heavy rain, heavy snow, hail, typhoon, strong cold wave, sandstorm, fog and other severe weather. In such severe weather, target detection is prone to many false detections and missed detections, resulting in low detection accuracy.

[0023] It can be understood that by training the teacher model on a small number of annotated extreme weather images, and then using the bounding box regression knowledge output by the trained teacher model to guide the output regression box of the student model, the student model has the regression ability of the teacher model. For example, the positioning performance of the student model is very poor. In order to improve the positioning performance of the student model, the knowledge distillation method is used to improve the regression accuracy of the model.

[0024] Specifically, the bounding box regression knowledge of the trained teacher model is distilled and transferred to the student model using a progressive knowledge distillation strategy to obtain a pre-trained detection model with regression capabilities under extreme weather conditions; wherein the progressive knowledge distillation strategy is a distillation strategy that gradually increases the weight of the regression distillation loss according to a preset loss weight adjustment strategy during the knowledge distillation process, so that the student model gradually has the regression capability under extreme weather conditions, and the initial weight of the regression distillation loss is not greater than the preset weight value. It is understandable that, due to the large difference between the student model and the teacher model in the early stages of training, in order to allow the student model to have an adaptation process, a progressive knowledge distillation method can be used so that the student model gradually has the regression capability of the trained teacher model under extreme weather conditions, and the preset weight value can be 0.1, that is, the initial weight of the regression distillation loss is not greater than 0.1.

[0025] For example, at the beginning of distillation, a smaller regression distillation loss weight, such as 0.05, is used to encourage the student model to learn only some basic features of the teacher model. As the student model gradually approaches the regression ability of the teacher model, more complex knowledge begins to be passed to the student model. The weight of the regression distillation loss gradually increases according to the preset loss weight adjustment strategy. The content learned by the student model gradually becomes more complex. The student model not only learns the bounding box regression of simple samples, but also begins to learn complex and difficult-to-distinguish border regression. Then, in the later stage of the distillation process, more high-level knowledge of the teacher model is introduced, such as fine-grained target detection, occlusion processing, and small object regression. At this time, the student model already has strong capabilities and can better handle target detection tasks, and learn the reasoning process, fine-grained features and decision-making capabilities of the teacher model through distillation.

[0026] And, the regression distillation loss is calculated as: ; in, represents the regression distillation loss, β represents the weight of the regression distillation loss, N represents the total number of samples, represents the regression output of the trained teacher model for the i-th sample, represents the regression output of the student model for the i-th sample, Represents the square of the L2 norm, that is, the mean square error (MSE), It represents the mean square error between the regression output of the trained teacher model for the i-th sample and the regression output of the student model for the i-th sample.

[0027] That is, during the training process, the weight of the regression distillation loss β can be dynamically adjusted according to the number of training rounds, so that the student model can gradually learn complex knowledge from the teacher model while avoiding excessive reliance on complex teacher model knowledge in the early stages.

[0028] It should be noted that the teacher model can be the RT-DETR (Real-Time Detection Transformers, i.e. real-time object detection based on the Transformer architecture) model, and the student model can be the YOLOv11 model. Given the advantages of the RT-DETR model in the regression (bounding box regression) part, i.e., it can handle more complex object distribution, occlusion, and small object detection, but its training complexity is high and it is difficult to deploy, see Figure 2As shown in the figure, the RT-DETR model (teacher model) trained on a small number of annotated extreme weather images outputs a relatively accurate regression bounding box to guide the output bounding box of the YOLOv11 model (student model) trained on a large number of normal weather images. That is, the regression part of the DETR model (bounding box regression knowledge) is distilled into the YOLOv11 model by knowledge distillation, so that the YOLOv11 model is still efficient in inference speed while learning the regression ability of the RT-DETR model in extreme weather. That is, the output bounding box of the RT-DETR model is gradually simulated by the student model to gradually fine-tune its output bounding box so that its regression box becomes more accurate. That is, the positioning ability of YOLOv11 in the target domain is still weak, so progressive bounding box distillation can be used to utilize the powerful bounding box regression ability of RT-DETR. After training on a small number of annotated extreme weather images, the regression knowledge of the trained RT-DETR is passed to YOLOv11.

[0029] Furthermore, when the teacher model may be an RT-DETR model and the student model may be a YOLOv11 model, the calculation formula for the above regression distillation loss may be: ; in, represents the regression output of the trained RT-DETR model for the i-th sample, represents the regression output of the YOLOv11 model for the i-th sample, It represents the mean square error between the regression output of the trained RT-DETR model for the i-th sample and the regression output of the YOLOv11 model for the i-th sample.

[0030] Step S12: input the original source domain images under normal weather conditions and the original target domain images under extreme weather conditions into the diffusion model respectively, and input the intermediate state images between the source domain and the target domain output by the diffusion model into the pre-trained detection model for instance-level feature adversarial training to obtain the detection model after adversarial training.

[0031] In this embodiment, through the knowledge distillation method, the pre-trained detection model already has the ability to detect extreme weather scenarios. Then, in the adversarial training stage, the original source domain images under normal weather and the original target domain images under extreme weather can be respectively input into the diffusion model, and the intermediate state images between the source domain and the target domain output by the diffusion model are input into the pre-trained detection model for instance-level feature adversarial training to obtain the detection model after adversarial training, thereby further narrowing the difference between the source domain and the target domain.

[0032] For example, see Figure 3As shown, the original source domain image (image under normal weather conditions) and the original target domain image (image under extreme weather conditions) are respectively passed into the diffusion model for training, and the diffusion model generates the target domain image in the source domain style and the source domain image in the target domain style, that is, the intermediate state image between the two domains, and then the target domain image in the source domain style and the source domain image in the target domain style are passed into the pre-trained detection model (YOLOv11 model with regression capability in extreme weather) for instance-level adversarial training, and the domain difference between normal weather and extreme weather is further reduced through adversarial learning, so that when the new teacher model is used to generate pseudo labels in the future, the quality of pseudo labels generated by the new teacher model in extreme weather can be further improved. Among them, the present application adopts a method based on the diffusion model for training, which is more stable, and the generated intermediate state image has less noise and is closer to the target domain, thereby reducing the model difference, improving the classification accuracy and further improving the positioning capability, and introducing instance-level feature adversarial training, which effectively reduces the interference of background information compared to the full-image feature alignment and pixel-level feature alignment of the prior art, greatly reduces the difference between the source domain and the target domain, and improves the detection performance of the model in the target domain.

[0033] It should be pointed out that the diffusion model is a type of probability-based generative model that generates images by simulating the gradual "noising" and "denoising" process of images. That is, the diffusion model generates intermediate images by adding noise and gradually denoising. In other words, in the image generation task, the diffusion model generates new images by learning the process of recovering real images from noise. Compared with the GAN (Generative Adversarial Network) method in the prior art, the training of the diffusion model is relatively stable. The training process of the GAN method is easily affected by mode collapse and training imbalance, and the generator may focus on generating a few styles of images while ignoring other types of image styles. However, since the optimization goal of the diffusion model is to restore the image by gradually denoising, and the objective function of the diffusion model is usually based on maximizing the likelihood estimate. Therefore, the loss function and the training process are relatively smooth, and it is not easy to have unstable training. Therefore, the diffusion model denoises and optimizes multiple steps to finally obtain the target domain image with the generated source domain style. It retains the content of the target domain image and integrates the style of the source domain image. The same is true for generating the source domain image with the target domain style.

[0034] Specifically, the diffusion model can adopt a denoising probabilistic diffusion model. In order to obtain a target domain image with the style of the source domain, the generated image is initialized from the target domain image, that is, initialized by adding a certain amount of noise to the target domain image, and then the image is restored through a step-by-step denoising process. In other words, the diffusion model learns the distribution of the image by adding noise and gradually denoising.

[0035] In order to achieve style transfer, the style features of the source domain or target domain are introduced in the denoising process of each step, that is, content loss and style loss are added in the denoising process of the diffusion model; among them, the content loss is calculated by comparing the features of the intermediate image (generated image) with the original target domain image or the original source domain image on the convolutional layer, and the style loss is calculated by comparing the Gram matrix of the intermediate image with the original source domain image or the original target domain image.

[0036] For example, it is hoped that the image generated by the diffusion model can maintain the content characteristics of the target domain image. Therefore, when generating a target domain image with the style of the source domain, the content loss between the image generated by the diffusion model and the target domain image is calculated, which can be done by comparing the features of the image on some convolutional layers, that is: ; in, Indicates content loss, Indicates the generated image, represents the target domain image, represents the features of the generated image extracted by the convolutional neural network, Represents the features of the target domain image extracted by the convolutional neural network, and in order to transform the style of the target domain image into the source domain style, it is necessary to calculate the style loss of the source domain image and the generated image.

[0037] Among them, the style loss can be calculated by comparing the Gram matrix of the image, that is: ; in, Represents the style loss, which is used to measure the style difference between the intermediate image and the source domain image. Indicates the generated image in the convolutional neural network The feature map of the layer, represents the source domain image, Represents the source domain image in the convolutional neural network The feature map of the layer, Indicates that the generated image is The Gram matrix of the layer feature map, Represents the source domain image in The Gram matrix of the layer feature map describes the correlation of features in the image. The Gram matrix is ​​used to capture image style features such as texture and style statistics.

[0038] It should be pointed out that, considering the large background difference between the target domain and the source domain, in order to prevent incorrect alignment, this application focuses on the target itself and implements instance-level feature alignment through the mask of the target area, that is, instance-level feature adversarial training. The core idea of ​​instance-level feature adversarial training is to reduce the feature distribution difference between the source domain and the target domain by introducing perturbations in the feature space. Different from the traditional full-image feature alignment, the instance-level alignment method only focuses on the target area in the image, that is, only the object area in the image is featured. Alignment.

[0039] Specifically, in the process of feature adversarial training, the target frame area is extracted from the intermediate image, and based on the mask of the target frame area, instance-level feature alignment is performed in an adversarial manner between the generator and the discriminator to reduce the feature distribution difference between the source domain and the target domain; wherein the discriminator is used to determine whether the features of the target frame areas corresponding to the source domain and the target domain are respectively distinguishable, and the generator is used to minimize the judgment ability of the discriminator so that the features of the target frame areas corresponding to the source domain and the target domain become indistinguishable.

[0040] For example, we first need to extract the target boxes from the intermediate image. These target boxes specify the areas of interest, and masks are used to represent these areas. The mask is usually a binary image, where the part corresponding to the target area is 1 and the rest is 0. Through the mask, we only focus on the features of the target area. Using the mask of the target box, we can shield the features of the non-target area in the feature map and retain the features of the target area, that is, the mask M. s and M t It is a feature map that acts on the source and target images so that only the target area is processed. Through the mask application, the attention is focused on the features of the target box area, so that the model can focus on the key features of the target while ignoring the background and other irrelevant parts when aligning features between the source and target domains. This method can significantly improve the detection performance of the model in the target domain, especially when the domain difference is large.

[0041] For another example, the maximum mean difference (MMD) can be used to measure the distribution difference between the source domain and the target domain. By maximizing the similarity of the feature distribution of the target region of the source domain and the target domain, their MMD values ​​are minimized, that is: ; ; ; in, represents the distribution alignment loss of maximum mean difference, Represents source domain data The characteristic distribution of expectations, Represents the target domain data The characteristic distribution of expectations, Represents a kernel function, such as a Gaussian kernel, which calculates the distribution difference through the masked features. represents the source domain mask feature map, represents the target domain mask feature map, represents the source domain mask matrix, represents the target domain mask matrix, represents the source domain feature extraction function, represents the source domain feature map, represents the target domain feature extraction function, represents the target domain feature map, Represents element-by-element multiplication, which is used to perform element-by-element multiplication of the feature map and the mask matrix, retaining the features of the key areas and suppressing irrelevant areas.

[0042] It should be noted that the L2 norm used in this application is a simplified form of the MMD formula, rather than the conventional Hilport space representation.

[0043] Then, by minimizing , the model is able to learn how to align the target box region features of the source domain and the target domain.

[0044] Moreover, adversarial training is to reduce the feature difference between the source domain and the target domain through the adversarial process between the generator and the discriminator. In the adversarial training of the target box area, the discriminator determines whether the features of the target box area in the source domain and the target domain are distinguishable, while the generator minimizes the judgment ability of the discriminator, making the target area features of the source domain and the target domain indistinguishable. The loss of adversarial training is Right now: ; Among them, D represents the discriminator, which is responsible for judging whether the features of the target box area in the source domain and the target domain are distinguishable. The feature alignment of the target box area is achieved through masking, which can effectively reduce the domain difference between the source domain and the target domain, especially in the target detection task.

[0045] Step S13: determine a new teacher model and a new student model based on the detection model after the adversarial training, and use a preset dynamic category pseudo-label screening method to screen the pseudo-labels generated by the new teacher model based on the original target domain image, and use the screened target domain pseudo-labels to fine-tune the new student model to obtain a trained target detection model.

[0046] In this embodiment, the performance of the model is improved through the distillation and adversarial training stages, and then a new teacher model and a new student model can be determined based on the detection model after adversarial training. For example, a detection model after adversarial training is used as a new teacher model, and a detection model after adversarial training is used as a new student model. Then, in the semi-supervised training stage, the pseudo-labels generated by the new teacher model based on the original target domain image are screened using a preset dynamic category pseudo-label screening method, and then the new student model is fine-tuned using the screened target domain pseudo-labels to obtain a trained target detection model. Among them, the dynamic category pseudo-label screening method adopted can not only screen out high-quality target domain pseudo-labels of all categories, but also effectively use low-quality pseudo-label samples for internal mining, improve the robustness of the model, and enhance the detection ability of the model in extreme weather scenarios.

[0047] Specifically, the category statistics and distribution statistics of the original target domain image are obtained to obtain the corresponding statistical results, and the original target domain image is subjected to targeted data enhancement processing based on the statistical results to obtain a new target domain image; the pseudo labels generated by the new teacher model based on the original target domain image and the new target domain image are screened using a preset dynamic category pseudo label screening strategy. It can be understood that the use of targeted data enhancement, that is, the head category does not increase the number of categories, and the tail category is targeted for data enhancement through mixup, copy-paste, cutout and other methods, effectively reduces the impact of category imbalance on the model compared to the general data enhancement of the prior art. The single threshold strategy of the existing method is difficult to screen out high-quality pseudo labels for the tail category.

[0048] Among them, the preset dynamic category pseudo-label screening method is: the classification confidence score corresponding to the pseudo-label indicating that the target to be detected belongs to the target category is compared with the first threshold and the second threshold corresponding to the target category to obtain the corresponding comparison result; wherein the first threshold is not less than the second threshold, and in the process of fine-tuning training, the first threshold and the second threshold are dynamically adjusted according to the preset threshold dynamic adjustment strategy; when the comparison result shows that the classification confidence score is higher than the first threshold, the pseudo-label is determined as a positive sample to obtain the target domain pseudo-label; when the comparison result shows that the classification confidence score is lower than the second threshold, the pseudo-label is determined as a negative sample. Samples are obtained, and the negative samples are regularized to obtain regularized negative samples for fine-tuning training of the new student model; when the comparison result shows that the classification confidence score is between the first threshold and the second threshold, it is determined whether the pseudo-label is consistent with the pseudo-label predicted by the preset large model; when the pseudo-label is consistent with the pseudo-label predicted by the preset large model, the pseudo-label is determined as a positive sample to obtain the target domain pseudo-label; when the pseudo-label is inconsistent with the pseudo-label predicted by the preset large model, the pseudo-label is determined as a negative sample, and the negative sample is regularized to obtain the regularized negative sample for fine-tuning training of the new student model.

[0049] It can be understood that the dynamic category pseudo-label screening strategy is an important strategy used in knowledge distillation or semi-supervised learning, and its purpose is to screen high-quality target domain pseudo-labels based on the classification confidence score of each category. During the training process, the new student model usually relies on the pseudo-labels provided by the new teacher model, and the quality of these pseudo-labels is crucial to the training of the new student model. In the dynamic category pseudo-label screening, a dynamic threshold is set for each category, which can be adjusted based on the classification confidence score of each category, and combined with the large model for auxiliary judgment, to screen out high-quality target domain pseudo-labels to guide the training of the new student model, thereby ensuring that only high-quality target domain pseudo-labels are used for the training of the new student model, further improving the performance of the new student model. And after regularization, low-quality negative samples can also be used for fine-tuning training of the new student model, so that the new student model can adapt to low-quality target domain samples.

[0050] For example, see Figure 4As shown in the figure, for the output of the new teacher model, the part with a classification confidence score less than the classification threshold is first regarded as the background and directly filtered. For example, the classification threshold can be 0.1. Then, for each category C, two thresholds are set for it, namely, a high threshold (first threshold) and a low threshold (second threshold). The high threshold will gradually increase as the training progresses, and samples above the high threshold will be regarded as positive samples and directly used as pseudo labels in the target domain for training the new student model; pseudo labels below the low threshold will be regarded as negative samples, which require negative sample regularization and internal mining. For those intermediate samples with classification confidence scores between the low threshold and the high threshold, a large model is introduced to assist in judgment, such as grounding Dino (target detection large model). Through the large model, if the pseudo label predicted by the large model is consistent with the prediction of the current model, the pseudo label can be used as a positive sample; if the prediction of the large model is inconsistent with that of the current model, the pseudo label is regarded as a negative sample, and negative sample regularization and internal mining are performed on it. Among them, through the auxiliary judgment of the large model, it is possible to avoid adding unreliable pseudo labels to the training, ensuring that the label selection of intermediate samples is more accurate.

[0051] It should be noted that the goal of adjusting the high threshold is to gradually improve the screening criteria as the training progresses, ensuring that the model only relies on high-quality pseudo-labels in the later stages of training. For example, the present application adopts an adjustment strategy based on model convergence. When the model has tended to converge on certain categories, that is, when the classification confidence scores for certain categories reach a high level, the increase in the high threshold can be accelerated, allowing the model to rely more on reliable pseudo-labels. When the model converges quickly, that is, the model is able to stably identify certain categories, the increase in the high threshold can be accelerated to reduce the impact of uncertain samples; when the convergence is slow, a lower high threshold can be maintained to allow more samples to participate in training, and the high threshold of category C. Right now: ; in, represents the initial high threshold for category C, Represents a dynamic adjustment factor, which maps the confidence level to the threshold increase ratio through the Sigmoid function to ensure a smooth transition. represents the exponential decay term, is the average confidence of the current category c, reflecting the prediction stability of the model for this category. is the confidence reference value, which is used to control the trigger point of threshold adjustment. k is the convergence rate factor, which is used to control the sensitivity of the high threshold as the confidence level increases. It is the maximum increase coefficient, which is used to limit the increase of the threshold.

[0052] In addition, the main function of the low threshold is to avoid misjudgment of low-confidence samples, so it also needs to be gradually adjusted according to the progress of training. For example, the low threshold can be dynamically adjusted based on the difficulty of each category by using a category sample difficulty adjustment strategy. That is, the low threshold can be dynamically adjusted according to the difficulty of each category. By analyzing the model's learning progress for each category, the low threshold can be dynamically lowered for more difficult-to-identify categories. This strategy ensures that those more difficult-to-identify categories can participate more in training and avoid the model's over-reliance on easily identifiable categories. Therefore, for more difficult-to-identify categories, that is, low-confidence categories, the training can be enhanced by lowering the low threshold. The low threshold of category C Right now: ; in, represents the initial low threshold for category C, represents the attenuation coefficient, Indicates the maximum confidence level of category C, Represents the average confidence of category C.

[0053] It should also be noted that in the regularization of negative samples, the goal is to prevent the model from over-relying on a small number of high-quality positive samples, i.e., high-confidence positive samples, by regularizing those negative samples whose classification confidence scores are lower than a low threshold. This regularization method helps the model avoid overfitting on high-quality positive samples that are easy to classify by introducing some constraints and techniques, thereby improving its learning ability for low-confidence (i.e., uncertain) samples and improving the generalization of the model. By enhancing the model's learning of low-confidence negative samples, it helps the model avoid over-reliance on a small number of high-quality positive samples, thereby improving its learning and generalization capabilities for difficult samples. For example, by assigning higher weights to low-confidence negative samples, these low-confidence negative samples contribute more to model training, so that the model cannot rely solely on high-confidence positive samples, but must learn low-confidence negative samples to improve its performance in difficult situations. The weights Right now: ; in, represents the weight of the i-th negative sample below the low threshold, M represents the total number of negative samples below the low threshold, represents the pseudo label of the i-th negative sample below the low threshold, represents the predicted output of the new student model for the i-th sample, represents the loss function of the i-th negative sample below the low threshold. Larger) gives higher loss weights, forcing the model to learn difficult samples and high confidence samples ( Smaller) reduces the weight to prevent the model from over-relying on simple samples.

[0054] It can be seen that the dynamic category pseudo-label screening strategy can more accurately screen pseudo-labels by setting dynamically adjusted high and low thresholds for each category, and combining large model-assisted judgment and negative sample regularization, avoiding the model's excessive reliance on high-quality positive samples and preventing the model from falling into the noise of pseudo-labels. That is, by introducing large model judgment and mining of intermediate samples, the stability of training and the quality of pseudo-labels are improved, so that the model can be better trained in unsupervised learning tasks and significantly reduce the model's missed detection in extreme weather.

[0055] It should be pointed out that in the average teacher model method, the parameters of the new teacher model are calculated through the parameters of the new student model, and the exponential moving average method is used for smooth update. Compared with the traditional method, this can reduce the fluctuation of the parameters during the training process, so that the new teacher model can provide learning signals more stably. However, due to the shortcomings of the exponential moving average transition smoothness and hysteresis, an adaptive double exponential moving average can be used to adaptively adjust the attenuation rate according to the learning state of the new student model, so that the new teacher model can better adapt to the changes of the new student model during the training process. Specifically, the parameters of the new student model in each training round are obtained in real time; based on the preset adaptive double exponential moving average strategy, the parameters of the new teacher model are dynamically updated using the parameters of the new student model in each training round; wherein, the attenuation rate in the preset adaptive double exponential moving average strategy is dynamically adjusted based on the training round to dynamically control the parameter update speed of the new teacher model; And, the calculation formula of the attenuation rate is: ; in, Indicates the current training round number, Indicates the decay rate under the current number of training rounds, represents the total number of training rounds, represents the minimum value of the attenuation rate, Indicates the maximum value of the decay rate.

[0056] That is, in order to make the decay rate of EMA (Exponential Moving Average) gradually adapt as the training progresses, the decay rate can be dynamically adjusted using the number of training rounds, thereby ensuring that in the early stages of training, the new teacher model can quickly respond to changes in the new student model, and in the later stages of training, the new teacher model tends to be stable.

[0057] It should also be noted that the preset adaptive double exponential moving average strategy is an improved exponential method. Its core idea is to introduce two exponential moving averages and enable them to be dynamically adjusted according to the progress of training. This improvement solves the "over-smoothing" or "slow response" problems that may exist in the traditional exponential moving average method and enhances the adaptability of the new teacher model in knowledge distillation. In the standard EMA, the update of the teacher model is based on the weighted average of historical parameters. The update speed of the parameters is controlled by a fixed decay rate α. Generally, a larger decay rate, such as a decay rate close to 1, means a slower update speed, which helps to smooth the training process of the model, but may also cause the new teacher model to respond too slowly and fail to keep up with the changes of the new student model in time. On the contrary, a smaller decay rate means that the new teacher model adapts to the new student model faster, but may cause the new teacher model to fluctuate too much, so the introduction of the adaptive decay rate aims to dynamically adjust this decay rate so that it can quickly follow up the changes of the new student model in the early stage of training, and smooth the update of the teacher model in the later stage of training to maintain the stability of training. By introducing an adaptive decay rate and two exponential moving update processes, the adaptability and stability of the new teacher model are enhanced. Compared with the traditional exponential moving average, the adaptive double exponential moving average can better balance the flexibility and stability of the new teacher model, adapt to the changes of the new student model during the training process, and optimize the training process by dynamically adjusting the decay rate.

[0058] For example, the adaptive double exponential moving average process uses two different exponential moving averages, namely fast exponential moving and slow exponential moving. The fast exponential moving responds faster to changes in the new student model and can quickly update the new teacher model parameters. The slow exponential moving updates more slowly to maintain the stability of the new teacher model and avoid excessive changes.

[0059] Quick Update Index As shown in the following formula: ; Slow Index Update As shown in the following formula: ; in, and Indicates the decay rate of the adaptive adjustment of the current number of training rounds, which changes dynamically with the training process; Represents the parameters of the new student model.

[0060] That is, in this embodiment, by introducing an adaptive decay rate and two exponential moving update processes, the adaptability and stability of the new teacher model are enhanced. Compared with the traditional exponential moving average, the adaptive double exponential moving average can better balance the flexibility and stability of the new teacher model, adapt to the changes of the new student model during the training process, and optimize the training process by dynamically adjusting the decay rate.

[0061] For example, see Figure 5 As shown, the category statistics and distribution statistics of the original target domain image are obtained by obtaining the corresponding statistical results, and the original target domain image is subjected to targeted data enhancement processing based on the statistical results to obtain a new target domain image, such as using mixup, copy-paste, cutout and other methods for targeted data enhancement. For the source domain image, it is processed by ordinary data enhancement methods to obtain a new source domain image, such as using conventional data enhancement methods such as rotation, translation, mosaic, etc. The original target domain image and the new target domain image are input into the new teacher model and the new student model, and the original source domain image and the new source domain image are input into the new student model for training, where During the training process, the new student model relies on the pseudo-labels provided by the new teacher model, and the quality of these pseudo-labels is crucial to the training of the new student model. Therefore, a preset dynamic category pseudo-label screening strategy is used to screen out high-quality target domain pseudo-labels output by the new teacher model for the training of the new student model. In order to make the new teacher model better adapt to the progress of training, an adaptive double exponential moving average strategy is adopted, which adaptively adjusts the decay rate according to the learning state of the new student model. In addition, in the process of screening pseudo-labels, it is also possible to combine the large model for auxiliary judgment and screen out high-quality pseudo-labels to guide the training of the new student model, thereby further improving the model performance.

[0062] It should be noted that in the training phase, all targeted data enhancement methods are used for target domain images, and conventional data enhancement methods such as rotation, translation, and mosaic are still used for source domain images. As the training progresses, the model gradually converges, and some data enhancement methods can be gradually discarded to stabilize the model. Finally, only mixup and translation enhancement methods are retained for target domain images and source domain images to ensure smooth convergence of the model.

[0063] Step S14: Use the trained target detection model to perform target detection on the image to be detected under extreme weather conditions to obtain corresponding detection results.

[0064] It can be understood that by training the student model for target detection under extreme weather conditions through additional bounding box distillation, diffusion model generation of intermediate images, dynamic category pseudo-label screening strategy and other techniques adopted in the distillation stage, adversarial training stage and fine-tuning training stage, the positioning and classification capabilities in extreme weather scenarios can be greatly improved with only a small increase in parameters and computational complexity, significantly reducing missed detections and false detections.

[0065] It can be seen that in the embodiment of the present invention, distillation enables the student model to have the regression ability of the trained teacher model under extreme weather conditions, and then an intermediate state image that is closer to the target domain and has less noise is generated through a diffusion model, and the intermediate state image is used to perform instance-level feature adversarial training on the pre-trained detection model that has learned the bounding box regression knowledge of the trained teacher model and has the regression ability under extreme weather conditions to reduce the feature difference between the source domain and the target domain to obtain the detection model after adversarial training, and then a new teacher model and a new student model are determined based on the detection model after adversarial training. When the new teacher model is used to generate pseudo labels in the subsequent process, the quality of the pseudo labels generated by the new teacher model can be improved. Then, in the fine-tuning stage, the pseudo labels generated by the new teacher model are screened using a dynamic category pseudo-label screening strategy, and then the high-quality target domain pseudo labels are screened out to train a new student model to obtain a trained target detection model for target detection of images to be detected under extreme weather conditions, thereby greatly reducing the missed detection and false detection phenomena of current technologies in extreme weather scenarios, and the inference speed remains basically unchanged.

[0066] That is, compared with the prior art, the present invention introduces an additional knowledge distillation stage based on the average teacher model, namely, a three-stage training paradigm. First, the positioning capability of the pre-trained detection model is significantly improved through progressive regression box distillation. Secondly, the prior art uses methods based on adversarial learning in the image migration stage, which is very unstable. The present invention uses a generation method based on a diffusion model to make training more stable. The generated intermediate images have less noise and are closer to the target domain, thereby reducing model differences, improving classification accuracy, and further improving positioning capabilities. It also introduces instance-level feature adversarial training, which effectively reduces the interference of background information compared to the global feature alignment and pixel-level feature alignment of the prior art, greatly reduces the difference between the source domain and the target domain, and improves the detection performance of the detection model in the target domain. Finally, in the fine-tuning training stage, the present invention uses targeted attenuation. Compared with the data enhancement of the existing technology, the subtractive data enhancement effectively reduces the impact of category imbalance on the model and ensures that the model can converge smoothly. It also better balances the flexibility and stability of the new teacher model through the adaptive double exponential moving average method, adapts to the changes of the new student model during the training process, and optimizes the training process by dynamically adjusting the attenuation rate. In addition, the single threshold strategy of the existing method is difficult to screen out high-quality pseudo-labels for the tail category. The present invention adopts a dynamic pseudo-label screening strategy with large model assisted judgment and negative sample regularization, which can not only screen out high-quality positive samples of all categories, but also effectively use low-quality negative samples for internal mining, thereby improving the robustness of the model and enhancing the detection capability of the model in extreme weather scenarios.

[0067] In one embodiment, if Figure 6 As shown, based on the above cross-domain target detection method based on semi-supervised learning under extreme weather, the present invention also provides a cross-domain target detection system based on semi-supervised learning under extreme weather, including: The model distillation module 11 is used to distill and transfer the bounding box regression knowledge of the trained teacher model to the student model to obtain a pre-trained detection model with regression capability under extreme weather conditions; wherein the student model is a detection model with detection capability under normal weather conditions, and the trained teacher model is a model obtained by training a detection model with a bounding box regression capability superior to that of the student model using a preset number of extreme weather images with labels; The adversarial training module 12 is used to input the original source domain image under normal weather and the original target domain image under extreme weather into the diffusion model respectively, and input the intermediate state image between the source domain and the target domain output by the diffusion model into the pre-trained detection model to perform instance-level feature adversarial training, so as to obtain the detection model after adversarial training; A model fine-tuning module 13 is used to determine a new teacher model and a new student model based on the detection model after the adversarial training, and use a preset dynamic category pseudo-label screening method to screen the pseudo-labels generated by the new teacher model based on the original target domain image, and use the screened target domain pseudo-labels to fine-tune the new student model to obtain a trained target detection model; The target detection module 14 is used to use the trained target detection model to perform target detection on the image to be detected under extreme weather conditions to obtain corresponding detection results.

[0068] Figure 7 A schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal may include: A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0069] When the processor 502 executes the program, the cross-domain target detection method based on semi-supervised learning under extreme weather conditions provided in the above embodiment is implemented.

[0070] Furthermore, the terminal further includes: The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0071] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0072] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0073] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0074] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0075] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0076] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cross-domain target detection method based on semi-supervised learning under extreme weather as described above.

[0077] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this application. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the claims.

[0078] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction. The logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or device (such as a computer-based system, a system including a processor or other system that can read instructions from an instruction execution system, device or device and execute instructions), or in conjunction with these instruction execution systems, devices or devices.

[0079] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA, Programmable Gate Array), a field programmable gate array (FPGA, Field-Programmable Gate Array), etc.

[0080] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A cross-domain target detection method based on semi-supervised learning in extreme weather, characterized in that: The method comprises: Distilling the bounding box regression knowledge of the trained teacher model to the student model, and obtaining a pre-trained detection model with regression capability under extreme weather conditions; wherein the student model is a detection model with detection capability under normal weather conditions, and the trained teacher model is a model obtained by training a detection model with a bounding box regression capability superior to the student model using a preset number of extreme weather images with labels; The original source domain image under normal weather and the original target domain image under extreme weather are respectively input into the diffusion model, and the intermediate state image between the source domain and the target domain output by the diffusion model is input into the pre-trained detection model for instance-level feature adversarial training to obtain the detection model after adversarial training; Determine a new teacher model and a new student model based on the detection model after the adversarial training, and use a preset dynamic category pseudo-label screening method to screen the pseudo-labels generated by the new teacher model based on the original target domain image, and use the screened target domain pseudo-labels to fine-tune the new student model to obtain a trained target detection model; The trained target detection model is used to perform target detection on the image to be detected under extreme weather conditions to obtain corresponding detection results.

2. The cross-domain target detection method based on semi-supervised learning under extreme weather conditions according to claim 1 is characterized in that: The bounding box regression knowledge distillation of the trained teacher model is transferred to the student model to obtain a pre-trained detection model with regression capability under extreme weather conditions, including: The bounding box regression knowledge of the trained teacher model is distilled and transferred to the student model using a progressive knowledge distillation strategy, thus obtaining a pre-trained detection model with regression capabilities under extreme weather conditions. Among them, the progressive knowledge distillation strategy is a distillation strategy that gradually increases the weight of the regression distillation loss according to the preset loss weight adjustment strategy during the knowledge distillation process, so that the student model gradually has the regression ability under extreme weather conditions, and the initial weight of the regression distillation loss is not greater than the preset weight value.

3. The cross-domain target detection method based on semi-supervised learning under extreme weather conditions according to claim 2 is characterized in that: The calculation formula of the regression distillation loss is: ； in, represents the regression distillation loss, β represents the weight of the regression distillation loss, N represents the total number of samples, represents the regression output of the trained teacher model for the i-th sample, represents the regression output of the student model for the i-th sample, It represents the mean square error between the regression output of the trained teacher model for the i-th sample and the regression output of the student model for the i-th sample.

4. The cross-domain target detection method based on semi-supervised learning under extreme weather conditions according to claim 1 is characterized in that: The teacher model is the RT-DETR model, and the student model is the YOLOv11 model.

5. The cross-domain target detection method based on semi-supervised learning under extreme weather conditions according to claim 1 is characterized in that: The diffusion model is a denoising probabilistic diffusion model, and the diffusion model generates the intermediate state image by adding noise and gradually denoising, and adds content loss and style loss in the denoising process of the diffusion model; The content loss is calculated by comparing the features of the intermediate image with the original target domain image or the original source domain image on the convolutional layer, and the style loss is calculated by comparing the Gram matrix of the intermediate image with the original source domain image or the original target domain image.

6. The cross-domain target detection method based on semi-supervised learning under extreme weather conditions according to claim 1 is characterized in that: Also includes: During the feature adversarial training, a target frame region is extracted from the intermediate state image, and instance-level feature alignment is performed based on a mask of the target frame region and in an adversarial manner between a generator and a discriminator; The discriminator is used to determine whether the features of the target frame area corresponding to the source domain and the target domain are distinguishable, and the generator is used to minimize the determination ability of the discriminator.

7. The cross-domain target detection method based on semi-supervised learning under extreme weather conditions according to claim 1 is characterized in that: In the process of fine-tuning the new student model using the filtered target domain pseudo labels to obtain a trained target detection model, the process also includes: Obtaining the parameters of the new student model in each training round in real time; Based on a preset adaptive double exponential moving average strategy, the parameters of the new teacher model are dynamically updated using the parameters of the new student model in each training round; wherein the decay rate in the preset adaptive double exponential moving average strategy is dynamically adjusted based on the training rounds to dynamically control the parameter update speed of the new teacher model; And, the calculation formula of the attenuation rate is: ； in, Indicates the current training round number, Indicates the decay rate under the current number of training rounds, represents the total number of training rounds, represents the minimum value of the attenuation rate, Indicates the maximum value of the decay rate.

8. The cross-domain target detection method based on semi-supervised learning in extreme weather according to any one of claims 1 to 7, characterized in that: The method of using a preset dynamic category pseudo-label screening method to screen the pseudo-labels generated by the new teacher model based on the original target domain image includes: Performing category statistics and distribution statistics on the original target domain image to obtain corresponding statistical results, and performing targeted data enhancement processing on the original target domain image based on the statistical results to obtain a new target domain image; The pseudo labels generated by the new teacher model based on the original target domain image and the new target domain image are screened using a preset dynamic category pseudo label screening method.

9. The cross-domain target detection method based on semi-supervised learning in extreme weather according to claim 8, characterized in that: The preset dynamic category pseudo label screening method is: Compare the classification confidence score corresponding to when the pseudo label indicates that the target to be detected belongs to the target category with the first threshold and the second threshold corresponding to the target category to obtain corresponding comparison results; wherein the first threshold is not less than the second threshold, and during the fine-tuning training, the first threshold and the second threshold are dynamically adjusted according to a preset threshold dynamic adjustment strategy; When the comparison result indicates that the classification confidence score is higher than the first threshold, the pseudo label is determined as a positive sample to obtain the target domain pseudo label; When the comparison result indicates that the classification confidence score is lower than the second threshold, the pseudo label is determined as a negative sample, and the negative sample is regularized to obtain a regularized negative sample for fine-tuning training of the new student model; When the comparison result shows that the classification confidence score is between the first threshold and the second threshold, determining whether the pseudo label is consistent with the pseudo label predicted by the preset large model; When the pseudo label is consistent with the pseudo label predicted by the preset large model, the pseudo label is determined as a positive sample to obtain the target domain pseudo label; When the pseudo label is inconsistent with the pseudo label predicted by the preset large model, the pseudo label is determined as a negative sample, and the negative sample is regularized to obtain a regularized negative sample for fine-tuning training of the new student model.

10. A cross-domain target detection system based on semi-supervised learning in extreme weather, characterized in that: The system comprises: A model distillation module, for distilling and transferring bounding box regression knowledge possessed by a trained teacher model to a student model, to obtain a pre-trained detection model with regression capability under extreme weather conditions; wherein the student model is a detection model with detection capability under normal weather conditions, and the trained teacher model is a model obtained by training a detection model with a bounding box regression capability superior to that of the student model using a preset number of extreme weather images with labels; An adversarial training module is used to input the original source domain images under normal weather conditions and the original target domain images under extreme weather conditions into the diffusion model respectively, and input the intermediate state images between the source domain and the target domain output by the diffusion model into the pre-trained detection model to perform instance-level feature adversarial training, so as to obtain a detection model after adversarial training; A model fine-tuning module is used to determine a new teacher model and a new student model based on the detection model after the adversarial training, and use a preset dynamic category pseudo-label screening method to screen the pseudo-labels generated by the new teacher model based on the original target domain image, and use the screened target domain pseudo-labels to fine-tune the new student model to obtain a trained target detection model; The target detection module is used to use the trained target detection model to perform target detection on the image to be detected under extreme weather conditions to obtain corresponding detection results.

Images