Passive domain adaptive target detection method based on adaptive information balance enhancement

Abstract

The invention relates to a passive domain self-adaptive target detection method based on self-adaptive information balance enhancement. The passive domain self-adaptive target detection method comprises the following steps of S1, establishing a target detection scene picture database; s2, pre-training a target detection model; s3, constructing a style feature extraction module; s4, training the style feature extraction module by using a target domain data set; s5, constructing a balance enhancement module based on the style feature extraction module; s6, constructing a prediction quality evaluator module; and S7, fusing the balance enhancement module and the prediction quality evaluator module based on a mean value teacher framework to carry out domain migration training. According to the method, the problems of inter-domain information imbalance and pseudo label noise are effectively solved based on the mean teacher network framework in cooperation with the adaptive information balance data enhancement module and the prediction quality evaluator module, and the detection performance of the model on a target domain is improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision and target detection technology, specifically to a passive domain adaptive target detection method based on adaptive information balance enhancement. Background Technology

[0002] With the rapid development of computing power and the continuous expansion of dataset size, object detection (OD) technology has made significant progress in performance. This progress is usually based on an idealized assumption: that the data distribution of the test set and training set remains highly consistent. However, in practical applications, especially in remote sensing scenarios, this assumption often fails due to domain bias, causing models carefully trained in the source domain to show a significant performance decline when applied to the target domain. While fine-tuning the target domain can alleviate this problem, this method often comes with high data collection and labeling costs. To address these challenges, unsupervised domain adaptation (UDA) has gradually become a research hotspot, aiming to enable models to adapt effectively without relying on labeled target domain data. Traditional UDA methods typically assume that both labeled source domain data and unlabeled target domain data can be obtained simultaneously. However, in practice, access to source domain data has become increasingly restricted due to data protection and storage space considerations, prompting the development of source-independent domain adaptation (SFDA) methods, particularly in remote sensing object detection tasks.

[0003] The SFDA (Simultaneous Faculty of Knowledge) approach for two-stage detectors has been extensively explored, but its high computational complexity limits its usability in practical applications. Therefore, recent research interest has shifted towards more efficient single-stage detectors, such as the YOLO series. SF-YOLO, for example, proposed an SFDA solution based on the YOLO architecture and demonstrated its superiority. Most current research on SFDA tends to employ a self-training strategy under the Mean Teacher (MT) model, using exponential moving average (EMA) to update the teacher model and guide the student model's learning process. While the MT framework theoretically ensures continuous optimization of the teacher model, thereby guiding the student model's gradual improvement, it still faces two major challenges in practice:

[0004] First, designing effective data augmentation mechanisms is one of the keys to the success of the MT framework. In particular, remote sensing images are rich in background information. Although style transfer data augmentation methods can improve model performance, improper application may lead to the loss of source domain information or even cause model collapse.

[0005] Secondly, the biases inherent in the source domain pre-trained model are accumulated by the teacher model with each EMA update during the transfer to the target domain, thereby exacerbating the instability of the training process and ultimately affecting model performance. Summary of the Invention

[0006] In view of the above-mentioned technical problems, this invention proposes a passive domain adaptive target detection method based on adaptive information balance enhancement. By introducing an adaptive information balance data augmentation module and a prediction quality evaluator module, combined with the mean teacher framework, the method effectively solves the problems of inter-domain information imbalance and pseudo-label noise, and improves the detection performance of the model in the target domain.

[0007] The technical solution to the technical problem of this invention is: a passive domain adaptive target detection method based on adaptive information balancing enhancement, comprising the following steps:

[0008] Step S1: Establish a database of scene images for target detection;

[0009] Step S2: Pre-train the object detection model;

[0010] Step S3: Construct a style feature extraction module;

[0011] Step S4: Train the style feature extraction module using the target domain dataset;

[0012] Step S5: Construct a balance enhancement module based on the style feature extraction module;

[0013] Step S6: Construct the predictive quality evaluator module;

[0014] Step S7: Based on the mean teacher framework, integrate the balanced enhancement module and the prediction quality evaluator module to perform domain transfer training.

[0015] According to one technical solution of the present invention, step S1 includes:

[0016] Step S11: Collect source domain scene datasets and target domain scene datasets;

[0017] Step S12: Label the source domain scene dataset for model pre-training;

[0018] Step S13: Divide the target domain scene dataset into a training subset and a test subset.

[0019] According to one technical solution of the present invention, step S2 includes:

[0020] Step S21: Set the number of training rounds N1, the mini-batch size m1, use the Adam optimization algorithm, and set the learning rate α1 and momentum parameter. , ;

[0021] Step S22: Initialize the model using YOLO weights pre-trained on the COCO dataset.

[0022] According to one technical solution of the present invention, step S3 includes:

[0023] Step S31: Use the AdaIN style feature extractor to build an encoder based on the VGG16 network;

[0024] Step S32: Calculate the mean of the dataset as a style feature;

[0025] Step S33: Encode the input image and style feature map, and calculate the mean of the features. and variance ;

[0026] Step S34: Calculate the style transfer feature values ​​according to the style transfer formula:

[0027]

[0028] Where x represents the target domain image and y represents the target domain style image. and Representing the features of an image, This represents the features obtained by style transfer of image x based on style image y. and It consists of two learnable neural network layers;

[0029] Step S35: Perform a secondary fusion of the style transfer feature values ​​and the original feature values, and calculate the fusion parameter λ:

[0030] in, Represents a learnable neural network layer;

[0031] Step S36: Perform feature fusion. ;

[0032] Step S37: Use the decoder to restore the fused features to the image, and supervise the training with a loss function.

[0033] According to one technical solution of the present invention, the loss function includes style loss and content loss:

[0034]

[0035]

[0036] in, The result of decoding the original image. For style feature diagrams, The image is the original image, and 'a' is a configurable weight parameter.

[0037] According to one technical solution of the present invention, step S4 includes:

[0038] Step S41: Set the number of training epochs N2, mini-batch size m2, use the Adam optimization algorithm, learning rate α2, and momentum parameter. , ;

[0039] Step S42: Initialize the encoder using pre-trained VGG16 weights, freeze the encoder parameters, and update only the fully connected layers and the decoder.

[0040] According to one technical solution of the present invention, step S5 includes:

[0041] Step S51: Train two style enhancers to learn target domain style enhancement and source domain style enhancement respectively;

[0042] Step S52: Perform full style enhancement on the input image to obtain the source domain style boundary and the target domain style boundary;

[0043] Step S53: Calculate the distance between the input image and the two style boundaries using a distance metric method based on image complexity.

[0044]

[0045] The compression function c uses the PNG compression algorithm; the distance between the input image and the style boundary of the target domain is obtained. and the distance between the input image and the style boundary of the source domain for:

[0046]

[0047] ;

[0048] Step S54: Calculate the adaptive style enhancement parameters:

[0049]

[0050] Step S55: Perform style enhancement based on adaptive style enhancement coefficients:

[0051]

[0052] in, and These are style enhancers for the source and target domains, respectively.

[0053] According to one technical solution of the present invention, step S6 includes:

[0054] Step S61: Divide the prediction results output by the YOLO model into reliable predictions and uncertain predictions, and classify them using the confidence threshold γ;

[0055] Step S62: Redefine the loss function:

[0056]

[0057] in, This represents the loss function computed on unlabeled and reliable images. This represents the loss function computed on an uncertain image, with α used to balance the weighting between the two.

[0058] Step S63: Calculate the loss function on the reliable image. The calculation is as follows:

[0059]

[0060] Where CE represents the cross-entropy loss function, This is the output of the student model. Indicates a pseudo-tag;

[0061] Step S64, Uncertain Prediction Loss The calculation is as follows:

[0062] .

[0063] According to one technical solution of the present invention, step S7 includes:

[0064] Step S71: Set the number of training epochs N3, mini-batch size m3, use the Adam optimization algorithm, learning rate α3, and momentum parameter. , ;

[0065] Step S72: The input image undergoes style enhancement through the balance enhancement module;

[0066] Step S73: Input the enhanced image into the student model for target detection;

[0067] Step S74: Input the original image into the teacher model for target detection;

[0068] Step S75: Classify the student model output results according to the prediction quality evaluator module and calculate the loss function;

[0069] Step S76: The teacher model is updated using an exponential moving average.

[0070]

[0071] Where θ is the teacher model parameter, Θ is the student model parameter, and a is the smoothing coefficient.

[0072] Compared with the prior art, the present invention has the following beneficial effects:

[0073] This invention presents a passive domain adaptive target detection method based on adaptive information balancing enhancement. By introducing an adaptive information balancing data augmentation module, it effectively solves the problem of source domain information loss caused by traditional style transfer methods. This method calculates adaptive style enhancement parameters based on image complexity distance metrics, and can adaptively balance the information between the source and target domains according to the style intensity of the original image, significantly mitigating the performance degradation caused by inter-domain information imbalance.

[0074] This invention designs a prediction quality evaluator module, which effectively reduces the impact of low-quality pseudo-labels on model training by evaluating prediction quality and redefining the loss function. This module categorizes YOLO model outputs into reliable and uncertain predictions, uses a confidence threshold for classification, and optimizes the training process through a weighted loss function. This significantly reduces the accumulation of bias in the source domain pre-trained model within the mean teacher framework, enhancing the model's robustness and generalization ability in the target domain.

[0075] This invention employs a complexity-based distance metric algorithm in the balanced enhancement module, enabling more accurate quantification of image style intensity and thus achieving finer style enhancement control. By calculating the distance between the input image and the style boundaries of the source and target domains and dynamically adjusting the enhancement parameters, this method better adapts to style changes in the target domain while preserving effective features of the source domain. In remote sensing target detection tasks, it improves the model's detection accuracy in complex scenarios such as fog and low light, while maintaining high real-time performance.

[0076] The overall framework of this invention is based on a mean teacher network, integrating adaptive information balancing enhancement and prediction quality assessment mechanisms to achieve efficient passive domain transfer. This method is not only applicable to remote sensing target detection but can also be extended to other domain transfer tasks such as scene classification and semantic segmentation, demonstrating broad applicability. It significantly improves model training stability, achieves fast convergence, and maintains consistent performance across different domains. Attached Figure Description

[0077] Figure 1 This is a flowchart illustrating a passive domain adaptive target detection method based on adaptive information balance enhancement in one embodiment of the present invention.

[0078] Figure 2This is an implementation flow of a passive domain adaptive target detection method based on adaptive information balance enhancement in one embodiment of the present invention;

[0079] Figure 3 This is a target detection effect diagram without domain migration in one embodiment of the present invention;

[0080] Figure 4 The image shows the target detection effect of the SFDA solution based on the YOLO architecture proposed using SF-YOLO in one embodiment of the present invention.

[0081] Figure 5 This is a target detection effect diagram using the adaptive information balance enhancement YOLO model of the present invention in one embodiment of the present invention. Detailed Implementation

[0082] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0083] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The embodiments cannot be described in detail here, but the embodiments of the present invention are not limited to the following embodiments.

[0084] like Figure 1 and Figure 2 As shown, the passive domain adaptive target detection method based on adaptive information balance enhancement of the present invention includes the following steps:

[0085] Step S1: Establish a target detection scene image database, including:

[0086] Step S11: Collect source domain scene datasets and target domain scene datasets;

[0087] Step S12: Label the source domain scene dataset for model pre-training;

[0088] Step S13: Divide the target domain scene dataset into a training subset and a test subset.

[0089] Specifically, two datasets are collected: a source domain scene dataset (such as the Cityscapes dataset) and a target domain scene dataset (such as the Cityscapes-foggy dataset). The target objects to be detected are identified, and data annotation is performed. The source domain scene dataset is used as pre-training data for the model. A portion of the target domain scene dataset is used as test data and does not participate in domain transfer training. This portion of data will be used for model performance evaluation after domain transfer training. The remaining portion is used as training data and participates in domain transfer training.

[0090] Step S2: Perform pre-training of the object detection model, including:

[0091] Step S21: Set the number of training rounds N1, the mini-batch size m1, use the Adam optimization algorithm, and set the learning rate α1 and momentum parameter. , ;

[0092] Step S22: Initialize the model using YOLO weights pre-trained on the COCO dataset;

[0093] The object detection model was pre-trained using a source domain dataset, with YOLO detectors as the base model. The training epochs were N1=200, and the mini-batch size was m1=16. The Adam optimization algorithm was used with a learning rate α1=0.0001 and a momentum parameter... =0.9, =0.999; The model initialization uses YOLO weights pre-trained on the COCO dataset, including a feature extraction network, a region proposal network (if a two-stage detector is used), and a detection head.

[0094] Step S3: Construct a style feature extraction module, including:

[0095] Step S31: Use the AdaIN style feature extractor to extract image style features and achieve style transfer, and build an encoder based on the VGG16 network;

[0096] Step S32: Calculate the mean of the dataset as a style feature;

[0097] The style extractor calculates the mean of the dataset as its style feature. For each image in the dataset, combined with the style feature map, AdaIN encodes both the style feature map and the original image, using a pre-trained VGG16 network as the encoder.

[0098] Step S33: Encode the input image and style feature map, and calculate the mean of the features. and variance Specifically, the feature values ​​of the style feature map and the original image are input into two learnable fully connected networks to calculate the mean of the features. and variance .

[0099] Step S34: Calculate the style transfer feature values ​​according to the style transfer formula:

[0100]

[0101] Where x represents the target domain image and y represents the target domain style image. and Representing the features of an image, This represents the features obtained by style transfer of image x based on style image y. and It consists of two learnable neural network layers;

[0102] Step S35: Perform a secondary fusion of the style transfer feature values ​​and the original feature values, and calculate the fusion parameter λ:

[0103] in, Represents a learnable neural network layer;

[0104] To prevent performance degradation during style enhancement, the feature values ​​of the original image and the style transfer feature values ​​are fused a second time.

[0105] Step S36: Perform feature fusion. ;

[0106] Step S37: Use the decoder to restore the fused features to the image, and supervise the training with the loss function;

[0107] In some embodiments of the present invention, the loss function includes style loss and content loss:

[0108]

[0109]

[0110] in, The result of decoding the original image. For style feature diagrams, The image is the original image, and 'a' is a configurable weight parameter.

[0111] The fused features and original image features are decoded. The decoder is a learnable fully connected network that is responsible for restoring the feature values ​​to the image format. A loss function is used to ensure that the style transfer is successful and that the original image information is not lost.

[0112] Step S4: Train the style feature extraction module using the target domain dataset, including:

[0113] Step S41: Set the number of training epochs N2, mini-batch size m2, use the Adam optimization algorithm, learning rate α2, and momentum parameter. , ;

[0114] The total number of training epochs is set to N² = 1600, the mini-batch size is m² = 16, and the learning rate α² and momentum parameter are used in the Adam optimization algorithm. , α2 = 0.0001, , .

[0115] Step S42: Initialize the encoder using pre-trained VGG16 weights, freeze the encoder parameters, and update only the fully connected layers and the decoder;

[0116] Step S5: Construct a balance enhancement module based on the style feature extraction module, including:

[0117] Step S51: Train two style enhancers to learn target domain style enhancement and source domain style enhancement respectively;

[0118] Two style enhancers are trained. The first style enhancer learns the style enhancement of the target domain scene, and the second style enhancer learns the style enhancement of the source domain scene. Here, the target domain scene data can be used for destylation training.

[0119] Step S52: Perform full style enhancement on the input image to obtain the source domain style boundary and the target domain style boundary;

[0120] Two style enhancers are used to perform full style enhancement on the input image, resulting in two style boundaries: one for the source domain and one for the target domain.

[0121] Step S53: Calculate the distance between the input image and the two style boundaries using a distance metric method based on image complexity.

[0122]

[0123] The compression function c uses the PNG compression algorithm; the distance between the input image and the style boundary of the target domain is obtained. and the distance between the input image and the style boundary of the source domain for:

[0124]

[0125] ;

[0126] Step S54: Calculate the adaptive style enhancement parameters:

[0127]

[0128] Step S55: Perform style enhancement based on adaptive style enhancement coefficients:

[0129]

[0130] in, and These are style enhancers for the source and target domains, respectively.

[0131] Step S6: Construct the predictive quality evaluator module, including:

[0132] Step S61: Divide the prediction results output by the YOLO model into reliable predictions and uncertain predictions, and classify them using the confidence threshold γ;

[0133] Step S62: Redefine the loss function:

[0134]

[0135] in, This represents the loss function computed on unlabeled and reliable images. This represents the loss function computed on an uncertain image, with α used to balance the weighting between the two.

[0136] Step S63: Calculate the loss function on the reliable image. The calculation is as follows:

[0137]

[0138] Where CE represents the cross-entropy loss function, This is the output of the student model. Indicates a pseudo-tag;

[0139] Step S64, Uncertain Prediction Loss The calculation is as follows:

[0140] ;

[0141] Step S7: Based on the mean teacher framework, integrate the balanced enhancement module and the prediction quality evaluator module for domain transfer training, including:

[0142] Step S71: Set the number of training rounds N3, the mini-batch size m3, and use the Adam optimization algorithm with a learning rate α3 and momentum parameters β5 and β6.

[0143] Set the total number of training rounds to N3=60, the mini-batch size to m3=16, and the learning rate α3 and momentum parameter in the Adam optimization algorithm. , α3 = 0.0001, , .

[0144] Step S72: The input image undergoes style enhancement via the balance enhancement module.

[0145] ;

[0146] Step S73: Input the enhanced image into the student model for target detection;

[0147] Step S74: Input the original image into the teacher model for target detection;

[0148] Step S75: Classify the student model output based on the prediction quality evaluator module and calculate the loss function:

[0149]

[0150]

[0151] ;

[0152] Step S76: The teacher model is updated using an exponential moving average.

[0153]

[0154] Where θ is the teacher model parameter, Θ is the student model parameter, and a is the smoothing coefficient.

[0155] To demonstrate the effectiveness and capability of this invention in handling transfer tasks in the field of object detection, a task involving the transfer of data from the Cityscape dataset to the Cityscape-Foggy dataset was performed, completing the object detection task for eight object classes. The Cityscapes and Cityscape-Foggy datasets contain eight categories: 1: person; 2: rider; 3: car; 4: truck; 5: bus; 6: train; 7: motorcycle; 8: bicycle. The images are from 50 different cities, taken in different seasons, and include daytime scenes. They contain approximately 5,000 finely annotated images. Cityscapes-Foggy is a derivative version generated based on the original Cityscapes dataset, designed to simulate scenes under foggy conditions. The Cityscapes-Foggy dataset was created by applying a synthetic fogging effect to the original Cityscapes images. This process preserves all annotation information from the original images. All images are high-resolution images of 1024x2048 pixels.

[0156] The target detection mAP after field migration in this invention can reach 53.9. The target detection results are compared as follows: Figures 3 to 5 As shown.

[0157] In summary, this invention proposes a novel adaptive information balancing enhancement method for object detection in passive domain transfer tasks based on the YOLO series detectors. In adaptive information balancing enhancement YOLO, the object detector can effectively adapt to new scenes. This invention introduces a balancing enhancement module that adaptively balances the information between the source and target domain scenes based on the style intensity of the original image. Balancing enhancement improves the information balance between the two domains, mitigating the source domain data loss problem caused by traditional style enhancement methods. The balancing enhancement module employs a complexity-based distance algorithm to more accurately quantify the intensity of image styles. Furthermore, this invention introduces a prediction quality evaluator to assess prediction quality and reduce the impact of low-entropy data on model training. Although this method primarily focuses on remote sensing object detection tasks, it is theoretically applicable to other tasks involving domain transfer, such as scene classification and semantic segmentation.

[0158] Finally, it should be noted that the above description represents a preferred embodiment of the present invention. It should be pointed out that although preferred embodiments have been described, those skilled in the art, once they understand the basic inventive concept of the present invention, can make various improvements and modifications without departing from the principles described herein. These improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present invention.

Claims

1. A passive domain adaptive target detection method based on adaptive information balancing enhancement, characterized in that, Includes the following steps: Step S1: Establish a database of scene images for target detection; Step S2: Pre-train the object detection model; Step S3: Construct a style feature extraction module; Step S4: Train the style feature extraction module using the target domain dataset; Step S5: Construct a balance enhancement module based on the style feature extraction module; Step S6: Construct the predictive quality evaluator module; Step S7: Based on the mean teacher framework, integrate the balanced enhancement module and the prediction quality evaluator module to perform domain transfer training.

2. The method according to claim 1, characterized in that, Step S1 includes: Step S11: Collect source domain scene datasets and target domain scene datasets; Step S12: Label the source domain scene dataset for model pre-training; Step S13: Divide the target domain scene dataset into a training subset and a test subset.

3. The method according to claim 1, characterized in that, Step S2 includes: Step S21: Set the number of training rounds N1, the mini-batch size m1, use the Adam optimization algorithm, and set the learning rate α1 and momentum parameter. , ; Step S22: Initialize the model using YOLO weights pre-trained on the COCO dataset.

4. The method according to claim 3, characterized in that, Step S3 includes: Step S31: Use the AdaIN style feature extractor to build an encoder based on the VGG16 network; Step S32: Calculate the mean of the dataset as a style feature; Step S33: Encode the input image and style feature map, and calculate the mean of the features. and variance ; Step S34: Calculate the style transfer feature values ​​according to the style transfer formula: Where x represents the target domain image and y represents the target domain style image. and Representing the features of an image, This represents the features obtained by style transfer of image x based on style image y. and It consists of two learnable neural network layers; Step S35: Perform a secondary fusion of the style transfer feature values ​​and the original feature values, and calculate the fusion parameter λ: in, Represents a learnable neural network layer; Step S36: Perform feature fusion. ; Step S37: Use the decoder to restore the fused features to the image, and supervise the training with a loss function.

5. The method according to claim 4, characterized in that, The loss function includes style loss and content loss: in, The result of decoding the original image. For style feature diagrams, The image is the original image, and 'a' is a configurable weight parameter.

6. The method according to claim 1, characterized in that, Step S4 includes: Step S41: Set the number of training epochs N2, mini-batch size m2, use the Adam optimization algorithm, learning rate α2, and momentum parameter. , ; Step S42: Initialize the encoder using pre-trained VGG16 weights, freeze the encoder parameters, and update only the fully connected layers and the decoder.

7. The method according to claim 6, characterized in that, Step S5 includes: Step S51: Train two style enhancers to learn target domain style enhancement and source domain style enhancement respectively; Step S52: Perform full style enhancement on the input image to obtain the source domain style boundary and the target domain style boundary; Step S53: Calculate the distance between the input image and the two style boundaries using a distance metric method based on image complexity. The compression function c uses the PNG compression algorithm; the distance between the input image and the style boundary of the target domain is obtained. and the distance between the input image and the style boundary of the source domain for: ； Step S54: Calculate the adaptive style enhancement parameters: Step S55: Perform style enhancement based on adaptive style enhancement coefficients: in, and These are style enhancers for the source and target domains, respectively.

8. The method according to claim 1, characterized in that, Step S6 includes: Step S61: Divide the prediction results output by the YOLO model into reliable predictions and uncertain predictions, and classify them using the confidence threshold γ; Step S62: Redefine the loss function: in, This represents the loss function computed on unlabeled and reliable images. This represents the loss function computed on an uncertain image, with α used to balance the weighting between the two. Step S63: Calculate the loss function on the reliable image. The calculation is as follows: Where CE represents the cross-entropy loss function, This is the output of the student model. Indicates a pseudo-tag; Step S64, Uncertain Prediction Loss The calculation is as follows: 。 9. The method according to claim 1, characterized in that, Step S7 includes: Step S71: Set the number of training epochs N3, mini-batch size m3, use the Adam optimization algorithm, learning rate α3, and momentum parameter. , ; Step S72: The input image undergoes style enhancement through the balance enhancement module; Step S73: Input the enhanced image into the student model for target detection; Step S74: Input the original image into the teacher model for target detection; Step S75: Classify the student model output results according to the prediction quality evaluator module and calculate the loss function; Step S76: The teacher model is updated using an exponential moving average. Where θ is the teacher model parameter, Θ is the student model parameter, and a is the smoothing coefficient.

Images