Target detection domain adaptation method based on random context consistency reasoning

Through the method of random context consistency inference, combined with semi-supervised learning and teacher-student model framework, the problem of ignoring the characteristics and context correlation of target domains in the existing technology is solved, and the performance of the object detection model in cross-domain adaptation is improved and the performance and reliance of the object detection model in cross-domain adaptation is improved. Great.

CN119313866AActive Publication Date: 2025-01-14HANGZHOU DIANZI UNIV

Patent Information

Application Number
CN202411159473.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-22
Publication Date
2025-01-14
Estimated Expiration
2044-08-22

AI Technical Summary

Technical Problem

Existing target detection pre-adaptation methods ignore specific features and contextual relevance of the target domain when learning domain invariant features, resulting in performance degradation in practical applications, especially in the case of large domain gaps.

Method used

Adopting the object detection domain adaptation method based on random context consistency inference, learning the context of the target domain through the random complementary mask module, the internal context consistency inference module and the external context consistency inference module, combined with the semi-supervised learning teacher-student model framework Relevant knowledge and reduce the dependence of pseudo-labels.

Benefits of technology

The performance of the object detection model in cross-domain adaptation is improved, the capture ability and feature representation ability of target domain-specific features are enhanced, and the robustness and generalization ability of the model are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119313866A_ABST
    Figure CN119313866A_ABST
Patent Text Reader

Abstract

The invention discloses a target detection domain adaptation method based on random context consistency reasoning. A semi-supervised learning teacher-student model framework is adopted, slow updating is performed through index moving average of a teacher model, so that the model can obtain information from main knowledge learned by student models, and a self-supervised effect is achieved. A random complementary mask module is introduced to mask partial local visual features which the network seriously depends on, so that the network is encouraged to utilize other visual clues in the context. In addition, the phenomenon that most self-training frameworks seriously depend on pseudo tags is avoided through an internal context consistency reasoning module, and the self context reasoning ability of the student model is improved by constructing a self-supervised training normal form of the student model. And finally, constructing a self-training framework based on a teacher-student model through an external context consistency reasoning module, and encouraging a student model to make correct prediction on a masked area by utilizing a pseudo tag, with global priori knowledge, of the teacher model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of domain adaptation target detection and image processing, and in particular relates to a target detection domain adaptation method based on random context consistency reasoning. Background Art

[0002] In the field of computer vision, object detection plays a vital role in applications such as autonomous driving, intelligent surveillance, and industrial automation. However, detectors trained on datasets in labeled source domains suffer from severe performance degradation in real-world applications due to domain shift caused by the appearance and texture differences between objects and backgrounds. To address this issue, researchers have turned their attention to object detection domain adaptation techniques, hoping to design a detector that can transfer knowledge learned from labeled source domain data to unlabeled target domains.

[0003] Different from traditional object detection, the key challenge of object detection pre-adaptation lies in learning domain-invariant features to bridge the domain gap while retaining domain-specific features of the target domain to facilitate detection. Mainstream methods attempt to adjust the feature distribution between the source and target domains through pixel-level, instance-level, and image-level adversarial learning. Other methods attempt to mitigate domain shift by modeling prototypes as category centers and leveraging graph matching theory to minimize each distance. However, the above methods focus on extracting domain-invariant features, which may lead to ignoring domain-specific features that reflect the discriminative information of objects in the target domain. In addition, they also ignore contextual correlations between objects and between objects and backgrounds, which can provide powerful discriminative cues for object detection, especially when the domain gap is large.

[0004] Recently, some object detection pre-adaptation methods have introduced a self-training framework and achieved significant performance improvements. The teacher model predicts unlabeled target images to obtain pseudo labels, thereby guiding the student model to generate prediction results close to the pseudo labels, achieving a consistency constraint relationship between the teacher and student models (such as Figure 1 (a). However, the quality of pseudo labels generated by the teacher model is usually very poor, and this single constraint relationship limits the domain adaptation ability of the student model, resulting in unsatisfactory learning results. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a method for object detection domain adaptation based on random context consistency reasoning. The present invention proposes a random context consistency reasoning network that learns context-related knowledge of the target domain (such as Figure 1(b)). First, we introduce a random complementary mask module to generate a pair of complementary masked views by randomly masking some local visual features, thereby encouraging the student model to exploit discriminative visual cues in the context. Second, we design an internal context consistency reasoning module to model the context consistency of the student model's predictions between a pair of complementary masked views, thereby supervising the student model to learn complementary context-related knowledge in the target domain. In addition, we also design an external context consistency reasoning module to establish context consistency between the pseudo-labels generated by the teacher model and the student's predictions, thereby allowing the student model to learn the semantic and spatial information of the context.

[0006] The present invention proposes to use the teacher-student model framework commonly used in the field of semi-supervised learning to perform cross-domain adaptation tasks. The teacher model is slowly updated by the exponential moving average of the student model. The update coefficient α determines the knowledge update speed of the teacher model for the student model. The teacher model is regarded as a robust set of the student model, and the main knowledge learned by the student model can be obtained. Therefore, the prediction output by the teacher model can be used as the pseudo label of the student model to achieve the effect of self-supervision. In the training phase, the source domain and target domain data are trained simultaneously. Half of the source domain data (I s ,y s ), half of the target domain data (I t ,y t ), where the source domain data uses its own labels for supervised training, while the target domain data uses the output of the teacher model to obtain pseudo labels after confidence screening The present invention proposes a random context consistency reasoning network suitable for target detection pre-adaptation technology, which is composed of a random complementary mask module, an internal context consistency reasoning module and an external context consistency reasoning module, such as Figure 2 shown.

[0007] Specifically, the random complementary mask module generates a pair of complementary mask views {V 1 ,V 2 Then, the internal context consistency reasoning module is designed to perform consistency reasoning between the two views predicted by the student model, thereby learning context-related knowledge in a specific domain. Meanwhile, in the external context consistency reasoning module, the teacher model learns context-related knowledge from the original view V with complete context information. 0 (target domain image) and supervise the student model to learn from the complementary mask view {V 1 ,V 2 Therefore, the student model can learn the contextual reasoning ability of the target domain.

[0008] A method for object detection domain adaptation based on random context consistency reasoning, the specific steps are as follows:

[0009] Step 1: Use ResNet50 pre-trained on ImageNet as the backbone network of the student model and the teacher model to extract image features. The detector uses Faster-RCNN. At the beginning of training, the student model is copied to the teacher model.

[0010] Step 2: The labeled source domain data (I s ,y s ) Input the student model and output the test result y' s With label y s Calculate the supervised loss L sup , and conduct supervised training; at the same time, the unlabeled target domain data (I t ,none) Input the teacher model and judge the output result: when the predicted result has a target with a confidence greater than δ, it will be screened out as a pseudo label And proceed to step 3, otherwise continue to step 2.

[0011] Step 3: Perform random context consistency reasoning: The random complementary mask module first generates a mask matrix M sampled from a uniform distribution, and then subtracts M from the all-ones matrix to obtain the complementary mask matrix By comparing with the input target image I t Calculate the Kadampa product to obtain a pair of complementary mask images, that is, generate a pair of complementary mask views {V 1 ,V 2}.

[0012] Step 4: Guide the student model from complementary and asymmetric complementary mask views {V 1 ,V 2}, which includes classification and regression consistency reasoning. Specifically, we first transform the complementary mask view {V 1 ,V 2 The student model trained in step 2 is input to generate the corresponding predictions. Subsequently, the internal context consistency reasoning loss is calculated for the predictions of the complementary mask view. Specifically divided into: classification consistency reasoning loss and regression consistency inference loss To avoid mismatched consistency reasoning, we set a larger IoU matching threshold τ = 0.75.

[0013] Step 5: The external context consistency reasoning module uses the pseudo labels generated by the teacher model With the student model for the complementary mask view {V 1 ,V 2}’s predictions compute the external context consistency inference loss Specifically, we use the pseudo-labels generated by the teacher model to calculate the classification consistency inference loss and regression consistency inference loss

[0014] Step 6: Comprehensively train and update the student model and teacher model by combining supervised loss, internal context consistency reasoning loss and external context consistency reasoning loss to improve the cross-domain adaptability of the teacher and student models.

[0015] After steps 2 to 5, an iterative training is completed. Repeat the iterative training until the set number of iterations is reached. In each iteration, by calculating L sup , as well as Perform gradient back propagation to update the student model, and then update the teacher model through exponential moving average. sup Enhanced the baseline capability of the student model in the source domain; A self-supervision paradigm was constructed using the student model, which effectively reduced the model's reliance on pseudo labels. The pseudo-labels of the teacher model are used to encourage the student model to predict the masked area, thereby enhancing the student model's ability to capture specific details in the target domain and enhancing the student model's feature representation ability, thereby improving the expression of domain-specific features. These loss functions work together to fully represent the domain-specific features in the target domain while enhancing the extraction of domain-invariant features, thereby improving the cross-domain adaptability of the teacher and student models.

[0016] The beneficial effects of the present invention are as follows:

[0017] The present invention innovatively proposes a teacher-student model network with random context consistency reasoning to solve the problem of domain adaptation in target detection. This method adopts a teacher-student model framework of semi-supervised learning, and slowly updates the teacher model through the exponential moving average of the teacher model, so that the model can obtain information from the main knowledge learned by the student model, thereby achieving a self-supervisory effect. By introducing a random complementary mask module, some local visual features that the network relies heavily on are masked, thereby encouraging the network to use other visual clues in the context. In addition, the internal context consistency reasoning module designed by the present invention effectively avoids the phenomenon of heavy reliance on pseudo-labels in most self-training frameworks, and improves the context reasoning ability of the student model itself by constructing a self-supervised training paradigm for the student model. Finally, the external context consistency reasoning module designed by the present invention effectively constructs a self-training framework based on the teacher-student model, and uses the pseudo-labels with global prior knowledge of the teacher model to encourage the student model to make correct predictions for the masked area. Through the above three modules, the network can have good cross-domain adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of the random context consistency reasoning strategy;

[0019] Figure 2 It is an overall framework diagram of an embodiment of the present invention;

[0020] Figure 3 It is a comparative test of the performance convergence curves between different methods;

[0021] Figure 4 The figure is a visualization of the detection result of the embodiment of the present invention. DETAILED DESCRIPTION

[0022] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0023] First, the specific details of the implementation of the embodiment are described:

[0024] In the embodiment of the present invention, we adopt the unsupervised domain adaptation standard. The training data is divided into two parts: source domain training data, where images and their labels (bounding boxes and instance categories) are provided, and target domain training data, where there are only unlabeled images. The adjusted model is tested on a test set with the same data distribution as the target domain training data. In all experiments, we use mean average precision (mAP) as the main evaluation metric with a threshold of 0.5.

[0025] The teacher-student model of the present invention is isomorphic. The present invention uses ResNet-50 as the backbone network of the teacher-student model and adopts FPN to divide the features into different scales. All training and test images are resized so that the length of the shorter side is 800 pixels to fit into the GPU memory. The teacher-student model is initialized using weights pre-trained on ImageNet. Each batch contains 1 source domain image and 1 target domain image. The model is trained for 60,000 iterations with an initial learning rate of 0.0025 and a weight decay of 0.0001. The exponential moving average EMA weight α=0.9 is used for color enhancement (brightness, contrast, saturation, hue and blur). The output confidence threshold δ=0.8 of the pseudo-label box is set in the experiment, and the experiment is conducted on RTX 4080Ti.

[0026] Then introduce the required data set:

[0027] The proposed method is evaluated by studying the domain differences between weather conditions. Weather is an important source of domain differences because the scene will be visually different as the weather conditions change. Whether the detection system can operate reliably in different weather conditions is crucial for a safe autonomous driving system. The experiment uses Cityscapes as the source domain, where images are mainly taken on sunny days. In this experiment, the ground-truth bounding box is obtained by fitting the most compact rectangle around the corresponding instance mask.

[0028] For the target domain, we used the Foggy Cityscapes dataset. Foggy Cityscapes is a synthetic foggy dataset that simulates fog in real scenes. Foggy images are created using original sunny images from Cityscapes as well as depth maps. The semantic annotations and data splitting of Foggy Cityscapes are inherited from Cityscapes, which makes it ideal for studying domain differences caused by visibility changes. Foggy Cityscapes contains multiple versions, each with a fixed fog density level, but in this experiment, we choose the version with the most challenging fog density level corresponding to the visibility distance of 150 meters. Therefore, we finally use the training set of Cityscapes as the (labeled) source domain training set, and the training set of Foggy Cityscapes as the (unlabeled) target domain training set.

[0029] Here are the specific implementation steps:

[0030] Step 1: Perform supervised training: Take the labeled source domain image I s Input the student model and convert the detection result y' of the student model s With label y s Calculate the loss, the supervised loss is:

[0031] L sup =L cla (C' S ,C s )+L reg (B' S ,B s )

[0032] Where L cla and L reg Respectively represent the classification loss and regression loss of Faster-CRNN, and the detection result y' s Contains the predicted bounding box location B' s and target box type C' s , B s , C s is the label y s Bounding box locations and categories in .

[0033] At the same time, the unlabeled target domain image I t Input the teacher model, and select the detection boxes with a classification confidence δ higher than δ>0.8 as pseudo labels according to the classification confidence δ output by the teacher model

[0034]

[0035] Step 2: The random complementary mask module first creates an all-zero matrix, and then samples the area greater than r from the uniform distribution and sets it to 1, thereby obtaining a mask matrix M:

[0036]

[0037] where u is sampled from a uniform distribution U(0,1), b is the size of the mask patch, r is the mask ratio in (0,1), is the following table of the mask block, H and W are the height and width of the input image.

[0038] Then use the all-ones matrix to subtract M to get the complementary mask matrix

[0039]

[0040] Among them, 1 represents an all-one matrix with the same dimension as M, and the gamma product between the mask matrix and the target domain image is used to obtain a pair of complementary mask views {V 1 ,V 2}:

[0041] V 1 =M⊙I t

[0042]

[0043] Step 3: Obtain a pair of complementary mask views {V 1 ,V 2}, the internal context consistency reasoning module guides the student model to reason about context-related information from complementary and asymmetric target views, including classification and regression consistency reasoning. To avoid mismatched consistency reasoning, we set the IoU matching threshold τ = 0.75. Specifically, the complementary mask view {V 1 ,V 2} input to the student model to generate the corresponding prediction, and the student model obtains the complementary mask view {V 1 ,V 2} Make predictions and get the predicted bounding box set calculate and The IoU matrix of the bounding boxes in is calculated and the maximum value on the longest edge is selected as their matching score. If the score exceeds the preset value τ, the two bounding boxes are considered to be matched successfully.

[0044] In order to reduce the class probability vector p of the complementary mask view M and The category distribution diverges between , so the classification consistency reasoning loss is calculated:

[0045]

[0046] Among them, K represents and The number of bounding boxes matched between yes The class probability vector of the i-th bounding box in , and represents the corresponding category prediction, w i is the confidence of the prediction between (0,1). For regression consistency loss, we use Huber loss to calculate {V 1 ,V 2 Deviation loss of the predicted bounding box matched in}:

[0047]

[0048] in, is a collection of bounding boxes A single bounding box in .

[0049] The Huber loss H σ is defined as:

[0050]

[0051] Among them, x represents the difference in bounding box coordinates and σ represents the turning point. Huber loss reduces the model's sensitivity to outliers during initial training and adaptively increases the penalty for the student model.

[0052] The total loss of the internal context consistency reasoning module is:

[0053]

[0054] Step 4: The external context consistency reasoning module uses the pseudo labels generated by the teacher model With the student model for the complementary mask view {V 1 ,V 2}’s predictions compute the external context consistency inference loss First, the student model is optimized using the classification consistency reasoning loss to reason about the contextual semantic information in the occluded view:

[0055]

[0056] Where N and M represent the complementary mask views {V 1 ,V 2 The number of matches between the bounding box set in} and the bounding box set in the pseudo-label, express The class probability vector of the i-th bounding box in, represents the corresponding target category in the pseudo-label, λ and μ are weights to balance the number of targets in the two views, and the regression consistency loss is:

[0057]

[0058] in They are respectively the student model for the complementary mask view {V 1 ,V 2}The set of predicted bounding boxes, is the i-th bounding box among them.

[0059] The total loss of the external context consistency reasoning module is:

[0060]

[0061] Step 5: In this step, we first use the calculated supervised loss L sup , internal context consistency reasoning loss and external context consistency reasoning loss Perform gradient back propagation to update the parameters of the student model. This process is a crucial part of deep learning training. Through gradient back propagation, the model is continuously optimized in each round of iteration, gradually improving its detection performance.

[0062] Next, the exponential moving average is introduced to update the teacher model.

[0063] The teacher model does not use gradient backpropagation to update parameters, but uses a smoothing method to update parameters. By taking the exponential moving average of the student model parameters, the teacher model gradually accumulates knowledge from multiple rounds of iterations. This accumulation of knowledge makes the teacher model a robust set of student models, which brings together the key knowledge learned by the student model and further improves the stability and robustness of the teacher model's cross-domain adaptation. The formula for each iterative update is as follows:

[0064]

[0065] in represents the parameters of the teacher model, represents the parameters of the student model, α is the weighted weight value, representing the update rate of the model, and the present invention adopts the default value of 0.9996.

[0066] After a series of operations from step 1 to step 5, one round of iteration is completed. According to the verification of the embodiment, the present invention adopts 60,000 iterations.

[0067] After training with the internal and external context consistency reasoning modules, the student model can learn a powerful feature representation capability, which is not only reflected in the extraction of domain-invariant features between different domains, but also in the representation of domain-specific features of the target domain, thereby acquiring powerful cross-domain adaptability.

[0068]

[0069]

[0070] Table 1 Cross-domain adaptation experiment of Cityscapes → Foggy Cityscapes (visible at 150 meters)

[0071] Table 1 shows the cross-domain quantitative experiments conducted by the embodiments of the present invention on the Cityscapes and Foggy Cityscapes datasets, where Cityscapes is the source domain and Foggy Cityscapes is the target domain. This embodiment uses mean average precision (mAP) as the main evaluation indicator with a threshold of 0.5. It can be found that the performance of the present invention is better than other state-of-the-art methods, thus proving the effectiveness of the random context consistency reasoning network.

[0072] Figure 3 This is a comparative test of the performance convergence curves between different methods. It can be found that the embodiments of the present invention can converge quickly and the performance is continuously improved. Figure 4 This is a visualization of the detection results of the embodiment. It can be found that the present invention can accurately identify easily confused categories, and has accurate positioning and good recognition effect.

[0073] Object detection is an indispensable core component of autonomous driving technology. In an autonomous driving system, driverless cars must obtain real-time information about the road and traffic environment through visual sensors (such as cameras, lidar, etc.) in order to make correct driving decisions. The main task of object detection technology is to identify and track various traffic elements during driving, such as traffic signs, other vehicles, pedestrians, bicycles, etc., to assist autonomous vehicles in making accurate driving decisions.

[0074] However, in the actual road environment, many factors (such as weather, road conditions, etc.) will affect the quality and feature distribution of the image, which may cause the captured image to differ from the dataset used when the model was trained. This difference causes the performance of the target detection model in the actual scene to be inferior to that on the standard dataset, which is the so-called domain adaptation problem.

[0075] Our proposed method aims to enhance the robustness and generalization ability of autonomous driving systems. This method uses unlabeled target domain data for domain adaptation and improves the robustness of cross-domain detection by training deep learning models, so that the model can be effectively applied to real-world scenarios with limited training data.

[0076] The above contents are further detailed descriptions of the present invention in combination with specific / preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, they can also make several substitutions or modifications to these described embodiments without departing from the concept of the present invention, and these substitutions or modifications should be regarded as belonging to the protection scope of the present invention.

[0077] Parts of the present invention that are not described in detail belong to the well-known technologies of those skilled in the art.

Claims

1. A domain adaptation method for object detection based on random context consistency reasoning, characterized in that: The specific steps are as follows: Step 1: Use ResNet50 pre-trained on ImageNet as the backbone network of the student model and the teacher model to extract image features. The detector uses Faster-RCNN. At the beginning of training, copy the student model to the teacher model. Step 2: The labeled source domain data (I s ,y s ) Input the student model and output the test result y' s With label y s Calculate the supervised loss L sup , and conduct supervised training; at the same time, the unlabeled target domain data (I t ,nome) inputs the teacher model and judges the output result: when the predicted result has a target with a confidence greater than δ, it is screened out as a pseudo label And proceed to step 3, otherwise continue to step 2. Step 3: Perform random context consistency reasoning: The random complementary mask module first generates a mask matrix M sampled from a uniform distribution, and then subtracts M from the all-ones matrix to obtain the complementary mask matrix By comparing with the input target image I t Calculate the Kadampa product to obtain a pair of complementary mask images, that is, generate a pair of complementary mask views {V1, V2}; Step 4: Guide the student model to infer context-relevant information from complementary and asymmetric complementary mask views {V1, V2} through the internal context consistency reasoning module, which includes classification and regression consistency reasoning; specifically, first input the complementary mask view {V1, V2} into the student model trained in step 2 to generate the corresponding prediction; then, calculate the internal context consistency reasoning loss for the prediction of the complementary mask view Specifically divided into: classification consistency reasoning loss and regression consistency inference loss Step 5: The external context consistency reasoning module uses the pseudo labels generated by the teacher model Compute the external context consistency inference loss with the student model’s predictions for the complementary mask views {V1, V2} Specifically, the classification consistency inference loss is calculated using the pseudo labels generated by the teacher model and regression consistency inference loss Step 6: Comprehensively train and update the student model and teacher model by using supervised loss, internal context consistency reasoning loss, and external context consistency reasoning loss to improve the cross-domain adaptability of the teacher and student models; After steps 2 to 5, an iterative training is completed. Repeat the iterative training until the set number of iterations is reached. In each iteration, by calculating L sup , as well as Perform gradient backpropagation to update the student model, and then update the teacher model through exponential moving average; supervised loss L sup Enhanced the baseline capability of the student model in the source domain; A self-supervision paradigm was constructed using the student model, which effectively reduced the model's reliance on pseudo labels. The pseudo-labels of the teacher model are used to encourage the student model to predict the masked area, thereby enhancing the student model's ability to capture specific details in the target domain and enhancing the feature representation ability of the student model, thereby improving the expression of domain-specific features; these loss functions cooperate with each other to fully characterize the domain-specific features in the target domain while enhancing the extraction of domain-invariant features, thereby improving the cross-domain adaptability of the teacher and student models.

2. According to claim 1, a method for object detection domain adaptation based on random context consistency reasoning is characterized in that: In step 4, the IoU matching threshold τ=0.75 is set.

3. According to claim 1, a method for object detection domain adaptation based on random context consistency reasoning is characterized in that: The specific method of step 2 is as follows: Perform supervised training: The labeled source domain image I s Input the student model and convert the detection result y' of the student model s With label y s Calculate the loss, the supervised loss is: L sup =L cla (C' S ,C s )+L reg (B' S ,B s ) Where L cla and L reg Respectively represent the classification loss and regression loss of Faster-CRNN, and the detection result y' s Contains the predicted bounding box location B' s and target box type C' s , B s , C s is the label y s The bounding box position and category in ; At the same time, the unlabeled target domain image I t Input the teacher model, and select the detection boxes with a classification confidence δ higher than δ>0.8 as pseudo labels according to the classification confidence δ output by the teacher model 4. The object detection domain adaptation method based on random context consistency reasoning according to any one of claims 1 to 3, characterized in that: The specific method of step 3 is as follows: The random complementary mask module first creates an all-zero matrix, and then samples regions larger than r from a uniform distribution and sets them to 1, thereby obtaining a mask matrix M: where u is sampled from a uniform distribution U(0,1), b is the size of the mask patch, r is the mask ratio in (0,1), is the following table of the mask block, H and W are the height and width of the input image; Then use the all-ones matrix to subtract M to get the complementary mask matrix Among them, 1 represents an all-one matrix with the same dimension as M, and the gamma product between the mask matrix and the target domain image is used to obtain a pair of complementary mask views {V1, V2}: V1=M⊙I t 5. According to claim 4, a method for object detection domain adaptation based on random context consistency reasoning is characterized in that: The specific method of step 4 is as follows: After obtaining a pair of complementary mask views {V1, V2}, the student model is guided to infer context-related information from complementary and asymmetric target views through the internal context consistency reasoning module, including classification and regression consistency reasoning; in order to avoid mismatched consistency reasoning, the IoU matching threshold τ = 0.75 is set; specifically, the complementary mask view {V1, V2} is input into the student model to generate the corresponding prediction, and the student model predicts the obtained complementary mask view {V1, V2} to obtain a set of predicted bounding boxes calculate and The IoU matrix of the bounding box in the image is calculated, and the maximum value on the longest edge is selected as their matching score; if the score exceeds the preset value τ, the two bounding boxes are considered to be matched successfully; In order to reduce the class probability vector p of the complementary mask view M and The category distribution diverges between , so the classification consistency reasoning loss is calculated: Among them, K represents and The number of bounding boxes matched between yes The class probability vector of the i-th bounding box in , and represents the corresponding category prediction, w i is the confidence of the prediction between (0,1); for the regression consistency loss, the Huber loss is used to calculate the deviation loss of the matching predicted bounding box in {V1,V2}: in, is a collection of bounding boxes A single bounding box in ; The Huber loss H σ is defined as: Among them, x represents the bounding box coordinate difference, σ represents the turning point; The total loss of the internal context consistency reasoning module is:

6. The object detection domain adaptation method based on random context consistency reasoning according to claim 5 is characterized in that: Step 5: The external context consistency reasoning module uses the pseudo labels generated by the teacher model Compute the external context consistency inference loss with the student model’s predictions for the complementary mask views {V1, V2} First, the student model is optimized using the classification consistency reasoning loss to reason about the contextual semantic information in the occluded view: Among them, N and M represent the number of matches between the bounding box set in the complementary mask view {V1, V2} inferred by the student model and the bounding box set in the pseudo-label, express The class probability vector of the i-th bounding box in, represents the corresponding target category in the pseudo-label, λ and μ are weights to balance the number of targets in the two views, and the regression consistency loss is: in are the set of bounding boxes predicted by the student model for the complementary mask views {V1, V2}, is the i-th bounding box; The total loss of the external context consistency reasoning module is:

7. The object detection domain adaptation method based on random context consistency reasoning according to claim 6 is characterized in that: The specific method of introducing exponential moving average to update the teacher model is as follows: The teacher model updates parameters in a smooth manner. By taking the exponential moving average of the student model parameters, the teacher model gradually accumulates knowledge from multiple rounds of iterations. The formula for each iteration update is as follows: in represents the parameters of the teacher model, represents the parameters of the student model, and α is the weighted weight value.

Citation Information

Patent Citations

  • Feature and instance joint transfer learning method in semi-supervised scene

    CN110533193A

  • Infrared weak and small target detection method and device based on semi-supervised mixed domain adaptation

    CN116310655A

  • Semi-supervised domain adaptive semantic segmentation method for double-level alignment active learning

    CN117058373A

  • Target detection domain adaptation method based on background and foreground cutting and interchanging

    CN118196371A

Cited By

  • Cross-domain target detection method and system based on semi-supervised learning in extreme weather

    CN119863615A

  • Cross-Domain Object Detection Method and System Based on Semi-Supervised Learning in Extreme Weather

    CN119863615B

  • Remote sensing image target detection method and system based on consistency relation reasoning

    CN119863617A

  • Image semantic segmentation active domain adaptation method and system based on segmentation all-in-one model

    CN120953731A

  • Domain adaptive target detection method and system based on complementary mask and loop training

    CN122135009A