Unsupervised domain adaptive target detection method and system based on double-path structure

Abstract

The invention provides an unsupervised domain adaptive target detection method and system based on a double-path structure, and the method comprises the steps: carrying out the target detection of an unmarked target domain data set through an unsupervised domain adaptive target detection model of the double-path structure, performing label filtering fusion on the invariant feature detection result set and the feature detection result set obtained by detection to obtain a target false label of the target domain data set; training a model by using the source domain data set, the target domain data set and the target pseudo-tag, and updating the target pseudo-tag once when a new target pseudo-tag obtained based on target detection is trained once; and inputting the to-be-detected target domain image into the trained model for detection to obtain a target detection result, performing label filtering fusion on the result obtained by dual-path detection to obtain a pseudo label with high confidence, and performing training fine tuning on the model by using the data set with the pseudo label. The recall ratio and precision ratio of model detection can be improved, and then the detection precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and machine learning, and in particular to an unsupervised domain adaptation target detection method and system based on a dual-path structure. Background Art

[0002] Deep convolutional neural networks have demonstrated excellent performance in object detection tasks, but these results are usually based on the assumption that the feature distribution of the source domain data used for training models is similar to that of the target domain data used for testing models. In the real world, due to changes in brightness, style, color, shooting angle, etc., there are often differences in the feature distribution of the source domain and the target domain. This phenomenon is called domain shift. The domain shift problem affects the flexibility and generalization ability of object detectors, bringing challenges to practical applications.

[0003] Through unsupervised domain adaptation target detection methods for target detection, knowledge is transferred from the source domain to the target domain to solve the domain shift problem. This type of method can effectively improve the domain adaptability of the detector without the need for target domain annotation.

[0004] At present, there are two methods to construct unsupervised domain adaptation target detection models in unsupervised domain adaptation target detection methods for target detection, one is based on adversarial learning, and the other is based on pseudo-label self-training. The adversarial learning-based method can only confuse and calibrate domain-invariant features at a computable granularity through training, but cannot achieve alignment for finer-grained and difficult-to-quantify features. The inherent differences between the source domain and the target domain still exist, and over-emphasizing the alignment of the edges of feature distribution may weaken the adaptability of the model, ultimately resulting in low accuracy of domain adaptation target detection. Learning target domain-specific features based on pseudo-label self-training can improve the professionalism of the target detection model and make it more adaptable to the target domain, but this method will inevitably be interfered by the internal noise of the pseudo-labels. The inaccuracy of classification and positioning labels will significantly hinder the convergence of self-training and affect the accuracy of domain adaptation target detection.

[0005] Therefore, both existing methods will affect the accuracy of domain adaptation target detection, resulting in inaccurate target detection results. Summary of the invention

[0006] In order to overcome the above-mentioned deficiencies of the prior art, the present invention proposes an unsupervised domain adaptation target detection method based on a dual-path structure, comprising:

[0007] Performing target detection on an unlabeled target domain dataset using an unsupervised domain adaptation target detection model with a dual-path structure, thereby obtaining an invariant feature detection result set and a feature detection result set of the target domain dataset;

[0008] Performing label filtering and fusion on the invariant feature detection result set and the feature detection result set of the target domain data set to obtain a target pseudo label of the target domain data set;

[0009] Using the labeled source domain dataset, the unlabeled target domain dataset, and the target pseudo-label, the unsupervised domain adaptation target detection model is trained to obtain a trained unsupervised domain adaptation target detection model;

[0010] The target domain image to be detected is input into the trained unsupervised domain adaptation target detection model to perform target detection, and obtain the target detection result of the target domain image to be detected.

[0011] Optionally, the labeled source domain dataset includes multiple source domain images, category labels and position labels of targets in the source domain images;

[0012] The unlabeled target domain dataset includes a plurality of target domain images.

[0013] Optionally, the unsupervised domain adaptation target detection model with a dual-path structure performs target detection on an unlabeled target domain dataset to obtain an invariant feature detection result set and a feature detection result set of the target domain dataset, including:

[0014] For each target domain image in the target domain dataset, using a feature extraction backbone network in the unsupervised domain adaptation target detection model to extract basic features of the target domain image;

[0015] Utilizing the source domain guided invariant feature detection path in the unsupervised domain adaptation target detection model, obtaining invariant features in the target domain image based on the basic features; performing target detection on the invariant features to obtain invariant feature detection results of the target domain image;

[0016] Utilizing the target domain guided feature detection path in the unsupervised domain adaptation target detection model, the target domain features of the target domain image are acquired based on the basic features; fusing the target domain features and the invariant features; performing target detection on the fused target domain features and the invariant features to obtain a feature detection result of the target domain image;

[0017] The invariant feature detection result and the feature detection result of each of the target domain images are summarized to obtain an invariant feature detection result set and a feature detection result set of the target domain data set.

[0018] Optionally, the using the source domain guided invariant feature detection path in the unsupervised domain adaptation target detection model to obtain invariant features in the target domain image based on the basic features; performing target detection on the invariant features to obtain invariant feature detection results of the target domain image includes:

[0019] Using the invariant feature differentiation network in the source domain guided invariant feature detection path, the basic features of the target domain image are converted into invariant features;

[0020] The invariant feature detection network in the invariant feature detection path guided by the source domain is used to detect the invariant features of the target domain image, and the invariant feature detection result of the target domain image is output.

[0021] Optionally, the step of utilizing the target domain guided feature detection path in the unsupervised domain adaptation target detection model to obtain target domain features of the target domain image based on the basic features; fusing the target domain features and invariant features; and performing target detection on the fused target domain features and invariant features to obtain feature detection results of the target domain image includes:

[0022] Using a target domain feature differentiation network in the target domain guided feature detection path, the basic features of the target domain image are converted into target domain features;

[0023] fusing the target domain features and invariant features of the target domain image to obtain fused features;

[0024] The target domain feature detection network in the target domain guided feature detection path is used to perform target detection on the fused features to obtain a feature detection result of the target domain image.

[0025] Optionally, the using the labeled source domain dataset, the unlabeled target domain dataset, and the target pseudo-label to train the unsupervised domain adaptation target detection model to obtain a trained unsupervised domain adaptation target detection model includes:

[0026] Inputting the labeled source domain dataset and the unlabeled target domain dataset into the unsupervised domain adaptation target detection model, obtaining an invariant feature detection result of each source domain image in the source domain dataset and each target domain image in the target domain dataset through the invariant feature detection path guided by the source domain, and obtaining a feature detection result of each target domain image in the target domain dataset through the feature detection path guided by the target domain;

[0027] Calculating source domain detection loss based on the category and position of the target in the invariant feature detection result of each source domain image, as well as the category label and position label of the target in each source domain image; iteratively updating the invariant feature detection network using the source domain detection loss;

[0028] Based on the category and position of the target in the feature detection result of each target domain image and the target pseudo-label of the target domain data set, a target domain detection loss is calculated; the target domain feature differentiation network and the target domain feature detection network are iteratively updated using the target domain detection loss; and based on the invariant feature detection result and feature detection result of each target domain image in the target domain data set, a new target pseudo-label of the target domain data set is obtained for updating the target pseudo-label of the target domain data set;

[0029] The domain discrimination network in the invariant feature detection path guided by the source domain is used to judge the domain category to which the image input into the invariant feature detection path guided by the source domain belongs, and the total discrimination loss is calculated based on the judgment result. The image is a source domain image or a target domain image. The feature extraction backbone network, the invariant feature differentiation network and the invariant feature detection network are updated based on the total discrimination loss.

[0030] Optionally, the domain discrimination network includes a pixel-level domain discrimination network, an instance-level domain discrimination network, and a category-level discrimination network; using the domain discrimination network to judge the domain category to which the image input into the invariant feature detection path guided by the source domain belongs, and calculating the total discrimination loss based on the judgment result, comprises:

[0031] Using the pixel-level domain discrimination network to determine the domain to which the pixel feature in the image input into the invariant feature detection path guided by the source domain belongs, and calculating the pixel-level domain discrimination loss based on the determination result;

[0032] Using the instance-level domain discrimination network, determining the domain to which the instance feature in the image input into the source domain guided invariant feature detection path belongs, and calculating the instance-level domain discrimination loss based on the determination result;

[0033] Using the category-level discriminant network, determining the category to which the instance feature in the image input into the source domain guided invariant feature detection path belongs, and calculating the category-level discriminant loss based on the determination result;

[0034] The pixel-level domain discrimination loss, the instance-level domain discrimination loss, and the category-level discrimination loss are superimposed to obtain the total discrimination loss.

[0035] Optionally, converting the basic features of the target domain image into invariant features includes:

[0036] Determining the channel attention weights of the basic features of the target domain image by using the channel attention mechanism of the invariant feature differentiation network;

[0037] Determine the spatial attention weights of the basic features of the target domain image by using the spatial attention mechanism of the invariant feature differentiation network;

[0038] Using the basic features of the target domain image, the channel attention weights of the basic features, and the spatial attention weights, the basic features are converted into invariant features;

[0039] The expression for converting the basic features into invariant features is:

[0040] F IF =W c (x)·x·W s (W c (x)·x)

[0041] Among them, F IF represents the unchanged features after the basic features are transformed; x represents the basic features; W c (x) represents the channel attention weight of the basic feature; W s (W c (x)·x) represents the spatial attention weight after the basic feature is multiplied by the channel attention weight.

[0042] Optionally, performing label filtering and fusion on the invariant feature detection result set and the feature detection result set of the target domain data set to obtain a target pseudo label of the target domain data set includes:

[0043] According to the intersection of the invariant feature detection result set and the feature detection result set, and the confidence of each invariant feature detection result in the invariant feature detection result set and the confidence of each feature detection result in the feature detection result set, the invariant feature detection result set and the feature detection result set are hierarchically filtered;

[0044] The invariant feature detection result set and the feature detection result set after hierarchical filtering are fused by non-maximum suppression to obtain the target pseudo label of the target domain data set.

[0045] Based on the same inventive concept, the present invention proposes an unsupervised domain adaptation target detection system based on a dual-path structure, comprising:

[0046] A target domain data set detection module is used to perform target detection on an unlabeled target domain data set through an unsupervised domain adaptation target detection model with a dual-path structure, and obtain an invariant feature detection result set and a feature detection result set of the target domain data set;

[0047] A pseudo-label acquisition module is used to perform label filtering and fusion on the invariant feature detection result set and the feature detection result set of the target domain data set to obtain a target pseudo-label of the target domain data set;

[0048] A model training module is used to train the unsupervised domain adaptation target detection model using the labeled source domain dataset, the unlabeled target domain dataset and the target pseudo-label to obtain a trained unsupervised domain adaptation target detection model;

[0049] The target domain image detection module is used to input the target domain image to be detected into the trained unsupervised domain adaptation target detection model to perform target detection and obtain the target detection result of the target domain image to be detected.

[0050] Optionally, the labeled source domain dataset includes multiple source domain images, category labels and position labels of targets in the source domain images; and the unlabeled target domain dataset includes multiple target domain images.

[0051] Optionally, the target domain data set detection module includes:

[0052] A feature extraction unit, configured to extract basic features of each target domain image in the target domain dataset by using a feature extraction backbone network in the unsupervised domain adaptation target detection model;

[0053] A source domain path detection unit is used to obtain the invariant features in the target domain image based on the basic features by using the invariant feature detection path guided by the source domain in the unsupervised domain adaptation target detection model; perform target detection on the invariant features to obtain the invariant feature detection result of the target domain image;

[0054] a target domain path detection unit, configured to obtain target domain features of the target domain image based on the basic features by using the target domain guided feature detection path in the unsupervised domain adaptation target detection model; fuse the target domain features and invariant features; perform target detection on the fused target domain features and invariant features to obtain feature detection results of the target domain image;

[0055] The summarizing unit is used to summarize the invariant feature detection results and feature detection results of each of the target domain images to obtain an invariant feature detection result set and a feature detection result set of the target domain data set.

[0056] Optionally, the source domain path detection unit includes:

[0057] an invariant feature differentiation subunit, configured to transform the basic features of the target domain image into invariant features by using the invariant feature differentiation network in the invariant feature detection path guided by the source domain;

[0058] The invariant feature detection subunit is used to detect the invariant features of the target domain image using the invariant feature detection network in the invariant feature detection path guided by the source domain, and output the invariant feature detection result of the target domain image.

[0059] Optionally, the target domain path detection unit includes:

[0060] a feature differentiation subunit, configured to transform the basic features of the target domain image into target domain features by using a target domain feature differentiation network in the target domain guided feature detection path;

[0061] A feature fusion subunit, used for fusing the target domain feature and the invariant feature of the target domain image to obtain a fused feature;

[0062] The feature detection subunit is used to perform target detection on the fused features using the target domain feature detection network in the target domain guided feature detection path to obtain a feature detection result of the target domain image.

[0063] Optionally, the model training module includes:

[0064] a detection unit, configured to input the labeled source domain dataset and the unlabeled target domain dataset into the unsupervised domain adaptation target detection model, obtain an invariant feature detection result of each source domain image in the source domain dataset and each target domain image in the target domain dataset through the invariant feature detection path guided by the source domain, and obtain a feature detection result of each target domain image in the target domain dataset through the feature detection path guided by the target domain;

[0065] an invariant feature detection network updating unit, configured to calculate a source domain detection loss based on the category and position of the target in the invariant feature detection result of each source domain image, and the category label and position label of the target in each source domain image; and iteratively update the invariant feature detection network using the source domain detection loss;

[0066] A target domain path updating unit calculates a target domain detection loss based on the category and position of the target in the feature detection result of each target domain image and the target pseudo-label of the target domain data set; iteratively updates the target domain feature differentiation network and the target domain feature detection network using the target domain detection loss; and obtains a new target pseudo-label of the target domain data set based on the invariant feature detection result and feature detection result of each target domain image in the target domain data set, for updating the target pseudo-label of the target domain data set;

[0067] A source domain path updating unit is used to use the domain discrimination network in the invariant feature detection path guided by the source domain to determine the domain category to which the image input into the invariant feature detection path guided by the source domain belongs, and calculate the total discrimination loss based on the judgment result, wherein the image is a source domain image or a target domain image, and update the feature extraction backbone network, the invariant feature differentiation network and the invariant feature detection network based on the total discrimination loss.

[0068] Optionally, the source domain path updating unit includes a total discrimination loss obtaining subunit, which is specifically used to:

[0069] Using the pixel-level domain discrimination network to determine the domain to which the pixel feature in the image input into the invariant feature detection path guided by the source domain belongs, and calculating the pixel-level domain discrimination loss based on the determination result;

[0070] Using the instance-level domain discrimination network, determining the domain to which the instance feature in the image input into the source domain guided invariant feature detection path belongs, and calculating the instance-level domain discrimination loss based on the determination result;

[0071] Using the category-level discriminant network, determining the category to which the instance feature in the image input into the source domain guided invariant feature detection path belongs, and calculating the category-level discriminant loss based on the determination result;

[0072] The pixel-level domain discrimination loss, the instance-level domain discrimination loss, and the category-level discrimination loss are superimposed to obtain the total discrimination loss.

[0073] Optionally, the invariant feature differentiation subunit is specifically used for:

[0074] Determining the channel attention weights of the basic features of the target domain image by using the channel attention mechanism of the invariant feature differentiation network;

[0075] Determine the spatial attention weights of the basic features of the target domain image by using the spatial attention mechanism of the invariant feature differentiation network;

[0076] Using the basic features of the target domain image, the channel attention weights of the basic features, and the spatial attention weights, the basic features are converted into invariant features;

[0077] The expression for converting the basic features into invariant features is:

[0078] F IF =W c (x)·x·W s (W c (x)·x)

[0079] Among them, F IF represents the unchanged features after the basic features are transformed; x represents the basic features; W c (x) represents the channel attention weight of the basic feature; W s (W c (x)·x) represents the spatial attention weight after the basic feature is multiplied by the channel attention weight.

[0080] Optionally, the pseudo-label acquisition module is specifically used to:

[0081] According to the intersection of the invariant feature detection result set and the feature detection result set, and the confidence of each invariant feature detection result in the invariant feature detection result set and the confidence of each feature detection result in the feature detection result set, the invariant feature detection result set and the feature detection result set are hierarchically filtered;

[0082] The invariant feature detection result set and the feature detection result set after hierarchical filtering are fused by non-maximum suppression to obtain the target pseudo label of the target domain data set.

[0083] In another aspect, the present application further provides an electronic device, comprising: at least one processor and a memory; the memory and the processor are connected via a bus;

[0084] The memory is used to store one or more programs;

[0085] When the one or more programs are executed by the at least one processor, an unsupervised domain adaptation target detection method based on a dual-path structure as described above is implemented.

[0086] On the other hand, the present application also provides a computer-readable storage medium having an execution program stored thereon, which, when executed, implements an unsupervised domain adaptation target detection method based on a dual-path structure as described above.

[0087] Compared with the closest prior art, the present invention has the following beneficial effects:

[0088] The present invention provides an unsupervised domain adaptation target detection method and system based on a dual-path structure, comprising: performing target detection on an unlabeled target domain data set through an unsupervised domain adaptation target detection model with a dual-path structure, performing label filtering and fusion on a set of invariant feature detection results and a set of feature detection results of the target domain data set obtained by detection, and obtaining a target pseudo-label of the target domain data set; training the unsupervised domain adaptation target detection model using a labeled source domain data set, an unlabeled target domain data set, and a target pseudo-label; inputting a target domain image to be detected into the trained model for target detection, and obtaining a target detection result of the target domain image to be detected. The present invention obtains a set of invariant feature detection results and a set of feature detection results of the target domain data set through dual-path detection, performs label filtering and fusion on the invariant feature detection results and the feature detection results, and obtains a pseudo-label with a high confidence level; and fine-tuning the model by using a continuously updated data set with pseudo-labels to train the model, which can improve the recall rate and precision rate of the model detection, thereby improving the detection accuracy of the model; and the dual paths in the unsupervised domain adaptation target detection model can improve the training and detection efficiency by simultaneous training. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 A schematic diagram of a flow chart of an unsupervised domain adaptation target detection method based on a dual-path structure provided by the present invention;

[0090] Figure 2 A schematic diagram of the target detection framework of the unsupervised domain adaptation target detection model based on a dual-path structure;

[0091] Figure 3 Schematic diagram of the structure of the invariant domain feature differentiation network and the target domain feature differentiation network;

[0092] Figure 4 It is a schematic diagram of the structure of an unsupervised domain adaptation target detection system based on a dual-path structure;

[0093] Figure 5 The present invention provides a schematic structural diagram of an electronic device. DETAILED DESCRIPTION

[0094] Currently, among the unsupervised domain adaptation methods for target detection, the adversarial learning-based methods can only confuse and calibrate domain-invariant features at a computable granularity through training, but cannot achieve alignment for finer-grained and difficult-to-quantify features. The inherent differences between the source domain and the target domain still exist, and over-emphasizing the alignment of the edge of the feature distribution will weaken the model's adaptability.

[0095] Learning domain-specific features can improve the professionalism of the model and make it more adaptable to the target domain. Self-training through pseudo-labels is a method to learn target domain-specific features. However, this method will inevitably be disturbed by the internal noise of pseudo-labels, and the inaccuracy of classification and positioning labels will significantly hinder the convergence of self-training.

[0096] The present invention proposes an unsupervised domain adaptation target detection method and system based on a dual-path structure, aiming to alleviate the adverse effects of coercion in adversarial learning on its adaptability and solve the problem of difficult convergence of self-training due to label noise, thereby improving the accuracy of unsupervised domain adaptation target detection by simultaneously learning domain-invariant features and target domain-specific features.

[0097] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0098] Example 1

[0099] The present invention provides an unsupervised domain adaptation target detection method based on a dual-path structure, such as Figure 1 As shown, including:

[0100] S1. Performing target detection on an unlabeled target domain dataset through an unsupervised domain adaptation target detection model with a dual-path structure to obtain an invariant feature detection result set and a feature detection result set of the target domain dataset;

[0101] S2, performing label filtering and fusion on the invariant feature detection result set and the feature detection result set of the target domain data set to obtain a target pseudo label of the target domain data set;

[0102] S3, using the labeled source domain data set, the unlabeled target domain data set and the target pseudo label, training the unsupervised domain adaptation target detection model to obtain a trained unsupervised domain adaptation target detection model;

[0103] S4. Input the target domain image to be detected into the trained unsupervised domain adaptation target detection model to perform target detection, and obtain a target detection result of the target domain image to be detected.

[0104] In step S1, before performing target detection on the unlabeled target domain dataset, the method further includes: obtaining a labeled source domain dataset and an unlabeled target domain dataset.

[0105] The labeled source domain dataset includes multiple source domain images, category labels and position labels of targets in the source domain images; the unlabeled target domain dataset only includes multiple target domain images.

[0106] Among them, the target categories of interest in the source domain images and the target domain images are the same, and the images in the two domain datasets are obviously different in brightness, style, color, shooting angle, etc.

[0107] like Figure 2 As shown, the unsupervised domain adaptation target detection model of the dual-path structure used in the present invention includes:

[0108] Feature extraction backbone network (i.e. Figure 2 A shared backbone network in the dataset is used to extract features from images in the dataset to obtain basic features in the images. In this embodiment, the feature extraction backbone network adopts VGG (Visual Geometry Group, a deep convolutional network) and ResNet (Residual Network) structures to extract features from the source domain images and the target domain images input therein, and output their basic features;

[0109] A source domain guided invariant feature detection path, used to detect a target based on invariant features between the source domain dataset and the target domain dataset;

[0110] A target domain guided feature detection path for detecting a target based on features of the target domain dataset;

[0111] The feature interaction enhancement network is used to fuse the target domain features extracted by the invariant feature detection path guided by the source domain (the invariant feature detection results of the target domain image) and the target domain features extracted by the feature detection path guided by the target domain (the feature detection results of the target domain image).

[0112] Furthermore, the source domain guided invariant feature detection path includes:

[0113] An invariant feature differentiation network, used to differentiate the basic features of the image extracted by the feature extraction backbone network into invariant features;

[0114] A domain discrimination network is used to distinguish the domain category to which the input image belongs, and align the features of the source domain and target domain images output by the feature extraction backbone network and the invariant feature differentiation network through gradient back propagation;

[0115] The invariant feature detection network is used to detect the invariant features output by the invariant feature differentiation network, and output the category and position of the target in the detected image (invariant feature detection result).

[0116] Furthermore, the target domain guided feature detection path includes:

[0117] A target domain feature differentiation network is used to differentiate the basic features of the image extracted by the feature extraction backbone network into target domain features;

[0118] The target domain feature detection network is used to detect the target domain features output by the target domain feature differentiation network, and output the category and position of the target in the detected image (feature detection result); in the present invention, the target domain feature detection network is used to perform target detection on the fused target domain features and invariant features, and output the feature detection result.

[0119] The steps for constructing the target domain feature differentiation network are consistent with the steps for constructing the invariant feature differentiation network, but the two networks do not share parameters.

[0120] Among them, the domain discrimination network is used to determine whether the input feature comes from the source domain or the target domain, and calculate the total discrimination loss (denoted as L). In other normal model training, the loss L is back-propagated, and after multiple iterations, the feature extraction backbone network tends to extract easily distinguishable source domain features and target domain features, so that the domain discrimination network can make judgments. However, the original intention of "adversarial training" is the opposite. In the present invention, "adversarial training" is embodied in that after the domain discrimination network calculates the loss L, it is normally propagated in the domain discrimination network during the back-propagation process, that is, it is expected that the domain discrimination network can more accurately distinguish between source domain features and target domain features. However, when passing through the gradient reversal layer, the gradient brought by the loss is reversed (for example, the original gradient is 2, and the gradient after reversal is -2), and the reversed gradient continues to propagate to the invariant feature detection network, the invariant feature differentiation network, and the backbone network, in the expectation that these networks confuse the processed source domain features and target domain features, making them difficult to distinguish, so as to achieve the purpose of aligning the feature edge distribution and migrating knowledge from the source domain to the target domain.

[0121] In step S1, the unsupervised domain adaptation target detection model with the dual-path structure is used to perform target detection on unlabeled target domain data.

[0122] S11. For each target domain image in the target domain dataset, extract basic features of the target domain image using a feature extraction backbone network in the unsupervised domain adaptation target detection model.

[0123] S12. Utilizing the invariant feature detection path guided by the source domain, based on the basic features, obtain the invariant features in the target domain image; and perform target detection on the invariant features to obtain an invariant feature detection result of the target domain image.

[0124] Specifically, the invariant feature differentiation network in the invariant feature detection path guided by the source domain is used to transform (differentiate) the basic features of the target domain image into invariant features, such as Figure 3 As shown:

[0125] In the channel attention mechanism of the invariant feature differentiation network ( Figure 3Part A in the figure), perform maximum pooling and average pooling on each feature channel of the basic feature, input the results of the maximum pooling and average pooling into the fully connected layer for learning, where ReLu represents an activation function, and the result of the fully connected layer output is processed by the activation function sigmoid mapping to obtain the channel attention weight of the basic feature; the specific calculation formula for calculating the channel attention weight of the input basic feature x is:

[0126] W c (x) = σ(F MLP (AP(x))+f MLP (MP(x)))

[0127] Among them, W c (x) represents the channel attention weight of the basic feature x, σ represents the sigmoid activation function; f MLP represents a fully connected layer, AP(·) and MP(·) represent average pooling and maximum pooling of feature channels.

[0128] In the spatial attention mechanism of the invariant feature differentiation network ( Figure 3 Part B in the above), perform maximum pooling and average pooling on each feature channel of the basic feature, splice the results of the maximum pooling and average pooling, perform a convolution operation on the spliced ​​result, and process the result of the convolution operation by an activation function sigmoid to obtain the spatial attention weight of the basic feature; the specific calculation formula for calculating the basic feature spatial attention weight is:

[0129]

[0130] Among them, W s (x) represents the spatial attention weight of the basic feature x; is a dilated convolution with kernel k=7×7 and dilation rate d=2; σ represents the sigmoid activation function; AP(·) and MP(·) represent average pooling and maximum pooling of feature channels.

[0131] The basic features are converted into invariant features by using the basic features of the target domain image, the channel attention weights of the basic features, and the spatial attention weights;

[0132] The expression for converting the basic features into invariant features is:

[0133] F IF =W c (x)·x·W s (W c (x)·x)

[0134] Among them, F IFrepresents the unchanged features after the basic features are transformed; x represents the basic features; W c (x) represents the channel attention weight of the basic feature; W s (W c (x)·x) represents the spatial attention weight after the basic feature is multiplied by the channel attention weight.

[0135] Similarly, if the basic features of the source domain image are to be differentiated (converted) into invariant features through the invariant feature differentiation network, the conversion is performed using the steps of converting the basic features of the target domain image into invariant features.

[0136] The invariant feature detection network in the invariant feature detection path guided by the source domain is used to detect the invariant features of the target domain image, and the invariant feature detection result of the target domain image is output.

[0137] S13. Utilizing the target domain-guided feature detection path, obtain the target domain features of the target domain image based on the basic features; fuse the target domain features and invariant features of the target domain image to obtain fused features; perform target detection on the fused features to obtain feature detection results of the target domain image.

[0138] Specifically, the basic features of the target domain image are converted into target domain features using a target domain feature differentiation network in the target domain guided feature detection path;

[0139] The target domain features and the invariant features of the target domain image are fused, and the calculation formula for feature fusion is:

[0140]

[0141] Among them, F t represents the fused features of the target domain feature detection network input, that is, the fused target domain features and invariant features, represents the invariant features of the target domain image output after being enhanced by the invariant feature differentiation network, The target domain features of the target domain image output after the target domain feature differentiation network is enhanced are shown. Represents element-wise summation of a vector.

[0142] The target domain feature detection network in the target domain guided feature detection path is used to perform target detection on the fused features to obtain a feature detection result of the target domain image.

[0143] S14. Summarize the invariant feature detection results and feature detection results of each of the target domain images to obtain an invariant feature detection result set and a feature detection result set of the target domain data set.

[0144] In step S2, label filtering and fusion are performed on the invariant feature detection result set and the feature detection result set of the target domain data set to obtain a target pseudo label of the target domain data set.

[0145] S21, according to the intersection of the invariant feature detection result set and the feature detection result set, and the confidence of each invariant feature detection result in the invariant feature detection result set and the confidence of each feature detection result in the feature detection result set, the invariant feature detection result set and the feature detection result set are hierarchically filtered; the expression of the hierarchical filtering is:

[0146]

[0147] in, represents the set of detection results after hierarchical filtering, which represents targets with low confidence but detected by both the source domain path and the target domain path (detection results), filtering out unreliable targets with low confidence and detected by only a single path (the invariant feature detection path guided by the source domain or the feature detection path guided by the target domain); It represents the second set of detection results after hierarchical filtering, indicating high confidence, but there may be only one target detected by a single path, filtering out unreliable targets with low confidence; m,q represents the initial detection result of the qth target output by the invariant feature detection path guided by the source domain; y m,k represents the initial detection result of the kth target output by the feature detection path guided by the target domain, and the detection result set output by the invariant feature detection path guided by the source domain is The detection result set output by the target domain guided feature detection path is T T(m) ={y m,k |k∈[1,K]}, Q and K represent the total number of detection results of the two detection paths respectively; c I(m,q) represents the detection category of the qth target output by the invariant feature detection path guided by the source domain; c T(m,k) represents the detection category of the kth target output by the feature detection path guided by the target domain; c I(m,q) =c T(m,k) Indicates the category of the predicted target obtained by dual-path detection of the same m image; It means to filter out all detection results of the same category, whose IoU (Intersection over Union) is greater than the threshold and whose confidence is greater than the threshold; represents the bounding box of the qth object output by the source domain guided invariant feature detection path; represents the bounding box of the kth target output by the target domain guided feature detection path; represents the bounding box, R 4 P represents a four-dimensional vector, the coordinates of the center point of the bounding box (horizontal and vertical) and the length and width of the bounding box; c It represents the confidence of the detection category c. The confidence threshold β0 is less than the confidence threshold β1. In this embodiment, the value of β0 is 0.65 and the value of β1 is 0.8.

[0148] S22, performing non-maximum suppression fusion on the invariant feature detection result set and the feature detection result set after hierarchical filtration to obtain a target pseudo label of the target domain data set.

[0149] The expression of the target pseudo label of the target domain dataset obtained by non-maximum suppression is:

[0150]

[0151] in, Represents the target pseudo label of the target domain dataset; NMS (Non-Maximum Suppression) represents non-maximum suppression. Indicates the first set of test results after hierarchical filtering; represents the second set of detection results after hierarchical filtering; θ represents the IoU threshold of non-maximum suppression. In this embodiment, θ is 0.1.

[0152] In step S3, the unsupervised domain adaptation target detection model is trained using the labeled source domain dataset, the unlabeled target domain dataset, and the target pseudo-label to obtain a trained unsupervised domain adaptation target detection model.

[0153] In this embodiment, VGG-16 (Visual Geometry Group-16 network), ResNet-50 (Residual Network, residual network-50) and ResNet-101 (Residual Network, residual network 101) are selected as the basic network, and Faster R-CNN (Fast Region-based Convolutional Neural Network, fast regional convolutional neural network) is used as the detector (detection network) to build an unsupervised domain adaptation target detection model and train it. The construction process refers to the structure of the unsupervised domain adaptation target detection model in step S1.

[0154] The labeled source domain dataset and the unlabeled target domain dataset are input into the unsupervised domain adaptation target detection model, and the invariant feature detection results of each source domain image in the source domain dataset and each target domain image in the target domain dataset are obtained through the invariant feature detection path guided by the source domain, and the feature detection results of each target domain image in the target domain dataset are obtained through the feature detection path guided by the target domain.

[0155] Specifically, Figure 2 As shown, the labeled source domain dataset and the unlabeled target domain dataset are input into the unsupervised domain adaptation target detection model, and the basic features of each source domain image in the source domain dataset and each target domain image in the target domain dataset are extracted using the feature extraction backbone network;

[0156] The source domain path (the invariant feature detection path guided by the source domain) and the target domain path (the feature detection path guided by the target domain) share the same feature extraction backbone network, the basic features of the source domain image extracted by the feature extraction backbone network are input into the source domain path, and the basic features of the target domain image extracted by the feature extraction backbone network are input into both the source domain path and the target domain path;

[0157] Using the invariant feature differentiation network in the source domain path, the basic features of the source domain image and the target domain image are differentiated into invariant features; using the invariant feature detection network in the source domain path, detection is performed based on the invariant features of the source domain image and the target domain image, and the invariant feature detection results of the source domain image and the target domain image are output, wherein the invariant feature detection results include the category and position of the target in the image;

[0158] The target domain feature differentiation network in the target domain path is used to differentiate the basic features of the target domain image into target domain features, the invariant feature domain and the target domain features of the target domain image are fused, the fused invariant feature domain and the target domain features are input into the target domain feature detection network in the target domain path, and the feature detection result of the target domain image is output, wherein the feature detection result includes the category and position of the target in the target domain image.

[0159] At the same time, the source domain path includes a domain discrimination network, which is used to distinguish the domain category to which the input image belongs, and align the features of the source domain image and the target domain image output by the feature extraction backbone network and the invariant feature differentiation network through gradient back propagation. That is, the domain discrimination network determines the domain category to which the image input into the source domain path belongs, obtains the total discrimination loss, performs gradient back propagation alignment features based on the total discrimination loss, and updates the feature extraction backbone network, the invariant feature differentiation network, and the invariant feature detection network.

[0160] Specifically, the domain discrimination network includes a pixel-level domain discrimination network, an instance-level domain discrimination network, and a category-level discrimination network;

[0161] The pixel-level domain discrimination network is used to determine the domain to which the pixel features in the image input into the invariant feature detection path guided by the source domain belong, and calculate the pixel-level domain discrimination loss based on the judgment result. The calculation formula of the pixel-level domain discrimination loss is:

[0162]

[0163] in, represents pixel-level domain discrimination loss; pix represents pixel; Represents the label of the pixel in the i-th image, the image feature from the source domain, that is, the label y of the pixel in the image i = 0, image feature y from the target domain i =1; represents the invariant features of the image in the source domain data extracted by the source domain guided invariant feature detection path, represents the invariant features of the image in the target domain data extracted by the invariant feature detection path guided by the source domain, (i, j) represents the jth local feature of the i-th image, which refers to the pixel feature here; D pix Represents a pixel-level domain discriminant network, which is used to predict (detect) the domain category of each pixel feature; represents the domain discrimination prediction result of the jth pixel feature in the i-th image from the source domain, It represents the domain discrimination prediction result of the j-th pixel feature in the i-th image from the target domain, and its value range is [0,1].

[0164] The instance-level domain discrimination network is used to determine the domain to which the instance feature in the image input into the invariant feature detection path guided by the source domain belongs, and calculate the instance-level domain discrimination loss based on the judgment result; the calculation formula of the instance-level domain discrimination loss is:

[0165]

[0166] in, represents the instance-level domain discrimination loss; Represents the label of the instance in the i-th image, the image feature from the source domain (i.e., the label of the instance in the image) y i = 0, image feature y from the target domain i =1; represents the invariant features of the image in the source domain data extracted by the source domain guided invariant feature detection path, represents the invariant features of the image in the target domain data extracted by the invariant feature detection path guided by the source domain, (i, j) represents the jth local feature of the i-th image, which refers to the instance feature here, and Dins represents the instance-level domain discriminator, predicting the domain category for each instance feature.

[0167] The category-level discriminant network is used to determine the class to which the instance features in the image of the invariant feature detection path guided by the input source domain belong, and the category-level discriminant loss is calculated based on the judgment result. The target category label of the strength feature in the target domain data comes from the detection result of the invariant feature detection network.

[0168] The pixel-level domain discrimination loss, the instance-level domain discrimination loss and the category-level discrimination loss are superimposed to obtain the total discrimination loss; based on the total discrimination loss, gradient back-propagation is performed to align features, and the feature extraction backbone network, the invariant feature differentiation network and the invariant feature detection network are updated. It is expected that these networks will confuse the processed source domain features and target domain features, making them difficult to distinguish, so as to achieve the purpose of aligning feature edge distribution and transferring knowledge from the source domain to the target domain. And compared with aligning edge distribution only through adversarial training, it provides a finer feature granularity alignment, that is, it considers the data differences between the two domains in a more detailed manner, optimizes the learning of target domain features, improves the adaptability of the model in the target domain, and makes the detection accuracy higher.

[0169] Based on the category and position of the target in the invariant feature detection result of the source domain image, as well as the category label and position label of the target in the source domain image, the source domain detection loss is calculated; the invariant feature detection network is iteratively updated using the source domain detection loss; through continuous training and continuous iterative updating, the invariant feature detection network is enabled to have the ability to detect targets; the invariant feature detection network has the ability to detect source domain targets before it has the ability to detect target domain targets.

[0170] Based on the category and position of the target in the feature detection result of the target domain image and the target pseudo label of the target domain data set, the target domain detection loss is calculated; the target domain feature differentiation network and the target domain feature detection network are iteratively updated using the target domain detection loss so that they are good at detecting targets in the target domain;

[0171] And based on the invariant feature detection result and the feature detection result of each target domain image in the target domain dataset, a new target pseudo-label of the target domain dataset is obtained, which is used to update the target pseudo-label of the target domain dataset, that is, each time the target domain feature differentiation network and the target domain feature detection network in the unsupervised domain adaptation target detection model are trained once, the target pseudo-label is also updated once, and the pseudo-label is continuously updated and generated in each batch during training. The generation method of the pseudo-label each time training is performed refers to step S1-step S2.

[0172] In step S4, the target domain image to be detected is input into the trained unsupervised domain adaptation target detection model to perform target detection, and obtain the target detection result of the target domain image to be detected.

[0173] Specifically, the target domain image to be detected is input into the trained unsupervised domain adaptation target detection model to perform target detection, and an invariant feature detection result and a feature detection result of the target domain image to be detected are obtained;

[0174] The invariant feature detection result and the feature detection result of the target domain image to be detected are subjected to label filtering and fusion to obtain the target detection result of the target domain image to be detected, wherein the method for performing label filtering and fusion on the invariant feature detection result and the feature detection result refers to the filtering and fusion method of step S2.

[0175] The present invention combines the advantages of adversarial learning and self-training, uses the invariant feature detection results and feature detection results to perform label filtering and fusion, obtains pseudo labels with high confidence, and uses the data set with pseudo labels to train and fine-tune the model, which can improve the recall and precision of model detection, and obtain target detection results with higher recall and precision; and the domain discrimination network obtains the total discrimination loss by analyzing the domain discrimination losses at the pixel level, instance level and category level, and updates multiple networks from the path, which provides a finer feature granularity alignment than aligning the edge distribution only through adversarial training, that is, it considers the data difference between the two domains more carefully, realizes knowledge transfer from the source domain to the target domain, optimizes the learning of the target domain features, improves the adaptability of the model in the target domain, and makes the detection accuracy higher; and in the present invention, the dual paths share the feature extraction backbone network and are trained at the same time, which can improve the training and detection efficiency; and the source domain path learns the invariant features between domains through adversarial training, and the target domain path learns the target domain features through training, which can improve the generalization ability of the model.

[0176] Example 2

[0177] An unsupervised domain adaptation target detection method based on a dual-path structure provided by the present invention is subjected to an ablation experiment in a domain adaptation scenario from the source domain Cityscapes (a widely used street view image dataset) to the target domain Foggy Cityscapes (a variant created based on the Cityscapes dataset, which changes the visual features of the Cityscapes image by simulating foggy conditions), respectively verifying the effects of the domain discrimination network, the invariant feature differentiation network, the target domain feature differentiation network, the target domain guided feature detection path, and the pseudo-label fusion strategy proposed in the present invention. The experimental results are shown in Table 1, where the values ​​shown are the average detection accuracy (mAP) and the average detection recall (mRec) of all categories of targets. The results show that the above-mentioned key technologies proposed by the present invention have an important improvement in improving the model performance in this unsupervised domain adaptation target detection scenario.

[0178]

[0179] Example 3

[0180] Based on the same inventive concept, the present invention also provides an unsupervised domain adaptation target detection system based on a dual-path structure, such as Figure 4 As shown, including:

[0181] A target domain data set detection module is used to perform target detection on an unlabeled target domain data set through an unsupervised domain adaptation target detection model with a dual-path structure, and obtain an invariant feature detection result set and a feature detection result set of the target domain data set;

[0182] A pseudo-label acquisition module is used to perform label filtering and fusion on the invariant feature detection result set and the feature detection result set of the target domain data set to obtain a target pseudo-label of the target domain data set;

[0183] A model training module is used to train the unsupervised domain adaptation target detection model using the labeled source domain dataset, the unlabeled target domain dataset and the target pseudo-label to obtain a trained unsupervised domain adaptation target detection model;

[0184] The target domain image detection module is used to input the target domain image to be detected into the trained unsupervised domain adaptation target detection model to perform target detection and obtain the target detection result of the target domain image to be detected.

[0185] The labeled source domain dataset includes multiple source domain images, category labels and position labels of targets in the source domain images; and the unlabeled target domain dataset includes multiple target domain images.

[0186] In a possible implementation, the target domain dataset detection module may include:

[0187] A feature extraction unit, configured to extract basic features of each target domain image in the target domain dataset by using a feature extraction backbone network in the unsupervised domain adaptation target detection model;

[0188] A source domain path detection unit is used to obtain the invariant features in the target domain image based on the basic features by using the invariant feature detection path guided by the source domain in the unsupervised domain adaptation target detection model; perform target detection on the invariant features to obtain the invariant feature detection result of the target domain image;

[0189] a target domain path detection unit, configured to obtain target domain features of the target domain image based on the basic features by using the target domain guided feature detection path in the unsupervised domain adaptation target detection model; fuse the target domain features and invariant features; perform target detection on the fused target domain features and invariant features to obtain feature detection results of the target domain image;

[0190] The summarizing unit is used to summarize the invariant feature detection results and feature detection results of each of the target domain images to obtain an invariant feature detection result set and a feature detection result set of the target domain data set.

[0191] In a possible implementation, the source domain path detection unit may include:

[0192] an invariant feature differentiation subunit, configured to transform the basic features of the target domain image into invariant features by using the invariant feature differentiation network in the invariant feature detection path guided by the source domain;

[0193] The invariant feature detection subunit is used to detect the invariant features of the target domain image using the invariant feature detection network in the invariant feature detection path guided by the source domain, and output the invariant feature detection result of the target domain image.

[0194] In a possible implementation, the target domain path detection unit may include:

[0195] a feature differentiation subunit, configured to transform the basic features of the target domain image into target domain features by using a target domain feature differentiation network in the target domain guided feature detection path;

[0196] A feature fusion subunit, used for fusing the target domain feature and the invariant feature of the target domain image to obtain a fused feature;

[0197] The feature detection subunit is used to perform target detection on the fused features using the target domain feature detection network in the target domain guided feature detection path to obtain a feature detection result of the target domain image.

[0198] In a possible implementation, the model training module may include:

[0199] a detection unit, configured to input the labeled source domain dataset and the unlabeled target domain dataset into the unsupervised domain adaptation target detection model, obtain an invariant feature detection result of each source domain image in the source domain dataset and each target domain image in the target domain dataset through the invariant feature detection path guided by the source domain, and obtain a feature detection result of each target domain image in the target domain dataset through the feature detection path guided by the target domain;

[0200] an invariant feature detection network updating unit, configured to calculate a source domain detection loss based on the category and position of the target in the invariant feature detection result of each source domain image, and the category label and position label of the target in each source domain image; and iteratively update the invariant feature detection network using the source domain detection loss;

[0201] A target domain path updating unit calculates a target domain detection loss based on the category and position of the target in the feature detection result of each target domain image and the target pseudo-label of the target domain data set; iteratively updates the target domain feature differentiation network and the target domain feature detection network using the target domain detection loss; and obtains a new target pseudo-label of the target domain data set based on the invariant feature detection result and feature detection result of each target domain image in the target domain data set, for updating the target pseudo-label of the target domain data set;

[0202] A source domain path updating unit is used to use the domain discrimination network in the invariant feature detection path guided by the source domain to determine the domain category to which the image input into the invariant feature detection path guided by the source domain belongs, and calculate the total discrimination loss based on the judgment result, wherein the image is a source domain image or a target domain image, and update the feature extraction backbone network, the invariant feature differentiation network and the invariant feature detection network based on the total discrimination loss.

[0203] In a possible implementation, the source domain path updating unit may include a total discrimination loss obtaining subunit, which is specifically used to:

[0204] Using the pixel-level domain discrimination network to determine the domain to which the pixel feature in the image input into the invariant feature detection path guided by the source domain belongs, and calculating the pixel-level domain discrimination loss based on the determination result;

[0205] Using the instance-level domain discrimination network, determining the domain to which the instance feature in the image input into the source domain guided invariant feature detection path belongs, and calculating the instance-level domain discrimination loss based on the determination result;

[0206] Using the category-level discriminant network, determining the category to which the instance feature in the image input into the source domain guided invariant feature detection path belongs, and calculating the category-level discriminant loss based on the determination result;

[0207] The pixel-level domain discrimination loss, the instance-level domain discrimination loss, and the category-level discrimination loss are superimposed to obtain the total discrimination loss.

[0208] In a possible implementation, the above-mentioned invariant feature differentiation subunit is specifically used for:

[0209] Determine the channel attention weights of the basic features of the target domain image by using the channel attention mechanism of the invariant feature differentiation network; determine the spatial attention weights of the basic features of the target domain image by using the spatial attention mechanism of the invariant feature differentiation network;

[0210] Using the basic features of the target domain image, the channel attention weights of the basic features, and the spatial attention weights, the basic features are converted into invariant features;

[0211] The expression for converting the basic features into invariant features is:

[0212] F IF =W c (x)·x·W s (W c (x)·x)

[0213] Among them, F IF represents the unchanged features after the basic features are transformed; x represents the basic features; W c (x) represents the channel attention weight of the basic feature; W s (W c (x)·x) represents the spatial attention weight after the basic feature is multiplied by the channel attention weight.

[0214] In a possible implementation, the pseudo-label acquisition module is specifically used to:

[0215] According to the intersection of the invariant feature detection result set and the feature detection result set, as well as the confidence of each invariant feature detection result in the invariant feature detection result set and the confidence of each feature detection result in the feature detection result set, the invariant feature detection result set and the feature detection result set are hierarchically filtered; and the invariant feature detection result set and the feature detection result set after the hierarchical filtering are fused by non-maximum suppression to obtain the target pseudo-label of the target domain data set.

[0216] Example 4

[0217] like Figure 5 As shown, the present invention also provides an electronic device, which may be a computer device, a single-chip device, an intelligent mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, the processor, and the transceiver component are connected via a bus; the memory may be used to store an execution program, and an exemplary execution program may include instructions; the processor is used to execute the instructions stored in the memory. The memory may also be used to store data, which may be called and / or modified when the instructions are executed.

[0218] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in a storage medium to implement the corresponding method flow or corresponding functions, so as to implement the steps of an unsupervised domain adaptation target detection method based on a dual-path structure in the above-mentioned embodiment.

[0219] Example 5

[0220] Based on the same inventive concept, the present invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It can be understood that the storage medium here can include both built-in storage media in electronic devices and, of course, extended storage media supported by electronic devices. The storage medium provides a storage space that stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more execution programs (including program codes). It should be noted that the storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage. The processor loads and executes one or more instructions stored in the storage medium, which can implement the steps of an unsupervised domain adaptation target detection method based on a dual-path structure in the above embodiment.

[0221] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit its protection scope. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that after reading the present invention, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the application, but these changes, modifications or equivalent substitutions are all within the protection scope of the claims to be approved.

Claims

1. An unsupervised domain adaptation target detection method based on a dual-path structure, characterized in that: include: Performing target detection on an unlabeled target domain dataset using an unsupervised domain adaptation target detection model with a dual-path structure, thereby obtaining an invariant feature detection result set and a feature detection result set of the target domain dataset; Performing label filtering and fusion on the invariant feature detection result set and the feature detection result set of the target domain data set to obtain a target pseudo label of the target domain data set; Using the labeled source domain dataset, the unlabeled target domain dataset, and the target pseudo-label, the unsupervised domain adaptation target detection model is trained to obtain a trained unsupervised domain adaptation target detection model; The target domain image to be detected is input into the trained unsupervised domain adaptation target detection model to perform target detection, and obtain the target detection result of the target domain image to be detected.

2. The method according to claim 1, characterized in that The labeled source domain dataset includes a plurality of source domain images, category labels and position labels of targets in the source domain images; The unlabeled target domain dataset includes a plurality of target domain images.

3. The method according to claim 2, characterized in that The unsupervised domain adaptation target detection model with a dual-path structure performs target detection on an unlabeled target domain dataset to obtain an invariant feature detection result set and a feature detection result set of the target domain dataset, including: For each target domain image in the target domain dataset, using a feature extraction backbone network in the unsupervised domain adaptation target detection model to extract basic features of the target domain image; Utilizing the source domain guided invariant feature detection path in the unsupervised domain adaptation target detection model, obtaining invariant features in the target domain image based on the basic features; performing target detection on the invariant features to obtain invariant feature detection results of the target domain image; Utilizing the target domain guided feature detection path in the unsupervised domain adaptation target detection model, the target domain features of the target domain image are acquired based on the basic features; fusing the target domain features and the invariant features; performing target detection on the fused target domain features and the invariant features to obtain a feature detection result of the target domain image; The invariant feature detection result and the feature detection result of each of the target domain images are summarized to obtain an invariant feature detection result set and a feature detection result set of the target domain dataset.

4. The method according to claim 3, characterized in that The invariant feature detection path guided by the source domain in the unsupervised domain adaptation target detection model is used to obtain the invariant features in the target domain image based on the basic features; Performing target detection on the invariant features to obtain an invariant feature detection result of the target domain image includes: Using the invariant feature differentiation network in the source domain guided invariant feature detection path, the basic features of the target domain image are converted into invariant features; The invariant feature detection network in the invariant feature detection path guided by the source domain is used to detect the invariant features of the target domain image, and the invariant feature detection result of the target domain image is output.

5. The method according to claim 4, characterized in that The target domain features of the target domain image are acquired based on the basic features by using the target domain guided feature detection path in the unsupervised domain adaptation target detection model; and the target domain features and the invariant features are fused; Performing target detection on the fused target domain features and invariant features to obtain a feature detection result of the target domain image includes: Using a target domain feature differentiation network in the target domain guided feature detection path, the basic features of the target domain image are converted into target domain features; fusing the target domain features and invariant features of the target domain image to obtain fused features; The target domain feature detection network in the target domain guided feature detection path is used to perform target detection on the fused features to obtain a feature detection result of the target domain image.

6. The method according to claim 5, characterized in that The step of training the unsupervised domain adaptation target detection model by using the labeled source domain dataset, the unlabeled target domain dataset, and the target pseudo-label to obtain a trained unsupervised domain adaptation target detection model includes: Inputting the labeled source domain dataset and the unlabeled target domain dataset into the unsupervised domain adaptation target detection model, obtaining an invariant feature detection result of each source domain image in the source domain dataset and each target domain image in the target domain dataset through the invariant feature detection path guided by the source domain, and obtaining a feature detection result of each target domain image in the target domain dataset through the feature detection path guided by the target domain; Calculating source domain detection loss based on the category and position of the target in the invariant feature detection result of each source domain image, as well as the category label and position label of the target in each source domain image; iteratively updating the invariant feature detection network using the source domain detection loss; Based on the category and position of the target in the feature detection result of each target domain image and the target pseudo-label of the target domain data set, a target domain detection loss is calculated; the target domain feature differentiation network and the target domain feature detection network are iteratively updated using the target domain detection loss; and based on the invariant feature detection result and feature detection result of each target domain image in the target domain data set, a new target pseudo-label of the target domain data set is obtained for updating the target pseudo-label of the target domain data set; The domain discrimination network in the invariant feature detection path guided by the source domain is used to judge the domain category to which the image input into the invariant feature detection path guided by the source domain belongs, and the total discrimination loss is calculated based on the judgment result. The image is a source domain image or a target domain image. The feature extraction backbone network, the invariant feature differentiation network and the invariant feature detection network are updated based on the total discrimination loss.

7. The method according to claim 6, characterized in that The domain discrimination network includes a pixel-level domain discrimination network, an instance-level domain discrimination network and a category-level discrimination network; the domain discrimination network is used to judge the domain category to which the image input into the invariant feature detection path guided by the source domain belongs, and the total discrimination loss is calculated based on the judgment result, including: Using the pixel-level domain discrimination network to determine the domain to which the pixel feature in the image input into the invariant feature detection path guided by the source domain belongs, and calculating the pixel-level domain discrimination loss based on the determination result; Using the instance-level domain discrimination network, determining the domain to which the instance feature in the image input into the source domain guided invariant feature detection path belongs, and calculating the instance-level domain discrimination loss based on the determination result; Using the category-level discriminant network, determining the category to which the instance feature in the image input into the source domain guided invariant feature detection path belongs, and calculating the category-level discriminant loss based on the determination result; The pixel-level domain discrimination loss, the instance-level domain discrimination loss, and the category-level discrimination loss are superimposed to obtain the total discrimination loss.

8. The method according to claim 4, characterized in that The converting the basic features of the target domain image into invariant features comprises: Determining the channel attention weights of the basic features of the target domain image by using the channel attention mechanism of the invariant feature differentiation network; Determine the spatial attention weights of the basic features of the target domain image by using the spatial attention mechanism of the invariant feature differentiation network; Using the basic features of the target domain image, the channel attention weights of the basic features, and the spatial attention weights, the basic features are converted into invariant features; The expression for converting the basic features into invariant features is: F IF =W c (x)·x·W s (W c (x)·x) Among them, F IF represents the unchanged features after the basic features are transformed; x represents the basic features; W c (x) represents the channel attention weight of the basic feature; W s (W c (x)·x) represents the spatial attention weight after the basic feature is multiplied by the channel attention weight.

9. The method according to claim 1, characterized in that The step of performing label filtering and fusion on the invariant feature detection result set and the feature detection result set of the target domain data set to obtain a target pseudo label of the target domain data set includes: According to the intersection of the invariant feature detection result set and the feature detection result set, and the confidence of each invariant feature detection result in the invariant feature detection result set and the confidence of each feature detection result in the feature detection result set, the invariant feature detection result set and the feature detection result set are hierarchically filtered; The invariant feature detection result set and the feature detection result set after hierarchical filtering are fused by non-maximum suppression to obtain the target pseudo label of the target domain data set.

10. An unsupervised domain adaptation target detection system based on a dual-path structure, characterized in that: include: A target domain data set detection module is used to perform target detection on an unlabeled target domain data set through an unsupervised domain adaptation target detection model with a dual-path structure, and obtain an invariant feature detection result set and a feature detection result set of the target domain data set; A pseudo-label acquisition module is used to perform label filtering and fusion on the invariant feature detection result set and the feature detection result set of the target domain data set to obtain a target pseudo-label of the target domain data set; A model training module is used to train the unsupervised domain adaptation target detection model using the labeled source domain dataset, the unlabeled target domain dataset and the target pseudo-label to obtain a trained unsupervised domain adaptation target detection model; The target domain image detection module is used to input the target domain image to be detected into the trained unsupervised domain adaptation target detection model to perform target detection and obtain the target detection result of the target domain image to be detected.

Images