Unsupervised adaptive target detection method and system based on cross-domain feature fusion
By introducing a cross-domain feature fusion method of adaptive network and adversarial network in object detection, the problem of insufficient generalization ability caused by domain differences is solved, unsupervised adaptive target detection is achieved, and detection accuracy and robustness are improved.
Patent Information
- Application Number
- CN202411398378.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2025-02-14
AI Technical Summary
When facing the domain differences between different data sets, the existing object detection methods lack generalization capabilities and require a large amount of manual labeling of data, which is difficult to effectively solve in practical applications.
Unsupervised adaptive object detection method based on cross-domain feature fusion is adopted, and the feature difference between domains is extracted by introducing the Yolov8 model of adaptive and adversarial networks, and the feature difference between domains is reduced through feature fusion and adversarial training. , realize target detection.
The detection accuracy of the object detection model on unlabeled target domain data is significantly improved, the dependence on a large number of labeled data is reduced, and the robustness and adaptability of the model is enhanced.
Smart Images
Figure CN119445300A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of deep learning technology, and in particular to an unsupervised adaptive target detection method and system based on cross-domain feature fusion. Background Art
[0002] With the rapid development of computer vision technology, object detection has been widely used in many fields, such as autonomous driving, security monitoring, industrial detection, etc. However, most existing object detection methods rely on a large amount of labeled data for training, which is difficult to achieve in many practical application scenarios, especially between different data sets. Due to differences in acquisition conditions, lighting, viewing angles and other factors, there will be significant differences in data distribution, which is called "domain gap". The existence of such domain differences seriously affects the generalization ability of the model in new environments. Traditional solutions include data enhancement and transfer learning, but these methods often require additional manually labeled data or assume that the distribution between the source domain and the target domain is similar, which is not always feasible in practical applications.
[0003] How to effectively address domain gaps and implement unsupervised domain adaptive object detection that does not require a large amount of manually labeled data is a technical problem that needs to be solved. Summary of the invention
[0004] The technical task of the present invention is to address the above shortcomings and provide an unsupervised adaptive target detection method and system based on cross-domain feature fusion to solve the technical problem of how to effectively deal with field gaps and unsupervised domain adaptive target detection that does not require a large amount of manually labeled data.
[0005] In a first aspect, the present invention provides an unsupervised adaptive target detection method based on cross-domain feature fusion, comprising the following steps:
[0006] Data collection: obtaining a source domain dataset and a target domain dataset, and preprocessing the source domain dataset and the target domain dataset as the target dataset to obtain the preprocessed source domain dataset and the target domain dataset;
[0007] Detection model construction: A target detection model is constructed based on the Yolov8 model that introduces an adaptive network and an adversarial network. The target detection model takes the input data set as input, and is used to extract source domain and target domain features based on the input data set. The source domain features and the target domain features are fused through the adaptive network, and the distribution difference between the source domain features and the target domain features is minimized through the adversarial network. The target is recognized based on the extracted features to obtain a category label for indicating the target type and an aiming frame position for indicating the target position.
[0008] Model training: A training set is constructed based on the source domain dataset, the target domain dataset, and the annotation information corresponding to the source domain dataset. The target detection model is trained based on the training set to obtain a trained target detection model.
[0009] Target detection: Collect images of the target to be tested to obtain the data set to be tested, preprocess the data set to be tested as the target data set to obtain the preprocessed data set, use the preprocessed data set as input, perform target detection and recognition through the trained target detection model, and obtain the category label used to represent the target type and the aiming frame position used to represent the target position.
[0010] Preferably, the target data set is preprocessed, and the images in the target data set are scaled and normalized by the preprocessing.
[0011] Preferably, the target detection model includes a backbone network, a neck, a head, and an adaptive network and an adversarial network entering the backbone network;
[0012] The backbone network is used to extract features from the input source domain data set and target domain data set to obtain source domain features and target domain features;
[0013] The adaptive network is used to adjust the adaptive weights of the source domain features and the target domain features based on an adaptive learning mechanism, and to fuse the source domain features and the target domain features based on the adaptive weights to obtain fused features;
[0014] The adversarial network includes a generator and a discriminator. The generator is used to take source domain features and target domain features as inputs to generate false features of source domain features and false features of target domain features. The discriminator is used to take source domain features, false features of source domain features, target domain features and false features of target domain features as input features, judge whether the category of the input features is true or false, and output the category discrimination result.
[0015] The neck is used to take the fused features output by the adaptive network as input, perform multi-scale feature fusion on the fused features at different stages, and obtain fused multi-scale features;
[0016] The head is used to perform target detection based on the fused multi-scale features output by the neck as input, and obtain a category label for indicating the type of the target and an aiming frame position for indicating the position of the target.
[0017] Preferably, the loss function includes a target detection loss constructed based on the annotation information corresponding to the source domain data set and the category labels and anchor box positions predicted and output by the target detection model, and an adversarial training loss constructed based on the source domain features and the target domain features.
[0018] Preferably, the anchor frame position includes the center point coordinates of the anchor frame and the width and height of the anchor frame.
[0019] In a second aspect, the present invention provides an unsupervised adaptive target detection system based on cross-domain feature fusion, which is used to implement unsupervised adaptive target detection by using an unsupervised adaptive target detection method based on cross-domain feature fusion as described in any one of the first aspects, wherein the system includes a data collection module, a detection model construction module, a model training module, and a target detection module;
[0020] The data collection module is used to perform the following: obtain a source domain data set and a target domain data set, and preprocess the source domain data set and the target domain data set as the target data set to obtain the preprocessed source domain data set and the target domain data set;
[0021] The detection model building module is used to perform the following: construct a target detection model based on the Yolov8 model that introduces an adaptive network and an adversarial network, wherein the target detection model takes an input data set as input, is used to extract source domain and target domain features based on the input data set, performs feature fusion of source domain features and target domain features through an adaptive network, minimizes the distribution difference between source domain features and target domain features through an adversarial network, and performs target recognition based on the extracted features to obtain a category label for indicating the type of the target object and an aiming frame position for indicating the position of the target object;
[0022] The model training module is used to perform the following: construct a training set based on the source domain data set, the target domain data set and the annotation information corresponding to the source domain data set, and perform model training on the target detection model based on the training set to obtain a trained target detection model;
[0023] The target detection module is used to perform the following: collect images of the target to be tested to obtain a data set to be tested, preprocess the data set to be tested as the target data set to obtain a preprocessed data set, use the preprocessed data set as input, perform target detection and recognition through the trained target detection model, and obtain a category label for indicating the target type and an aiming frame position for indicating the target position.
[0024] Preferably, the target data set is preprocessed, and the data acquisition module is used to scale and normalize the images in the target data set through preprocessing.
[0025] Preferably, the target detection model includes a backbone network, a neck, a head, and an adaptive network and an adversarial network entering the backbone network;
[0026] The backbone network is used to extract features from the input source domain data set and target domain data set to obtain source domain features and target domain features;
[0027] The adaptive network is used to adjust the adaptive weights of the source domain features and the target domain features based on an adaptive learning mechanism, and to fuse the source domain features and the target domain features based on the adaptive weights to obtain fused features;
[0028] The adversarial network includes a generator and a discriminator. The generator is used to take source domain features and target domain features as inputs to generate false features of source domain features and false features of target domain features. The discriminator is used to take source domain features, false features of source domain features, target domain features and false features of target domain features as input features, judge whether the category of the input features is true or false, and output the category discrimination result.
[0029] The neck is used to take the fused features output by the adaptive network as input, perform multi-scale feature fusion on the fused features at different stages, and obtain fused multi-scale features;
[0030] The head is used to perform target detection based on the fused multi-scale features output by the neck as input, and obtain a category label for indicating the type of the target and an aiming frame position for indicating the position of the target.
[0031] Preferably, the loss function includes a target detection loss constructed based on the annotation information corresponding to the source domain data set and the category labels and anchor box positions predicted and output by the target detection model, and an adversarial training loss constructed based on the source domain features and the target domain features.
[0032] Preferably, the anchor frame position includes the center point coordinates of the anchor frame and the width and height of the anchor frame.
[0033] The unsupervised adaptive target detection method and system based on cross-domain feature fusion of the present invention have the following advantages:
[0034] 1. Generalization: Through cross-domain feature fusion technology, the detection accuracy of the target detection model on unlabeled target domain data is significantly improved, especially when there is a large difference between the source domain and the target domain data;
[0035] 2. Low cost: Traditional supervised learning methods require a large amount of labeled data, while unsupervised domain adaptive target detection methods can maintain high detection performance even in the absence of target domain annotations;
[0036] 3. Strong robustness: By introducing the conditional adaptive network (CDAN) and adversarial network to minimize the difference between the feature distribution of the source domain and the target domain, the model can better adapt to data changes in different environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0038] The present invention is further described below in conjunction with the accompanying drawings.
[0039] Figure 1 This is a flowchart of an unsupervised adaptive target detection method based on cross-domain feature fusion in Example 1. DETAILED DESCRIPTION
[0040] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments may be combined with each other.
[0041] The embodiments of the present invention provide an unsupervised adaptive target detection method and system based on cross-domain feature fusion, which are used to solve the technical problem of how to effectively address domain gaps and unsupervised domain adaptive target detection that does not require a large amount of manually labeled data.
[0042] Embodiment 1:
[0043] The present invention discloses an unsupervised adaptive target detection method based on cross-domain feature fusion, which includes four steps: data collection, detection model construction, model training and target detection.
[0044] Step S100: Data collection: obtaining a source domain dataset and a target domain dataset, and preprocessing the source domain dataset and the target domain dataset as target datasets to obtain preprocessed source domain dataset and target domain dataset.
[0045] In step S100 of this embodiment, the target data set is preprocessed, and images in the target data set are scaled and normalized through the preprocessing.
[0046] Step S200 detection model construction: a target detection model is constructed based on the Yolov8 model that introduces an adaptive network and an adversarial network. The target detection model takes an input data set as input, and is used to extract source domain and target domain features based on the input data set, and perform feature fusion of source domain features and target domain features through an adaptive network, and minimize the distribution difference between source domain features and target domain features through an adversarial network, and perform target recognition based on the extracted features to obtain a category label for representing the target type and an aiming frame position for representing the target position.
[0047] In this embodiment, the target detection model includes a backbone network, a neck, a head, and an adaptive network and an adversarial network entering the backbone network.
[0048] The backbone network is used to extract features from the input source domain data set and target domain data set to obtain source domain features and target domain features.
[0049] The adaptive network is used to adjust the adaptive weights of the source domain features and the target domain features based on an adaptive learning mechanism, and to fuse the source domain features and the target domain features based on the adaptive weights to obtain fused features.
[0050] The adversarial network includes a generator and a discriminator. The generator is used to take source domain features and target domain features as input to generate false features of source domain features and false features of target domain features. The discriminator is used to take source domain features, false features of source domain features, target domain features and false features of target domain features as input features, judge whether the category of the input features is true or false, and output the category discrimination result.
[0051] The neck is used to take the fused features output by the adaptive network as input, perform multi-scale feature fusion on the fused features at different stages, and obtain the fused multi-scale features.
[0052] The head is used to perform target detection based on the fused multi-scale features output by the neck as input, and obtain a category label for indicating the type of the target and an aiming frame position for indicating the position of the target.
[0053] The anchor box position includes the center point coordinates of the anchor box and the width and height of the anchor box.
[0054] Step S300: Model training: construct a training set based on the source domain data set, the target domain data set and the annotation information corresponding to the source domain data set, and perform model training on the target detection model based on the training set to obtain a trained target detection model.
[0055] As a specific implementation of model training, when training the target detection model, the loss function includes the target detection loss constructed based on the annotation information corresponding to the source domain dataset and the category label and anchor box position predicted by the target detection model output, and the adversarial training loss constructed based on the source domain features and the target domain features.
[0056] In this embodiment, the loss function includes target detection loss and adversarial training loss to ensure that the performance of the model is balanced in the two domains. End-to-end training is performed based on minimization of the loss function, and the parameters of the target detection model are updated until convergence.
[0057] Step S400 target detection: collect images of the target to be tested to obtain a data set to be tested, preprocess the data set to be tested as the target data set to obtain a preprocessed data set, use the preprocessed data set as input, perform target detection and recognition through the trained target detection model, and obtain a category label for indicating the target type and an aiming frame position for indicating the target position.
[0058] The method of this embodiment minimizes the difference between the source domain and the target domain feature distribution through adversarial training, significantly improving the performance of the target detection model in the target domain, especially for cases where the data distribution is inconsistent. This method can not only be applied to target detection tasks in specific fields, but can also be widely used in various computer vision tasks, such as autonomous driving, security monitoring and other fields.
[0059] Embodiment 2:
[0060] The present invention discloses an unsupervised adaptive target detection system based on cross-domain feature fusion, comprising a data collection module, a detection model construction module, a model training module and a target detection module.
[0061] The data collection module is used to perform the following: obtain a source domain data set and a target domain data set, and preprocess the source domain data set and the target domain data set as target data sets to obtain the preprocessed source domain data set and the target domain data set.
[0062] This embodiment preprocesses the target data set, and the data collection module is used to scale and normalize the images in the target data set through preprocessing.
[0063] The detection model construction module is used to perform the following: construct a target detection model based on the Yolov8 model that introduces an adaptive network and an adversarial network. The target detection model takes an input data set as input, and is used to extract source domain and target domain features based on the input data set, perform feature fusion on the source domain features and the target domain features through an adaptive network, and minimize the distribution difference between the source domain features and the target domain features through an adversarial network, and perform target recognition based on the extracted features to obtain a category label for representing the target type and an aiming frame position for representing the target position.
[0064] In this embodiment, the target detection model includes a backbone network, a neck, a head, and an adaptive network and an adversarial network entering the backbone network.
[0065] The backbone network is used to extract features from the input source domain data set and target domain data set to obtain source domain features and target domain features.
[0066] The adaptive network is used to adjust the adaptive weights of the source domain features and the target domain features based on an adaptive learning mechanism, and to fuse the source domain features and the target domain features based on the adaptive weights to obtain fused features.
[0067] The adversarial network includes a generator and a discriminator. The generator is used to take source domain features and target domain features as input to generate false features of source domain features and false features of target domain features. The discriminator is used to take source domain features, false features of source domain features, target domain features and false features of target domain features as input features, judge whether the category of the input features is true or false, and output the category discrimination result.
[0068] The neck is used to take the fused features output by the adaptive network as input, perform multi-scale feature fusion on the fused features at different stages, and obtain the fused multi-scale features.
[0069] The head is used to perform target detection based on the fused multi-scale features output by the neck as input, and obtain a category label for indicating the type of the target and an aiming frame position for indicating the position of the target.
[0070] The anchor box position includes the center point coordinates of the anchor box and the width and height of the anchor box.
[0071] The model training module is used to perform the following: construct a training set based on the source domain data set, the target domain data set and the annotation information corresponding to the source domain data set, and perform model training on the target detection model based on the training set to obtain a trained target detection model.
[0072] As a specific implementation of model training, when training the target detection model, the loss function includes the target detection loss constructed based on the annotation information corresponding to the source domain dataset and the category label and anchor box position predicted by the target detection model output, and the adversarial training loss constructed based on the source domain features and the target domain features.
[0073] In this embodiment, the loss function includes target detection loss and adversarial training loss to ensure that the performance of the model is balanced in the two domains. End-to-end training is performed based on minimization of the loss function, and the parameters of the target detection model are updated until convergence.
[0074] The target detection module is used to perform the following: collect images of the target to be tested to obtain a data set to be tested, preprocess the data set to be tested as the target data set to obtain a preprocessed data set, use the preprocessed data set as input, perform target detection and recognition through the trained target detection model, and obtain a category label for indicating the target type and an aiming frame position for indicating the target position.
[0075] The system of this embodiment can execute the method disclosed in Example 1 to realize unsupervised adaptive target detection.
[0076] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the means in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.
Claims
1. An unsupervised adaptive target detection method based on cross-domain feature fusion, characterized in that: The steps include: Data collection: obtaining a source domain dataset and a target domain dataset, and preprocessing the source domain dataset and the target domain dataset as the target dataset to obtain the preprocessed source domain dataset and the target domain dataset; Detection model construction: A target detection model is constructed based on the Yolov8 model that introduces an adaptive network and an adversarial network. The target detection model takes the input data set as input, and is used to extract source domain and target domain features based on the input data set. The source domain features and the target domain features are fused through the adaptive network, and the distribution difference between the source domain features and the target domain features is minimized through the adversarial network. The target is recognized based on the extracted features to obtain a category label for indicating the target type and an aiming frame position for indicating the target position. Model training: A training set is constructed based on the source domain dataset, the target domain dataset, and the annotation information corresponding to the source domain dataset. The target detection model is trained based on the training set to obtain a trained target detection model. Target detection: Collect images of the target to be tested to obtain the data set to be tested, preprocess the data set to be tested as the target data set to obtain the preprocessed data set, use the preprocessed data set as input, perform target detection and recognition through the trained target detection model, and obtain the category label used to represent the target type and the aiming frame position used to represent the target position.
2. The unsupervised adaptive target detection method based on cross-domain feature fusion according to claim 1 is characterized in that: The target data set is preprocessed by scaling and normalizing the images in the target data set.
3. The unsupervised adaptive target detection method based on cross-domain feature fusion according to claim 1 is characterized in that: The target detection model includes a backbone network, a neck, a head, and an adaptive network and an adversarial network entering the backbone network; The backbone network is used to extract features from the input source domain data set and target domain data set to obtain source domain features and target domain features; The adaptive network is used to adjust the adaptive weights of the source domain features and the target domain features based on an adaptive learning mechanism, and to fuse the source domain features and the target domain features based on the adaptive weights to obtain fused features; The adversarial network includes a generator and a discriminator. The generator is used to take source domain features and target domain features as inputs to generate false features of source domain features and false features of target domain features. The discriminator is used to take source domain features, false features of source domain features, target domain features and false features of target domain features as input features, judge whether the category of the input features is true or false, and output the category discrimination result. The neck is used to take the fused features output by the adaptive network as input, perform multi-scale feature fusion on the fused features at different stages, and obtain fused multi-scale features; The head is used to perform target detection based on the fused multi-scale features output by the neck as input, and obtain a category label for indicating the type of the target and an aiming frame position for indicating the position of the target.
4. The unsupervised adaptive target detection method based on cross-domain feature fusion according to claim 1 is characterized in that: The loss function includes the target detection loss constructed based on the annotation information corresponding to the source domain dataset and the category label and anchor box position predicted by the target detection model, as well as the adversarial training loss constructed based on the source domain features and the target domain features.
5. The unsupervised adaptive target detection method based on cross-domain feature fusion according to claim 1 is characterized in that: The anchor box position includes the center point coordinates of the anchor box and the width and height of the anchor box.
6. An unsupervised adaptive target detection system based on cross-domain feature fusion, characterized in that: Used to realize unsupervised adaptive target detection by an unsupervised adaptive target detection method based on cross-domain feature fusion as described in any one of claims 1 to 5, the system comprising a data collection module, a detection model construction module, a model training module and a target detection module; The data collection module is used to perform the following: obtain a source domain data set and a target domain data set, and preprocess the source domain data set and the target domain data set as the target data set to obtain the preprocessed source domain data set and the target domain data set; The detection model building module is used to perform the following: construct a target detection model based on the Yolov8 model that introduces an adaptive network and an adversarial network, wherein the target detection model takes an input data set as input, is used to extract source domain and target domain features based on the input data set, performs feature fusion of source domain features and target domain features through an adaptive network, minimizes the distribution difference between source domain features and target domain features through an adversarial network, and performs target recognition based on the extracted features to obtain a category label for indicating the type of the target object and an aiming frame position for indicating the position of the target object; The model training module is used to perform the following: construct a training set based on the source domain data set, the target domain data set and the annotation information corresponding to the source domain data set, and perform model training on the target detection model based on the training set to obtain a trained target detection model; The target detection module is used to perform the following: collect images of the target to be tested to obtain a data set to be tested, preprocess the data set to be tested as the target data set to obtain a preprocessed data set, use the preprocessed data set as input, perform target detection and recognition through the trained target detection model, and obtain a category label for indicating the target type and an aiming frame position for indicating the target position.
7. The unsupervised adaptive target detection system based on cross-domain feature fusion according to claim 6 is characterized in that: The target data set is preprocessed, and the data acquisition module is used to scale and normalize the images in the target data set through preprocessing.
8. The unsupervised adaptive target detection system based on cross-domain feature fusion according to claim 6, characterized in that: The target detection model includes a backbone network, a neck, a head, and an adaptive network and an adversarial network entering the backbone network; The backbone network is used to extract features from the input source domain data set and target domain data set to obtain source domain features and target domain features; The adaptive network is used to adjust the adaptive weights of the source domain features and the target domain features based on an adaptive learning mechanism, and to fuse the source domain features and the target domain features based on the adaptive weights to obtain fused features; The adversarial network includes a generator and a discriminator. The generator is used to take source domain features and target domain features as inputs to generate false features of source domain features and false features of target domain features. The discriminator is used to take source domain features, false features of source domain features, target domain features and false features of target domain features as input features, judge whether the category of the input features is true or false, and output the category discrimination result. The neck is used to take the fused features output by the adaptive network as input, perform multi-scale feature fusion on the fused features at different stages, and obtain fused multi-scale features; The head is used to perform target detection based on the fused multi-scale features output by the neck as input, and obtain a category label for indicating the type of the target and an aiming frame position for indicating the position of the target.
9. The unsupervised adaptive target detection system based on cross-domain feature fusion according to claim 6, characterized in that: The loss function includes the target detection loss constructed based on the annotation information corresponding to the source domain dataset and the category label and anchor box position predicted by the target detection model, as well as the adversarial training loss constructed based on the source domain features and the target domain features.
10. The unsupervised adaptive target detection system based on cross-domain feature fusion according to claim 6, characterized in that: The anchor box position includes the center point coordinates of the anchor box and the width and height of the anchor box.