Method, apparatus, equipment and media for constructing a target detection model based on dual dictionaries
By constructing a dual-dictionary object detection model and using the CycleGan network for sample style transfer and Intersection over Union (IoU) to determine pseudo-label weights, the domain offset problem is solved, and the object detection model's detection performance in the target domain is improved.
Patent Information
- Application Number
- CN202411121765.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2044-08-15
AI Technical Summary
In existing technologies, object detection models suffer from domain bias when training and test samples come from different distributions, leading to a decrease in detector performance. In pseudo-label training methods, reliable pseudo-labels lack target domain knowledge, which limits the performance improvement of the detector in the target domain.
By constructing a dual-dictionary-based object detection model, the CycleGAN network is used to perform style transfer on source and target domain samples, generating source and target domain dictionaries with complementary characteristics. Based on the intersection-union ratio (IoU), pseudo-labels and their confidence weights are determined, and weighted self-training is performed to construct the object detection model.
It effectively reduces the domain bias, improves the detection performance of the object detection model in the target domain, generates more accurate pseudo-labels, and improves the detection accuracy of the model in the target domain.
Smart Images

Figure CN119152180B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer vision technology, specifically to a method, apparatus, device, and medium for constructing a target detection model based on a dual dictionary. Background Technology
[0002] Object detection is a challenging task that aims to identify and locate all instances of a specific category of objects in an image. In recent years, this field has made significant progress with supervised methods, which have been widely used in image applications such as face recognition, autonomous driving, and medical detection. However, when training and test samples come from different distributions, a neighborhood bias problem arises, causing a sharp decline in detector performance. While this neighborhood bias problem can be mitigated by manually collecting and annotating training samples in the target domain, this is very expensive and time-consuming.
[0003] In related technologies, an unsupervised domain adaptive object detection method based on pseudo-label training is used to extract and map features from two domains and reduce the distribution difference between the source and target domains. That is, by learning the feature representation of the source domain, pseudo-labels are assigned to target domain samples with high confidence, and then the detector is trained in a supervised manner to improve accuracy and thus enhance the performance of the detector.
[0004] However, reliable pseudo-labels used in pseudo-label training methods typically lack target domain knowledge, thus limiting the detector's performance improvement in the target domain. Therefore, mitigating domain bias to enhance detection model performance is a pressing issue that needs to be addressed. Summary of the Invention
[0005] This application provides a method, apparatus, device, and medium for constructing a target detection model based on a dual dictionary, which can solve the technical problem of model performance degradation caused by the existence of domain offset in the prior art.
[0006] In a first aspect, embodiments of this application provide a method for constructing a target detection model based on a dual-dictionary framework, comprising the following steps:
[0007] Transform source domain samples into target domain style samples, and then transform target domain samples into source domain style samples;
[0008] A source domain dictionary is constructed based on source domain samples, and a target domain dictionary is constructed based on target domain style samples;
[0009] Target domain candidate boxes are constructed using source domain style samples and target domain dictionary, and source domain candidate boxes are constructed using target domain style samples and source domain dictionary;
[0010] The pseudo-labels and their corresponding confidence weights are determined based on the intersection-union ratio (IoU) between the target domain candidate boxes and the source domain candidate boxes.
[0011] A target detection model is constructed by performing weighted self-training based on the pseudo-labels, the confidence weights, the source domain samples, and the target domain samples.
[0012] In conjunction with the first aspect, in one implementation, the step of constructing a source domain dictionary based on source domain samples and a target domain dictionary based on target domain style samples includes:
[0013] Feature extraction is performed on source domain samples and target domain style samples respectively to determine the source domain feature matrix and the target domain feature matrix;
[0014] Dictionary learning is performed on the source domain feature matrix and the target domain feature matrix respectively based on L1 sparse constraints to generate source domain dictionary and target domain dictionary.
[0015] In conjunction with the first aspect, in one implementation, constructing the target domain candidate box using source domain style samples and the target domain dictionary includes:
[0016] The source domain style samples are fed into the region generation network to output the target domain detection box;
[0017] Target domain detection boxes with confidence scores greater than a first score threshold are added to the first set, and target domain detection boxes with confidence scores greater than a second score threshold are added to the third set as target domain candidate boxes, wherein the first score threshold is less than the second score threshold;
[0018] The target domain detection boxes in the first set are fed into the region of interest alignment network to output the first feature matrix. The first feature matrix is then encoded into the target domain dictionary based on L2 regularization to obtain the target domain encoding coefficient vector.
[0019] The target domain residual matrix is calculated based on the target domain coding coefficient vector, the first feature matrix, and the target domain dictionary, and the sensitivity is calculated using the target domain residual matrix.
[0020] The target domain detection boxes with sensitivity greater than the sensitivity threshold in the first set are added to the second set as target domain candidate boxes, and the union of the third set and the second set is taken as the fourth set.
[0021] In conjunction with the first aspect, in one implementation, constructing the source domain candidate box using the target domain style sample and the source domain dictionary includes:
[0022] The target domain style samples are fed into the region generation network to output source domain detection boxes;
[0023] Source domain detection boxes with confidence scores greater than the first score threshold are added to the fifth set, and source domain detection boxes with confidence scores greater than the second score threshold are added to the seventh set as source domain candidate boxes, where the first score threshold is less than the second score threshold.
[0024] The source domain detection boxes in the fifth set are fed into the region of interest alignment network to output the second feature matrix. The second feature matrix is then encoded into the source domain dictionary based on L2 regularization to obtain the source domain encoding coefficient vector.
[0025] The source domain residual matrix is calculated based on the source domain coding coefficient vector, the second feature matrix, and the source domain dictionary, and the sensitivity is calculated using the source domain residual matrix.
[0026] Source domain detection boxes with sensitivity greater than the sensitivity threshold in the fifth set are added to the sixth set as source domain candidate boxes, and the union of the sixth and seventh sets is taken as the eighth set.
[0027] In conjunction with the first aspect, in one implementation, determining the pseudo-labels and their corresponding confidence weights based on the Intersection over Union (IoU) between the target domain candidate boxes and the source domain candidate boxes includes:
[0028] When the predicted category of the first target domain candidate box in the third set is the same as the predicted category of the first source domain candidate box in the seventh set, the first IoU between the first target domain candidate box and the first source domain candidate box is calculated.
[0029] Pseudo-labels are determined from the first target domain candidate box and the first source domain candidate box based on the relationship between the first IoU and the first IoU threshold.
[0030] Calculate the second IoU between the second target domain candidate box and the second source domain candidate box in the fourth set;
[0031] When the second IoU is greater than the second IoU threshold, the second IoU is used as the confidence weight of the pseudo-label.
[0032] When the second IoU is not greater than the second IoU threshold, the smallest second IoU is selected as the confidence weight of the pseudo-label based on iterative training.
[0033] In conjunction with the first aspect, in one implementation, the weighted self-training loss function L st for:
[0034]
[0035] In the formula, This represents the bounding box regression loss. I represents the classification loss. s Represents source domain samples, Represents source domain sample I s The true label of the i-th foreground element, N represents the number of true labels of the source domain samples, and I T Let M represent the number of pseudo-labels for the target domain samples, μ represent the parameters used to balance the source and target domain losses, and w represent the target domain samples. j This represents the confidence weight of the j-th pseudo-label. This represents the j-th pseudo-tag.
[0036] In conjunction with the first aspect, in one implementation, the target loss function in the weighted self-training process is:
[0037] L total =βL st +L adv
[0038] In the formula, L total Let L represent the objective loss function, β represent the parameter used to balance the losses from multiple tasks, and L represent the target loss function. adv This represents the adversarial loss function, which consists of a source domain class alignment loss function and a target domain class alignment loss function.
[0039] Secondly, embodiments of this application provide a dual-dictionary-based target detection model construction apparatus, comprising:
[0040] The style transfer module is used to convert source domain samples into target domain style samples and vice versa.
[0041] The dictionary building module is used to build a source domain dictionary based on source domain samples and a target domain dictionary based on target domain style samples.
[0042] The candidate box determination module is used to construct target domain candidate boxes using source domain style samples and target domain dictionary, and to construct source domain candidate boxes using target domain style samples and source domain dictionary;
[0043] The label weight determination module is used to determine the pseudo-labels and their corresponding confidence weights based on the intersection-union ratio (IoU) between the target domain candidate boxes and the source domain candidate boxes.
[0044] The model building module is used to perform weighted self-training based on the pseudo-labels, the confidence weights, the source domain samples, and the target domain samples to build an object detection model.
[0045] In conjunction with the second aspect, in one implementation, the dictionary construction module is specifically used for:
[0046] Feature extraction is performed on source domain samples and target domain style samples respectively to determine the source domain feature matrix and the target domain feature matrix;
[0047] Dictionary learning is performed on the source domain feature matrix and the target domain feature matrix respectively based on L1 sparse constraints to generate source domain dictionary and target domain dictionary.
[0048] In conjunction with the second aspect, in one implementation, the candidate box determination module is specifically used for:
[0049] The source domain style samples are fed into the region generation network to output the target domain detection box;
[0050] Target domain detection boxes with confidence scores greater than a first score threshold are added to the first set, and target domain detection boxes with confidence scores greater than a second score threshold are added to the third set as target domain candidate boxes, wherein the first score threshold is less than the second score threshold;
[0051] The target domain detection boxes in the first set are fed into the region of interest alignment network to output the first feature matrix. The first feature matrix is then encoded into the target domain dictionary based on L2 regularization to obtain the target domain encoding coefficient vector.
[0052] The target domain residual matrix is calculated based on the target domain coding coefficient vector, the first feature matrix, and the target domain dictionary, and the sensitivity is calculated using the target domain residual matrix.
[0053] The target domain detection boxes with sensitivity greater than the sensitivity threshold in the first set are added to the second set as target domain candidate boxes, and the union of the third set and the second set is taken as the fourth set.
[0054] In conjunction with the second aspect, in one implementation, the candidate box determination module is further configured to:
[0055] The target domain style samples are fed into the region generation network to output source domain detection boxes;
[0056] Source domain detection boxes with confidence scores greater than the first score threshold are added to the fifth set, and source domain detection boxes with confidence scores greater than the second score threshold are added to the seventh set as source domain candidate boxes, where the first score threshold is less than the second score threshold.
[0057] The source domain detection boxes in the fifth set are fed into the region of interest alignment network to output the second feature matrix. The second feature matrix is then encoded into the source domain dictionary based on L2 regularization to obtain the source domain encoding coefficient vector.
[0058] The source domain residual matrix is calculated based on the source domain coding coefficient vector, the second feature matrix, and the source domain dictionary, and the sensitivity is calculated using the source domain residual matrix.
[0059] Source domain detection boxes with sensitivity greater than the sensitivity threshold in the fifth set are added to the sixth set as source domain candidate boxes, and the union of the sixth and seventh sets is taken as the eighth set.
[0060] In conjunction with the second aspect, in one implementation, the label weight determination module is specifically used for:
[0061] When the predicted category of the first target domain candidate box in the third set is the same as the predicted category of the first source domain candidate box in the seventh set, the first IoU between the first target domain candidate box and the first source domain candidate box is calculated.
[0062] Pseudo-labels are determined from the first target domain candidate box and the first source domain candidate box based on the relationship between the first IoU and the first IoU threshold.
[0063] Calculate the second IoU between the second target domain candidate box and the second source domain candidate box in the fourth set;
[0064] When the second IoU is greater than the second IoU threshold, the second IoU is used as the confidence weight of the pseudo-label.
[0065] When the second IoU is not greater than the second IoU threshold, the smallest second IoU is selected as the confidence weight of the pseudo-label based on iterative training.
[0066] In conjunction with the second aspect, in one implementation, the weighted self-training loss function L st for:
[0067]
[0068] In the formula, This represents the bounding box regression loss. I represents the classification loss. s Represents source domain samples, Represents source domain sample I s The true label of the i-th foreground element, N represents the number of true labels of the source domain samples, and I T Let M represent the number of pseudo-labels for the target domain samples, μ represent the parameters used to balance the source and target domain losses, and w represent the target domain samples. j This represents the confidence weight of the j-th pseudo-label. This represents the j-th pseudo-tag.
[0069] In conjunction with the second aspect, in one implementation, the target loss function in the weighted self-training process is:
[0070] L total =βL st +L adv
[0071] In the formula, L total Let L represent the objective loss function, β represent the parameter used to balance the losses from multiple tasks, and L represent the target loss function. advThis represents the adversarial loss function, which consists of a source domain class alignment loss function and a target domain class alignment loss function.
[0072] Thirdly, embodiments of this application provide a target detection model construction device based on a dual dictionary. The target detection model construction device based on a dual dictionary includes a processor, a memory, and a target detection model construction program based on a dual dictionary stored in the memory and executable by the processor. When the target detection model construction program based on a dual dictionary is executed by the processor, it implements the steps of the target detection model construction method based on a dual dictionary as described above.
[0073] Fourthly, embodiments of this application provide a computer-readable storage medium storing a target detection model construction program based on a dual dictionary, wherein when the target detection model construction program based on a dual dictionary is executed by a processor, it implements the steps of the target detection model construction method based on a dual dictionary as described above.
[0074] The beneficial effects of the technical solutions provided in this application include:
[0075] By transforming source domain samples into target domain style samples and vice versa, and then constructing source domain dictionaries and target domain dictionaries based on the source and target domain style samples respectively, a dual dictionary with complementary characteristics is generated to compensate for the domain offset problem between the source and target domains. This results in a dictionary with more information, laying the foundation for obtaining more accurate pseudo-labels. Then, with the help of the target domain dictionary and the source domain dictionary, high-confidence source domain candidate boxes and target domain candidate boxes are mined based on the source and target domain style samples respectively. Pseudo-labels and confidence weights are generated for unlabeled target domain samples based on the IoU between the candidate boxes. Finally, weighted self-training is performed on the target domain samples to build an object detection model, thereby effectively improving the model's detection performance in the target domain. Attached Figure Description
[0076] Figure 1 This is a flowchart illustrating an embodiment of the object detection model construction method based on dual dictionaries in this application;
[0077] Figure 2 This is a schematic diagram illustrating the overall model construction process involved in the embodiments of this application;
[0078] Figure 3 This is a schematic diagram of the pseudo-labeling and weight allocation process guided by a dual dictionary in the embodiments of this application;
[0079] Figure 4This is a schematic diagram of the hardware structure of the target detection model construction device based on dual dictionaries involved in the embodiments of this application. Detailed Implementation
[0080] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.
[0081] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0082] In a first aspect, embodiments of this application provide a method for constructing a target detection model based on a dual-dictionary approach.
[0083] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the object detection model construction method based on a dual-dictionary in this application. Figure 1 As shown, the method for constructing a target detection model based on a dual-dictionary includes:
[0084] Step S10: Convert the source domain sample into a target domain style sample, and convert the target domain sample into a source domain style sample.
[0085] As an example, it is understandable that while pseudo-labels in the target domain can be assigned by extracting a dictionary for each category in the source domain and using the learned dictionary, directly constructing a dictionary in the source domain and applying it to the target domain has the following drawbacks: due to the entanglement of domain-invariant and domain-specific features, the constructed dictionary contains a large amount of source domain-specific information; and during feature reconstruction, a large amount of domain-specific information in the target domain can be regarded as noise. Therefore, when the signal-to-noise ratio is low, the class sensitivity of the reconstructed residual will decrease, making it easy to generate incorrect pseudo-labels in the target domain.
[0086] To address the aforementioned issues, this embodiment will assign pseudo-tags by constructing a dual dictionary with complementary characteristics. For details, see [link to documentation]. Figure 2As shown, it is preferable to use a CycleGan network (i.e., a recurrent generative adversarial network, which is an unsupervised generative adversarial network that mainly trains two pairs of generator-discriminator models to transform images from one domain to another) to transform target domain samples into source domain style samples and source domain samples into target domain style samples, so as to construct a source domain dictionary from the source domain samples and generate a target domain dictionary from the transformed target domain style samples.
[0087] Step S20: Construct a source domain dictionary based on the source domain samples, and construct a target domain dictionary based on the target domain style samples.
[0088] As an example, it should be understood that CycleGAN networks can preserve some domain-invariant information while performing style transfer on image samples through cycle consistency loss, i.e., transferring certain domain-specific information. Therefore, the transferred data will contain less source-specific information. However, since its transfer process is unsupervised, some negative transfers may compromise domain invariance. Therefore, to compensate for the feature loss caused by this negative transfer, this embodiment constructs dictionaries not only in the target domain but also in the source domain. That is, a target domain dictionary is constructed in the target domain using style samples from the target domain, and another complementary source domain dictionary is built in the source domain using labeled training samples from the source domain. The target category to be detected is stored through the source domain dictionary and the target domain dictionary.
[0089] Further, in one embodiment, the step of constructing a source domain dictionary based on source domain samples and constructing a target domain dictionary based on target domain style samples includes:
[0090] Feature extraction is performed on source domain samples and target domain style samples respectively to determine the source domain feature matrix and the target domain feature matrix;
[0091] Dictionary learning is performed on the source domain feature matrix and the target domain feature matrix respectively based on L1 sparse constraints to generate source domain dictionary and target domain dictionary.
[0092] As an example, in this embodiment, the source domain sample is represented as S0 and the target domain style sample is represented as T1; see also Figure 2 As shown, a preferred approach is to extract various feature matrices from source domain samples and target domain style samples using a feature extraction network; wherein, the k-th class feature matrix corresponding to the target domain style sample is represented as... And the k-th class feature matrix corresponding to the source domain sample is represented as It should be noted that each column in the feature matrix represents a feature vector of the k-th class after L2 regularization (a technique to prevent overfitting in machine learning and statistical modeling), R l×nThis represents an l×n dimensional real matrix, where l represents the feature dimension and n represents the total number of samples.
[0093] For each category k, dictionary learning is performed to obtain the target domain dictionary for the k-th category. and the source domain dictionary of the kth category m represents dictionary D k The number of atoms in the dictionary; it should be understood that, for the k-th category, this embodiment will perform dictionary learning by solving the following L1 (i.e., L1 norm, which is used to introduce sparsity in optimization problems) sparse optimization problem;
[0094]
[0095]
[0096] in, Describes the sparse matrix of the target domain for the k-th category and Let the source domain sparse matrix of the k-th class be represented and λ1, λ2 represent regularization hyperparameters, d j express and The atoms in the array are all L2 norm normalized.
[0097] After performing equations (1) and (2) on all categories respectively, two matrices are obtained and labeled as follows: as well as That is, generate the source domain dictionary. and target domain dictionary
[0098] Step S30: Construct candidate bounding boxes for the target domain using source domain style samples and the target domain dictionary, and construct candidate bounding boxes for the source domain using target domain style samples and the source domain dictionary.
[0099] As an example, in this embodiment, a Faster-RCNN (Faster Region-based Convolutional Neural Network) model can be preferably used as the baseline detector to output the detection boxes. See also... Figure 3As shown, the Faster-RCNN model mainly includes a Region Proposal Network (RPN) and a Region of Interest Alignment Network (ROI-Align). Specifically, source domain style samples and target domain dictionaries are fed into the RPN for prediction, and feature maps and their corresponding feature matrices are extracted based on the ROI-Alignment Network and the ground truth values of source domain samples. This is used to calculate a set of candidate objects corresponding to the target domain through sensitivity and other methods. Conversely, target domain style images and source domain dictionaries are fed into the RPN for prediction, and feature maps and their corresponding feature matrices are extracted based on the ROI-Alignment Network and the ground truth values of source domain style samples. This is used to calculate a set of candidate objects corresponding to the source domain through sensitivity and other methods.
[0100] Further, in one embodiment, constructing the target domain candidate box using source domain style samples and the target domain dictionary includes:
[0101] The source domain style samples are fed into the region generation network to output the target domain detection box;
[0102] Target domain detection boxes with confidence scores greater than a first score threshold are added to the first set, and target domain detection boxes with confidence scores greater than a second score threshold are added to the third set as target domain candidate boxes, wherein the first score threshold is less than the second score threshold;
[0103] The target domain detection boxes in the first set are fed into the region of interest alignment network to output the first feature matrix. The first feature matrix is then encoded into the target domain dictionary based on L2 regularization to obtain the target domain encoding coefficient vector.
[0104] The target domain residual matrix is calculated based on the target domain coding coefficient vector, the first feature matrix, and the target domain dictionary, and the sensitivity is calculated using the target domain residual matrix.
[0105] The target domain detection boxes with sensitivity greater than the sensitivity threshold in the first set are added to the second set as target domain candidate boxes, and the union of the third set and the second set is taken as the fourth set.
[0106] As an example, in this embodiment, the specific values of the first score threshold θ1, the second score threshold θ3, and the sensitivity threshold θ2 can be determined according to actual needs and are not limited here, as long as the first score threshold θ1 is less than the second score threshold θ3.
[0107] For details, see Figure 3 As shown, source domain style samples are fed into Faster-RCNN to output target domain detection boxes that include location information, category information, and confidence scores. This yields the output of the Faster-RCNN detector. in Let n be the number of target domain detection boxes after non-maximum suppression (NMS); then determine the target domain detection boxes. If the confidence score is greater than the first score threshold θ1, discard it; otherwise, keep the target domain detection box. Add to the first set In the first set, m1 represents the number of elements in the first set; then the first set... Target domain detection box in The data is fed into the ROI-Align network to obtain its feature map and extract its corresponding feature matrix x. T Then, L2 regularization is applied to the feature matrix x. T Encoding in the target domain dictionary The optimized target domain coding coefficient vector is obtained by minimizing the objective function of equation (3).
[0108] In the formula, λ3 is the regularization hyperparameter, and its specific value can be determined according to the actual situation; This represents the initial target domain coding coefficient vector, the specific value of which can be determined according to the actual situation.
[0109] Next, based on the target domain coding coefficient vector Feature matrix x T and target domain dictionary D T Calculate the target domain residual vector for each category. That is, the target domain residual matrix r is obtained through the following equation (4). T :
[0110]
[0111] Among them, (D) T ) T It is D T transpose,
[0112] For the residual matrix r in the target domain T Each target domain residual vector For the residual vector in the target domain Sort in ascending order to obtain the corresponding ordered result. After sorting all the residual vectors of the target domain, an ordered set of target domains can be obtained. And it satisfies The sensitivity is then calculated using the following formula (5):
[0113]
[0114] Among them, the class sensitivity Sen is used to measure the feature matrix x. T The higher the matching degree with the k-th category, i.e., the higher the sensitivity, the higher the category uniqueness of the sample information, which further means that the sample has a high probability of belonging to a certain category of foreground (foreground refers to the object to be detected and identified). Therefore, based on the first set... Target detection box The corresponding x T The calculated sensitivity Sen(x) T The second set is constructed based on the relationship between the magnitude of the two values and the sensitivity threshold θ2. In the middle; specifically, select the first set. In which Sen(x) is satisfied T The target region detection box with a value greater than θ2 is added to the second set as a target region candidate box.
[0115] At the same time, it is also necessary to set Target detection boxes with a confidence score higher than the second score threshold θ3 Added as a candidate bounding box for the target domain to the third set. In the middle, m2 represents the number of elements in the third set; finally, the second set is... and the third set The union of the four sets is used as the fourth set. Right now and the fourth set It is a set of candidate boxes obtained by mining pseudo-labels in the target domain.
[0116] Further, in one embodiment, the step of constructing source domain candidate boxes using target domain style samples and source domain dictionaries includes:
[0117] The target domain style samples are fed into the region generation network to output source domain detection boxes;
[0118] Source domain detection boxes with confidence scores greater than the first score threshold are added to the fifth set, and source domain detection boxes with confidence scores greater than the second score threshold are added to the seventh set as source domain candidate boxes, where the first score threshold is less than the second score threshold.
[0119] The source domain detection boxes in the fifth set are fed into the region of interest alignment network to output the second feature matrix. The second feature matrix is then encoded into the source domain dictionary based on L2 regularization to obtain the source domain encoding coefficient vector.
[0120] The source domain residual matrix is calculated based on the source domain coding coefficient vector, the second feature matrix, and the source domain dictionary, and the sensitivity is calculated using the source domain residual matrix.
[0121] Source domain detection boxes with sensitivity greater than the sensitivity threshold in the fifth set are added to the sixth set as source domain candidate boxes, and the union of the sixth and seventh sets is taken as the eighth set.
[0122] As an example, in this embodiment, the specific values of the first score threshold θ1, the second score threshold θ3, and the sensitivity threshold θ2 can be determined according to actual needs and are not limited here, as long as the first score threshold θ1 is less than the second score threshold θ3.
[0123] For details, see Figure 3 As shown, source domain style samples are fed into Faster-RCNN to output source domain detection boxes that include location information, category information, and confidence scores. This yields the output of the Faster-RCNN detector. in Let n be the number of source region detection boxes after non-maximum suppression (NMS); then determine the source region detection boxes. If the confidence score is greater than the first score threshold θ1, discard it; otherwise, keep the source domain detection box. Added to the fifth set In the context of the fifth set, m3 represents the number of elements in the fifth set; then the fifth set... Source domain detection box in The data is fed into the ROI-Align network to obtain its feature map and extract its corresponding feature matrix x. S Then, L2 regularization is applied to the feature matrix x. S Encoding in the source domain dictionary The optimized source domain coding coefficient vector is obtained by minimizing the objective function of equation (6).
[0124] In the formula, λ3 is the regularization hyperparameter, and its specific value can be determined according to the actual situation; This represents the initial source domain coding coefficient vector, the specific value of which can be determined according to the actual situation.
[0125] Next, based on the source domain coding coefficient vector Feature matrix x S Dictionary of source domains S Calculate the source domain residual vector for each category. That is, the source domain residual matrix r is obtained by the following equation (7). S :
[0126]
[0127] Among them, (D) S ) T It is D Stranspose,
[0128] For the source domain residual matrix r S Each source domain residual vector in For source domain residual vector Sort in ascending order to obtain the corresponding ordered result. After sorting all the residual vectors of the target domain, an ordered set of target domains can be obtained. And it satisfies The sensitivity is then calculated using the following formula (8):
[0129]
[0130] Among them, the class sensitivity Sen is used to measure the feature matrix x. S The higher the matching degree with the k-th category, i.e., the higher the sensitivity, the higher the category uniqueness of the sample information, which further means that the sample has a high probability of belonging to a certain category of foreground (foreground refers to the object to be detected and identified). Therefore, it can be based on the fifth set. Mid-source domain detection frame The corresponding x S The calculated sensitivity Sen(x) S The sixth set is constructed based on the relationship between the magnitude of the ) and the sensitivity threshold θ2. In the middle; specifically, the fifth set is selected. In which Sen(x) is satisfied S The source region detection boxes with a value greater than θ2 are added as source region candidate boxes to the sixth set.
[0131] At the same time, it is also necessary to set Source domain detection boxes with a confidence score higher than the second score threshold θ3 Added to set 7 as a candidate source domain bounding box In the middle, m4 represents the number of elements in the seventh set; finally, the sixth set is... and the seventh set The union of the sets is the eighth set. Right now And the eighth set It is a set of candidate boxes obtained by mining pseudo-labels in the source domain.
[0132] Step S40: Determine the pseudo-labels and their corresponding confidence weights based on the Intersection over Union (IoU) between the target domain candidate boxes and the source domain candidate boxes.
[0133] Exemplary and understandable, the feature distributions of source domain samples and source domain style samples transformed from target domain samples are different, resulting in different positions of candidate boxes generated by the two dictionary mining processes. Since the foreground contains domain-invariant features, the higher the overlap between the candidate boxes generated by the two dictionaries, the greater the probability of foreground presence in that region, indicating a more reliable predicted box. Therefore, this embodiment will be based on a third set. The target domain candidate boxes and the seventh set Pseudo-labels are assigned based on the IoU values between candidate source domain boxes in the dataset, and based on the fourth set. The target domain candidate boxes and the eighth set The pseudo-label confidence weight is generated by cross-joining the IoU values between the candidate boxes in the source domain.
[0134] Further, in one embodiment, determining the pseudo-labels and their corresponding confidence weights based on the Intersection over Union (IoU) between the target domain candidate boxes and the source domain candidate boxes includes:
[0135] When the predicted category of the first target domain candidate box in the third set is the same as the predicted category of the first source domain candidate box in the seventh set, the first IoU between the first target domain candidate box and the first source domain candidate box is calculated.
[0136] Pseudo-labels are determined from the first target domain candidate box and the first source domain candidate box based on the relationship between the first IoU and the first IoU threshold.
[0137] Calculate the second IoU between the second target domain candidate box and the second source domain candidate box in the fourth set;
[0138] When the second IoU is greater than the second IoU threshold, the second IoU is used as the confidence weight of the pseudo-label.
[0139] When the second IoU is not greater than the second IoU threshold, the smallest second IoU is selected as the confidence weight of the pseudo-label based on iterative training.
[0140] As an example, it should be noted that the specific values of the first IoU threshold δ1 and the second IoU threshold δ2 can be determined according to actual needs and are not limited here. In this embodiment, if the third set Candidate boxes for the target domain in b T With the seventh set Source domain candidate box b S When the predicted categories are consistent, the IoU value of the predicted object's location information is calculated using the following formula:
[0141]
[0142] Intersection area Represents two candidate boxes b T and b S The area of the overlapping bounding boxes of Union area Represents two candidate boxes b T and b S The total area covered.
[0143] If IoU(b T ,b S If IoU(b) > δ1, it means that the two candidate boxes are the results of detecting the same target, so one of the candidate boxes can be used as the pseudo-label of the target; while if IoU(b) > δ1, it means that the two candidate boxes are the results of detecting the same target, so one of the candidate boxes can be used as the pseudo-label of the target; T ,b S If )≤δ1, it means that the two candidate boxes are detection results of different targets, and both candidate boxes are regarded as pseudo labels.
[0144] It should be understood that IoU cannot measure the reliability of pseudo-labels. However, when two pseudo-labels are highly similar, it indicates that the foreground portion of the target box corresponding to the detection result is large, meaning it mainly represents domain-invariant features. In other words, the detection box accurately encloses the target region, resulting in less background. Therefore, such detection results can be assigned a higher confidence weight. Conversely, when two pseudo-labels are low similar, it indicates that the target box corresponding to the detection result has a large background portion, meaning it contains a large number of domain-specific features. Understandably, due to the domain transformation gap, the consistency of pseudo-label assignments between the two dictionaries for this detection result is affected. Therefore, such detection results should be assigned a lower confidence weight to train the domain-adaptive detection model, thereby reducing the impact of these pseudo-label samples on the accuracy of the detection box.
[0145] Specifically, in this embodiment, the fourth set will be used. Candidate boxes for the target domain in b T and the eighth set Source domain candidate box b S The IoU value between the pseudo-labels is used to generate the confidence weights for each pseudo-label.
[0146]
[0147] It should be noted that when IoU(b) in equation (10) T ,b S If )>δ2, then the IoU(b) T ,b S ) as the confidence weight of the pseudo-label; and if IoU(b) in equation (10) T ,b S If )≤δ2, then the IoU(b) during the iterative training process will be...T ,b S The minimum value is used as the confidence weight for the pseudo-label. It can be understood that the confidence weight reflects the quality of the pseudo-label; that is, higher-quality candidate boxes are assigned greater weight, and vice versa.
[0148] Step S50: Perform weighted self-training based on the pseudo-labels, the confidence weights, the source domain samples, and the target domain samples to construct an object detection model. The loss function L for the weighted self-training is... st for:
[0149]
[0150] In the formula, This represents the bounding box regression loss. I represents the classification loss. s Represents source domain samples, Represents source domain sample I s The true label of the i-th foreground sample, where N represents the number of true labels in the source domain samples, and I T Let M represent the number of pseudo-labels for the target domain samples, μ represent the parameters used to balance the source and target domain losses, and w represent the target domain samples. j This represents the confidence weight of the j-th pseudo-label. This represents the j-th pseudo-tag.
[0151] The objective loss function in the weighted self-training process is:
[0152] L total =βL st +L adv
[0153] In the formula, L total Let L represent the objective loss function, β represent the parameter used to balance the losses from multiple tasks, and L represent the target loss function. adv This represents the adversarial loss function, which consists of a source domain class alignment loss function and a target domain class alignment loss function.
[0154] Exemplary, in this embodiment, a category-aware alignment process and a weighted self-training process are performed based on pseudo-labels, confidence weights, source domain samples, and target domain samples. The network parameters are updated using a weighted self-training loss and a category alignment loss, ultimately generating an object detection model. See also... Figure 2 As shown, in the IoU-based weighted self-training process, the weighted self-training loss function L is used. st for:
[0155]
[0156] In the formula, This represents the bounding box regression loss. I represents the classification loss. s Represents source domain samples, Represents source domain sample I s The true label of the i-th foreground sample, where N represents the number of true labels in the source domain samples, and I T Let M represent the number of pseudo-labels for the target domain samples, and μ represent the parameter used to balance the source and target domain losses, the specific value of which can be determined according to actual needs. j This represents the confidence weight of the j-th pseudo-label. This represents the j-th pseudo-tag.
[0157] In this embodiment, K category-aware discriminators are used to adjust the features of specific categories during category-based alignment, and a Focal Loss loss function is used to encourage the domain classifier to focus on instances that are difficult to classify during alignment. It should be noted that the working principle and method of the category-aware discriminator are common knowledge in the field and will not be elaborated upon here for the sake of brevity. The loss function for the k-th category-guided alignment includes the source domain category alignment loss function. Alignment loss function with target domain category and and It can be written as:
[0158]
[0159]
[0160] in, This represents the feature map after ROI alignment of the k-th class in the source domain, which can be obtained from the true label of the source domain; This represents the feature map of the k-th class in the target domain after ROI alignment, which can be obtained from the pseudo-label of the target domain; and These represent the number of the k-th class foreground elements in the input source domain samples and the target domain samples, respectively. This represents the discrimination result of the k-th category-aware discriminator on the input image; γ is a hyperparameter used to control the weights of instances that are difficult to classify. It should be noted that this embodiment also employs a gradient inversion layer when processing the feature matrices of different categories through multi-path data processing. This introduces adversarial training into the feature representation learning process, and the gradient is inverted before being output as a dictionary D for each category before propagating the gradient back to the feature extraction network. k If we output the source domain dictionary and target domain dictionary for each category, then the adversarial loss can be summarized as follows:
[0161]
[0162] In the formula, L adv This represents the adversarial loss function.
[0163] Combining all the above losses, we obtain the overall objective loss function, which can be expressed as:
[0164] L total =βL st +L adv (15)
[0165] Here, β represents a parameter used to balance the loss of multiple tasks, and its specific value can be determined according to actual needs.
[0166] In summary, this embodiment addresses the problem of poor model transfer performance in unsupervised adaptive object detection caused by significant style differences between domains. It provides an object detection framework based on a dual-domain dictionary. Specifically, it learns two dictionaries representing specific categories in the source and target domains respectively, and reconstructs candidate boxes in both domains using these dictionaries, resulting in two sets of candidate boxes with complementary characteristics. Then, pseudo-labels are assigned to the candidate boxes based on the Intersection over Union (IoU) between the two sets of candidate boxes. Finally, confidence weights are assigned based on the IoU of the pseudo-labels, and weighted self-training is performed on target domain samples. Therefore, this embodiment compensates for the domain offset problem between the source and target domains by constructing a dual-domain dictionary, and generates pseudo-labels and confidence weights for unlabeled target domain samples in a complementary manner. This effectively improves the model's detection performance in the target domain and enhances its generalization ability.
[0167] Secondly, embodiments of this application also provide a target detection model construction device based on a dual dictionary.
[0168] In one embodiment, the dual-dictionary-based target detection model construction device includes:
[0169] The style transfer module is used to convert source domain samples into target domain style samples and vice versa.
[0170] The dictionary building module is used to build a source domain dictionary based on source domain samples and a target domain dictionary based on target domain style samples.
[0171] The candidate box determination module is used to construct target domain candidate boxes using source domain style samples and target domain dictionary, and to construct source domain candidate boxes using target domain style samples and source domain dictionary;
[0172] The label weight determination module is used to determine the pseudo-labels and their corresponding confidence weights based on the intersection-union ratio (IoU) between the target domain candidate boxes and the source domain candidate boxes.
[0173] The model building module is used to perform weighted self-training based on the pseudo-labels, the confidence weights, the source domain samples, and the target domain samples to build an object detection model.
[0174] Furthermore, in one embodiment, the dictionary construction module is specifically used for:
[0175] Feature extraction is performed on source domain samples and target domain style samples respectively to determine the source domain feature matrix and the target domain feature matrix;
[0176] Dictionary learning is performed on the source domain feature matrix and the target domain feature matrix respectively based on L1 sparse constraints to generate source domain dictionary and target domain dictionary.
[0177] Furthermore, in one embodiment, the candidate box determination module is specifically used for:
[0178] The source domain style samples are fed into the region generation network to output the target domain detection box;
[0179] Target domain detection boxes with confidence scores greater than a first score threshold are added to the first set, and target domain detection boxes with confidence scores greater than a second score threshold are added to the third set as target domain candidate boxes, wherein the first score threshold is less than the second score threshold;
[0180] The target domain detection boxes in the first set are fed into the region of interest alignment network to output the first feature matrix. The first feature matrix is then encoded into the target domain dictionary based on L2 regularization to obtain the target domain encoding coefficient vector.
[0181] The target domain residual matrix is calculated based on the target domain coding coefficient vector, the first feature matrix, and the target domain dictionary, and the sensitivity is calculated using the target domain residual matrix.
[0182] The target domain detection boxes with sensitivity greater than the sensitivity threshold in the first set are added to the second set as target domain candidate boxes, and the union of the third set and the second set is taken as the fourth set.
[0183] Furthermore, in one embodiment, the candidate box determination module is specifically used for:
[0184] The target domain style samples are fed into the region generation network to output source domain detection boxes;
[0185] Source domain detection boxes with confidence scores greater than the first score threshold are added to the fifth set, and source domain detection boxes with confidence scores greater than the second score threshold are added to the seventh set as source domain candidate boxes, where the first score threshold is less than the second score threshold.
[0186] The source domain detection boxes in the fifth set are fed into the region of interest alignment network to output the second feature matrix. The second feature matrix is then encoded into the source domain dictionary based on L2 regularization to obtain the source domain encoding coefficient vector.
[0187] The source domain residual matrix is calculated based on the source domain coding coefficient vector, the second feature matrix, and the source domain dictionary, and the sensitivity is calculated using the source domain residual matrix.
[0188] Source domain detection boxes with sensitivity greater than the sensitivity threshold in the fifth set are added to the sixth set as source domain candidate boxes, and the union of the sixth and seventh sets is taken as the eighth set.
[0189] Furthermore, in one embodiment, the label weight determination module is specifically used for:
[0190] When the predicted category of the first target domain candidate box in the third set is the same as the predicted category of the first source domain candidate box in the seventh set, the first IoU between the first target domain candidate box and the first source domain candidate box is calculated.
[0191] Pseudo-labels are determined from the first target domain candidate box and the first source domain candidate box based on the relationship between the first IoU and the first IoU threshold.
[0192] Calculate the second IoU between the second target domain candidate box and the second source domain candidate box in the fourth set;
[0193] When the second IoU is greater than the second IoU threshold, the second IoU is used as the confidence weight of the pseudo-label.
[0194] When the second IoU is not greater than the second IoU threshold, the smallest second IoU is selected as the confidence weight of the pseudo-label based on iterative training.
[0195] Furthermore, in one embodiment, the weighted self-training loss function L st for:
[0196]
[0197] In the formula, This represents the bounding box regression loss. I represents the classification loss. s Represents source domain samples, Represents source domain sample I s The true label of the i-th foreground sample, where N represents the number of true labels in the source domain samples, and I T Let M represent the number of pseudo-labels for the target domain samples, μ represent the parameters used to balance the source and target domain losses, and w represent the target domain samples. j This represents the confidence weight of the j-th pseudo-label. This represents the j-th pseudo-tag.
[0198] Furthermore, in one embodiment, the target loss function in the weighted self-training process is:
[0199] L total =βL st +L adv
[0200] In the formula, L total Let L represent the objective loss function, β represent the parameter used to balance the losses from multiple tasks, and L represent the target loss function. adv This represents the adversarial loss function, which consists of a source domain class alignment loss function and a target domain class alignment loss function.
[0201] The functions of each module in the above-mentioned dual-dictionary-based target detection model construction device correspond to the steps in the above-mentioned dual-dictionary-based target detection model construction method embodiment, and their functions and implementation processes will not be described in detail here.
[0202] Thirdly, embodiments of this application provide a target detection model construction device based on a dual dictionary. The target detection model construction device based on a dual dictionary can be a personal computer (PC), a laptop computer, a server, or other devices with sample processing capabilities.
[0203] Reference Figure 4 , Figure 4 This is a schematic diagram of the hardware structure of the target detection model construction device based on a dual dictionary involved in the embodiments of this application. In the embodiments of this application, the target detection model construction device based on a dual dictionary may include a processor, a memory, a communication interface, and a communication bus.
[0204] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.
[0205] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces. These interfaces facilitate interconnection of devices within the dual-dictionary-based target detection model building device, and also enable interconnection between the dual-dictionary-based target detection model building device and other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.
[0206] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.
[0207] The processor can be a general-purpose processor, which can call the object detection model construction program based on the dual dictionary stored in memory and execute the object detection model construction method based on the dual dictionary provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the object detection model construction program based on the dual dictionary is called can be referred to in the various embodiments of the object detection model construction method based on the dual dictionary of this application, and will not be repeated here.
[0208] Those skilled in the art will understand that Figure 4 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0209] Fourthly, embodiments of this application also provide a computer-readable storage medium.
[0210] This application stores a target detection model construction program based on a dual dictionary on a readable storage medium, wherein when the target detection model construction program based on a dual dictionary is executed by a processor, it implements the steps of the target detection model construction method based on a dual dictionary as described above.
[0211] The method implemented when the object detection model construction program based on dual dictionaries is executed can be referred to in the various embodiments of the object detection model construction method based on dual dictionaries in this application, and will not be repeated here.
[0212] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0213] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.
[0214] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.
[0215] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.
[0216] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.
[0217] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.
[0218] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A method for constructing a target detection model based on a dual-dictionary approach, characterized in that, Includes the following steps: Transform source domain samples into target domain style samples, and then transform target domain samples into source domain style samples; A source domain dictionary is constructed based on source domain samples, and a target domain dictionary is constructed based on target domain style samples; Target domain candidate boxes are constructed using source domain style samples and target domain dictionary, and source domain candidate boxes are constructed using target domain style samples and source domain dictionary; The pseudo-labels and their corresponding confidence weights are determined based on the intersection-union ratio (IoU) between the target domain candidate boxes and the source domain candidate boxes. A target detection model is constructed by performing weighted self-training based on the pseudo-labels, the confidence weights, the source domain samples, and the target domain samples. The step of constructing a target domain candidate box using source domain style samples and a target domain dictionary includes: The source domain style samples are fed into the region generation network to output the target domain detection box; Target domain detection boxes with confidence scores greater than a first score threshold are added to the first set, and target domain detection boxes with confidence scores greater than a second score threshold are added to the third set as target domain candidate boxes, wherein the first score threshold is less than the second score threshold; The target domain detection boxes in the first set are fed into the region of interest alignment network to output the first feature matrix. The first feature matrix is then encoded into the target domain dictionary based on L2 regularization to obtain the target domain encoding coefficient vector. The target domain residual matrix is calculated based on the target domain coding coefficient vector, the first feature matrix, and the target domain dictionary, and the sensitivity is calculated using the target domain residual matrix. The target domain detection boxes with sensitivity greater than the sensitivity threshold in the first set are added to the second set as target domain candidate boxes, and the union of the third set and the second set is used as the fourth set; The step of constructing source domain candidate boxes using target domain style samples and source domain dictionaries includes: The target domain style samples are fed into the region generation network to output source domain detection boxes; Source domain detection boxes with confidence scores greater than the first score threshold are added to the fifth set, and source domain detection boxes with confidence scores greater than the second score threshold are added to the seventh set as source domain candidate boxes, where the first score threshold is less than the second score threshold. The source domain detection boxes in the fifth set are fed into the region of interest alignment network to output the second feature matrix. The second feature matrix is then encoded into the source domain dictionary based on L2 regularization to obtain the source domain encoding coefficient vector. The source domain residual matrix is calculated based on the source domain coding coefficient vector, the second feature matrix, and the source domain dictionary, and the sensitivity is calculated using the source domain residual matrix. Source domain detection boxes with sensitivity greater than the sensitivity threshold in the fifth set are added to the sixth set as source domain candidate boxes, and the union of the sixth and seventh sets is taken as the eighth set. The step of determining the pseudo-labels and their corresponding confidence weights based on the Intersection over Union (IoU) between the target domain candidate boxes and the source domain candidate boxes includes: When the predicted category of the first target domain candidate box in the third set is the same as the predicted category of the first source domain candidate box in the seventh set, the first IoU between the first target domain candidate box and the first source domain candidate box is calculated. Pseudo-labels are determined from the first target domain candidate box and the first source domain candidate box based on the relationship between the first IoU and the first IoU threshold. Calculate the second IoU between the second target domain candidate box and the second source domain candidate box in the fourth set; When the second IoU is greater than the second IoU threshold, the second IoU is used as the confidence weight of the pseudo-label. When the second IoU is not greater than the second IoU threshold, the smallest second IoU is selected as the confidence weight of the pseudo-label based on iterative training. The weighted self-training loss function for: In the formula, This represents the bounding box regression loss. Represents classification loss. Represents source domain samples, Represents source domain samples The Middle N represents the number of true labels for the foreground samples, where N is the number of true labels for the source domain samples. Let M represent the target domain samples, and M represent the number of pseudo-labels for the target domain samples. This represents the parameter used to balance the losses in the source and target domains. Indicates the first j The confidence weight of each pseudo-label Indicates the first j A pseudo-label.
2. The method for constructing a target detection model based on a dual dictionary as described in claim 1, characterized in that, The construction of a source domain dictionary based on source domain samples and a target domain dictionary based on target domain style samples includes: Feature extraction is performed on source domain samples and target domain style samples respectively to determine the source domain feature matrix and the target domain feature matrix; Dictionary learning is performed on the source domain feature matrix and the target domain feature matrix respectively based on L1 sparse constraints to generate source domain dictionary and target domain dictionary.
3. The method for constructing a target detection model based on a dual dictionary as described in claim 1, characterized in that, The objective loss function in the weighted self-training process is: In the formula, Represents the target loss function. This represents the parameter used to balance the loss from multiple tasks. This represents the adversarial loss function, which consists of a source domain class alignment loss function and a target domain class alignment loss function.
4. A device for constructing a target detection model based on a dual-dictionary system, characterized in that, include: The style transfer module is used to convert source domain samples into target domain style samples and vice versa. The dictionary building module is used to build a source domain dictionary based on source domain samples and a target domain dictionary based on target domain style samples. The candidate box determination module is used to construct target domain candidate boxes using source domain style samples and target domain dictionary, and to construct source domain candidate boxes using target domain style samples and source domain dictionary; The label weight determination module is used to determine the pseudo-labels and their corresponding confidence weights based on the intersection-union ratio (IoU) between the target domain candidate boxes and the source domain candidate boxes. The model building module is used to perform weighted self-training based on the pseudo-labels, the confidence weights, the source domain samples, and the target domain samples to build an object detection model; Specifically, the candidate box determination module is used to feed source domain style samples into a region generation network to output target domain detection boxes; add target domain detection boxes with confidence scores greater than a first score threshold to a first set, and add target domain detection boxes with confidence scores greater than a second score threshold as target domain candidate boxes to a third set, where the first score threshold is less than the second score threshold; feed the target domain detection boxes in the first set into a region of interest alignment network to output a first feature matrix, and encode the first feature matrix into a target domain dictionary based on L2 regularization to obtain a target domain encoding coefficient vector; calculate the target domain residual matrix based on the target domain encoding coefficient vector, the first feature matrix, and the target domain dictionary, and calculate the sensitivity using the target domain residual matrix; add target domain detection boxes in the first set with sensitivity greater than a sensitivity threshold as target domain candidate boxes to a second set, and add the third set and... The union of the second set is used as the fourth set; target domain style samples are fed into the region generation network to output source domain detection boxes; source domain detection boxes with confidence scores greater than the first score threshold are added to the fifth set, and source domain detection boxes with confidence scores greater than the second score threshold are added to the seventh set as source domain candidate boxes, where the first score threshold is less than the second score threshold; source domain detection boxes in the fifth set are fed into the region of interest alignment network to output the second feature matrix, and the second feature matrix is encoded into the source domain dictionary based on L2 regularization to obtain the source domain encoding coefficient vector; the source domain residual matrix is calculated based on the source domain encoding coefficient vector, the second feature matrix, and the source domain dictionary, and the sensitivity is calculated through the source domain residual matrix; source domain detection boxes in the fifth set with sensitivity greater than the sensitivity threshold are added to the sixth set as source domain candidate boxes, and the union of the sixth and seventh sets is used as the eighth set; The label weight determination module is specifically used to calculate the first IoU between the first target domain candidate box and the first source domain candidate box when the predicted category corresponding to the first target domain candidate box in the third set is the same as the predicted category corresponding to the first source domain candidate box in the seventh set; determine the pseudo label from the first target domain candidate box and the first source domain candidate box according to the relationship between the first IoU and the first IoU threshold; calculate the second IoU between the second target domain candidate box and the second source domain candidate box in the fourth set; and use the second IoU as the confidence weight of the pseudo label when the second IoU is greater than the second IoU threshold. When the second IoU is not greater than the second IoU threshold, the smallest second IoU is selected as the confidence weight of the pseudo-label based on iterative training. The weighted self-training loss function for: In the formula, This represents the bounding box regression loss. Represents classification loss. Represents source domain samples, Represents source domain samples The Middle N represents the number of true labels for the foreground samples, where N is the number of true labels for the source domain samples. Let M represent the target domain samples, and M represent the number of pseudo-labels for the target domain samples. This represents the parameter used to balance the losses in the source and target domains. Indicates the first j The confidence weight of each pseudo-label Indicates the first j A pseudo-label.
5. A target detection model construction device based on a dual-dictionary system, characterized in that, The dual-dictionary-based target detection model construction device includes a processor, a memory, and a dual-dictionary-based target detection model construction program stored in the memory and executable by the processor, wherein when the dual-dictionary-based target detection model construction program is executed by the processor, it implements the steps of the dual-dictionary-based target detection model construction method as described in any one of claims 1 to 3.
6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a target detection model construction program based on a dual dictionary, wherein when the target detection model construction program based on a dual dictionary is executed by a processor, it implements the steps of the target detection model construction method based on a dual dictionary as described in any one of claims 1 to 3.
Citation Information
Patent Citations
Image super-resolution reconstruction method based on double-dictionary learning
CN102800076A
Human movement recognition method based on cross-domain dictionary learning
CN104063684A
