Target detection model construction method and device based on double dictionaries, equipment and medium

Abstract

The invention discloses a target detection model construction method and device based on double dictionaries, equipment and a medium, and relates to the technical field of computer vision, and the method comprises the steps: converting a source domain sample into a target domain style sample, and converting the target domain sample into a source domain style sample; constructing a source domain dictionary based on the source domain sample, and constructing a target domain dictionary based on the target domain style sample; constructing a target domain candidate frame through the source domain style sample and the target domain dictionary, and constructing a source domain candidate frame through the target domain style sample and the source domain dictionary; determining a false label and a confidence coefficient weight corresponding to the false label according to the intersection-to-union ratio IoU between the target domain candidate frame and the source domain candidate frame; and performing weighted self-training based on the pseudo tag, the confidence coefficient weight, the source domain sample and the target domain sample to construct a target detection model. According to the method and the device, the field offset problem between the source domain and the target domain can be compensated, and the detection performance of the model on the target domain is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, device, equipment and medium for constructing a target detection model based on a dual dictionary. Background Art

[0002] Object detection is a challenging task that aims to identify and locate all object instances of a specific category in an image. In recent years, this direction has made great progress in supervised methods and has been widely used in image applications, such as face recognition, autonomous driving, and medical detection. However, when the training samples and test samples come from different distributions, there is a domain shift problem, which causes the performance of the detector to drop sharply. Although the domain shift problem can be alleviated by manually collecting and annotating target domain training samples, this is very expensive and time-consuming.

[0003] In the related technology, an unsupervised domain adaptive target detection method based on pseudo-label training is used to extract and map the features of the two domains and narrow the distribution difference between the source domain and the target domain. That is, by learning the feature representation of the source domain, a high-confidence pseudo-label is assigned to the target domain samples, and then the detector is supervised trained to achieve the effect of improving the accuracy, thereby improving the performance of the detector.

[0004] However, the reliable pseudo-labels in the pseudo-label training method usually do not have the target domain knowledge, which limits the performance improvement of the detector in the target domain. Therefore, how to reduce the domain shift to improve the performance of the detection model is an urgent problem to be solved. Summary of the invention

[0005] The present application provides a method, device, equipment and medium for constructing a target detection model based on a dual dictionary, which can solve the technical problem of model performance degradation caused by domain offset in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a method for constructing a target detection model based on a dual dictionary, comprising the following steps:

[0007] Convert source domain samples into target domain style samples, and convert target domain samples into source domain style samples;

[0008] A source domain dictionary is constructed based on source domain samples, and a target domain dictionary is constructed based on target domain style samples;

[0009] Construct target domain candidate boxes through source domain style samples and target domain dictionary, and construct source domain candidate boxes through target domain style samples and source domain dictionary;

[0010] Determine the pseudo label and its corresponding confidence weight based on the intersection over union (IoU) between the target domain candidate box and the source domain candidate box;

[0011] Weighted self-training is performed based on the pseudo labels, the confidence weights, the source domain samples, and the target domain samples to construct a target detection model.

[0012] In combination with the first aspect, in one implementation, constructing a source domain dictionary based on source domain samples and constructing a target domain dictionary based on target domain style samples includes:

[0013] Feature extraction is performed on the source domain samples and the target domain style samples respectively to determine the source domain feature matrix and the target domain feature matrix;

[0014] According to the L1 sparse constraint, dictionary learning is performed on the source domain feature matrix and the target domain feature matrix respectively to generate a source domain dictionary and a target domain dictionary.

[0015] In combination with the first aspect, in one implementation, constructing a target domain candidate box using a source domain style sample and a target domain dictionary includes:

[0016] Feed the source domain style sample into the region generation network to output the target domain detection box;

[0017] Adding target domain detection boxes whose confidence scores are greater than a first score threshold to the first set, and adding target domain detection boxes whose confidence scores are greater than a second score threshold to the third set as target domain candidate boxes, wherein the first score threshold is less than the second score threshold;

[0018] The target domain detection box in the first set is sent to the region of interest alignment network, the first feature matrix is ​​output, and the first feature matrix is ​​encoded into the target domain dictionary based on L2 regularization to obtain the target domain encoding coefficient vector;

[0019] A target domain residual matrix is ​​calculated according to the target domain coding coefficient vector, the first characteristic matrix and the target domain dictionary, and sensitivity is calculated through the target domain residual matrix;

[0020] The target domain detection boxes in the first set whose sensitivity is greater than the sensitivity threshold are added to the second set as target domain candidate boxes, and the union of the third set and the second set is taken as the fourth set.

[0021] In combination with the first aspect, in one implementation, constructing a source domain candidate box using a target domain style sample and a source domain dictionary includes:

[0022] Send the target domain style sample to the region generation network to output the source domain detection box;

[0023] adding source domain detection boxes whose confidence scores are greater than a first score threshold to the fifth set, and adding source domain detection boxes whose confidence scores are greater than a second score threshold as source domain candidate boxes to the seventh set, where the first score threshold is less than the second score threshold;

[0024] The source domain detection box in the fifth set is sent to the region of interest alignment network, the second feature matrix is ​​output, and the second feature matrix is ​​encoded into the source domain dictionary based on L2 regularization to obtain the source domain encoding coefficient vector;

[0025] A source domain residual matrix is ​​calculated according to the source domain coding coefficient vector, the second characteristic matrix and the source domain dictionary, and the sensitivity is calculated through the source domain residual matrix;

[0026] The source domain detection boxes in the fifth set whose sensitivity is greater than the sensitivity threshold are added to the sixth set as source domain candidate boxes, and the union of the sixth set and the seventh set is taken as the eighth set.

[0027] In combination with the first aspect, in one implementation, determining the pseudo label and its corresponding confidence weight according to the intersection over union (IoU) ratio between the target domain candidate box and the source domain candidate box includes:

[0028] When the prediction category corresponding to the first target domain candidate box in the third set is the same as the prediction category corresponding to the first source domain candidate box in the seventh set, calculating a first IoU between the first target domain candidate box and the first source domain candidate box;

[0029] Determine a pseudo label from the first target domain candidate box and the first source domain candidate box according to a size relationship between the first IoU and the first IoU threshold;

[0030] Calculate a second IoU between the second target domain candidate box and the second source domain candidate box in the fourth set;

[0031] When the second IoU is greater than the second IoU threshold, the second IoU is used as the confidence weight of the pseudo label;

[0032] When the second IoU is not greater than the second IoU threshold, the smallest second IoU is selected based on iterative training as the confidence weight of the pseudo label.

[0033] In combination with the first aspect, in one embodiment, the loss function L of the weighted self-training st for:

[0034]

[0035] In the formula, represents the bounding box regression loss, represents the classification loss, I s represents the source domain sample, Represents the source domain sample I s The true label of the i-th foreground in , N represents the number of true labels of source domain samples, and I T represents the target domain sample, M represents the number of pseudo labels of the target domain sample, μ represents the parameter used to balance the loss of the source domain and the target domain, and w j represents the confidence weight of the jth pseudo label, represents the jth pseudo label.

[0036] In combination with the first aspect, in one implementation, the target loss function in the weighted self-training process is:

[0037] L total =βL st +L adv

[0038] Where, L total represents the target loss function, β represents the parameter used to balance the multi-task loss, and L adv Denotes the adversarial loss function, which is composed of the source domain category alignment loss function and the target domain category alignment loss function.

[0039] In a second aspect, an embodiment of the present application provides a dual-dictionary-based target detection model construction device, comprising:

[0040] A style conversion module, which is used to convert source domain samples into target domain style samples, and convert target domain samples into source domain style samples;

[0041] A dictionary construction module, which is used to construct a source domain dictionary based on source domain samples, and to construct a target domain dictionary based on target domain style samples;

[0042] A candidate box determination module is used to construct a target domain candidate box through a source domain style sample and a target domain dictionary, and to construct a source domain candidate box through a target domain style sample and a source domain dictionary;

[0043] The label weight determination module is used to determine the pseudo label and its corresponding confidence weight according to the intersection over union (IoU) between the target domain candidate box and the source domain candidate box;

[0044] A model building module is used to perform weighted self-training based on the pseudo-label, the confidence weight, the source domain sample and the target domain sample to build a target detection model.

[0045] In conjunction with the second aspect, in one implementation, the dictionary construction module is specifically used to:

[0046] Feature extraction is performed on the source domain samples and the target domain style samples respectively to determine the source domain feature matrix and the target domain feature matrix;

[0047] According to the L1 sparse constraint, dictionary learning is performed on the source domain feature matrix and the target domain feature matrix respectively to generate a source domain dictionary and a target domain dictionary.

[0048] In conjunction with the second aspect, in one implementation, the candidate box determination module is specifically used to:

[0049] Feed the source domain style sample into the region generation network to output the target domain detection box;

[0050] Adding target domain detection boxes whose confidence scores are greater than a first score threshold to the first set, and adding target domain detection boxes whose confidence scores are greater than a second score threshold to the third set as target domain candidate boxes, wherein the first score threshold is less than the second score threshold;

[0051] The target domain detection box in the first set is sent to the region of interest alignment network, the first feature matrix is ​​output, and the first feature matrix is ​​encoded into the target domain dictionary based on L2 regularization to obtain the target domain encoding coefficient vector;

[0052] A target domain residual matrix is ​​calculated according to the target domain coding coefficient vector, the first characteristic matrix and the target domain dictionary, and sensitivity is calculated through the target domain residual matrix;

[0053] The target domain detection boxes in the first set whose sensitivity is greater than the sensitivity threshold are added to the second set as target domain candidate boxes, and the union of the third set and the second set is taken as the fourth set.

[0054] In conjunction with the second aspect, in one implementation, the candidate box determination module is further configured to:

[0055] Send the target domain style sample to the region generation network to output the source domain detection box;

[0056] adding source domain detection boxes whose confidence scores are greater than a first score threshold to the fifth set, and adding source domain detection boxes whose confidence scores are greater than a second score threshold as source domain candidate boxes to the seventh set, where the first score threshold is less than the second score threshold;

[0057] The source domain detection box in the fifth set is sent to the region of interest alignment network, the second feature matrix is ​​output, and the second feature matrix is ​​encoded into the source domain dictionary based on L2 regularization to obtain the source domain encoding coefficient vector;

[0058] A source domain residual matrix is ​​calculated according to the source domain coding coefficient vector, the second characteristic matrix and the source domain dictionary, and the sensitivity is calculated through the source domain residual matrix;

[0059] The source domain detection boxes in the fifth set whose sensitivity is greater than the sensitivity threshold are added to the sixth set as source domain candidate boxes, and the union of the sixth set and the seventh set is taken as the eighth set.

[0060] In conjunction with the second aspect, in one implementation, the tag weight determination module is specifically used to:

[0061] When the prediction category corresponding to the first target domain candidate box in the third set is the same as the prediction category corresponding to the first source domain candidate box in the seventh set, calculating a first IoU between the first target domain candidate box and the first source domain candidate box;

[0062] Determine a pseudo label from the first target domain candidate box and the first source domain candidate box according to a size relationship between the first IoU and the first IoU threshold;

[0063] Calculate a second IoU between the second target domain candidate box and the second source domain candidate box in the fourth set;

[0064] When the second IoU is greater than the second IoU threshold, the second IoU is used as the confidence weight of the pseudo label;

[0065] When the second IoU is not greater than the second IoU threshold, the smallest second IoU is selected based on iterative training as the confidence weight of the pseudo label.

[0066] In conjunction with the second aspect, in one implementation, the loss function L of the weighted self-training st for:

[0067]

[0068] In the formula, represents the bounding box regression loss, represents the classification loss, I s represents the source domain sample, Represents the source domain sample I s The true label of the i-th foreground in , N represents the number of true labels of source domain samples, and I T represents the target domain sample, M represents the number of pseudo labels of the target domain sample, μ represents the parameter used to balance the loss of the source domain and the target domain, and w j represents the confidence weight of the jth pseudo label, represents the jth pseudo label.

[0069] In conjunction with the second aspect, in one implementation, the target loss function in the weighted self-training process is:

[0070] L total =βL st +L adv

[0071] Where, L total represents the target loss function, β represents the parameter used to balance the multi-task loss, and L advDenotes the adversarial loss function, which is composed of the source domain category alignment loss function and the target domain category alignment loss function.

[0072] In the third aspect, an embodiment of the present application provides a dual-dictionary based target detection model building device, which includes a processor, a memory, and a dual-dictionary based target detection model building program stored in the memory and executable by the processor, wherein when the dual-dictionary based target detection model building program is executed by the processor, the steps of the dual-dictionary based target detection model building method are implemented as described above.

[0073] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which is stored a target detection model construction program based on a dual dictionary. When the target detection model construction program based on a dual dictionary is executed by a processor, the steps of the target detection model construction method based on a dual dictionary are implemented as described above.

[0074] The beneficial effects brought by the technical solution provided in the embodiments of the present application include:

[0075] The source domain samples are converted into target domain style samples and the target domain samples are converted into source domain style samples. Then, a source domain dictionary and a target domain dictionary are constructed based on the source domain samples and the target domain style samples, respectively. That is, a dual dictionary with complementary characteristics is generated to make up for the domain offset problem between the source domain and the target domain, and then a dictionary with more information is constructed, thereby laying the foundation for obtaining more accurate pseudo labels. Then, with the assistance of the target domain dictionary and the source domain dictionary, high-confidence source domain candidate boxes and target domain candidate boxes are mined based on the source domain style samples and the target domain style samples, respectively, and pseudo labels and confidence weights are generated for unlabeled target domain samples based on the IoU between the candidate boxes. Finally, weighted self-training is performed on the target domain samples to construct a target detection model, thereby effectively improving the detection performance of the model in the target domain. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 A schematic diagram of a flow chart of an embodiment of a method for constructing a dual-dictionary target detection model in the present application;

[0077] Figure 2 A schematic diagram of the overall process of constructing the model involved in the embodiment of the present application;

[0078] Figure 3 A schematic diagram of a dual-dictionary-guided pseudo-label and weight allocation process involved in an embodiment of the present application;

[0079] Figure 4A schematic diagram of the hardware structure of a dual-dictionary based target detection model building device involved in the embodiment of the present application. DETAILED DESCRIPTION

[0080] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0081] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0082] In a first aspect, an embodiment of the present application provides a method for constructing a target detection model based on a dual dictionary.

[0083] In one embodiment, referring to Figure 1 , Figure 1 This is a flow chart of an embodiment of the method for constructing a dual-dictionary target detection model in this application. Figure 1 As shown, the method for constructing a dual-dictionary target detection model includes:

[0084] Step S10: Convert source domain samples into target domain style samples, and convert target domain samples into source domain style samples.

[0085] Exemplarily, it can be understood that although the pseudo-labels in the target domain can be assigned by extracting a dictionary for each category in the source domain and using the learned dictionary, directly constructing the dictionary in the source domain and applying it to the target domain has the following disadvantages: due to the entanglement of domain-invariant features and domain-specific features, the constructed dictionary contains a large amount of source domain-specific information; and in the feature reconstruction process, a large amount of specific domain information present in the target domain can be regarded as noise. Therefore, when the signal-to-noise ratio is low, the class sensitivity of the reconstructed residual will be reduced, so that it is easy to generate erroneous pseudo-labels in the target domain.

[0086] In order to solve the above problems, in this embodiment, a dual dictionary with complementary characteristics is constructed to assign pseudo labels. Figure 2As shown, the target domain samples are preferably converted into source domain style samples and the source domain samples are converted into target domain style samples through a CycleGan network (i.e., a cyclic generative adversarial network, which is an unsupervised generative adversarial network that mainly trains two pairs of generator-discriminator models to convert images from one domain to another), so as to build a source domain dictionary from the source domain samples and generate a target domain dictionary from the converted target domain style samples.

[0087] Step S20: constructing a source domain dictionary based on the source domain samples, and constructing a target domain dictionary based on the target domain style samples.

[0088] Exemplarily, it should be understood that the CycleGan network can retain some domain-invariant information while performing style conversion on the image samples through cycle consistency loss, that is, converting some domain-specific information, so there will be less specific source information in the converted data; however, since its conversion process is unsupervised, there are some negative conversions that may destroy domain invariance. Therefore, in order to compensate for the feature loss caused by this negative transfer, this embodiment will not only build a dictionary in the target domain, but also build a dictionary in the source domain, that is, use the target domain style samples to build a target domain dictionary in the target domain, and use the labeled training samples in the source domain to establish another complementary source domain dictionary in the source domain, so as to store the target categories to be detected through the source domain dictionary and the target domain dictionary.

[0089] Furthermore, in one embodiment, constructing a source domain dictionary based on source domain samples and constructing a target domain dictionary based on target domain style samples includes:

[0090] Feature extraction is performed on the source domain samples and the target domain style samples respectively to determine the source domain feature matrix and the target domain feature matrix;

[0091] According to the L1 sparse constraint, dictionary learning is performed on the source domain feature matrix and the target domain feature matrix respectively to generate a source domain dictionary and a target domain dictionary.

[0092] Exemplarily, in this embodiment, the source domain sample is denoted as S0 and the target domain style sample is denoted as T1; see Figure 2 As shown, it is preferred to extract various feature matrices from the source domain samples and the target domain style samples respectively through the feature extraction network; wherein the k-th feature matrix corresponding to the target domain style sample is expressed as And the k-th feature matrix corresponding to the source domain sample is expressed as It should be noted that each column in the feature matrix represents the feature vector of the kth class after L2 regularization (L2 regularization, which is a technique to prevent overfitting in machine learning and statistical modeling), R l×nRepresents an l×n dimensional real matrix, l represents the feature dimension, and n represents the total number of samples.

[0093] Dictionary learning is performed for each category k to obtain the target domain dictionary of the kth category and the source domain dictionary for the kth category m represents the dictionary D k The number of atoms in; It should be understood that, for the k-th category, this embodiment will perform dictionary learning by solving the following L1 (i.e., L1 norm, which is used to introduce sparsity in the optimization problem) sparse optimization problem;

[0094]

[0095]

[0096] in, represents the target domain sparse matrix of the kth category and represents the source domain sparse matrix of the kth category and λ1,λ2 represent regularization hyperparameters, d j express and The atoms in and each atom is normalized by the L2 norm.

[0097] After executing equation (1) and equation (2) for all categories, two matrices can be obtained and marked as as well as Generate source domain dictionary and the target domain dictionary

[0098] Step S30: construct a target domain candidate box through the source domain style sample and the target domain dictionary, and construct a source domain candidate box through the target domain style sample and the source domain dictionary.

[0099] For example, in this embodiment, the Faster-RCNN (Faster Region-based Convolutional Neural Network) model can be preferably used as the baseline detector to output the detection frame. Figure 3As shown in the figure, the Faster-RCNN model mainly includes a region proposal network (RPN) and a region of interest alignment network (ROI-Align), etc. Specifically, the source domain style sample and the target domain dictionary are sent to the region proposal network for prediction, and the feature map and its corresponding feature matrix are extracted based on the region of interest alignment network and the true value of the source domain sample, so as to obtain a set of candidate objects corresponding to the target domain through calculations such as sensitivity; the target domain style image and the source domain dictionary are sent to the region proposal network for prediction, and the feature map and its corresponding feature matrix are extracted based on the region of interest alignment network and the true value of the source domain style sample, so as to obtain a set of candidate objects corresponding to the source domain through calculations such as sensitivity.

[0100] Furthermore, in one embodiment, constructing a target domain candidate box using a source domain style sample and a target domain dictionary includes:

[0101] Feed the source domain style sample into the region generation network to output the target domain detection box;

[0102] Adding target domain detection boxes whose confidence scores are greater than a first score threshold to the first set, and adding target domain detection boxes whose confidence scores are greater than a second score threshold to the third set as target domain candidate boxes, wherein the first score threshold is less than the second score threshold;

[0103] The target domain detection box in the first set is sent to the region of interest alignment network, the first feature matrix is ​​output, and the first feature matrix is ​​encoded into the target domain dictionary based on L2 regularization to obtain the target domain encoding coefficient vector;

[0104] A target domain residual matrix is ​​calculated according to the target domain coding coefficient vector, the first characteristic matrix and the target domain dictionary, and sensitivity is calculated through the target domain residual matrix;

[0105] The target domain detection boxes in the first set whose sensitivity is greater than the sensitivity threshold are added to the second set as target domain candidate boxes, and the union of the third set and the second set is taken as the fourth set.

[0106] Exemplarily, in this embodiment, specific values ​​of the first score threshold θ1, the second score threshold θ3 and the sensitivity threshold θ2 can be determined according to actual needs and are not limited here, as long as the first score threshold θ1 is less than the second score threshold θ3.

[0107] For details, see Figure 3 As shown in Figure 1, the source domain style sample is fed into Faster-RCNN to output the target domain detection box including location information, category information, and confidence score, that is, the output of the Faster-RCNN detector is in is the i-th target domain detection box after non-maximum suppression (NMS), n is the number of detections; then determine the target domain detection box Is the confidence score of the target domain greater than the first score threshold θ1? If not, discard it; if so, Add to first collection In which m1 represents the number of elements in the first set; then the first set The target domain detection box in Send it to the ROI-Align network to obtain its feature map and extract the corresponding feature matrix x T ; Then L2 regularization is used to transform the feature matrix x T Encoded in the target domain dictionary That is, the optimized target domain coding coefficient vector is obtained by minimizing the objective function of formula (3)

[0108] In the formula, λ3 is a regularization hyperparameter, and its specific value can be determined according to actual conditions; Represents the initial target domain encoding coefficient vector, and its specific value can be determined according to actual conditions.

[0109] Next, based on the target domain encoding coefficient vector Feature matrix x T and the target domain dictionary D T Calculate the target domain residual vector for each category That is, the target domain residual matrix r is obtained by the following formula (4): T :

[0110]

[0111] Among them, (D T ) T Yes D T The transpose of

[0112] For the target domain residual matrix r T Each target domain residual vector in The residual vector in the target domain Arrange in ascending order to get the corresponding ordered After all target domain residual vectors are sorted, an ordered target domain set can be obtained. And its satisfaction Then the sensitivity is calculated according to the following formula (5):

[0113]

[0114] Among them, the sensitivity of the class Sen is used to measure the feature matrix x T The degree of matching with the k-th category, that is, the higher the sensitivity, means that the sample information has a higher category uniqueness, which further means that this sample has a high probability of belonging to the foreground of a certain category (the foreground refers to the object that needs to be detected and identified). Target area detection box The corresponding x T The calculated sensitivity Sen(x T ) and the sensitivity threshold θ2 to construct the second set Specifically, select the first set In the case of satisfying Sen(x T )>θ2’s target domain detection box is added to the second set as the target domain candidate box

[0115] At the same time, you also need to collect Target domain detection box with confidence score higher than the second score threshold θ3 Added to the third set as a target domain candidate box In which m2 represents the number of elements in the third set; finally, the second set and the third set The fourth set is the union of Right now And the fourth collection As a set of candidate boxes obtained by pseudo-label mining in the target domain.

[0116] Furthermore, in one embodiment, constructing a source domain candidate box using the target domain style sample and the source domain dictionary includes:

[0117] Send the target domain style sample to the region generation network to output the source domain detection box;

[0118] adding source domain detection boxes whose confidence scores are greater than a first score threshold to the fifth set, and adding source domain detection boxes whose confidence scores are greater than a second score threshold as source domain candidate boxes to the seventh set, where the first score threshold is less than the second score threshold;

[0119] The source domain detection box in the fifth set is sent to the region of interest alignment network, the second feature matrix is ​​output, and the second feature matrix is ​​encoded into the source domain dictionary based on L2 regularization to obtain the source domain encoding coefficient vector;

[0120] A source domain residual matrix is ​​calculated according to the source domain coding coefficient vector, the second characteristic matrix and the source domain dictionary, and the sensitivity is calculated through the source domain residual matrix;

[0121] The source domain detection boxes in the fifth set whose sensitivity is greater than the sensitivity threshold are added to the sixth set as source domain candidate boxes, and the union of the sixth set and the seventh set is taken as the eighth set.

[0122] Exemplarily, in this embodiment, specific values ​​of the first score threshold θ1, the second score threshold θ3 and the sensitivity threshold θ2 can be determined according to actual needs and are not limited here, as long as the first score threshold θ1 is less than the second score threshold θ3.

[0123] For details, see Figure 3 As shown in Figure 2, the source domain style sample is fed into Faster-RCNN to output a source domain detection box including location information, category information, and confidence score. The output of the Faster-RCNN detector is in is the i-th source domain detection box after non-maximum suppression (NMS), n is the number of detections; then determine the source domain detection box Is the confidence score of the source domain detection box greater than the first score threshold θ1? If not, discard it; if so, Added to the fifth collection In which m3 represents the number of elements in the fifth set; then the fifth set The source domain detection box in Send it to the ROI-Align network to obtain its feature map and extract the corresponding feature matrix x S ; Then L2 regularization is used to transform the feature matrix x S Encoded in the source domain dictionary That is, the optimized source domain coding coefficient vector is obtained by minimizing the objective function of formula (6)

[0124] In the formula, λ3 is a regularization hyperparameter, and its specific value can be determined according to actual conditions; Represents the initial source domain coding coefficient vector, and its specific value can be determined according to actual conditions.

[0125] Next, based on the source domain encoding coefficient vector Feature matrix x S and source domain dictionary D S Calculate the source domain residual vector for each category That is, the source domain residual matrix r is obtained by the following formula (7): S :

[0126]

[0127] Among them, (D S ) T Yes D SThe transpose of

[0128] For the source domain residual matrix r S Each source domain residual vector in The source domain residual vector Arrange in ascending order to get the corresponding ordered After all target domain residual vectors are sorted, an ordered target domain set can be obtained. And its satisfaction Then the sensitivity is calculated according to the following formula (8):

[0129]

[0130] Among them, the sensitivity of the class Sen is used to measure the feature matrix x S The degree of matching with the k-th category, that is, the higher the sensitivity, means that the sample information has a higher category uniqueness, which further means that this sample has a high probability of belonging to the foreground of a certain category (the foreground refers to the object that needs to be detected and identified). Source domain detection box The corresponding x S The calculated sensitivity Sen(x S ) and the sensitivity threshold θ2 to construct the sixth set Specifically, select the fifth set In the case of satisfying Sen(x S )>θ2's source detection box is added to the sixth set as a source candidate box

[0131] At the same time, you also need to collect Source domain detection boxes whose confidence scores are higher than the second score threshold θ3 Added to the seventh set as source domain candidate boxes In which m4 represents the number of elements in the seventh set; finally, the sixth set and the seventh set The eighth set is the union of Right now And the eighth set As a set of candidate boxes obtained by pseudo-label mining in the source domain.

[0132] Step S40: Determine a pseudo label and its corresponding confidence weight according to the intersection over union (IoU) between the target domain candidate box and the source domain candidate box.

[0133] For example, it can be understood that the feature distribution between the source domain sample and the source domain style sample obtained by converting the target domain sample is different, so the positions of the candidate boxes generated by the two dictionary mining processes are also different; and since the foreground contains domain-invariant features, the higher the overlap of the candidate boxes generated by the two dictionaries, the greater the probability that the foreground exists in the area, indicating that the predicted box is more reliable. Therefore, this embodiment will be based on the third set The target domain candidate box and the seventh set in The IoU values ​​between the source domain candidate boxes in the source domain are used to assign pseudo labels, and based on the fourth set The target domain candidate box and the eighth set in The pseudo-label confidence weights are generated by cross-joining the IoU values ​​between the source domain candidate boxes.

[0134] Furthermore, in one embodiment, determining the pseudo label and its corresponding confidence weight according to the intersection over union (IoU) ratio between the target domain candidate box and the source domain candidate box includes:

[0135] When the prediction category corresponding to the first target domain candidate box in the third set is the same as the prediction category corresponding to the first source domain candidate box in the seventh set, calculating a first IoU between the first target domain candidate box and the first source domain candidate box;

[0136] Determine a pseudo label from the first target domain candidate box and the first source domain candidate box according to a size relationship between the first IoU and the first IoU threshold;

[0137] Calculate a second IoU between the second target domain candidate box and the second source domain candidate box in the fourth set;

[0138] When the second IoU is greater than the second IoU threshold, the second IoU is used as the confidence weight of the pseudo label;

[0139] When the second IoU is not greater than the second IoU threshold, the smallest second IoU is selected based on iterative training as the confidence weight of the pseudo label.

[0140] For example, it should be noted that the specific values ​​of the first IoU threshold δ1 and the second IoU threshold δ2 can be determined according to actual needs and are not limited here. The target domain candidate box b in T With the seventh collection The source domain candidate box b in S When the predicted categories are consistent, the IoU value of the predicted object location information is calculated using the following formula:

[0141]

[0142] Among them, Intersection area Represents two candidate boxes b T and b S The area of ​​the bounding box overlap, Union area Represents two candidate boxes b T and b S Total area covered.

[0143] If IoU(b T ,b S )>δ1, indicating that the two candidate boxes are the results of detecting the same target, then one of the candidate boxes can be used as the pseudo label of the target; and if IoU(b T ,b S )≤δ1, indicating that the two candidate boxes are detection results of different targets, and both candidate boxes are regarded as pseudo labels.

[0144] It should be understood that IoU cannot measure the reliability of pseudo-labels. However, when the similarity between the two pseudo-label expressions is high, it means that the foreground part of the target box corresponding to the detection result is larger, that is, it is mainly expressed by domain-invariant features; in other words, the detection box accurately frames the target area, so the background part is smaller, so a higher confidence weight can be assigned to this type of detection result. On the contrary, when the similarity between the two pseudo-labels is low, it means that the background part of the target box corresponding to the detection result is larger, that is, there are a large number of domain-specific feature expressions. It is understandable that due to the existence of the domain transformation gap, the consistency of the pseudo-label assignment of the two dictionaries to the detection result is affected; therefore, a lower confidence weight should be assigned to this type of detection result for training the domain adaptive detection model, thereby reducing the impact of this part of the pseudo-label samples on the accuracy of the detection box.

[0145] Specifically, in this embodiment, the fourth set The target domain candidate box b in T and the eighth set The source domain candidate box b in S The IoU value between generates the confidence weight of each pseudo label:

[0146]

[0147] It should be noted that when IoU(b T ,b S )>δ2, then the IoU(b T ,b S ) as the confidence weight of the pseudo label; and if IoU(b T ,b S )≤δ2, then the IoU(bT ,b S )The minimum value is taken as the confidence weight of the pseudo label. It can be understood that the confidence weight reflects the quality of the pseudo label, that is, the candidate box with higher quality is assigned a larger weight, and vice versa.

[0148] Step S50: Perform weighted self-training based on the pseudo-label, the confidence weight, the source domain sample and the target domain sample to construct a target detection model. The loss function L of the weighted self-training is st for:

[0149]

[0150] In the formula, represents the bounding box regression loss, represents the classification loss, I s represents the source domain sample, Represents the source domain sample I s The true label of the i-th foreground in , N represents the number of true labels of source domain samples, and I T represents the target domain sample, M represents the number of pseudo labels of the target domain sample, μ represents the parameter used to balance the loss of the source domain and the target domain, and w j represents the confidence weight of the jth pseudo label, represents the jth pseudo label.

[0151] The objective loss function in the weighted self-training process is:

[0152] L total =βL st +L adv

[0153] Where, L total represents the target loss function, β represents the parameter used to balance the multi-task loss, and L adv Denotes the adversarial loss function, which is composed of the source domain category alignment loss function and the target domain category alignment loss function.

[0154] Exemplarily, in this embodiment, a category-aware alignment process and a weighted self-training process are performed based on pseudo labels, confidence weights, source domain samples, and target domain samples, and network parameters are updated through weighted self-training loss and category alignment loss to finally generate an object detection model. Figure 2 As shown in the figure, the loss function L of the weighted self-training used in the weighted self-training process based on IoU is st for:

[0155]

[0156] In the formula, represents the bounding box regression loss, represents the classification loss, I s represents the source domain sample, Represents the source domain sample I s The true label of the i-th foreground in , N represents the number of true labels of source domain samples, and I T represents the target domain sample, M represents the number of pseudo labels of the target domain sample, μ represents the parameter used to balance the loss of the source domain and the target domain, and its specific value can be determined according to actual needs, w j represents the confidence weight of the jth pseudo label, represents the jth pseudo label.

[0157] In this embodiment, K category-aware discriminators are used to adjust the features of a specific category when performing alignment based on category features, and a loss function Focal Loss is used to encourage the domain classifier to focus on difficult-to-classify instances when performing alignment. It should be noted that the working principle and method of the category-aware discriminator are common knowledge in this field, and for the sake of brevity, they are not described here. Among them, the loss function of the k-th category-guided alignment includes the source domain category alignment loss function And the target domain category alignment loss function and and can be written as:

[0158]

[0159]

[0160] in, It represents the feature map of the k-th category in the source domain after ROI alignment, which can be obtained from the true label of the source domain; It represents the feature map of the k-th category in the target domain after ROI alignment, which can be obtained from the pseudo label of the target domain; and Represents the number of k-th category foreground in the input source domain samples and target domain samples respectively; represents the discrimination result of the k-th category-aware discriminator on the input image; γ is a hyperparameter used to control the weight of instances that are difficult to classify. It should be noted that when the present embodiment processes the feature matrices of different categories respectively through multi-channel data processing, a gradient reversal layer is also used to introduce adversarial training in feature representation learning, and before the gradient is propagated back to the feature extraction network, the gradient is reversed before outputting the dictionary D corresponding to each category. k , that is, output the source domain dictionary and target domain dictionary for each category, then the adversarial loss can be summarized as:

[0161]

[0162] Where, L adv represents the adversarial loss function.

[0163] Combining all the above losses to obtain the overall target loss function, it can be expressed as:

[0164] L total =βL st +L adv (15)

[0165] Among them, β represents the parameter used to balance the multi-task loss, and its specific value can be determined according to actual needs.

[0166] In summary, this embodiment aims to solve the problem of poor model transfer effect caused by significant style differences between domains in unsupervised domain adaptive target detection, and provides a target detection framework based on a dual-domain dictionary, that is, learning two dictionaries of specific category representations in the source domain and the target domain respectively, and using the learned dictionaries to reconstruct candidate boxes in the source domain and the target domain respectively, to obtain two groups of candidate boxes with complementary characteristics; then assign pseudo labels to the candidate boxes according to the IoU between the two groups of candidate boxes, and finally assign confidence weights to the pseudo labels based on the IoU, and perform weighted self-training on the target domain samples. It can be seen that this embodiment compensates for the domain offset problem between the source domain and the target domain by constructing a dual-domain dictionary, and generates pseudo labels and confidence weights for unlabeled target domain samples in a complementary manner, so that the detection performance of the model in the target domain is effectively improved, and the generalization ability of the model can be enhanced.

[0167] In a second aspect, an embodiment of the present application also provides a device for constructing a target detection model based on a dual dictionary.

[0168] In one embodiment, a dual-dictionary based target detection model construction device includes:

[0169] A style conversion module, which is used to convert source domain samples into target domain style samples, and convert target domain samples into source domain style samples;

[0170] A dictionary construction module, which is used to construct a source domain dictionary based on source domain samples, and to construct a target domain dictionary based on target domain style samples;

[0171] A candidate box determination module is used to construct a target domain candidate box through a source domain style sample and a target domain dictionary, and to construct a source domain candidate box through a target domain style sample and a source domain dictionary;

[0172] The label weight determination module is used to determine the pseudo label and its corresponding confidence weight according to the intersection over union (IoU) between the target domain candidate box and the source domain candidate box;

[0173] A model building module is used to perform weighted self-training based on the pseudo-label, the confidence weight, the source domain sample and the target domain sample to build a target detection model.

[0174] Furthermore, in one embodiment, the dictionary construction module is specifically used for:

[0175] Feature extraction is performed on the source domain samples and the target domain style samples respectively to determine the source domain feature matrix and the target domain feature matrix;

[0176] According to the L1 sparse constraint, dictionary learning is performed on the source domain feature matrix and the target domain feature matrix respectively to generate a source domain dictionary and a target domain dictionary.

[0177] Furthermore, in one embodiment, the candidate box determination module is specifically used to:

[0178] Feed the source domain style sample into the region generation network to output the target domain detection box;

[0179] Adding target domain detection boxes whose confidence scores are greater than a first score threshold to the first set, and adding target domain detection boxes whose confidence scores are greater than a second score threshold to the third set as target domain candidate boxes, wherein the first score threshold is less than the second score threshold;

[0180] The target domain detection box in the first set is sent to the region of interest alignment network, the first feature matrix is ​​output, and the first feature matrix is ​​encoded into the target domain dictionary based on L2 regularization to obtain the target domain encoding coefficient vector;

[0181] A target domain residual matrix is ​​calculated according to the target domain coding coefficient vector, the first characteristic matrix and the target domain dictionary, and sensitivity is calculated through the target domain residual matrix;

[0182] The target domain detection boxes in the first set whose sensitivity is greater than the sensitivity threshold are added to the second set as target domain candidate boxes, and the union of the third set and the second set is taken as the fourth set.

[0183] Furthermore, in one embodiment, the candidate box determination module is further configured to:

[0184] Send the target domain style sample to the region generation network to output the source domain detection box;

[0185] adding source domain detection boxes whose confidence scores are greater than a first score threshold to the fifth set, and adding source domain detection boxes whose confidence scores are greater than a second score threshold as source domain candidate boxes to the seventh set, where the first score threshold is less than the second score threshold;

[0186] The source domain detection box in the fifth set is sent to the region of interest alignment network, the second feature matrix is ​​output, and the second feature matrix is ​​encoded into the source domain dictionary based on L2 regularization to obtain the source domain encoding coefficient vector;

[0187] A source domain residual matrix is ​​calculated according to the source domain coding coefficient vector, the second characteristic matrix and the source domain dictionary, and the sensitivity is calculated through the source domain residual matrix;

[0188] The source domain detection boxes in the fifth set whose sensitivity is greater than the sensitivity threshold are added to the sixth set as source domain candidate boxes, and the union of the sixth set and the seventh set is taken as the eighth set.

[0189] Furthermore, in one embodiment, the tag weight determination module is specifically used to:

[0190] When the prediction category corresponding to the first target domain candidate box in the third set is the same as the prediction category corresponding to the first source domain candidate box in the seventh set, calculating a first IoU between the first target domain candidate box and the first source domain candidate box;

[0191] Determine a pseudo label from the first target domain candidate box and the first source domain candidate box according to a size relationship between the first IoU and the first IoU threshold;

[0192] Calculate a second IoU between the second target domain candidate box and the second source domain candidate box in the fourth set;

[0193] When the second IoU is greater than the second IoU threshold, the second IoU is used as the confidence weight of the pseudo label;

[0194] When the second IoU is not greater than the second IoU threshold, the smallest second IoU is selected based on iterative training as the confidence weight of the pseudo label.

[0195] Furthermore, in one embodiment, the loss function L of the weighted self-training st for:

[0196]

[0197] In the formula, represents the bounding box regression loss, represents the classification loss, I s represents the source domain sample, Represents the source domain sample I s The true label of the i-th foreground in , N represents the number of true labels of source domain samples, and I T represents the target domain sample, M represents the number of pseudo labels of the target domain sample, μ represents the parameter used to balance the loss of the source domain and the target domain, and w j represents the confidence weight of the jth pseudo label, represents the jth pseudo label.

[0198] Furthermore, in one embodiment, the target loss function in the weighted self-training process is:

[0199] L total =βL st +L adv

[0200] Where, L total represents the target loss function, β represents the parameter used to balance the multi-task loss, and L adv Denotes the adversarial loss function, which is composed of the source domain category alignment loss function and the target domain category alignment loss function.

[0201] Among them, the functional implementation of each module in the above-mentioned dual-dictionary based target detection model construction device corresponds to the various steps in the above-mentioned dual-dictionary based target detection model construction method embodiment, and its functions and implementation processes will not be repeated here one by one.

[0202] In a third aspect, an embodiment of the present application provides a dual-dictionary-based target detection model construction device, which may be a personal computer (PC), a laptop computer, a server, or other device with sample processing capabilities.

[0203] Reference Figure 4 , Figure 4 The hardware structure diagram of the dual-dictionary-based target detection model construction device involved in the embodiment of the present application is shown in FIG. In the embodiment of the present application, the dual-dictionary-based target detection model construction device may include a processor, a memory, a communication interface, and a communication bus.

[0204] The communication bus may be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0205] The communication interface includes an input / output (I / O) interface, a physical interface, and a logical interface, etc., which are used to interconnect devices within the dual-dictionary target detection model building device, and an interface for interconnecting the dual-dictionary target detection model building device with other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, an optical fiber interface, an ATM interface, etc.; the user device can be a display, a keyboard, etc.

[0206] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0207] The processor may be a general-purpose processor, and the general-purpose processor may call the dual-dictionary-based target detection model construction program stored in the memory, and execute the dual-dictionary-based target detection model construction method provided in the embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the dual-dictionary-based target detection model construction program is called may refer to the various embodiments of the dual-dictionary-based target detection model construction method of the present application, which will not be repeated here.

[0208] Those skilled in the art will understand that Figure 4 The hardware structure shown in the figure does not constitute a limitation on the present application, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0209] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.

[0210] The readable storage medium of the present application stores a target detection model construction program based on a dual dictionary, wherein when the target detection model construction program based on a dual dictionary is executed by a processor, the steps of the target detection model construction method based on a dual dictionary as described above are implemented.

[0211] Among them, the method implemented when the dual-dictionary based target detection model construction program is executed can refer to the various embodiments of the dual-dictionary based target detection model construction method of the present application, and will not be repeated here.

[0212] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0213] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0214] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0215] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or. For example, A / B can mean A or B. The “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0216] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0217] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0218] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for constructing a target detection model based on dual dictionaries. It is characterized in that The following steps are involved: Convert source domain samples into target domain style samples, and convert target domain samples into source domain style samples; A source domain dictionary is constructed based on source domain samples, and a target domain dictionary is constructed based on target domain style samples; Construct target domain candidate boxes through source domain style samples and target domain dictionary, and construct source domain candidate boxes through target domain style samples and source domain dictionary; Determine the pseudo label and its corresponding confidence weight based on the intersection over union (IoU) between the target domain candidate box and the source domain candidate box; Weighted self-training is performed based on the pseudo labels, the confidence weights, the source domain samples, and the target domain samples to construct a target detection model.

2. The method for constructing a dual-dictionary target detection model according to claim 1, It is characterized in that The step of constructing a source domain dictionary based on source domain samples and constructing a target domain dictionary based on target domain style samples includes: Feature extraction is performed on the source domain samples and the target domain style samples respectively to determine the source domain feature matrix and the target domain feature matrix; According to the L1 sparse constraint, dictionary learning is performed on the source domain feature matrix and the target domain feature matrix respectively to generate a source domain dictionary and a target domain dictionary.

3. The method for constructing a dual-dictionary target detection model according to claim 1, It is characterized in that The target domain candidate box is constructed by using the source domain style sample and the target domain dictionary, including: Feed the source domain style sample into the region generation network to output the target domain detection box; Adding target domain detection boxes whose confidence scores are greater than a first score threshold to the first set, and adding target domain detection boxes whose confidence scores are greater than a second score threshold to the third set as target domain candidate boxes, wherein the first score threshold is less than the second score threshold; The target domain detection box in the first set is sent to the region of interest alignment network, the first feature matrix is ​​output, and the first feature matrix is ​​encoded into the target domain dictionary based on L2 regularization to obtain the target domain encoding coefficient vector; A target domain residual matrix is ​​calculated according to the target domain coding coefficient vector, the first characteristic matrix and the target domain dictionary, and sensitivity is calculated through the target domain residual matrix; The target domain detection boxes in the first set whose sensitivity is greater than the sensitivity threshold are added to the second set as target domain candidate boxes, and the union of the third set and the second set is taken as the fourth set.

4. The method for constructing a dual-dictionary target detection model according to claim 3, It is characterized in that The source domain candidate box is constructed by using the target domain style sample and the source domain dictionary, including: Send the target domain style sample to the region generation network to output the source domain detection box; adding source domain detection boxes whose confidence scores are greater than a first score threshold to the fifth set, and adding source domain detection boxes whose confidence scores are greater than a second score threshold as source domain candidate boxes to the seventh set, where the first score threshold is less than the second score threshold; The source domain detection box in the fifth set is sent to the region of interest alignment network, the second feature matrix is ​​output, and the second feature matrix is ​​encoded into the source domain dictionary based on L2 regularization to obtain the source domain encoding coefficient vector; A source domain residual matrix is ​​calculated according to the source domain coding coefficient vector, the second characteristic matrix and the source domain dictionary, and the sensitivity is calculated through the source domain residual matrix; The source domain detection boxes in the fifth set whose sensitivity is greater than the sensitivity threshold are added to the sixth set as source domain candidate boxes, and the union of the sixth set and the seventh set is taken as the eighth set.

5. The method for constructing a dual-dictionary target detection model according to claim 4, It is characterized in that The method of determining a pseudo label and its corresponding confidence weight according to the intersection over union (IoU) ratio between the target domain candidate box and the source domain candidate box includes: When the prediction category corresponding to the first target domain candidate box in the third set is the same as the prediction category corresponding to the first source domain candidate box in the seventh set, calculating a first IoU between the first target domain candidate box and the first source domain candidate box; Determine a pseudo label from the first target domain candidate box and the first source domain candidate box according to a size relationship between the first IoU and the first IoU threshold; Calculate a second IoU between the second target domain candidate box and the second source domain candidate box in the fourth set; When the second IoU is greater than the second IoU threshold, the second IoU is used as the confidence weight of the pseudo label; When the second IoU is not greater than the second IoU threshold, the smallest second IoU is selected based on iterative training as the confidence weight of the pseudo label.

6. The method for constructing a dual-dictionary target detection model according to claim 1, It is characterized in that The loss function L of the weighted self-training st for: In the formula, represents the bounding box regression loss, represents the classification loss, I s represents the source domain sample, Represents the source domain sample I s The true label of the i-th foreground in , N represents the number of true labels of source domain samples, and I T represents the target domain sample, M represents the number of pseudo labels of the target domain sample, μ represents the parameter used to balance the loss of the source domain and the target domain, and w j represents the confidence weight of the jth pseudo label, represents the jth pseudo label.

7. The method for constructing a dual-dictionary target detection model according to claim 6, It is characterized in that The objective loss function in the weighted self-training process is: L total =βL st +L adv Where, L total represents the target loss function, β represents the parameter used to balance the multi-task loss, and L adv Denotes the adversarial loss function, which is composed of the source domain category alignment loss function and the target domain category alignment loss function.

8. A device for constructing a target detection model based on a dual dictionary. It is characterized in that include: A style conversion module, which is used to convert source domain samples into target domain style samples, and convert target domain samples into source domain style samples; A dictionary construction module, which is used to construct a source domain dictionary based on source domain samples, and to construct a target domain dictionary based on target domain style samples; A candidate box determination module is used to construct a target domain candidate box through a source domain style sample and a target domain dictionary, and to construct a source domain candidate box through a target domain style sample and a source domain dictionary; The label weight determination module is used to determine the pseudo label and its corresponding confidence weight according to the intersection over union (IoU) between the target domain candidate box and the source domain candidate box; A model building module is used to perform weighted self-training based on the pseudo-label, the confidence weight, the source domain sample and the target domain sample to build a target detection model.

9. A device for building a target detection model based on a dual dictionary, It is characterized in that The dual-dictionary based target detection model building device includes a processor, a memory, and a dual-dictionary based target detection model building program stored in the memory and executable by the processor, wherein when the dual-dictionary based target detection model building program is executed by the processor, the steps of the dual-dictionary based target detection model building method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a dual-dictionary-based target detection model construction program, wherein when the dual-dictionary-based target detection model construction program is executed by a processor, the steps of the dual-dictionary-based target detection model construction method according to any one of claims 1 to 7 are implemented.

Images