Cross-dataset target detection method

Abstract

The invention provides a cross-dataset target detection method. The method comprises the following steps: step 1, determining a task demand and detecting a baseline network; step 2, constructing a unified label space; constructing a uniform label space, wherein the uniform label space comprises label categories of all to-be-detected data sets; constructing a unified label space through a semantic relationship of the labels or an experience database; step 3, constructing a loss function associated with the data category; step 4, constructing a unified index function; and 5, carrying out model training. According to the method, only one uniform detection head is finally arranged, the structure of the detection head is the same as that of a single-data-set detection model, detection types are basically increased under the condition that consumed time is not increased, negative sample use between different data sets is fully considered at the same time, and the detection precision can be better improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a cross-dataset target detection method. Background Art

[0002] With the development of deep neural network model structures and the introduction of various scale data sets, the detection accuracy and efficiency of target detection tasks have improved rapidly. However, such standard target detectors usually need to be trained on training data sets with fixed categories that have been annotated in advance. These data sets are constructed for different conditions and purposes and may not meet the usage scenarios. In practical applications, it is not possible to know all possible detection categories in advance, and new object classes often need to be added gradually. Data labeling is a tedious task, so using existing standard data sets to improve the performance of existing tasks has become one of the goals of current target detection tasks.

[0003] There are two main approaches to achieve target detection of new and old objects. One solution is to re-label and transform the problem into a standard object detection task. However, with massive data and an increasing number of categories of interest, it is quite tedious to annotate new category instances in all images of the original dataset.

[0004] The other is cross-dataset target detection. This method does not require re-labeling and can be trained in multiple data sets at the same time to achieve multi-dataset multi-category target detection. At present, there are also certain research results, such as patent CN113947148 A: "A cross-dataset target detection joint training method", but this patent only considers the detection of new categories, and does not consider the use of existing data sets to improve the detection effect; and only considers the positive sample category of the new data, and does not consider the impact of negative samples in different data sets on the training process; for example, patent CN 117079074 A: "A regression separation type multi-detection head cross-dataset target detection method", only considers the positive sample category of different data sets, and does not consider the impact of negative samples in different data sets on the training process; and the multi-detection head model requires multiple post-processing operations, which increases the time consumption compared to the single-head detection model. Summary of the invention

[0005] This application provides a cross-dataset target detection method, which can be used to solve the technical problem that a single-head detection model is extremely time-consuming.

[0006] The present application provides a cross-dataset target detection method, the method comprising:

[0007] Step 1: Determine the task requirements and test the baseline network;

[0008] Step 2: Construct a unified label space. Construct a unified label space that includes the label categories of all the data sets to be tested. Construct a unified label space through the semantic relationship of labels or the experience database.

[0009] Step 3: Construct a loss function associated with the data category;

[0010] Step 4: Construct a unified indicator function;

[0011] Step 5: Perform model training.

[0012] Furthermore, in step 1, the task requirements and the detection baseline network are determined, including:

[0013] Determining the task requirements includes determining the existing data sets and their respective annotation categories, and clarifying the final detection category requirements;

[0014] Determine the baseline network, that is, use the target detection network model suitable for a single dataset

[0015] Furthermore, in step 2, a unified label space is constructed, including:

[0016] Step 21, first merge the labels with the same semantics between the data sets; the labels with the same semantics actually refer to the same category of things, not just whether the labels are consistent;

[0017] Step 22, construct a unified data label space and retain the original dataset information of each image, that is, which dataset it comes from;

[0018] Step 23, constructing conflict information of positive and negative samples;

[0019] For the original label category, there will be only one category corresponding to the original label category in the unified label space, and the label category in the unified label space can find corresponding labels in multiple data sets or a single data set.

[0020] Furthermore, the positive and negative sample conflict information is defined as follows: assuming that there are only labels of the first category in the first data set, the labels of the first category are positive samples, and the labels of the second category corresponding to the labels of the first category are negative samples; assuming that there are only labels of the second category in the second data set, the labels of the second category are positive samples, and the labels of the first category are negative samples; the positive and negative samples of the first data set and the second data set are conflicting.

[0021] Furthermore, in step three, a loss function associated with the data category is constructed, including:

[0022] The loss function of target detection includes the detection box regression loss and the detection category loss, namely:

[0023]

[0024] in is the total loss function, represents the regression loss, represents the detection category loss, and represents the coefficient; Improvements have been made based on the focal loss function FocalLoss related to the label category. The formula is as follows:

[0025]

[0026] in,

[0027] In the formula, is the true value of the category, is the estimated probability of the prediction box, and is a constant set based on experience, ,N is the number of predicted target boxes of the model; Represents the category prediction probability of the tth box; when the negative samples come from different data sets, will make , thereby avoiding the conflict of negative samples in the dataset.

[0028] Furthermore, in step 4, a unified indicator function is constructed, including:

[0029] Use label mask to unify the evaluation function; the label mask formula is as follows:

[0030]

[0031]

[0032] Where label is the real label of the image; pred is the output value predicted by the model, which is an n-dimensional vector; mask is the label mask vector, which has the same dimension as pred; Represents the jth element in the model prediction output value; Represents a dataset; Indicates that there is a corresponding label in the data set; mAP is a common indicator function in target detection. is a unified indicator function; when the true value corresponding to the j-th predicted value is not labeled, mask(j)=0, indicating that this predicted value does not participate in the calculation of the relevant evaluation function value, and only the category with mask(j)=1 in the model is calculated.

[0033] Furthermore, in step five, model training is performed, including:

[0034] Different data sets will be iteratively trained through a unified backbone network and a unified detection head until the expected model is obtained.

[0035] This application uses a cross-dataset target detection method to train a unified detection model on multiple target detection datasets. This method mainly includes constructing a unified label space, a loss function associated with the data category, and a unified evaluation function, thereby realizing a cross-dataset target detection network structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flow chart of target detection across data sets provided in an embodiment of the present application;

[0037] Figure 2 A schematic diagram of the construction result of the unified label space provided in the embodiment of the present application;

[0038] Figure 3 A schematic diagram of negative sample loss related to the embodiment of the present application;

[0039] Figure 4 A network framework diagram provided for an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0041] The following first introduces the embodiments of the present application in conjunction with the accompanying drawings.

[0042] Step 1: Determine the task requirements and test baseline network.

[0043] Determining the task requirements includes determining the existing data sets and their respective annotation categories, and clarifying the final detection category requirements;

[0044] Determining the baseline network means adopting a target detection network model suitable for a single dataset. The cross-dataset target detection method proposed in this paper is to make appropriate modifications to it so that it can undertake cross-dataset target detection tasks; the baseline network can be any one of the YOLO, RetinaNet, and R-CNN target detection models.

[0045] Step 2: Build a unified label space.

[0046] It is necessary to construct a unified label space for the dataset to be tested. The label space refers to the set of labels annotated on the dataset, that is, a set of detection target categories. The label spaces of different datasets are generally different. The purpose of this invention is to construct a unified label space that can contain the label categories of all datasets to be tested; to construct a unified label space through the semantic relationship of labels or empirical databases;

[0047] In view of the overlap of labels between different data sets and the possible conflict between positive and negative samples, the present invention proposes the following unified label space construction method, the steps are as follows:

[0048] Step 21, first merge the labels with the same semantics between datasets; semantically the same means that the labels actually refer to the same category of things, not just whether the labels are consistent; for example, the labels human and pedestrian, pedestrian specifically refers to people walking, and the semantics of these two labels are different, such as vehicle and car, although the names are different, they refer to vehicles in some datasets, so the semantics are the same.

[0049] Step 22, construct a unified data label space and retain the original data set information of each image, that is, which data set it comes from;

[0050] Step 23, construct the conflict information of positive and negative samples for subsequent loss function calculation.

[0051] The positive and negative sample conflict information is defined as follows: assuming that there are only labels of the first category in the first data set, the labels of the first category are positive samples, and the labels of the second category corresponding to the labels of the first category are negative samples; if there are only labels of the second category in the second data set, the labels of the second category are positive samples, and the labels of the first category are negative samples; the positive and negative samples of the first data set and the second data set are conflicting.

[0052] For example, in dataset 1, only the human head is labeled as a positive sample, so the human head is a positive sample and the human body is a negative sample. In dataset 2, only the human body is labeled as a positive sample, so the human body is a positive sample and the human head is a negative sample. The positive and negative samples between the two datasets are conflicting. For the fusion of similar datasets, the conflict information of positive and negative samples should be retained.

[0053] After the above steps, for the original label category, there will be only one category in the unified label space that corresponds to the original label category. The label category in the unified label space may find corresponding labels in multiple datasets or only in a single dataset. This is because the label categories in the original dataset may overlap. The construction results of the unified label space can be referred to Figure 2 As shown in the figure, there are two original data sets on the left, with labels l1, l2, l3, l4, m1, m2, m3 respectively, where l3 and m2 represent labels with the same category or semantics (for example, both are window categories). They will be merged into n3 in the unified label space, and the original data set labels will be mapped to n1...n6, where the conflict information of positive and negative samples is not shown in this figure.

[0054] Step three: construct a loss function associated with the data category.

[0055] The loss function of target detection includes the detection box regression loss and the detection category loss, namely:

[0056]

[0057] in is the total loss function, represents the regression loss, represents the detection category loss, and represents the coefficient; Only focus on the prediction of positive samples and use the original calculation method. However, in the category loss function, attention will be paid to negative samples, so the original calculation method cannot be used.

[0058] In order to solve the problem that the original category loss function cannot be used due to incomplete image category labels, and to better solve the problem of imbalanced positive and negative samples and imbalanced number of difficult and easy samples in target detection, this network does not directly use the CrossEntropyLoss loss function commonly used in target detection. Instead, it makes improvements based on the focal loss function FocalLoss related to the label category. The formula is as follows:

[0059]

[0060] in,

[0061] In the formula, is the true value of the category, is the estimated probability of the prediction box, and is a constant set based on experience, , N is the number of predicted target boxes of the model; this ensures that negative samples from the same dataset only affect the loss function of this dataset and datasets that do not cause conflicts. When negative samples come from different datasets, will make , thereby avoiding the conflict of negative samples in the data set; the role of negative samples in the loss function is as follows Figure 4 As shown, the dotted line indicates no correlation, thereby effectively utilizing the negative category labels in the original data set and effectively improving the model detection accuracy.

[0062] Step 4: Construct a unified indicator function.

[0063] For the existing multi-detection head cross-dataset target detection method, since a single detection head only corresponds to a single data set, the loss function is also consistent with the single data set target detection calculation method, but it also results in multiple calculations of the loss function and then summing them up to get the total loss function. The method proposed in this article allows the loss function to be calculated at one time, and the process is more concise and clear. For the method described in this article, the validation set also performs the label merging operation as shown in step 2. Figure 2 For example, there are only l1, l2, l3, l4 labels in the original dataset 1, which are mapped to the unified label space of n1, n2, n3, n4. However, this network will make predictions for all categories on the validation set, and will make predictions for all label categories of n1...n6 on the validation set of dataset 1, but the labels of n5 and n6 do not exist in the original dataset, which will cause problems in the calculation of the indicator function.

[0064] This paper proposes a unified evaluation function with the help of label mask; the label mask formula is as follows:

[0065]

[0066]

[0067] Where label is the real label of the image; pred is the output value predicted by the model, which is an n-dimensional vector; mask is the label mask vector, which has the same dimension as pred; Represents the jth element in the model prediction output value; mAP is a common indicator function in target detection. It is a unified indicator function; when the true value corresponding to the j-th predicted value is not labeled, mask(j)=0, indicating that this predicted value does not participate in the calculation of the relevant evaluation function value, and only the category with mask(j)=1 in the model is calculated, thereby realizing the unified calculation of the evaluation function.

[0068] Step 5: Model training

[0069] In this paper, Yolov8 is selected as the baseline model. The improved network framework structure diagram after the above steps is as follows Figure 4 As shown in the figure, different data sets will be iteratively trained through a unified backbone network and a unified detection head until the expected model is obtained, ultimately achieving the target detection task across data sets.

[0070] The network design in this paper takes into account the category overlap between different data sets, and can handle the problem of newly added detection categories and the reuse of existing public data sets very well. In the end, the network has only one unified detection head, which is the same as the single data set detection model structure. It basically realizes the addition of detection categories without increasing time. This network also fully considers the use of negative samples between different data sets, which can better improve the detection accuracy.

[0071] The above-described embodiments of the present application do not constitute a limitation on the protection scope of the present application.

Claims

1. A cross-dataset target detection method, characterized in that: The method comprises: Step 1: Determine the task requirements and test the baseline network; Step 2: Build a unified label space. Build a unified label space that includes the label categories of all the data sets to be tested. Build a unified label space through the semantic relationship of labels or the experience database. Step 3: Construct a loss function associated with the data category; Step 4: Construct a unified indicator function; Step 5: Perform model training.

2. The method according to claim 1, characterized in that: Step 1: Determine the task requirements and test baseline network, including: Determining the task requirements includes determining the existing data sets and their respective annotation categories, and clarifying the final detection category requirements; Determine the baseline network, that is, use a target detection network model suitable for a single dataset.

3. The method according to claim 1, characterized in that Step 2: Build a unified label space, including: Step 21, first merge the labels with the same semantics between the data sets; the labels with the same semantics actually refer to the same category of things, not just whether the labels are consistent; Step 22, construct a unified data label space and retain the original dataset information of each image, that is, which dataset it comes from; Step 23, constructing conflict information of positive and negative samples; For the original label category, there will be only one category corresponding to the original label category in the unified label space, and the label category in the unified label space can find corresponding labels in multiple data sets or only in a single data set.

4. The method according to claim 3, characterized in that: The positive and negative sample conflict information is defined as follows: assuming that there are only labels of the first category in the first data set, the labels of the first category are positive samples, and the labels of the second category corresponding to the labels of the first category are negative samples; if there are only labels of the second category in the second data set, the labels of the second category are positive samples, and the labels of the first category are negative samples; the positive and negative samples of the first data set and the second data set are conflicting.

5. The method according to claim 1, characterized in that Step 3: Construct a loss function associated with the data category, including: The loss function of target detection includes the detection box regression loss and the detection category loss, namely: ； in is the total loss function, represents the regression loss, represents the detection category loss, and represents the coefficient; Improvements have been made based on the focal loss function FocalLoss related to the label category. The formula is as follows: ； in, ; In the formula, is the true value of the category, is the estimated probability of the prediction box, and is a constant set based on experience, ,N is the number of predicted target boxes of the model; Represents the category prediction probability of the tth box; when the negative samples come from different data sets, will make , thereby avoiding the conflict of negative samples in the dataset.

6. The method according to claim 1, characterized in that Step 4: Construct a unified indicator function, including: Use label mask to unify the evaluation function; the label mask formula is as follows: ； ； Where label is the real label of the image; pred is the output value predicted by the model, which is an n-dimensional vector; mask is the label mask vector, which has the same dimension as pred; Represents the jth element in the model prediction output value; Represents a dataset; Indicates that there is a corresponding label in the data set; mAP is a common indicator function in target detection. is a unified indicator function; when the true value corresponding to the j-th predicted value is not labeled, mask(j)=0, indicating that this predicted value does not participate in the calculation of the relevant evaluation function value, and only the category with mask(j)=1 in the model is calculated.

7. The method according to claim 1, characterized in that Step 5: Model training, including: Different data sets will be iteratively trained through a unified backbone network and a unified detection head until the expected model is obtained.

Images