An Independence-Guided Sparse Labeled Object Detection Method

By employing image independence decomposition and joint confidence to filter pseudo-labels, the problems of branch correlation and pseudo-label bias in sparsely labeled datasets are solved, thereby improving the training effect and detection accuracy of sparsely labeled object detection models.

CN118115727BActive Publication Date: 2026-11-17CHONGQING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410348632.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-26
Publication Date
2026-11-17
Estimated Expiration
2044-03-26

AI Technical Summary

Technical Problem

The correlation between different branches and pseudo-label bias caused by sparse labeled datasets in object detection affect the training effect and detection accuracy of the model.

Method used

The sparsely labeled image is decomposed into multiple independent component maps using the image independence decomposition method. Multiple branch networks are used for training, and pseudo-labels are filtered by joint confidence. The real labels are combined as positive supervision signals, and the total loss is calculated to adjust the model parameters.

Benefits of technology

It improves the model's learning efficiency and detection accuracy on sparse labeled datasets, solves the correlation problem between different branches, enhances the effect of pseudo-label screening, and strengthens the model's detection performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118115727B_ABST
    Figure CN118115727B_ABST
Patent Text Reader

Abstract

This invention belongs to the field of computer vision computing, specifically relating to an independence-guided sparsely labeled object detection method. The method includes: acquiring sparsely labeled image data and performing image independence decomposition to obtain three component images; inputting the original image and the three component images into four branch networks for processing, with each branch obtaining a classification result; calculating the Intersection over Union (IOU) between the predicted bounding box and the labeled bounding box based on the prediction result, and calculating the joint confidence score based on the IOU and classification score; filtering the predicted bounding boxes based on the joint confidence score to obtain pseudo-labels; combining the true label of the current branch with the pseudo-labels of other branches as a positive supervision signal; calculating the total model loss based on the positive supervision signal and classification result, and adjusting the model parameters based on the total loss to obtain a trained sparsely labeled object detection model. This invention achieves better model training results and effectively improves the detection accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of computer vision computing, specifically relating to an independence-guided sparse labeled target detection method. Background Technology

[0002] Object detection is a computer vision technique that identifies the type of objects and their location within images or videos. Unlike traditional object recognition tasks, object detection requires the model to simultaneously provide information on the type and location of multiple objects within the network, thus placing higher demands on the network's feature extraction and localization capabilities.

[0003] While object detection has indeed achieved remarkable results in many fields, its performance is not stable. Most object detectors are trained on fully labeled datasets; however, when the number of images and categories in the dataset is large, it is difficult, if not impossible, to label all existing object instances. In the OpenImages Dataset v4 (OID), there are 14 million objects across 500 categories that need to be detected. However, as a manually labeled dataset, OID inevitably has some issues with completeness. Studies have shown that its labeled recall rate is 43%, meaning that more than half of the objects in the images lack labeling. In industrial manufacturing, the cost of data labeling is enormous, further amplifying this problem. Therefore, training object detection on an incompletely labeled dataset becomes a necessary challenge, unlike semi-supervised training sets where some images are fully labeled and others are completely blank, resulting in sparse labeling.

[0004] By default, each image in a dataset may contain unlabeled objects; we call such incompletely labeled datasets sparsely annotated datasets. When such datasets are applied to traditional supervised object detection and trained using ground-truth methods, unlabeled instances provide the detector with incorrect supervision. These unlabeled instances and their surrounding bounding boxes are incorrectly treated as difficult negative samples, thus misleading the weights updated during gradient backpropagation. The harder the detector tries to learn positive samples, the worse the final result becomes.

[0005] In existing technologies, the best-performing method is using jointly trained models for sparsely labeled object detection. However, this approach suffers from two problems: the correlation problem between different branches and the bias problem of pseudo-labels. The correlation problem between different branches leads to convergence of inputs in multi-branch networks, and joint training cannot fully utilize multiple viewpoints of the image. The bias problem of pseudo-labels results in low-quality pseudo-labels being selected, affecting the accuracy of supervision. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes an independence-guided sparsely labeled target detection method, which includes: acquiring the image to be detected and inputting it into a trained sparsely labeled target detection model to obtain the target detection result;

[0007] The training process of a sparsely labeled object detection model includes:

[0008] S1: Obtain sparsely labeled image data and perform image independence decomposition on it to obtain 3 component images;

[0009] S2: Input the original image and the three component images into the four branch networks for processing. Each branch obtains a classification result. The classification result includes a predicted bounding box and a classification score.

[0010] S3: Calculate the IOU between the predicted bounding box and the labeled bounding box based on the prediction results, and calculate the joint confidence score based on the IOU and the classification score;

[0011] S4: Filter the predicted boxes based on the joint confidence score to obtain pseudo-labels;

[0012] S5: Combine the real label of the current branch with the pseudo label of other branches as a positive supervision signal;

[0013] S6: Calculate the total loss of the model based on the positive supervision signal and classification results, and adjust the model parameters according to the total loss to obtain the trained sparse labeled target detection model.

[0014] Preferably, the process of performing image independence decomposition on sparsely labeled images includes:

[0015] The sparsely labeled image is converted into a one-dimensional signal with three channels according to the RGB color channels;

[0016] The one-dimensional signals from the three channels are combined into a mixing matrix and subjected to independent component analysis transformation to obtain the transformed image.

[0017] The three dimensions of the transformed image are restored to the original image size and converted into grayscale images to obtain three component images.

[0018] Preferably, the branch network includes a backbone network and a detection head, and the backbone networks of the four branch networks share network parameters.

[0019] Furthermore, the detection head includes a classification branch, a regression branch, and an IOU prediction branch. The classification branch and the regression branch each include four convolutional layers. The IOU prediction branch is derived from the regression branch and includes a 3×3 convolutional layer.

[0020] Preferably, the formula for calculating the joint confidence level is:

[0021]

[0022] in, Indicates the joint confidence level. Represents the category score, This indicates the IOU predicted by the IOU prediction branch.

[0023] Preferably, the process of filtering predicted boxes includes: setting a joint confidence threshold, removing predicted boxes with a joint confidence score lower than the joint confidence threshold; removing duplicate predicted boxes; removing predicted boxes corresponding to previously labeled ground truth labels, and using the classification results corresponding to the remaining predicted boxes to form pseudo-labels.

[0024] Preferably, the formula for calculating the total loss of the model is:

[0025]

[0026] Where L represents the total model loss, This represents the classification loss for branch i. This represents the regression loss of branch i. C represents the prediction result for branch i. i This represents the positive supervisory signal for branch i. This represents the IOU loss of branch i.

[0027] Furthermore, the formula for calculating the IOU loss is:

[0028]

[0029] Among them, L iou This represents IOU loss, and BCE() represents binary cross-entropy loss. This indicates the IOU predicted by the IOU prediction branch. IOU represents the IOU between the predicted bounding box and the label box.

[0030] The beneficial effects of this invention are as follows: This invention uses image independence decomposition to improve the input of a sparse dataset object detection model. Image independence decomposition ensures the independence of each joint training branch, aiming to comprehensively capture different perspectives in the image. It can identify missing annotations and incorporate them as positive supervision into the training process, improving the model's learning efficiency. This invention sets joint confidence as a new metric for evaluating the quality of pseudo-labels. Pseudo-labels are selected based on joint confidence, improving the effectiveness of pseudo-label screening. Pseudo-labels and real labels are combined as positive supervision signals to train the model, effectively improving the model's detection accuracy and showing promising application prospects. Attached Figure Description

[0031] Figure 1This is a flowchart of the training process for the sparsely labeled target detection model in this invention;

[0032] Figure 2 This is a schematic diagram of the detection head in this invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] This invention proposes an independence-guided sparsely labeled target detection method, such as... Figure 1 As shown, the method includes the following:

[0035] The image to be detected is acquired and input into a trained sparsely labeled object detection model to obtain the object detection result. In this invention, the sparsely labeled object detection model is obtained through joint training.

[0036] The training process of a sparsely labeled object detection model includes:

[0037] S1: Obtain sparsely labeled image data and perform image independence decomposition to obtain 3 component images.

[0038] To obtain sparsely annotated image data, preferably, sparsely annotated versions of the MS COCO and Pascal VOC datasets are selected as sparsely annotated image data.

[0039] Perform image independence decomposition on sparsely labeled images, specifically:

[0040] The sparsely labeled image is converted into a one-dimensional signal with three channels according to the RGB color channels.

[0041] The one-dimensional signals from the three channels are combined into a mixing matrix and then subjected to independent component analysis (ICA). Preferably, ICA is used for the transformation. By maximizing the non-Gaussianity of the independent components, the initial mixing matrix is ​​iteratively adjusted to gradually improve the non-Gaussianity of the components. Through this iterative adjustment, ICA can quickly converge to the independent components.

[0042] The three dimensions of the transformed image are independent of each other. Each dimension is treated as an independent component of the image and restored to the original image size. In order to maintain the original image format, the image is converted into a grayscale image, resulting in three component images.

[0043] S2: Input the original image and the three component images into the four branch networks for processing. Each branch will obtain a classification result. The classification result includes the predicted bounding box and the classification score.

[0044] Multi-branch networks utilize multiple perspectives of the same image to train multiple learners, achieving mutual complementarity and learning.

[0045] Each of the four branch networks includes a backbone network and a detection head. Preferably, the backbone network uses the single-stage RetinaNet network as the baseline network and employs a feature pyramid network to extract features of different scales from feature maps at different levels, enabling the model to detect targets of different sizes. RetinaNet also introduces Focal Loss as a loss function, which addresses the problem of declining foreground detection performance due to an excessive number of background class samples in traditional object detection models by giving greater weight to samples that are difficult to classify. In addition, RetinaNet uses Anchor technology to generate a series of candidate boxes at different locations and scales, thereby improving the recall rate of object detection. The detection head adopts a dual-branch structure of classification and regression found in traditional object detection networks, with an IOU prediction branch derived from the regression branch. Both the classification and regression branches include four convolutional layers, and the detection head does not share parameters. The backbone networks of the four branch networks share network parameters.

[0046] S3: Calculate the IOU between the predicted bounding box and the labeled bounding box based on the prediction results, and calculate the joint confidence score based on the IOU and the classification score.

[0047] like Figure 2 As shown, the detection head includes three branches: a classification branch, a regression branch, and an IoU (Intersection over Union) prediction branch. The IoU prediction branch is calculated from the regression branch using a 3*3 convolution and is used to predict the IoU between the predicted bounding box and the ground truth bounding box.

[0048] This invention designs a joint confidence score as a metric to reflect the quality of false labels, thereby addressing the false label bias problem. The joint confidence score method of this invention combines the two branches of the object detection head (the classification branch and the bounding box regression branch), using the IoU predicted by the IoU prediction branch as a score, and combining it with the classification score to form the joint confidence score, expressed as:

[0049]

[0050] in, Indicates the joint confidence level. Represents the category score, This indicates the IOU predicted by the IOU prediction branch.

[0051] To focus the IoU prediction branch on IoU prediction, an additional IoU loss was added to the model:

[0052]

[0053] Among them, L iou The IoU loss is represented by , and BCE() represents the binary cross-entropy loss, a commonly used evaluation metric for binary classification problems. It is based on the idea of ​​cross-entropy and measures the difference between the model output and the true label.

[0054] S4: Filter the predicted boxes based on the joint confidence level to obtain pseudo-labels.

[0055] The output of each branch network is filtered by a joint generator. The process includes: setting a joint confidence threshold, typically around 0.5; removing predicted boxes with a joint confidence score below the threshold; removing duplicate predicted boxes; removing the predicted boxes corresponding to the true labels, and using the classification results of the remaining predicted boxes to form pseudo-labels.

[0056] During the first filtering, the IOU scores of all prediction boxes are initialized.

[0057] S5: Combine the real label of the current branch with the pseudo label of other branches as a positive supervision signal.

[0058] The true label of the current branch is combined with the pseudo labels of other branches as a positive supervision signal, which is used in the calculation of the training loss for this round. For each branch, the positive supervision signal C... i From the true label G l The set of pseudo-labels G of other branches pl Composition, represented as:

[0059]

[0060] G pl Represented as:

[0061]

[0062] Where n represents the branch index of the joint training network, which is 3 in this invention. n is 0-3, representing the original image and the three independent component branches after image independence decomposition. This indicates the pseudo-labels generated by branches j other than the current branch.

[0063] By filtering through the joint generator, pseudo-labels with high confidence are obtained. These pseudo-labels will serve as positive supervision signals for other branches and participate in the training of the model, providing more reliable positive sample annotations for sparsely labeled images.

[0064] S6: Calculate the total loss of the model based on the positive supervision signal and classification results, and adjust the model parameters according to the total loss to obtain the trained sparsely labeled object detection model.

[0065] The formula for calculating the total loss L of the model is:

[0066]

[0067]

[0068] Among them, L i The total loss of branch i, This represents the classification loss for branch i. This represents the regression loss of branch i. C represents the prediction result for branch i. i This represents the positive supervisory signal for branch i. This represents the IOU loss of branch i.

[0069] w b and w h ... i The classification loss and regression loss are calculated for each, and these losses are summed to obtain the total loss L. Wherein, It is based on x i ,w b ,w h The prediction results are calculated from these three parameters.

[0070] Based on the total loss, the backbone network and the classification and regression networks in the detection head are trained through backpropagation gradients. This process is repeated iteratively until the performance of both the classification and regression networks stabilizes or reaches the preset number of iterations.

[0071] After the sparsely labeled object detection model is trained, the image to be detected is acquired and input into the trained sparsely labeled object detection model to obtain the object detection result.

[0072] Evaluation of the present invention:

[0073] Table 1 shows the performance of this invention on sparsely labeled datasets. This invention is compared with other state-of-the-art methods in the same field, and experiments were conducted on three different dataset splitting methods, including:

[0074] Split-1: For each class c in the training set, randomly remove p% annotations, where p = {50, 70};

[0075] Split-2: For each class c in the training set, first select images containing class c. Then, for each image, simultaneously remove all annotations for class c with a removal probability of p%, where p = {50, 70}. We ensure that each image contains at least one annotation.

[0076] Split-3: Randomly removes comments p% in a class-agnostic manner, where p = {50, 70}.

[0077] Table 1. Detection performance of three different dataset splits.

[0078]

[0079] The evaluation metric in Table 1 is mAP (mean Average Precision), an important metric for object detection tasks in computer vision. AP (Average Precision) is the area under the precision and recall curves, while mAP is the average of AP across all classes. In other words, it is a measure of the average precision for all classes, used to evaluate the overall performance of the model across different classes. As can be seen from Table 1, the present invention achieves the best results, verifying its effectiveness.

[0080] In summary, this invention improves the input of the sparse dataset object detection model and proposes a sparse dataset object detection model based on image independence decomposition to solve the correlation problem of different branches in joint training, thus achieving better model training results. In addition, this invention proposes joint confidence as a new pseudo-label quality assessment metric, which improves the effect of pseudo-label screening and effectively enhances the detection accuracy.

[0081] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An independence-guided sparsely labeled target detection method, characterized in that, include: The image to be detected is acquired and input into a trained sparsely labeled object detection model to obtain the object detection result; The training process of a sparsely labeled object detection model includes: S1: Obtain sparsely labeled image data and perform image independence decomposition on it to obtain 3 component images; the process of performing image independence decomposition on sparsely labeled images includes: The sparsely labeled image is converted into a one-dimensional signal with three channels according to the RGB color channels; The one-dimensional signals from the three channels are combined into a mixing matrix and subjected to independent component analysis transformation to obtain the transformed image. The three dimensions of the transformed image are restored to the original image size and converted into grayscale images to obtain three component images; S2: Input the original image and the three component images into the four branch networks for processing. Each branch obtains a classification result. The classification result includes a predicted bounding box and a classification score. S3: Calculate the IOU between the predicted bounding box and the labeled bounding box based on the prediction results, and calculate the joint confidence score based on the IOU and the classification score; S4: Filter the predicted boxes based on the joint confidence score to obtain pseudo-labels; the process of filtering the predicted boxes includes: setting a joint confidence score threshold, removing predicted boxes with a joint confidence score lower than the joint confidence score threshold; removing duplicate predicted boxes; removing predicted boxes corresponding to the already labeled true labels, and the classification results corresponding to the remaining predicted boxes form pseudo-labels; S5: Combine the real label of the current branch with the pseudo label of other branches as a positive supervision signal; S6: Calculate the total model loss based on the positive supervision signal and classification results, and adjust the model parameters according to the total loss to obtain the trained sparsely labeled object detection model; the formula for calculating the total model loss is: ; in, This represents the total loss of the model. This represents the classification loss for branch i. This represents the regression loss of branch i. This represents the prediction result for branch i. This represents the positive supervisory signal for branch i. This represents the IOU loss of branch i.

2. The independence-guided sparse labeled target detection method according to claim 1, characterized in that, The branch network includes a backbone network and a detection head, and the backbone networks of the four branch networks share network parameters.

3. The independence-guided sparse labeled target detection method according to claim 2, characterized in that, The detection head includes a classification branch, a regression branch, and an IOU prediction branch. Both the classification and regression branches consist of four convolutional layers. The IOU prediction branch is derived from the regression branch and includes... The convolutional layer.

4. The independence-guided sparse labeled target detection method according to claim 1, characterized in that, The formula for calculating the joint confidence level is: ; in, Indicates the joint confidence level. Represents the category score, This indicates the IOU predicted by the IOU prediction branch.

5. The independence-guided sparse labeled target detection method according to claim 1, characterized in that, The formula for calculating IOU loss is: ; in, Indicates IOU loss. Represents the binary cross-entropy loss. This represents the IOU predicted by the IOU prediction branch. This represents the Interchange of Union (IOU) between the predicted bounding box and the labeled bounding box.