Computer vision learning and object detection under long-tail data distribution

Abstract

Systems and methods for long-tailed distribution object detection using an improved framework using multi-stage supervised and / or semi-supervised training are disclosed. The disclosed techniques effectively utilize unlabeled data through pseudo-labeling and robust augmentation techniques, eliminating the reliance on large labeled datasets. The disclosed techniques enhance the detection accuracy of high-frequency and rare classes without introducing unnecessary complexity such as knowledge distillation or meta-learning. Certain implementations of the disclosed techniques include multi-stage training with external unlabeled images. Certain implementations of the disclosed techniques include pre-training a model on (high-frequency) head classes and transferring the learning to (rare) tail classes, providing significant benchmark improvements over other methods.

Description

Cross-reference to related applications

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 727,882, filed December 4, 2024, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0002] This disclosure relates to object detection in computer vision, and more specifically, to techniques for processing long-tailed data distributions using supervised and semi-supervised learning methods. Background Technology

[0003] Despite significant advancements in modern visual recognition systems, many visual models struggle to learn object detection effectively when few training examples are available and / or the dataset exhibits severe class imbalance. Existing methods rely on external labels (e.g., via ImageNet) to augment low-shot training instances, large-scale labeled datasets, or complex multi-stage training paradigms when object classes follow a natural long-tail distribution. This can lead to inefficiency or failure to scale to practical applications. The reliance on such massive, labeled databases is impractical and has limited utility in real-world scenarios. Previous research on multi-stage training involves overly complex and cumbersome processes that yield poor results. A simplified yet efficient solution is urgently needed to address the long-tail distribution in detection tasks. Summary of the Invention

[0004] Embodiments of this disclosure include systems and methods for long-tailed object detection using an improved framework employing multi-stage supervised and / or semi-supervised training. This disclosure effectively utilizes unlabeled data through pseudo-labeling and robust augmentation techniques, eliminating reliance on large labeled datasets. This disclosure enhances detection accuracy for both high-frequency and rare classes without introducing unnecessary complexity such as knowledge distillation or meta-learning.

[0005] According to certain exemplary embodiments of the present disclosure, a computer-implemented method for detecting long-tailed objects is provided. The method may include acquiring a dataset comprising labeled head classes appearing in more than a threshold number of samples and labeled tail classes appearing in fewer than a threshold number of samples; pre-training a detection model on head class images in the dataset; wherein the detection model may include a backbone network for feature extraction and a detection head, the detection head including a classifier module and a regressor module for bounding box prediction. The method may further include augmenting the pre-trained detection model with pseudo-labeled unlabeled images; adapting the detection model to tail classes by initializing tail parameters of the detection model from the pre-trained head detection model and fine-tuning it using tail class images; and fine-tuning the detection model on a balanced dataset comprising head and tail classes, wherein the fine-tuning updates the classifier and regressor modules while freezing the parameters of the backbone network.

[0006] According to certain exemplary embodiments of the present disclosure, a computer system for long-tail object detection is provided. The system includes a memory and a processing unit; the memory stores a dataset having labeled head categories, labeled tail categories, and unlabeled images; the processing unit executes instructions to pre-train a detection model on head category images by optimizing feature extraction and head component detection, generating pseudo-labels for unlabeled images using a teacher model, wherein pseudo-labels are filtered based on confidence scores, enhancing tail category training data by copying and pasting rare category instances onto different background images, fine-tuning the detection model on a balanced dataset including sample head and tail category images, and merging and unifying head and tail category detection parameters into a single inference model.

[0007] According to certain exemplary embodiments of the present disclosure, a non-transitory computer-readable medium is disclosed, which stores instructions that, when executed by a processor, cause a system to perform a method comprising the following steps: acquiring a dataset comprising labeled head classes appearing in more than a threshold number of samples and labeled tail classes appearing in fewer than a threshold number of samples; pre-training a detection model on head class images of the dataset, wherein the detection model may include a backbone network for feature extraction and a detection head, the detection head including a classifier module and a regressor module for bounding box prediction. The method may further include augmenting the pre-trained detection model with pseudo-labeled unlabeled images, adapting the detection model to tail classes by initializing tail parameters of the detection model from the pre-trained detection model and fine-tuning it using tail class images, and fine-tuning the detection model on a balanced dataset comprising head and tail classes, wherein the fine-tuning updates the classifier module and the regressor module while freezing the parameters of the backbone network.

[0008] Other embodiments, features, and aspects of the present disclosure are described in detail herein and are considered part of the present disclosure. These other embodiments, features, and aspects can be understood with reference to the following detailed description, drawings, and claims. Attached Figure Description

[0009] The following description will be provided with reference to the accompanying drawings and flowcharts, which may not be drawn to scale.

[0010] Figure 1 An overview of prior art related to long-tail detection of the present disclosure is presented, which achieves excellent results on the LVIS v1 benchmark without the need for auxiliary image-level supervision.

[0011] Figure 2 The motivation behind this disclosure is illustrated by an improved head-tail model transfer framework for long-tail detection by incorporating unlabeled images into the representation and transfer learning stages. While more LVIS instance samples can be found in ImageNet, such auxiliary databases may not exist in other scenarios such as aerial images. The implementation of the disclosed framework does not rely on using additional image-level labels to advance long-tail detection, thus extending the practical application of the disclosed method beyond LVIS.

[0012] Figure 3A Models and detectors pre-trained using head categories with unlabeled images are shown according to certain exemplary embodiments of the present disclosure.

[0013] Figure 3B The present disclosure illustrates certain exemplary embodiments that can be implemented (from...) Figure 3A The head representation of the tail category is migrated.

[0014] Figure 3C Some exemplary embodiments of the technology according to this disclosure are shown in (from) Figure 3B A detector fine-tuned on a sample set of head and tail categories. In this example implementation, the "model" may include encoder-decoder transition blocks based on convolutional or transformer networks. In some implementations, the model can only be updated during pre-training.

[0015] Figure 4 An example of transfer learning from the COCO representation (solid bar) is shown that helps improve rare class detection on LVIS (shaded lines).

[0016] Figure 5A The effects of the techniques disclosed herein on pseudo-labeling are illustrated, and it is shown that enhancing unlabeled images by randomly pasting rare instances from the training set helps to facilitate pseudo-labeling in semi-supervised LTDs.

[0017] Figure 5B An example of an enhanced unlabeled image is shown, subjected to high light intensity and geometric perturbations related to the cut.

[0018] Figure 5C Additional examples of enhanced unlabeled images are shown, subjected to high light intensity and geometric perturbations related to the cut.

[0019] Figure 5D This demonstrates that leveraging rare instances can improve robustness and generalization to natural scenes (boxes).

[0020] Figure 6A The detection using the technique of this disclosure on an LVIS v1 verification image with a shot count of k=1 is shown. The visualization of the training set containing extremely rare k-shot examples (in this case, a subwoofer device) is highlighted in the dashed box. The technique of this disclosure performs well under extremely low-shot conditions using as few as a single training instance.

[0021] Figure 6B The detection using the techniques of this disclosure on an LVIS v1 verification image with k=3 is shown. The visualization of the training set containing extremely rare k-shot samples (martinis in this case) is highlighted in the dashed box.

[0022] Figure 6C The detection using the techniques of this disclosure on an LVIS v1 verification image with k=4 is shown. The visualization of the extremely rare k-shot examples (cocoa drinks in this case) in the training set is highlighted in the dashed box.

[0023] Figure 6D The detection using the techniques disclosed herein is shown on an LVIS v1 verification image with k=5. The visualization of the training set containing extremely rare k-shot examples (Sharpie pens in this case) is highlighted in the dashed box.

[0024] Figure 7A The evaluation of Objects365 is shown, grouped by the number of training images for each category group. All models use the ResNet-50 backbone.

[0025] Figure 7B The results demonstrate that the techniques disclosed herein are comprehensively superior to existing baselines in the verification results.

[0026] Figure 8A Ablation experiments are shown to evaluate the performance impact of the present disclosure techniques based on transfer learning of tail categories according to certain exemplary embodiments.

[0027] Figure 8B This illustrates how, according to certain exemplary embodiments, changing the number of sample shots can be used to utilize Dk Ablation experiments were conducted to fine-tune the performance impact of the disclosed technology.

[0028] Figure 9 This is a flowchart of a method according to certain exemplary embodiments of the present disclosure.

[0029] Figure 10 Results of LVIS v1 validation according to certain exemplary embodiments of the present disclosure are shown. GPU hours represent the total training clock time of 640K iterations and are a surrogate measure of model complexity. ResNet and Swin backbones were pre-trained on ImageNet-1K and ImageNet-22K, respectively. Results for Seesaw Loss and Detic are referenced from EFL and RichSem, respectively. Shaded lines indicate the models we implemented.

[0030] Based on the detailed description taken in conjunction with the accompanying drawings, the various features of the technology described herein will become more apparent to those skilled in the art. Those skilled in the art will recognize that alternative embodiments can be employed without departing from the principles of the present technology. Therefore, although specific embodiments are shown in the accompanying drawings, various modifications can be made to this technology. Detailed Implementation

[0031] This disclosure includes systems and methods for improving long-tailed object detection, which utilizes an improved framework of multi-stage supervised and / or semi-supervised training, which involves pre-training a model on the head category and then performing transfer learning on the tail category.

[0032] Some embodiments of this disclosure include a simplified multi-stage training framework for efficient long-tail detection. Unlike existing multi-stage methods, the framework in this disclosure allows for semi-supervised long-tail detection using optional unlabeled images. Some embodiments of this disclosure include techniques for pasting rare instances from the training set to unlabeled images. Unlike existing methods that use additional external data, this disclosure does not rely on any additional human annotations at the image level (e.g., full-image labels from ImageNet) or at the instance level (e.g., bounding boxes). Some embodiments of this disclosure can use only a human-labeled training set (like other methods) and originally collected unlabeled images that are not human-labeled in any way. When tested against competing methods on the challenging LVIS v1 benchmark, this disclosure builds new state-of-the-art results in both supervised and semi-supervised settings without the need for additional image-level labels.

[0033] In some implementations, the framework may include high-frequency representation learning for head categories, optionally incorporating unlabeled data to improve feature extraction; transfer learning of rare tail categories using pre-trained head representations, adapted for fine-tuning on augmented datasets; and unified detector fine-tuning for balanced subsets of head and tail categories.

[0034] This disclosed technique can be viewed as an improved head-to-tail model transfer paradigm without increasing the complexity of meta-learning or knowledge distillation. By utilizing supplementary unlabeled images without requiring additional image labels, this disclosed technique can provide improved benchmarks in supervised and semi-supervised settings.

[0035] Some embodiments of this disclosure may utilize pseudo-labels: unlabeled images pseudo-labeled during training. The pseudo-labels may originate from the teacher model, ensuring effective learning of rare classes. Some embodiments of this disclosure may include data augmentation: copy-paste augmentation, which ensures a balance of training samples between the head and tail distributions. Additional geometric and photometric augmentations may be utilized to enhance robustness. Some embodiments of this disclosure may be compatible with convolutional and transformer-based architectures, thus providing scalability across hardware and dataset configurations. Some embodiments of this disclosure may achieve state-of-the-art results on benchmarks such as LVIS v1 and Objects365 without auxiliary image-level labels.

[0036] The unique approach of this disclosure differs from previous methods in that it does not rely on any labels other than those provided by the training set. In contrast, methods like RichSem, Detic, and MosaicOS require additional manually annotated labels from external ImageNet databases to enhance their training protocols. This disclosure can utilize raw, unlabeled images without accompanying manual annotations. During training, this disclosure is able to generate pseudo-labels. Furthermore, rare instances can be algorithmically pasted onto unlabeled images without manual intervention. By comparison, methods using unlabeled images for single-stage training (such as CascadMatch) perform worse than the multi-stage training using unlabeled images disclosed herein.

[0037] Some embodiments of the technology disclosed herein can be further understood and supported by the following description and accompanying drawings.

[0038] Detecting, locating, and classifying object instances in images and videos is a long-standing problem in computer vision, which can be addressed using certain implementations of the techniques disclosed herein. Prior to this disclosure, recent advances in modern object detection systems have been primarily driven by models relying on robust neural architectures that are measured on relatively balanced, small-vocabulary benchmarks, such as PASCAL VOC or MS-COCO. However, when evaluated on more complex and imbalanced datasets with larger vocabularies, the same models exhibit a considerable drop in detection accuracy.

[0039] Some embodiments of the present disclosure enhance the capabilities of product detection systems, with particular emphasis on evaluating challenging large-vocabulary LVIS benchmarks that can represent real-world scenarios where object categories follow a natural long-tailed distribution. It is precisely at the tail end of this long-tailed distribution that most data-hungry models encounter performance difficulties, as this distribution can be represented by many rare categories with only a single training example.

[0040] Figure 1 Figure 100 shows the recognition performance 102 and class rarity 104 of existing methods developed to address the extreme differences in class distribution in long-tailed detection (LTD). This disclosure (by...) Figure 1 The “SimLTD” benchmark data point (represented by 106 points in the upper right corner) can leverage a process that combines unlabeled data with an intuitive multi-stage training strategy (discussed further below) to deliver superior overall performance while also optimizing accuracy for rare classes. The techniques disclosed herein achieve excellent results on the LVIS v1 benchmark without requiring auxiliary image-level supervision.

[0041] In contrast, previous methods divided the entire dataset into several stages, each containing increasingly smaller but balanced data samples. In these methods, the model could be trained incrementally via network expansion and knowledge distillation. However, these previous methods could be overly complex, requiring the maintenance of knowledge transfer across many sub-components and stages, which could lead to catastrophic forgetting and thus poor-quality solutions.

[0042] Previous methods have attempted to leverage external data to augment training instances in LVIS. However, these previous methods require access to large databases containing labeled images of thousands of object categories, which is not necessarily suitable for many practical settings outside of LVIS.

[0043] Figure 2This is an example graph of the long-tail distribution 200 of image counts 202 and categories 204 in an aerial image example, including a high-frequency category 206, a common category 208, and a rare category 210 with unlabeled data. The example distribution graph 200 can be divided into head categories 212 and tail categories 214. Head categories 212 may include high-frequency categories 206 and common categories 208, and tail categories 214 may include rare categories 210. In some implementations, representation learning can be performed on head categories 212, and then the learning can be transferred to tail categories 214.

[0044] In some implementations, industrial applications may focus on well-defined and readily available general objects (such as those that can be represented in head category 212); however, constructing new object-centric datasets to supplement the main training dataset can be expensive and time-consuming. Furthermore, the current challenge is how to handle truly rare unlabeled categories 210 (e.g., a rare fish species), which may severely limit observation and have limited availability.

[0045] According to certain exemplary embodiments of the present disclosure, the novel framework and / or process can be used to generally involve (i) pre-training on head category 212, (ii) transfer learning on tail category 214, and (iii) fine-tuning on a small sample set consisting of head category 212 and tail category 214. Certain embodiments of the present disclosure allow the use of unlabeled images to further improve the results. According to certain exemplary embodiments of the present disclosure, learning can be achieved in a semi-supervised manner using supplementary unlabeled data via pseudo-labels, thus not explicitly relying on any additional instance-level or image-level supervision.

[0046] Certain implementations of the techniques disclosed herein can address several challenges associated with the head-tail model transfer paradigm. For example, when the head class distribution is also skewed, conventional transfer from data-rich head representations to data-scarce tail classes may not yield sufficient performance improvements. Furthermore, the simple application of unlabeled data can exacerbate inductive bias in the model, as unlabeled samples may follow a similar long-tail distribution. In this case, the trained model may tend to generate more pseudo-labels for the head classes, leading to more pseudo-class imbalance.

[0047] This disclosed technique overcomes the aforementioned obstacles by integrating data augmentations specifically designed to mitigate class imbalance during the head and tail training phases. Extensive experiments demonstrate that stronger pre-trained models on the head category, with or without unlabeled images, transfer well to the tail category, thus providing an ideal solution for long-tail detection.

[0048] This disclosure provides a general and versatile framework for efficient semi-supervised long-tail distributions for unlabeled images. This framework leverages a simple and intuitive workflow, compatible with a variety of backbones and detectors based on classical convolutional and modern transformer architectures. When tested against competing methods on the challenging LVIS v1 benchmark, this disclosure demonstrates superior performance and scalability, building new state-of-the-art results with significant advantages.

[0049] Some embodiments of this disclosure can utilize pre-trained representations to provide a superior initialization than random weights for fine-tuning tail categories. Some embodiments of this disclosure can employ different but related methods for pre-training on high-frequency and common head instances, where the head representation is replicated and fine-tuned for tail categories without relying on knowledge distillation or meta-learning required by conventional methods. Notably, this disclosure may employ only three learning stages compared to the seven stages of prior art, which may exacerbate catastrophic forgetting. It is worth reiterating that the innovation of this disclosure lies in introducing unlabeled images in both the pre-training and transfer learning stages to augment LTD, resulting in a more general and flexible solution for using unlabeled data as an auxiliary supervision source. This solution can be easily collected without the burden of manual annotation and, due to its multi-stage training strategy, may be more effective than competing solutions, as will be further discussed below.

[0050] Long-tail detection (LTD) is related to the few-shot detection (FSD) task, which aims to adapt a base detector (trained from multiple-shot instances) to learn new concepts from a few-shot examples. Both tasks aim to improve the detection of categories with very few training instances. However, LTD faces unique challenges beyond FSD. For LTD, tail categories are indeed rare in natural scenes and follow a Zipf distribution. In contrast, novel few-shot examples in FSD datasets are randomly sampled and are not necessarily rare, but can include objects with varying degrees of observation frequency. Therefore, multi-stage training methods suitable for FSD may not be directly applicable to LTD and achieve the same expected results. The techniques disclosed in this paper provide specific adaptations to the LTD problem, resulting in improvements over existing techniques.

[0051] Continue to refer to Figure 2 Given a long-tailed dataset D with C classes and 204 subclasses. lt 200, the dataset D lt 200 can be divided into two disjoint subsets: one containing C elements that appear in >M images. head High-frequency and common categories of D head 212, and C appearing in ≤M images tail A rare category of Dtail 214. An unlabeled dataset D with unknown class distribution. u It may be available. In some implementations, the goal is to use a combination of labeled and unlabeled images to train a tool for targeting C. head ∪C tail A unified model that optimizes accuracy on test sets for both categories.

[0052] This disclosed technology can utilize a framework that includes: (i) in D head The representation of learning on, (ii) in D tail Transfer learning on, and (iii) in D k Fine-tuning on D k It is from D lt A reduced dataset consisting of k randomly sampled instances from each class. Note that D head D tail and D k They are still unbalanced, but to a far lesser degree than the original long-tailed D. lt That's not the case. In steps 1 and 2, optional unlabeled images can be utilized, but are not explicitly required for effective LTD. Experiments show that the fully supervised baseline model still exhibits superior performance and scalability even without unlabeled data.

[0053] Figures 3A-3C Three main example steps of the techniques disclosed herein are illustrated. The “model” discussed with reference to these figures may include encoder-decoder transition blocks based on convolutional networks or transformer networks. In some implementations, the model may be updated only during pre-training and remained frozen for the rest of the time.

[0054] Figure 3A The first step (step 1) of an example training process according to certain exemplary embodiments of the present disclosure may include pre-training the model 302 and detector 304 using unlabeled image 308 in head category 306. The output of detector 304 may be utilized in a next step, such as... Figure 3B As shown.

[0055] Figure 3B The second step (step 2) of an example training process according to certain exemplary embodiments of the present disclosure is shown, wherein model 302 can be frozen. Figure 3A The training of the head representation can be transferred to the tail category 310. An example of head-tail category fusion can be processed according to the algorithm, which will be discussed below in conjunction with training the learnable detection function Ψ for image-object pairs. det (D) head This will be discussed further below.

[0056] Figure 3C A third step (step 3) of an example training process according to certain exemplary embodiments of the present disclosure is shown, wherein the detector 304 may be fine-tuned on a sample set 312 of head and tail categories. It should be understood that the k example instances (20 in this example) and the number of shots (30 in this example) are for illustrative purposes only and other values ​​may be used.

[0057] One of the unresolved challenges in the LTD problem lies in how to learn and train an effective model from a small number of examples associated with the tail category. Empirical analysis demonstrates that, in the LVIS setting, few-shot learning can significantly benefit from pre-trained representations. For example, following the traditional LVIS protocol, the example dataset is divided into C groups that appear in M ​​> 10 images. head =866 common categories, and C appearing in M≤10 images tail =337 rare categories, to determine the category in C using various commodity detectors pre-trained on the COCO dataset. tail Improved detection performance.

[0058] Figure 4 It is shown in C tail The effectiveness of the pre-trained representations on detection was evaluated. For each model, the detection head, consisting of a regressor module and a bounding box classifier learned on the COCO dataset, was truncated, reinitialized with random values, and transfer learning was performed on the LVIS tail classes. Only the box classifier and regressor were updated, while the rest of the architecture was kept frozen, essentially treating the pre-trained model as a fixed detector. In addition to the pre-trained network, a Faster R-CNN detector initialized from scratch (excluding the pre-trained backbone) was evaluated as a lower bound baseline. Figure 4 This demonstrates a clear trend indicating stronger pre-trained representations, as measured by AP scores on COCO, which typically lead to improved rare class detection. This result is surprising considering that these models were pre-trained on COCO, a dataset smaller in scope and size than LVIS. Figure 4 As shown, training from scratch can lead to the worst performance, with accuracy only half. This experiment has dual significance: (1) we demonstrate that low-shot learning can be improved through transfer representation; and (2) the technique disclosed herein opens new avenues for self-supervised learning, semi-supervised learning, and multimodal learning, all of which demonstrate significantly better performance than supervised pre-training. This shows that the technique disclosed herein can improve low-shot learning in D... head Learn powerful representations on top of D and transfer these representations to D tailThis approach is designed for long-tail detection, whereas previous attempts at head-tail model transfer could only be achieved by incorporating additional meta-learning or knowledge distillation modules. A three-step method for implementing long-tail detection (e.g., without considering the additional complexity of previous methods) will be discussed in detail below. Figures 3A-3B (as shown) for detailed information.

[0059] Refer again Figure 3A The first step in the training process can begin by using training data points (x i y i )∈D head The supervision settings, where x i Let y represent the i-th input image. i It is the i-th ground fact annotation, including box labels and coordinates. Let Ψ det (D) head ) is defined as training on image-object pairs to produce supervised loss L. sup Learnable detection function:

[0060] (1) Here, h(x) i ) represents the forward pass of the input image, (L cls , L reg The classification loss (e.g., cross-entropy) and regression loss (e.g., L1) of the detector are used. Commodity convolutions and transformer-based networks can be considered for Ψ. det For convolutional networks, Faster R-CNN can be compared with previous studies using similar detectors (i.e., Mask R-CNN and RetinaNet). For modern transformer-based networks, improved detection transformers (DETRs) can be employed, such as deformable DETRs and DINO. Table 1

[0061] As shown in Table 1, data augmentation can be used to train Ψ. det This includes random image resizing, horizontal flipping, and photometric distortion. In some implementations, simple copy-paste (SCP) combined with repeat factor sampling (RFS) can be used to mitigate D. head The class imbalance at the image and instance levels is used for image and instance resampling in LTD. In some implementations, Ψ can be trained in a semi-supervised manner using unlabeled images with the aid of pseudo-labels. det Come and learn D headA stronger representation on top. For example, some implementations can leverage methods that increase effectiveness to advance D. head Semi-supervised representation learning can include SoftER Teacher, MixTeacher, and / or MixPL. For example, SoftER Teacher builds upon the end-to-end pseudo-labeling method of Soft Teacher by incorporating an auxiliary loss for consistency learning in region proposals and exhibits superior performance in semi-supervised few-shot detection. MixTeacher introduces a mixed-scale feature pyramid to generate more accurate pseudo-labels for objects with extreme scale variables, resulting in a more robust detector overall. While SoftER Teacher and MixTeacher are primarily suited for two-stage detectors such as Faster R-CNN, MixPL opens the door to semi-supervised learning for single-stage and DETR-based models by integrating blending and mosaic enhancements with pseudo-labels. One or more of the SoftER Teacher, MixTeacher, and / or MixPL methods may employ a student-teacher semi-supervised training paradigm, where the teacher may be an exponential moving average of the students. In a semi-supervised setting, the model can be derived from the labeled D via a composite objective function. head Image and unlabeled D u Learning from a joint dataset of images: (2) Here, α > 0 controls the contribution of pseudo-label loss originating from unlabeled data. pseudo The functional form of L in equation (1) sup The difference lies in the fact that the actual ground target y may be a false target predicted by the teacher model during the self-training process. What it replaced.

[0062] Refer again Figure 3B The second step in the training process can be to instantiate the tail model by copying parameters from the pre-trained head model for transfer learning. Let It is a supervised tail model. It is a semi-supervised correspondence model. In some implementations, the tail model can be trained in the same way using equations (1) and (2), but only the classifier and regressor modules need to be updated to fit the tail category, while the rest of the network is frozen. In this implementation, the pre-trained representation can be used as a bootstrap initializer to train an accurate tail model according to step 1 as described above.

[0063] Unlike common head objects, tail categories may be inherently rare even if they do not appear in labeled or unlabeled sources, which makes training Ψ′ using unlabeled images difficult. semi-detThis raises the barrier because the scarcity of rare classes in unlabeled scenes makes it difficult for the teacher model to propose reliable spurious targets. Some implementations of this disclosure circumvent this barrier by copying and pasting a random subset of rare instances from the labeled training set into unlabeled images—a novel process unique to the field. In each training iteration, the teacher model is guaranteed to observe an enhanced view of the sampled rare objects in different background scenes, facilitating spurious label supervision for the student model.

[0064] Figure 5A The effects of the techniques disclosed herein on pseudo-labeling are illustrated, and it is shown that enhancing unlabeled images by randomly pasting rare instances from the training set helps to facilitate pseudo-labeling in semi-supervised LTDs. Figure 5B An example of an enhanced unlabeled image is shown, subjected to high light intensity and geometric perturbations related to the cut. Figure 5C Additional examples of enhanced unlabeled images are shown, subjected to high light intensity and geometric perturbations related to the cut. Figure 5D This demonstrates that leveraging rare instances can improve robustness and generalization to natural scenes (boxes). While the process disclosed in this paper may inevitably lead to redundancy and overfitting, ablation experiments show that it is surprisingly helpful in adapting head representations to tail models, which will be discussed further below.

[0065] Refer again Figure 3C The third step of the training process can employ two independent models sharing a representation, one optimized for the head category and the other for the tail category. Certain embodiments of this disclosure can unify the two models into a single model for efficient single-pass inference on test samples that simultaneously contain both head and tail categories.

[0066] According to certain embodiments of the present disclosure, the following PyTorch pseudocode can be used for head and tail category merging: # HEAD_IDS: sorted list of head IDs, length 866 # head_ckpt: model checkpoint on head classes # TAIL_IDS: sorted list of tail IDs, length 337 # tail_ckpt: model checkpoint on tail classes ALL_IDS = sorted(HEAD_IDS + TAIL_IDS) # length 1203 ID2LABEL = { ID: label for label, ID in enumerate(ALL_IDS) } # mapping from category ID to integer label head_det = head_ckpt["state_dict"]["detector"] tail_det = tail_ckpt["state_dict"]["detector"] fused_det = torch.randn(len(ALL_IDS)) for label, ID in enumerate(HEAD_IDS): fused_det[ID2LABEL[ID]] = head_det[label] for label, ID in enumerate(TAIL_IDS): fused_det[ID2LABEL[ID]] = tail_det[label] head_ckpt["state_dict"]["detector"]= fused_det torch.save(head_ckpt, save_filename) # to fine-tune

[0067] According to certain exemplary embodiments of the present disclosure, parameters in the detector module can be merged, and the pre-trained head representation can be reused for the remainder of the network. In some embodiments, the unified detector can be targeted at a D consisting of k instances or images. k Fine-tuning is performed on these instances or images sampled by category from the long-tail training set. In some implementations, a regressor with a reduced learning rate is used to update only the box classifier, progressively adapting it to the tail categories while maintaining pre-trained accuracy for the head categories. Similar to class-incremental learning, the techniques disclosed herein can be applied to D... k It includes both head and tail categories for sample replay, avoiding catastrophic forgetting. k It can be formed through random sampling, while previous studies relied on complex schemes that guided sample replay based on confidence.

[0068] According to certain exemplary embodiments of the present disclosure, the present disclosure can be empirically evaluated using benchmarks on the challenging LVISv1 dataset, which has 100,170 training images and 19,809 validation images covering 1,203 classes. Standard LVIS evaluators can be used to compute the detection metric mAP for all classes separately without augmentation during testing. box APs for rare, common, and high-frequency categories. r AP c and AP f In some implementations, different random seeds can be used to pair D. k Triple sampling is performed, and a mean metric can be reported to capture statistical variability. This is achieved by focusing on mAP. box and AP r The performance improvement allows for comparative analysis between different technologies, demonstrating that the disclosed technology exhibits superior performance and scalability across various backbone networks, detectors, and unlabeled images.

[0069] The model disclosed herein has been trained on an 8×A6000 GPU using PyTorch and MMDetection. In the example implementation, in step 1 (as... Figure 3A In step 2 (as shown), the model can be pre-trained for 540K iterations. Figure 3B In the example shown), the model can perform transfer learning using 20K iterations. In step 3 (as shown...) Figure 3C As shown, the model can be fine-tuned through 80,000 iterations. In some implementations, training can be completed in between 8 hours and 15 days, depending on the training scope.

[0070] In some implementations, a high-quality supervised baseline can be constructed by combining the disclosed multi-stage training with diverse data augmentations. Some implementations of this disclosure may include utilizing multiple ResNet backbones for Faster R-CNN and DETR-based models. In some implementations, FPN can be used for additional feature extraction using Faster R-CNN. According to some exemplary implementations of this disclosure, the supervised baseline can serve as the foundation for a teacher model that generates reliable pseudo-objectives for semi-supervised learning.

[0071] According to certain exemplary embodiments of the present disclosure, one or more of SoftER Teacher, MixTeacher, and MixPL can be used for semi-supervised LTD. Some embodiments of the present disclosure may include inheriting all hyperparameters initially tuned on the COCO dataset without any modification. In an example embodiment, approximately 123K COCO-unlabeled2017 images can be used to improve the representation and transfer learning in steps 1 and 2 of the present disclosure. In an example embodiment, Objects365 can be used to remove all label information from the training set, further validating the effectiveness of the present disclosure in another relevant domain using approximately 1.7M original unlabeled images.

[0072] In comparing the effectiveness of the techniques disclosed herein with existing methods representing the state of the LVIS field, careful efforts were made to ensure a fair comparison across multiple dimensions, including the backbone, detector, and external data sources, as not all methods follow a single established experimental protocol. The results show that the techniques disclosed herein (SimLTD supervised baseline with Faster R-CNN) outperform all methods using relevant detectors. This improvement is convincing, reaching up to +4.1 AP. box and +5.6 AP r A similar trend was observed when using deformable DETR to compare with our disclosed supervisory baseline technique: the technique disclosed in AP box It is on par with RichSem, while in AP r It surpassed competitors including RichSem, achieving a significant improvement of up to +10.6 AP. r point.

[0073] For methods requiring external data, certain implementations of this disclosure using a semi-supervised approach also demonstrate superior performance. For example, when combined with MixTeacher and Faster R-CNN, this disclosure (SimLTD) outperforms competitors by up to +7.9 AP. box Meanwhile, in AP r The technology is competitive. Furthermore, by coupling with the MixPL and DINO detectors, the present disclosure extends well, achieving state-of-the-art results using only unlabeled images. Interestingly, the performance difference between COCO and Objects365 unlabeled images is small, suggesting that the present disclosure can leverage meaningful pseudo-label supervision from a large, unfiltered database with a distribution different from the training dataset.

[0074] Figure 6AThe detection using the technique of this disclosure on an LVIS v1 verification image with k=1 is shown. The visualization of the training set containing extremely rare k-shot examples (in this case, a subwoofer device) is highlighted in the dashed box. The technique of this disclosure performs well under extremely low shooting conditions using as few as a single training instance.

[0075] Figure 6B The detection using the techniques of this disclosure on an LVIS v1 verification image with k=3 is shown. The visualization of the training set containing extremely rare k-shot samples (martinis in this case) is highlighted in the dashed box.

[0076] Figure 6C The detection using the techniques of this disclosure on an LVIS v1 verification image with k=4 is shown. The visualization of the extremely rare k-shot examples (cocoa drinks in this case) in the training set is highlighted in the dashed box.

[0077] Figure 6D The detection using the techniques disclosed herein is shown on an LVIS v1 verification image with k=5. The visualization of the training set containing extremely rare k-shot examples (Sharpie pens in this case) is highlighted in the dashed box.

[0078] Figure 7A The generality of the disclosed framework evaluated against Objects365 by grouping based on the number of training images in each category group is shown. In this example, 30% of the training set was sampled and divided into C categories that appear in M ​​> 100 images. head =332 categories, and C appearing in M≤100 images tail =33 categories. Based on the number of training images for each category, the evaluation was divided into four category groups. All models used the ResNet-50 backbone.

[0079] like Figure 7A As shown, there is a significant difference in instance counts between LVIS and Objects365, with Objects365 having approximately 36 times more labels than LVIS for its richest objects. Figure 7B As shown, the technology disclosed herein comprehensively outperforms existing baselines in the verification results.

[0080] In some implementations, the techniques disclosed herein can be fine-tuned using a shooting k=30000 (a parameter that can be changed based on the dataset).

[0081] Significant differences exist when comparing the present disclosure technique with CascadeMatch. For example, while both the present disclosure technique and CascadeMatch can utilize COCO-unlabeled2017 for semi-supervised LTD, CascadeMatch performs end-to-end training in a single stage, while the present disclosure technique employs a decoupled approach. CascadeMatch further utilizes a stronger Cascade R-CNN compared to Faster R-CNN, which can be used in the present disclosure technique. Furthermore, CascadeMatch adheres to the APFixed protocol, which replaces the standard maximum of 300 detections per image with an upper limit of 10K detections per class from the entire validation set. Despite the limitations of the simpler model, Table 2 shows that the present disclosure technique (SimLTD) significantly outperforms CascadeMatch on almost every metric, further supporting the superiority of multi-stage training. Table 2

[0082] The techniques disclosed herein have been further compared with other techniques to determine the performance differences in multi-stage learning. For example, Dong et al., “Boosting Long-Tailed Object Detection via Step-Wise Learning on Smooth-Tail Data,” in ICCV, 2023, utilizes a three-step process: pre-training, fine-tuning, and knowledge distillation. Analysis was performed on deformable DETR models, and the results are similar to the simpler Faster R-CNN model used in this disclosure. When trained using the same efficient deformable DETR architecture, the results show that the techniques disclosed herein achieve an increase in performance of +6.3 AP. box and +10.2 AP r The model significantly outperforms the model by Dong et al. These significant improvements can be directly attributed to the multi-stage learning method of this disclosure, which is carefully designed to optimize the accuracy of both head and tail categories.

[0083] In terms of recall using additional data, this disclosure further compares to RichSem, which relies on the CLIP classifier (pre-trained on approximately 400M image caption pairs) and image supervision from an additional approximately 150M images to produce state-of-the-art results. However, such a strong dependency becomes difficult to maintain when the method is applied to customized datasets beyond general objects. In contrast, this disclosure utilizes optional unlabeled images to achieve superior performance to RichSem without relying on CLIP or auxiliary ImageNet labels. When external data is removed, for example in a fully supervised setting, this disclosure achieves +6.0 AP. r This significantly outperforms RichSem, suggesting that RichSem's success may be sensitive to contributions from CLIP and ImageNet supervision. In contrast, the techniques and frameworks disclosed herein demonstrate robustness across various settings with and without external data.

[0084] According to certain exemplary embodiments of the present disclosure, the present disclosure can be regarded as a multi-stage training strategy that combines standard data augmentation, effectively solving the class imbalance in the head and tail datasets while avoiding the complexity of previous methods. Table 3

[0085] Table 3 presents the ablation experiment results, quantifying the effectiveness of each component in the supervised baseline of this disclosure. According to an exemplary implementation of this disclosure, the single-stage model can be trained end-to-end on the entire long-tail dataset. The contributions of this disclosure establish a more robust baseline than potentially previous Faster R-CNN detectors using ResNet-50. These results also demonstrate the feasibility of multi-stage learning over a simple single-stage training process on the entire long-tail dataset, where the simple single-stage training process produces significantly worse results.

[0086] As discussed above, certain implementations of the technology disclosed herein can transfer pre-trained head representations to (e.g.) Figure 3B The tail category in step 2 (as shown) is optimized. However, in some implementations, this step may be optional and may be skipped, since it is possible to initialize the tail category with random values ​​before fine-tuning.

[0087] Figure 8A Ablation experiments are shown to evaluate the performance impact of the present disclosure techniques based on transfer learning of tail categories according to certain exemplary embodiments.

[0088] Figure 8B This illustrates how, according to certain exemplary embodiments, changing the number of sample shots can be used to utilize Dk Ablation experiments were conducted to fine-tune the performance impact of the disclosed technology.

[0089] Figure 8A and Figure 8B This demonstrates that transfer learning on tail categories is a worthwhile endeavor. Transfer learning yields up to +4.7 AP in both supervised and semi-supervised settings. r The improvement also brings the additional advantage of reducing the number of iterations required for fine-tuning to two-thirds of the original.

[0090] Figure 8A and Figure 8B It also shows the effect of mAP box and AP r The influence of D k The function for fine-tuning sample captures ranges from 10 to "all," i.e., the entire long-tail training set. The aim of this experiment is to determine how many times to sample Dk and optimize accuracy for rare classes while mitigating catastrophic forgetting of the pre-trained head representations. In this example implementation, analysis can be performed... Figure 8B The "inflection point on the curve" is used to determine whether 30 shots can balance the coordination between two metrics. For example, Figure 3C This demonstrates that with 30 shots, the entire tail distribution can be sampled, and 20 or fewer instances can be included in each category, including a mix of head categories for sample replay. Therefore, for this example, 30 shots can be considered the "optimal shot." (See again...) Figure 8B mAP was observed when 10 or 20 shots were taken. box The significantly reduced percentage (%) (left scale) indicates that insufficient samples lead to unfavorable forgetting of the head category. Furthermore, as... Figure 8B As shown, in response to an excessive number of head samples, using more than 30 shots resulted in AP (Advanced Persistent Threat) errors. r (%) (right scale) decreased sharply.

[0091] Figure 9This is a flowchart of a method 900 for long-tail object detection according to certain exemplary embodiments of the present disclosure. In block 902, method 900 includes acquiring a dataset comprising labeled head categories appearing in more than a threshold number of samples and labeled tail categories appearing in fewer than a threshold number of samples. In block 904, method 900 includes pre-training a detection model on head category images in the dataset, wherein the detection model includes a backbone network for feature extraction and a detection head, the detection head including a classifier module and a regressor module for bounding box prediction. In block 906, method 900 includes augmenting the pre-trained detection model with pseudo-labeled unlabeled images and adapting the detection model to the tail categories by initializing the tail parameters of the detection model from the pre-trained detection model and fine-tuning it using tail category images. In block 908, method 900 includes fine-tuning the detection model on a balanced dataset comprising head and tail categories, wherein the fine-tuning updates the classifier module and the regressor module while freezing the parameters of the backbone network.

[0092] In some implementations, the detection model may also include a loss function for combining classification loss and regression loss.

[0093] In some implementations, the backbone network can be selected from the ResNet family of architectures. In some implementations, the backbone network can be selected from a converter-based architecture.

[0094] According to certain exemplary embodiments of the present disclosure, the detection head may be combined with a Feature Pyramid Network (FPN) for multi-scale feature extraction.

[0095] In some implementations, pseudo-labeling can be performed using a teacher-student framework, where the teacher model can be used to generate pseudo-labels for the student model. In other implementations, pseudo-labeling can be performed by using a confidence threshold to select reliable pseudo-labels.

[0096] In some implementations, pseudo-marking may also include enhancing the unmarked image by one or more of the following: automatically copying rare instances from the marked image, and automatically pasting the copied rare instances onto the unmarked image to create a composite scene. In some implementations, pseudo-marking may also include and automatically apply one or more geometric and photometric transformations. One or more geometric and photometric transformations may include one or more of resizing, rotation, translation, and contrast adjustment.

[0097] In some implementations, pre-training may include training on an imbalanced head dataset using a data sampling strategy or procedure. In some implementations, the data sampling strategy or procedure may include simple copy-paste (SCP) augmentation to generate additional labeled instances of rare head classes. In some implementations, the data sampling strategy or procedure may include repeated factor sampling (RFS) to augment representations of underrepresented head classes.

[0098] According to certain exemplary embodiments of the present disclosure, fine-tuning may include freezing the backbone network and updating the classifier and regressor weights of the detection head. In some embodiments, fine-tuning may include incorporating additional unlabeled data containing rare class instances by enhancing the scene with rare objects. In some embodiments, fine-tuning may include optimizing the combined loss function. In some embodiments, optimizing the combined loss function may include determining a supervised loss for labeled tail images and a pseudo-label loss for unlabeled data.

[0099] According to certain exemplary embodiments of the present disclosure, the balanced dataset for fine-tuning may include a subset of head category images randomly sampled based on a predetermined shot count. In some embodiments, if the tail portion of the dataset has fewer samples than the shot count, the balanced dataset for fine-tuning may include all available tail category images.

[0100] This disclosure provides a simple and general framework for supervised and semi-supervised long-tail detection. Standing out from existing work, this disclosure offers superior performance and scalability, achieving state-of-the-art results on the challenging LVIS v1 benchmark without requiring auxiliary image-level supervision, and providing improved performance on long-tail detection problems.

[0101] Some embodiments of the present disclosure can be applied to practical applications and use cases that may include one or more of the following: (1) general object detection based on images and videos; (2) object detection based on aerial images; (3) defect detection in manufacturing processes; (4) pedestrian detection; (5) drone detection; (6) supervision; (7) face detection and recognition; (8) object detection in autonomous vehicles; (9) sports analytics detection, etc. For example, in a practical application use case of aerial images for determining roof conditions, roof stripes, markings, or deterioration indicate potential current or future problems related to the roof. However, the available aerial image datasets may only contain a small number of examples of such instances, which have a class imbalance between high-frequency and rare classes. As discussed herein, embodiments of the present disclosure can utilize image datasets with few labels to train models for enhanced recognition of rare instances.

[0102] Figure 10Results of LVIS v1 validation according to certain exemplary embodiments of the present disclosure are shown. GPU hours represent the total training clock time of 640K iterations and are a surrogate measure of model complexity. ResNet and Swin backbones were pre-trained on ImageNet-1K and ImageNet-22K, respectively. Seesaw Loss and Detic results are referenced from EFL and RichSem, respectively. Shaded lines indicate the models implemented according to the present disclosure.

[0103] Figure 10 The results listed demonstrate the effectiveness of the techniques disclosed herein compared to existing methods representing the state of the art in the LVIS field. For example, the SimLTD supervised baseline with Faster R-CNN outperforms all methods using correlation detectors. This improvement includes an amplitude of up to +3.9 AP. box and +5.6 AP r Similar trends were observed when comparing baseline techniques using DETR-based models with those disclosed herein. SimLTD demonstrates compelling performance and scalability across numerous backbones and detectors without requiring additional data.

[0104] For methods requiring external data, the semi-supervised model of this disclosure also demonstrates excellent performance. When combined with MixTeacher and Faster R-CNN, SimLTD outperforms competitors by up to +7.9 AP. box Meanwhile, in AP r It is competitive in this regard. Furthermore, through coupling with MixPL and a converter-based detector, SimLTD scales well, achieving state-of-the-art results using only unlabeled images. SimLTD also performs admirably between COCO and Objects365 unlabeled images, demonstrating the model's ability to leverage meaningful pseudo-label supervision from a large, unfiltered database with a distribution different from the training dataset.

[0105] The above description of various embodiments of the claimed subject matter is intended for illustration and description. These descriptions are not intended to be exhaustive, nor do they limit the claimed subject matter to the specific forms disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to most appropriately describe the principles of the invention and its practical application, thereby enabling those skilled in the art to understand the claimed subject matter, the various embodiments, and the various modifications suitable for the particular purpose considered.

[0106] Although certain embodiments are described in detail, the technology can be practiced in many ways, however detailed the description may appear. The embodiments may differ considerably in the details of their implementation, but are still included in this specification. Specific terms used in describing certain features or aspects of the various embodiments should not be construed as implying that these terms are redefined as limited to any particular characteristic, feature, or aspect of the technology associated with that term. Generally, the terms used in the following claims should not be construed as limiting the technology to the specific embodiments disclosed in the specification, unless such terms are expressly defined herein. Therefore, the actual scope of this technology includes not only the disclosed embodiments but also all equivalent methods of implementing or practicing said embodiments.

[0107] The language used in this specification has been chosen primarily for readability and pedagogical purposes and may not have been selected to depict or define the subject matter. Therefore, the scope of this technology is not limited by this detailed description, but rather by any claims published on the application based on this detailed description. Thus, the disclosure of various embodiments is intended to illustrate, but not limit, the scope of the technology described in the following claims.

Claims

1. A computer-implemented method for detecting long-tailed objects, comprising: Obtain a dataset comprising the label head category appearing in more than a threshold number of samples and the label tail category appearing in fewer than a threshold number of samples; A detection model is pre-trained on head category images in the dataset, wherein the detection model includes: The backbone network used for feature extraction; and The detection head includes a classifier module and a regressor module for bounding box prediction; The pre-trained detection model is enhanced by pseudo-labeling unlabeled images, and the detection model is adapted to tail categories by initializing the tail parameters of the detection model from the pre-trained model and fine-tuning it using tail category images; and The detection model is fine-tuned on a balanced dataset, which includes head and tail categories, wherein the fine-tuning updates the classifier module and the regressor module while freezing the parameters of the backbone network.

2. The computer-implemented method according to claim 1, wherein, The detector model also includes a loss function that combines classification loss and regression loss.

3. The computer-implemented method according to claim 1, wherein, The backbone network is selected from the ResNet family or a converter-based architecture.

4. The computer-implemented method according to claim 1, wherein, The detection head incorporates a Feature Pyramid Network (FPN) for multi-scale feature extraction.

5. The computer-implemented method according to claim 1, wherein, The pseudo-marker is executed through the following steps: A teacher-student framework is adopted, where the teacher model generates pseudo-labels for the student model; and Use a confidence threshold to select reliable pseudo-labels.

6. The computer-implemented method according to claim 1, wherein, The pseudo-tags also include enhancing the untagted image through the following automatic operations: Copy rare instances from tagged images; Paste the copied rare instances onto an unlabeled image to create a composite scene; and Apply one or more of the geometric and photometric transformations.

7. The computer-implemented method according to claim 6, wherein, Applying one or more of the geometric and photometric transformations includes: resizing, rotation, translation, and contrast adjustment.

8. The computer-implemented method according to claim 1, wherein, The pre-training includes training on an imbalanced head dataset using a data sampling strategy that combines the following operations: A simple copy-paste (SCP) enhancement for generating additional tagged instances of rare head classes; and Add repeat factor sampling (RFS) to the representation of underrepresented head categories.

9. The computer-implemented method according to claim 1, wherein, Fine-tuning includes: Freeze the backbone network and update the classifier and regressor weights of the detection head; By enhancing scenes with rare objects, additional unlabeled data containing rare category instances can be merged. as well as Loss function for optimal combination.

10. The computer-implemented method according to claim 9, wherein, The loss functions for optimizing the combination include: Determine the supervised loss for the labeled tail image; and The pseudo-label loss of unlabeled data.

11. The computer-implemented method according to claim 1, wherein, The balanced datasets used for fine-tuning include: A subset of head category images randomly sampled based on a predetermined shot count; and If the tail portion of the dataset has fewer samples than the shot count, the balanced dataset used for fine-tuning includes images of all available tail categories.

12. A computer-implemented system for detecting long-tailed objects, comprising: A memory that stores a dataset with labeled head categories, labeled tail categories, and unlabeled images; Processing unit, the processing unit executes instructions to: Pre-train a detection model on head category images by optimizing feature extraction and head component detection; The teacher model is used to generate pseudo-labels for unlabeled images, wherein the pseudo-labels are filtered based on confidence scores; Tail category training data can be enhanced by copying and pasting rare category instances onto different background images; The detection model was fine-tuned on a balanced dataset that included sampled head and tail category images; as well as The head and tail category detection parameters are merged and unified into a single inference model.

13. The system according to claim 12, wherein, The detection model includes a converter-based architecture, which includes: Encoder-decoder structure used to generate object queries; Multi-scale feature maps for detecting objects at different scales; and Configured to output the bounding box regressor and class probability predictor for the final detection.

14. The system of claim 12, further comprising a teacher-student framework for pseudo-labeling, the teacher-student framework comprising: Teacher model updated via exponential moving average of student model parameters; A mechanism for applying data augmentation to unlabeled images to generate robust pseudo-labels; as well as An adaptive weighting factor for pseudo-label loss that balances the contributions of labeled and unlabeled data.

15. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause a system to perform a method comprising the following operations: Obtain a dataset comprising the label head category appearing in more than a threshold number of samples and the label tail category appearing in fewer than a threshold number of samples; A detection model is pre-trained on head category images in the dataset, wherein the detection model includes: The backbone network used for feature extraction; and The detection head includes a classifier module and a regressor module for bounding box prediction; The pre-trained detection model is enhanced by pseudo-labeling unlabeled images, and the detection model is adapted to tail categories by initializing the tail parameters of the detection model from the pre-trained model and fine-tuning it using tail category images; and The detection model is fine-tuned on a balanced dataset, which includes head and tail categories, wherein the fine-tuning updates the classifier module and the regressor module while freezing the parameters of the backbone network.

16. The non-transitory computer-readable medium according to claim 15, wherein: The detector model also includes a loss function that combines classification loss and regression loss; The backbone network is selected from the ResNet family or a converter-based architecture; and The detection head incorporates a Feature Pyramid Network (FPN) for multi-scale feature extraction.

17. The non-transitory computer-readable medium according to claim 15, wherein, The pseudo-marker is executed through the following steps: A teacher-student framework is adopted, where the teacher model generates pseudo-labels for the student model; and Use a confidence threshold to select reliable pseudo-labels.

18. The non-transitory computer-readable medium according to claim 15, wherein, The pseudo-tags also include enhancing the untagted image through the following operations: Copy rare instances from tagged images; Paste the copied rare instances onto an unlabeled image to create a composite scene; and Apply one or more of the geometric and photometric transformations.

19. The non-transitory computer-readable medium according to claim 15, wherein, The pre-training includes training on an imbalanced head dataset using a data sampling strategy that combines the following operations: A simple copy-paste (SCP) enhancement for generating additional tagged instances of rare head classes; and Add repeat factor sampling (RFS) to the representation of underrepresented head categories.

20. The non-transitory computer-readable medium according to claim 15, wherein, Fine-tuning includes: Freeze the backbone network and update the classifier and regressor weights of the detection head; By enhancing scenes with rare objects, additional unlabeled data containing rare category instances can be merged. as well as The loss function for optimizing a combination includes: Determine the supervised loss for the labeled tail image; as well as The pseudo-label loss of unlabeled data.

Images