Small sample target detection method and device, computer device and storage medium
By performing feature extraction and clustering on labeled datasets of existing and new scenarios, a small-sample object detection model is constructed and fine-tuned, which solves the problem of poor model adaptability in new scenarios and achieves efficient object detection in new scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SPEEDBOT ROBOTICS CO LTD
- Filing Date
- 2023-03-21
- Publication Date
- 2026-07-24
AI Technical Summary
Existing object detection methods struggle to effectively transfer knowledge from existing scenarios to new ones, leading to a decline in detection performance in new environments.
By acquiring labeled datasets of existing and new scenarios, feature extraction and clustering are performed to construct a few-sample object detection model that includes a general object detection model, a foreground enhancement branch, and an attention module. The model is then fine-tuned using a small amount of data to improve its adaptability to new scenarios.
In new scenarios, rapid model deployment can be achieved with a small amount of labeled data, improving target detection performance and enhancing the model's adaptability and detection capabilities for new scenarios.
Smart Images
Figure CN116452822B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of machine learning technology, and particularly relates to a method, apparatus, computer device, and storage medium for small-sample object detection. Background Technology
[0002] With the development of computer technology and the widespread application of computer vision principles, research on real-time target tracking using computer image processing techniques has become increasingly popular. Existing target detection algorithms for small sample sizes mainly include metric learning-based models and meta-learning-based models. Metric learning-based models train new models by comparing the similarity of different samples, and training is highly dependent on the quality of positive and negative sample pairs. Meta-learning-based models can learn the learning process of new scenes, but it is difficult to maintain performance in existing scenes.
[0003] Existing object detection methods are primarily designed for detecting new object categories. However, in many real-world industrial scenarios, the challenge lies in addressing the issue of small-sample deployment in these new environments. New industrial scenarios, not yet deployed, typically have very limited labeled data, making it difficult for existing methods to transfer knowledge from existing scenarios to new ones. Consequently, the trained models are not accurately suited for object detection in these new scenarios, actually reducing detection performance. Therefore, improving object detection performance in new scenarios is a crucial challenge. Summary of the Invention
[0004] Therefore, it is necessary to provide a small-sample target detection method, apparatus, computer equipment, and storage medium that can improve the target detection effect in response to the above-mentioned technical problems.
[0005] A small-sample target detection method, comprising:
[0006] Obtain the first labeled dataset for the existing scenario and the second labeled dataset for the new scenario;
[0007] Feature extraction is performed on the first labeled dataset and the second labeled dataset respectively, and the extracted features are clustered to obtain prior features;
[0008] A few-shot object detection model is trained using the first labeled dataset and the prior features. The few-shot object detection model includes a general object detection model, a foreground enhancement branch, and an attention module.
[0009] A preset number of fine-tuning data are selected from the first labeled dataset and the second labeled dataset respectively, and the trained small sample target detection model is fine-tuned using the fine-tuning data;
[0010] The finely tuned small sample target detection model is invoked to perform target detection on the image data in the new scene.
[0011] In one embodiment, the step of extracting features from the first labeled dataset and the second labeled dataset respectively, and clustering the extracted features to obtain prior features, includes:
[0012] The first labeled dataset and the second labeled dataset are respectively input into the general object detection model to obtain the features output before the output layer in the general object detection model;
[0013] K-means clustering is performed on the features to obtain prior features.
[0014] In one embodiment, the foreground enhancement branch is positioned after the downsampling structure in the general object detection model, with the input being the deep features output by the downsampling in the general object detection model and the output being a mask of the target bounding box.
[0015] In one embodiment, the attention module is positioned before the output layer in the general object detection model to enhance the input features using a cross-attention mechanism.
[0016] In one embodiment, the foreground enhancement branch includes a 3x3 convolutional kernel.
[0017] In one embodiment, the general object detection model includes any one of YOLO, YOLOX, and Faster R-CNN.
[0018] A small-sample target detection device, comprising:
[0019] The acquisition module is used to acquire the first labeled dataset for an existing scenario and the second labeled dataset for a new scenario.
[0020] The feature extraction module is used to extract features from the first labeled dataset and the second labeled dataset respectively, and to cluster the extracted features to obtain prior features;
[0021] The pre-training module is used to train a few-shot object detection model using the first labeled dataset and the clustering features. The few-shot object detection model includes a general object detection model, a foreground enhancement branch, and an attention module.
[0022] The fine-tuning module is used to select a preset number of fine-tuning data from the first labeled dataset and the second labeled dataset respectively, and to fine-tune the trained small sample target detection model using the fine-tuning data;
[0023] The object detection module is used to call the fine-tuned small sample object detection model to perform object detection on the image data in the new scene.
[0024] In one embodiment, the feature extraction module is further configured to input the first labeled dataset and the second labeled dataset into a general object detection model, respectively, to obtain the features output before the output layer in the general object detection model; and to perform K-means clustering on the features to obtain prior features.
[0025] The present invention also provides a computer device, the computer device including a processor and a memory, the memory storing a computer program, the processor executing the computer program to implement the steps of the small sample target detection method described in any of the above claims.
[0026] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the small sample target detection method described in any of the preceding claims.
[0027] The aforementioned few-shot object detection method, apparatus, computer equipment, and storage medium acquire a first labeled dataset for an existing scene and a second labeled dataset for a new scene. Features are then extracted from both datasets, and the extracted features are clustered to obtain prior features. A few-shot object detection model, including a general object detection model, a foreground enhancement branch, and an attention module, is then trained using the first labeled dataset and the prior features. Finally, a small number of data points are selected from both datasets for fine-tuning to obtain the fully trained few-shot object detection model. This method enables rapid model deployment with limited labeled data when facing new scenes with only a few samples. Furthermore, by using prior information for clustering and associating the clustered features with image features through attention, the prior information is linked to the image before prediction, thereby improving the model's adaptability to new scenes and enhancing the object detection performance in new scenes. Attached Figure Description
[0028] Figure 1 This is a flowchart illustrating a small-sample target detection method in one embodiment.
[0029] Figure 2 This is a schematic diagram of scene data in one embodiment.
[0030] Figure 3 This is a schematic diagram of the structure of a small sample target detection network in one embodiment.
[0031] Figure 4 This is a structural block diagram of a small sample target detection device in one embodiment. Detailed Implementation
[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0033] In one embodiment, such as Figure 1 As shown, a small-sample target detection method is provided, including the following steps:
[0034] Step S101: Obtain the first labeled dataset under the existing scene and the second labeled dataset under the new scene.
[0035] The first labeled dataset refers to the labeled dataset under the existing scenario, and the second labeled dataset refers to the labeled dataset under the new scenario.
[0036] Specifically, taking an industrial scenario as an example, firstly, images of components in a known scenario are acquired, and then these images are labeled to create a labeled dataset for the existing scenario, resulting in the first labeled dataset. Similarly, images of components in a new scenario are collected and manually labeled to obtain the second labeled dataset. Since the labeled data for the new scenario is very limited, the second labeled dataset only requires a small number of images. For example, assuming known scenarios A, B, and C, and a new scenario D, then we can obtain the first labeled dataset DA, the first labeled dataset DB, the first labeled dataset DC, and the second labeled dataset DD. Figure 2 As shown, a scenario data diagram is provided.
[0037] Step S102: Extract features from the first labeled dataset and the second labeled dataset respectively, and cluster the extracted features to obtain prior features.
[0038] Specifically, after obtaining the first and second labeled datasets, feature extraction is performed on both datasets. Then, a clustering algorithm is used to cluster the extracted features, and the clustered features are used as prior features. In this embodiment, any existing clustering algorithm can be used. Furthermore, the labeled dataset for feature extraction can be the entire dataset, or a certain number of data points can be randomly sampled from it. For example, 100 images can be randomly selected for feature extraction.
[0039] In one embodiment, step S102 includes: inputting the first labeled dataset and the second labeled dataset into the general object detection model respectively, obtaining the features output before the output layer in the general object detection model; performing K-means clustering on the features to obtain prior features.
[0040] Specifically, for the labeled dataset, an existing general object detection model is called, i.e., a model commonly used for object detection, such as YOLO, YOLOX, or Faster R-CNN. The labeled dataset is then input into the called model, and the existing object detection model extracts features from this data. In this embodiment, the extracted features are those before the prediction layer of the general object detection model, i.e., before the output layer. For the existing object detection model structure, this can be understood as the feature result after the pyramid structure. Then, the extracted features are clustered using a clustering algorithm. This embodiment preferably uses the K-means clustering algorithm. The clustered features are denoted as prior features. For example, features FA, FB, FC, and FD are extracted from the datasets DA, DB, DC, and DD. Then, features FA, FB, FC, and FD are clustered to obtain prior features CA, CB, CC, and CD.
[0041] Step S103: Train a few-shot object detection model using the first labeled dataset and prior features. The few-shot object detection model includes a general object detection model, a foreground enhancement branch, and an attention module.
[0042] Specifically, a small-sample object detection model is pre-constructed. In this embodiment, the small-sample object detection model consists of a general object detection model, a foreground enhancement branch, and an attention module. The general object detection model is preferably the YOLOX model. The foreground enhancement branch is positioned after the downsampling structure in the general object detection model. Its input features are the deep features output after downsampling in the general object detection model, and its output is the mask of the target bounding box. That is, by drawing deep features from the backbone structure of the general object detection model, the foreground object features are enhanced. Its input is the deep features output by the downsampling structure, and its output label is the mask generated from the target bounding box. In this embodiment, the foreground enhancement branch uses the DICE loss function. The foreground enhancement branch is preferably a single convolutional layer with a kernel size of 3x3. The attention module is positioned before the output layer of the general object detection model, that is, before the prediction layer. The attention module enhances the input features before they are input into the prediction layer for processing. In this embodiment, the attention mechanism used by the attention module is the cross-attention mechanism of the transformer. Figure 3 The diagram shows a schematic of a small-sample target detection network structure. Figure 3 The model shown is formed by adding a 3x3 convolutional layer after the backbone network of the general object detection model, and adding an attention module after the pyramid structure of the general object detection model.
[0043] Furthermore, since the few-sample object detection model in this embodiment adds a foreground enhancement branch to the traditional general object detection model, the few-sample object detection module in this embodiment has a total of three output branches. One branch is used to predict the position and size of the detection box, denoted by p. box =(x,y,h,w) represents the center coordinates and width and height of the detection box. Another branch is used to predict the category of the target, using p. cls Similarly, y box and y cls This represents the bounding box label and class label of the target. There is also a foreground enhancement branch, which predicts the foreground binary mask of the region represented by the target bounding box, using p... fg and y fg These represent the model prediction and the label, respectively. Therefore, the loss function used consists of the loss of these three parts: L det =λ1L box (p box ,y box )+λ2L cls (p cls ,y cls )+λ3L fg (p fg ,y fg ), where L box The loss function used to represent the detection box is typically L2 loss or smoothL1 loss; L cls To represent classification loss, cross-entropy loss is generally used; L fg This represents the loss function for the foreground mask, calculated using the Dice loss. λ1, λ2, and λ3 represent the weight coefficients of the three losses, predefined before training, with a default value of 1. The network is optimized using stochastic gradient descent until the network loss converges.
[0044] Step S104: Select a preset number of fine-tuning data from the first labeled dataset and the second labeled dataset respectively, and use the fine-tuning data to fine-tune the trained small sample target detection model.
[0045] Specifically, after pre-training the few-shot object detection model, a predetermined number of images (ideally no more than 50) are used on the new scene dataset DD. Simultaneously, to prevent performance degradation on the DA, DB, and DC datasets and to enhance the model's generalization ability, the same small amount of data is also selected on DA, DB, and DC. Then, the trained few-shot object detection model is fine-tuned using the selected data. During the fine-tuning phase, except for the prediction layer, all other parameters are fixed, the learning rate is halved, and the same loss function is used to fine-tune the model. In other words, the training process of the few-shot object detection model in this embodiment includes two stages: pre-training and fine-tuning. Pre-training uses a large amount of labeled data from existing scenes and prior features obtained through clustering, without directly accessing labeled data from the new scene. Fine-tuning uses a small amount of labeled data from both existing and new scenes.
[0046] Step S105: Call the fine-tuned small sample target detection model to perform target detection on the image data in the new scene.
[0047] Specifically, after the pre-training and fine-tuning of the few-sample object detection model are completed, the model is fixedly deployed for object detection. When new image data for a new scene requiring object detection is obtained, the model is invoked to perform object detection on that image data.
[0048] This embodiment also compares the object detection performance of the few-shot object detection model through experiments. During the experiments, for known factory-collected parts datasets A, B, and C, approximately 5000 images were manually labeled to construct training sets A, B, and C, and test sets A, B, and C, respectively. Simultaneously, for a new factory-collected dataset D, 50 images were labeled as the training set D, and another 100 images were labeled as the test set D. Then, experiments were conducted using a general object detection model and the few-shot object detection model constructed in this embodiment, with YOLOX-S as the general model. The comparison results are shown in Table 1 below:
[0049] Table 1
[0050]
[0051] As shown in Table 1 above, the model was trained and compared using a large amount of data from existing scenarios. Since the model constructed in this embodiment includes a foreground enhancement component and a feature association component, it already outperforms the original YOLOX-S across all scenarios. Both foreground enhancement and feature association are effective. Compared to the original YOLOX-S, the foreground enhancement component achieves performance improvements of 1.2, 1.0, and 2.3 in the three scenarios, respectively, while the feature association module achieves improvements of 2.1, 2.0, and 4.1. By combining these two modules, the model constructed in this paper achieves performance improvements of 3.1, 2.9, and 5.0 in the three scenarios, respectively.
[0052] Next, fine-tuning and comparison were performed on a new scene D with only 50 training images, and the results are shown in Table 2 below. It can be seen that the few-shot model constructed in this paper maintains its performance on test sets A, B, and C, and surpasses the original YOLO-S model by 4.3 points on the new scene test set D. This indicates that the constructed few-shot model can learn prior knowledge from known scenes and transfer this prior knowledge to new scenes, thus enhancing the object detection effect.
[0053]
[0054] The aforementioned few-shot object detection method acquires a first-label dataset for an existing scene and a second-label dataset for a new scene. Features are extracted from both datasets, and the extracted features are clustered to obtain prior features. Then, a few-shot object detection model, including a general object detection model, a foreground enhancement branch, and an attention module, is trained using the first-label dataset and the prior features. Subsequently, a small amount of data is selected from both datasets for fine-tuning to obtain the fully trained few-shot object detection model. This method enables rapid model deployment with limited labeled data when facing new scenes with only a few samples. Furthermore, by using prior information for clustering and associating the clustered features with image features through attention, the prior information is linked to the image before prediction, thereby improving the model's adaptability to new scenes and enhancing the object detection performance in new scenes.
[0055] In one embodiment, such as Figure 4 As shown, a small sample target detection device is provided, comprising:
[0056] The acquisition module 401 is used to acquire the first labeled dataset under the existing scene and the second labeled dataset under the new scene.
[0057] The feature extraction module 402 is used to extract features from the first labeled dataset and the second labeled dataset respectively, and to cluster the extracted features to obtain prior features.
[0058] The pre-training module 403 is used to train a few-shot object detection model using the first labeled dataset and clustering features. The few-shot object detection model includes a general object detection model, a foreground enhancement branch, and an attention module.
[0059] The fine-tuning module 404 is used to select a preset number of fine-tuning data from the first labeled dataset and the second labeled dataset respectively, and to fine-tune the trained small sample target detection model using the fine-tuning data.
[0060] The object detection module 405 is used to call the fine-tuned small sample object detection model to perform object detection on image data in new scenes.
[0061] In one embodiment, the feature extraction module 402 is further configured to input the first labeled dataset and the second labeled dataset into the general object detection model respectively, obtain the features output before the output layer in the general object detection model; and perform K-means clustering on the features to obtain prior features.
[0062] Specific limitations regarding the small-sample target detection device can be found in the limitations of the small-sample target detection method described above, and will not be repeated here. Each module in the aforementioned small-sample target detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various small-sample target detection method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms.
[0063] In one embodiment, a computer device is provided, which may be a server, including a processor, memory, and a network interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a small-sample target detection method. Exemplarily, the computer program may be divided into one or more modules, one or more of which are stored in memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the computer device. The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the computer device, connecting various parts of the computer device via various interfaces and lines.
[0064] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0065] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0066] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. A method for detecting targets in small samples, characterized in that, include: Obtain the first labeled dataset for the existing scenario and the second labeled dataset for the new scenario; Feature extraction is performed on the first labeled dataset and the second labeled dataset respectively, and the extracted features are clustered to obtain prior features; A few-shot object detection model is trained using the first labeled dataset and the prior features. The few-shot object detection model includes a general object detection model, a foreground enhancement branch, and an attention module. The foreground enhancement branch is positioned after the downsampling structure in the general object detection model. Its input is the deep features output by the downsampling in the general object detection model, and its output is a mask of the target box. The attention module is positioned before the output layer in the general object detection model and uses a cross-attention mechanism to enhance the input features. A preset number of fine-tuning data are selected from the first labeled dataset and the second labeled dataset respectively, and the trained small sample target detection model is fine-tuned using the fine-tuning data; The finely tuned small sample target detection model is invoked to perform target detection on the image data in the new scene.
2. The method according to claim 1, characterized in that, The step of extracting features from the first labeled dataset and the second labeled dataset respectively, and clustering the extracted features to obtain prior features, includes: The first labeled dataset and the second labeled dataset are respectively input into the general object detection model to obtain the features output before the output layer in the general object detection model; Clustering the features yields prior features.
3. The method according to claim 1, characterized in that, The few-shot object detection module has three branches of output: one branch predicts the position and size of the detection box, another branch predicts the object category, and the third branch, the foreground enhancement branch, predicts the foreground binary mask of the region represented by the object box. The loss function used by the few-shot object detection module consists of the loss from these three branches.
4. The method according to claim 3, characterized in that, The loss function is expressed as: ； in, This represents the center point coordinates (x, y) and the length and width (h, w) of the detection box. The label of the detection box representing the target. The loss function representing the detection box; Indicates the category of the predicted target. Category labels representing the target Indicates classification loss; This indicates the model's prediction. Indicates model label, The loss function representing the foreground mask; , and These represent the weighting coefficients for the three losses.
5. The method according to claim 4, characterized in that, The loss functions are as follows: L2 loss or smoothL1 loss is used for the detection box loss; cross-entropy loss is used for the classification loss; and Dice loss is used for the foreground mask loss.
6. The method according to claim 1, characterized in that, The foreground enhancement branch includes a 3*3 convolutional kernel; the general object detection model includes any one of YOLO, YOLOX, or Faster R-CNN.
7. The method according to any one of claims 1-6, characterized in that, During the fine-tuning phase, except for the prediction layer, the other parameters are fixed, the learning rate is halved, and the same loss function is used to fine-tune the model.
8. A small sample target detection device, characterized in that, The apparatus for implementing the method of any one of claims 1-7 comprises: The acquisition module is used to acquire the first labeled dataset for an existing scenario and the second labeled dataset for a new scenario. The feature extraction module is used to extract features from the first labeled dataset and the second labeled dataset respectively, and to cluster the extracted features to obtain prior features; The pre-training module is used to train a few-shot object detection model using the first labeled dataset and the prior features. The few-shot object detection model includes a general object detection model, a foreground enhancement branch, and an attention module. The fine-tuning module is used to select a preset number of fine-tuning data from the first labeled dataset and the second labeled dataset respectively, and to fine-tune the trained small sample target detection model using the fine-tuning data; The object detection module is used to call the fine-tuned small sample object detection model to perform object detection on the image data in the new scene.
9. A computer device comprising a processor and a memory, the memory storing a computer program, characterized in that, The processor is used to implement the small sample target detection method according to any one of claims 1-7 when executing the computer program.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the small sample target detection method according to any one of claims 1-7.