Online dynamic learning model based on YOLO target detection model

By using an online dynamic learning model based on the YOLO object detection model, combined with a text-image interaction feature enhancement network of a large language model and a teacher-student model, automatic label generation and high-quality object data augmentation, the problems of object detection model adaptability and storage cost on new data are solved, and efficient self-supervised training and dynamic learning are achieved.

CN120894646AActive Publication Date: 2025-11-04SHENZHEN UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202510991324.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-07-17
Publication Date
2025-11-04
Estimated Expiration
2045-07-17

AI Technical Summary

Technical Problem

Existing object detection models perform poorly when faced with new data, are prone to catastrophic forgetting problems, and have tedious and time-consuming data labeling, resulting in high storage costs and making them difficult to adapt to complex scenarios and fine-grained object detection.

Method used

We employ an online dynamic learning model based on the YOLO object detection model, combined with a text-image interaction feature enhancement network of a large language model and a teacher-student model, automatic label generation and high-quality object data augmentation, and self-supervised training by constraining the model output through a loss function.

Benefits of technology

It effectively alleviates the problem of catastrophic forgetting, reduces data annotation costs, improves the model's dynamic adaptability and generalization ability, and reduces storage costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120894646A_ABST
    Figure CN120894646A_ABST
Patent Text Reader

Abstract

The invention relates to the field of target detection, and particularly discloses an online dynamic learning model based on a YOLO target detection model. According to the model, a text-image interaction feature enhancement network based on a language large model and a teacher-student model, automatic tag generation based on the large model and data enhancement based on a detected high-quality target are used for constructing an overall architecture of the YOLO online dynamic learning model; and online dynamic learning of the YOLO target detection model is carried out based on the overall architecture of the online dynamic learning model of the YOLO. A text-image interaction network is enhanced in combination with a large language model and a teacher-student model framework, and a high-quality target frame is screened for data enhancement. A bimodal large model driven labeling method is used, a language large model and a teacher and student model are combined to improve generalization ability, and a data enhancement strategy based on a high-quality target is provided. The method effectively relieves the problem of disastrous forgetting, reduces the marking cost, improves the model adaptability, and is widely applied to real-time target detection tasks in complex scenes.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and particularly to an online dynamic learning model based on a YOLO target detection model. BACKGROUND

[0002] In recent years, deep learning has made significant progress in target detection, and the YOLO (You Only Look Once) series algorithm has become a popular choice in the field of target detection due to its efficient and real-time detection performance. However, as the application scenarios become more diverse and complex, target detection models need to continuously adapt to new data and new environments, which puts higher requirements on the dynamic learning ability and generalization ability of the models. In the training and optimization process of target detection models, traditional methods often rely on a large amount of labeled data, and once the model is trained, it is difficult to effectively adapt to new data. At the same time, for the detection of complex scenes and fine-grained targets, existing models still have deficiencies.

[0003] Currently, deep learning networks based on CNN and Transformer have become very mature in target detection, and many works have improved the YOLO network, such as introducing more complex feature extraction networks and optimizing the design of the detection head. However, these methods still have some defects:

[0004] 1) Traditional target detection models often perform poorly when facing new data, and after fine-tuning, they are prone to catastrophic forgetting problems;

[0005] 2) The target detection task requires a large amount of labeled data, especially for new things, and the labeling process is tedious and time-consuming, and it is difficult to ensure the accuracy of the labeling;

[0006] 3) To alleviate the catastrophic forgetting problem, existing dynamic learning methods often use data playback, which greatly increases the storage cost. SUMMARY

[0007] The present application proposes an online dynamic learning model based on a YOLO target detection model to address the above problems in the prior art, which can alleviate the catastrophic forgetting problem that traditional target detection models are prone to in the incremental learning process, effectively reduce data storage costs, and enhance the model's ability to adapt to dynamic environments and learning ability.

[0008] To achieve the above object, the application provides an online dynamic learning model based on a YOLO target detection model, which uses a text-image interaction feature enhancement network based on a language large model and a teacher-student model, automatic label generation based on a large model, and data enhancement based on detected high-quality targets to construct the overall architecture of the YOLO online dynamic learning model, and performs online dynamic learning of the YOLO target detection model based on the overall architecture of the YOLO online dynamic learning model.

[0009] Preferably, the model as a whole uses a loss function to constrain the output result of the model, the loss function of the model includes a contrast loss function of image features and text features, a teacher-student model consistency loss function and a YOLO loss function, the YOLO loss function includes a multi-classification loss function BCE and a positioning loss CIoU function.

[0010] Preferably, the expression of the contrast loss function of the image features and the text features is:

[0011] s j,k =α·L 2-Norm (e j )·L 2-Norm (w k ) T +β;

[0012] In the formula, s j,k is the contrast loss function of the image features and the text features, L 2-Norm (·) is the L2 normalization, e j ∈J is the image feature of the jth target, J represents the number of image target object features, w k ∈K is the kth text feature, K represents the number of text target object features, alpha is a weight coefficient, beta is a displacement factor, and j and k are the indices of J and K.

[0013] J is determined by the positioning loss function CIoU to determine the boundary regression frame and the image features in the corresponding frame, and J is regarded as the image target object feature; the number K of text target object features is directly obtained from the target object explicitly stated in the text;

[0014] The expression of the teacher-student model consistency loss function is a mean square error loss:

[0015]

[0016] In the formula, y i is the feature output by the teacher model, is the feature output by the student model, and n is the number of feature channels.

[0017] Preferably, the step of dynamic learning of the model using the overall architecture of the YOLO-based online dynamic learning model comprises:

[0018] S1, using two large models of Grounding-DINO and Segment-Anything to generate pseudo-label annotations, intercepting target images with high accuracy of model output, and storing them into a database, and generating new image data by image synthesis;

[0019] S2, using a large model BLIP to generate corresponding text from images;

[0020] S3, the text is encoded by the text encoder CLIP-Encoder of the CLIP model to obtain text features, and the image is feature-extracted by the backbone network YOLO-Backbone of YOLO to obtain image features;

[0021] S4, input the text features and image features into two text-image feature enhancement modules for processing; wherein the two text-image feature enhancement modules are divided into a teacher model of a frozen feature enhancement module, and another is a student model trained with new data, and the outputs of the teacher model and the student model are constrained by a teacher-student model consistency loss function;

[0022] S5, calculating the similarity of text features and image features in the text area matching, and using a contrast loss function of image features and text features to constrain the text features and image features of the student model;

[0023] S6, outputting results by YOLO-Head, and using a multi-classification loss function BCE and a positioning loss function CIoU in a YOLO loss function for constraint.

[0024] Preferably, in S4, the teacher model of the frozen feature enhancement module retains the parameters obtained by pre-training the model using the initial annotation data set, and the neural network layer parameters are not updated with training.

[0025] Preferably, in S4, the specific steps of inputting the text features and image features into the two text-image feature enhancement modules for processing are:

[0026] S41, using the text features and the image features of each scale to perform feature enhancement on the image features of each scale based on the cross-attention mechanism for the multi-scale image features output by the YOLO encoder;

[0027] S42, for the multi-scale image features output by the YOLO encoder, the multi-scale image features are pooled to the same scale for splicing, and the text features are feature-enhanced based on the cross-attention mechanism.

[0028] Preferably, in S6, the calculation formula of the multi-classification loss function BCE is:

[0029]

[0030] In the formula, N is the number of target boxes, C is the number of classes, y i,c is the true label 0 or 1 of the ith target box belonging to the cth class, p i,c is the probability of the ith target box predicted by the model belonging to the cth class.

[0031] The calculation formula of the positioning loss CIoU is:

[0032]

[0033] In the formula, IoU is the intersection over union between the predicted box and the real box, p 2 (b,b gt ) is the square of the Euclidean distance between the center point of the predicted box b and the center point of the real box b gt , c is the length of the diagonal of the smallest closed region covering the predicted box and the real box, a is a weight coefficient, and v is a parameter for measuring the consistency of the width-height ratio, defined as:

[0034]

[0035] where w gt and h gt are the width and height of the real box, and w and h are the width and height of the predicted box.

[0036] Preferably, the automatic label generation based on the large model constitutes a dual-modal large model driven labeling method framework, and the labeling method framework is composed of a text driven labeling generation system and an online learning feedback loop, wherein the specific steps of the text driven labeling generation system and the online learning feedback loop are:

[0037] Step one, cooperate with the target detection based on Grounding-DINO to realize open vocabulary target recognition, perform open vocabulary target recognition and instance segmentation, encode the text prompt into spatial attention weight, and interact with the image feature to obtain semantic guided target positioning;

[0038] Step two, use the encoder of Segment-Anything to receive the image and the coordinates of the detection box to obtain position-semantics encoded instance segmentation.

[0039] Step three, after obtaining the final automatic labeling image containing the labeling box and the mask, the picture is converted into YOLO standard labeling information.

[0040] Preferably, the data enhancement based on the detected high-quality target is realized by the following steps:

[0041] Step one, screening the target frame whose accuracy based on the YOLO target detection model output exceeds the preset threshold, and extracting the high-quality local image quality;

[0042] Step two, extracting the high-quality local image from the original image, and retaining the complete information of the target object;

[0043] Step three, using the method of splicing multiple target images or weighted sum of the target image and a complete image non-target region, combining multiple high-quality local images into a new image data set, increasing the diversity and complexity of the data;

[0044] Step four, using the generated new image data set for model training.

[0045] Preferably, the formula used for weighted sum of the intercepted target image and a complete image is:

[0046]

[0047] In the formula, I is the original image, b is the intercepted target image, is the generated new image, u, v, is the coordinate of the image pixel point, and φ is the weight.

[0048] Therefore, the present application proposes an online dynamic learning model based on a YOLO target detection model, which has the following beneficial effects:

[0049] (1) The present application uses a language large model and a teacher-student model, and the student model can quickly adapt to new data, improve the generalization ability and robustness while maintaining the understanding ability of old data, and effectively alleviate the catastrophic forgetting problem.

[0050] (2) The automatic label generation method based on the large model of the present application can generate high-quality pseudo labels, realize self-supervised training, reduce data labeling cost and time, and improve model adaptability and flexibility.

[0051] (3) The data enhancement method based on high-quality target image of the present application generates diverse and complex new image data, reduces storage cost, and significantly improves the dynamic learning ability of the model.

[0052] The technical solutions of the present application will be further described in detail below with the help of the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is the overall architecture diagram of an online dynamic learning model based on a YOLO target detection model of the present application;

[0054] Figure 2 is a text-image feature enhancement module architecture of an online dynamic learning model based on a YOLO target detection model of the application;

[0055] Figure 3 is a double-modal large model driven labeling method framework diagram of an online dynamic learning model based on a YOLO target detection model of the application;

[0056] Figure 4 is a loss function composition structure diagram of an online dynamic learning model based on a YOLO target detection model of the application. DETAILED DESCRIPTION

[0057] In order to make the technical solutions, advantages and purposes of the application clearer, the technical solutions of the embodiments of the application will be described clearly and completely below. The described embodiments are part of the embodiments of the application, rather than all the embodiments. Based on the described embodiments of the application, all other embodiments obtained by those of ordinary skill in the art without any creative effort fall within the protection scope of the present application.

[0058] Unless otherwise defined, the technical terms or scientific terms used in the present application shall have the usual meanings understood by those skilled in the art to which the present application belongs.

[0059] As shown in Figures 1-4 , according to the online dynamic learning model based on the YOLO target detection model provided by the application, the model uses a text-image interaction feature enhancement network based on a language large model and a teacher-student model, automatic label generation based on a large model, and data enhancement based on detected high-quality targets to construct the overall architecture of the YOLO online dynamic learning model, and performs online dynamic learning of the YOLO target detection model based on the overall architecture of the YOLO online dynamic learning model.

[0060] The overall model uses a loss function to constrain the output results of the model, and the loss function of the model includes a contrast loss function of image features and text features, a teacher-student model consistency loss function, and a YOLO loss function, wherein the YOLO loss function includes a multi-classification loss function BCE and a positioning loss CIoU function.

[0061] The role of the contrast loss function of image features and text features is to align the image features and the text features.

[0062] The expression of the contrast loss function of image features and text features is:

[0063] s j,k =α·L 2-Norm (e j )·L2-Norm (w k ) T +β;

[0064] In the formula, s j,k is the contrast loss function of image features and text features, L 2-Norm (·) is the L2 normalization, e j is the image feature of the jth target, J represents the number of image target object features, w k is the kth text feature, K represents the number of text target object features, α is the weight coefficient, β is the displacement factor, and j and k are the indices of J and K;

[0065] wherein J is determined by the positioning loss function CIoU to determine the boundary regression frame and the image features in the corresponding frame, and J is regarded as the image target object feature; the number K of text target object features is directly obtained from the target object explicitly stated in the text;

[0066] The teacher-student model consistency loss function is the expression of the mean square error loss:

[0067]

[0068] In the formula, y i is the feature output by the teacher model, is the feature output by the student model, and n is the number of feature channels.

[0069] The steps of dynamic learning of the model using the overall architecture of the online dynamic learning model based on YOLO include:

[0070] S1, use the two large models Grounding-DINO and Segment-Anything to generate pseudo-label annotations, intercept the target images with high accuracy of model output, and store them into the database, and generate new image data by image synthesis;

[0071] S2, use the large model BLIP to generate the corresponding text from the image;

[0072] S3, encode the text by the text encoder CLIP-Encoder of the CLIP model to obtain the text feature, and extract the image feature by the backbone network YOLO-Backbone of YOLO to obtain the image feature;

[0073] S4, input the text feature and the image feature into two text-image feature enhancement modules for processing;

[0074] The two text-image feature enhancement modules are divided into a teacher model of a frozen feature enhancement module and a student model of new parameters trained by new data, and the outputs of the teacher model and the student model are constrained by a teacher-student model consistency loss function.

[0075] The teacher-student model consistency loss function is to constrain the parameter update of the student model (a new model trained by using the pseudo-labels generated by the two large models Grounding-DINO and Segment-Anything to annotate new data) by the teacher model (an old model pre-trained by using the initial labeled data set).

[0076] In S4, the teacher model of the frozen feature enhancement module retains the parameters obtained by pre-training the model by using the initial labeled data set, and the neural network layer parameters are not updated with the training, wherein the frozen module does not work in the target detection of the current task, but works in the training of the new model in the background.

[0077] In S4, the specific steps of inputting the text feature and the image feature to the two text-image feature enhancement modules for processing are as follows:

[0078] In S41, the text feature and the image feature of each scale are used to enhance the image feature of each scale based on the cross-attention mechanism.

[0079] In S42, the multi-scale image features output by the YOLO encoder are pooled to the same scale for splicing, and the text feature is enhanced based on the cross-attention mechanism.

[0080] In S5, the similarity between the text feature and the image feature is calculated in the text region matching, and the contrast loss function of the image feature and the text feature is used to constrain the text feature and the image feature of the student model.

[0081] In S6, the output result is output by the YOLO-Head, and the multi-classification loss function BCE and the positioning loss function CIoU in the YOLO loss function are used for constraint.

[0082] In S6, the calculation formula of the multi-classification loss function BCE is as follows:

[0083]

[0084] In the formula, N is the number of target boxes, C is the number of classes, y i,c is the true label 0 or 1 of the i-th target box belonging to the c-th class, p i,c is the probability of the i-th target box predicted by the model belonging to the c-th class.

[0085] The calculation formula of the positioning loss CIoU is as follows:

[0086]

[0087] In the formula, IoU is the intersection over union between the prediction box and the real box, p 2 (b, b gt ) is the squared Euclidean distance between the center point of the prediction box b and the center point of the real box b gt , c is the length of the diagonal of the smallest closed region covering the prediction box and the real box, a is a weight coefficient, and v is a parameter for measuring the consistency of the width-height ratio, which is defined as:

[0088]

[0089] where w gt and h gt are the width and height of the real box, and w and h are the width and height of the prediction box.

[0090] The automatic label generation based on the large model constitutes a dual-modal large model driven labeling method framework, and the labeling method framework is composed of a text driven labeling generation system and an online learning feedback loop, wherein the specific steps of the text driven labeling generation system and the online learning feedback loop are as follows:

[0091] Step one, cooperate with the Grounding-DINO based target detection to realize open vocabulary target recognition, perform open vocabulary target recognition and instance segmentation, encode the text prompt into spatial attention weight, and interact with the image feature to obtain semantic guided target positioning;

[0092] Step two, use the encoder of Segment-Anything to receive the image and the coordinates of the detection box, and obtain the position-semantic encoded instance segmentation;

[0093] Step three, after obtaining the final automatic labeling image containing the labeling box and the mask, the picture is converted into YOLO standard labeling information.

[0094] Data augmentation based on the detected high-quality target is realized through the following steps:

[0095] Step one, filter the target box whose accuracy output by the YOLO target detection model exceeds the preset threshold, and extract the high-quality local image quality;

[0096] Step two, extract the high-quality local image from the original image to retain the complete information of the target object;

[0097] Step three, use the method of splicing multiple target images or weighting sum of target images and non-target regions of a complete image to combine multiple high-quality local images into a new image dataset, increase the diversity and complexity of the data; the formula used for weighting sum of the intercepted target image and a complete image is:

[0098]

[0099] In the formula, I is the original image, b is the intercepted target image, is the generated new image, u, v, is the coordinate of the image pixel point, and φ is the weight;

[0100] Step four, use the generated new image dataset for model training.

[0101] Embodiment one,

[0102] According to the online dynamic learning model based on the YOLO target detection model provided by the application, the initial labeling dataset is used to train the model to obtain a preliminary target detection model. The model is used for the present target detection task, and new data and confidence are stored. In the background, two large models Grounding-DINO and Segment-Anything are used to generate pseudo-label annotations of new data, and another model with the same parameters as the present model is trained. When the average confidence of the output results of the model is higher than the average of the present model, the parameters of the present model can be updated by the parameters of the model, wherein the parameters of the model freezing module remain unchanged.

[0103] The specific experimental process is as follows:

[0104] I. Experimental setup

[0105] (1) Dataset: COCO2017

[0106] (2) Comparative model: set the YOLOv5s model (trained using COCO2017 labeled data), the YOLOv5s model improved by the text-image interaction feature enhancement module (trained using COCO2017 labeled data), the online dynamic learning model based on the YOLOv5s target detection model without using high-quality target data enhancement (trained using large model DINO+SAM labeled data), and the online dynamic learning model based on the YOLOv5s target detection model using high-quality target data enhancement (trained using large model DINO+SAM labeled data) as the comparison.

[0107] (3) Evaluation index:

[0108]

[0109] In the formula, TP is a positive example, and the number of instances classified as a positive example by the classifier; TN is a negative example, and the number of instances classified as a negative example by the classifier; FP is a negative example, but is classified as a positive example by the classifier; FN is a positive example, but is classified as a negative example by the classifier;

[0110] Experimental results

[0111]

[0112]

[0113] II. Result analysis

[0114] From the experimental results, it can be seen that the model using only the text-image interaction feature enhancement network has a significant improvement in AP compared to the YOLOv5s model. The network can better capture the semantic association between text and image by combining the language large model and the teacher-student model architecture, enhance the understanding and recognition ability of the model to the target, and thus improve the detection accuracy.

[0115] The automatic label generation method provides more high-quality labeled data for the model, although it is not as good as the original manual labeling data of the COCO2017 dataset, but it also solves the dependence on manual labeling in the unlabeled target detection and optimization process. The complete model combined with this method further improves the AP, indicating that the automatically generated pseudo-labels effectively enrich the training data, enabling the model to learn more comprehensive feature representations.

[0116] The high-quality target data enhancement method increases the diversity and complexity of the data by intercepting high-accuracy target images and synthesizing new data. This not only helps the model better learn the features of the target, but also improves the model's ability to adapt to different scenes and target changes. From the experimental results, it can be seen that the improvement of the complete model in AP also benefits from this data enhancement method, and the optimization of training time and inference time shows that this method improves performance without significantly increasing computational burden.

[0117] Therefore, the present application provides an online dynamic learning model based on a YOLO target detection model, which significantly enhances the generalization and robustness through a language large model and a teacher-student model, effectively alleviating the catastrophic forgetting problem. At the same time, the automatic label generation method based on the large model reduces the labeling cost and realizes self-supervised training. In addition, the data enhancement method improves the dynamic learning ability, reduces the storage cost, and effectively improves the adaptability and flexibility of the model.

[0118] It should be pointed out finally that the above examples are only used to illustrate the technical solutions of the present application but not to limit it, and although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can still be modified or replaced equivalently, and these modifications or equivalent replacements should not make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present application.

Claims

1. An online dynamic learning model based on the YOLO object detection model, characterized in that, The model uses three parts to construct the overall architecture of the YOLO online dynamic learning model: a text-image interaction feature enhancement network based on a large language model and a teacher-student model, automatic label generation based on the large model, and data augmentation based on detected high-quality targets. Based on the overall architecture of the YOLO online dynamic learning model, the YOLO object detection model is dynamically learned online.

2. The online dynamic learning model based on the YOLO object detection model according to claim 1, characterized in that, The model as a whole uses loss functions to constrain the output results of the model. The loss functions of the model include the contrast loss function of image features and text features, the teacher-student model consistency loss function, and the YOLO loss function. The YOLO loss function includes the multi-class loss function BCE and the localization loss function CIoU.

3. The online dynamic learning model based on the YOLO object detection model according to claim 2, characterized in that, The expression for the contrast loss function between the image features and the text features is as follows: s j,k =α·L 2-Norm (e j )·L 2-Norm (w k ) T +b; In the formula, s j,k L is the contrastive loss function for image features and text features. 2-Norm (·) represents the normalization of L2, e j ∈J represents the image features of the j-th target, where J represents the number of image target object features, w k ∈K is the k-th text feature, K represents the number of text target object features, α is the weight coefficient, β is the shift factor, and j and k are the indices of J and K; Where J is determined by the localization loss function CIoU to define the bounding regression box and the corresponding image features within the box, and J is regarded as the image target object feature; the number of text target object features K is directly obtained from the target objects that are clearly defined in the text. The expression for the mean squared error loss function of the teacher-student model consistency loss function is as follows: In the formula, y i Features output by the teacher model The features output by the student model are denoted by n, where n is the number of feature channels.

4. The online dynamic learning model based on the YOLO object detection model according to claim 1, characterized in that, The steps for dynamic learning using the overall architecture of the YOLO-based online dynamic learning model include: S1. Use two large models, Grounding-DINO and Segment-Anything, to generate pseudo-label annotations, extract the target images with high accuracy from the model output, store them in the database, and generate new image data through image synthesis. S2. Use the large model BLIP to generate corresponding text from the input image; S3. Text is encoded by CLIP-Encoder of CLIP model to obtain text features, and images are extracted by YOLO-Backbone network to obtain image features. S4. Input the text features and image features into two text-image feature enhancement modules for processing; wherein, the two text-image feature enhancement modules are a teacher model of the frozen feature enhancement module and a student model with new parameters trained by new data, and constrain the output of the teacher model and the student model through the teacher-student model consistency loss function; S5. Calculate the similarity between text features and image features in text region matching, and use the contrastive loss function of image features and text features to constrain the text features and image features of the student model. S6. Output the results from YOLO-Head and use the multi-class loss function BCE and the localization loss function CIoU from the YOLO loss function for constraints.

5. The online dynamic learning model based on the YOLO object detection model according to claim 4, characterized in that, In S4, the teacher model of the frozen feature enhancement module retains the parameters obtained by pre-training the model using the initial labeled dataset, and the neural network layer parameters are not updated with training.

6. The online dynamic learning model based on the YOLO object detection model according to claim 5, characterized in that, In S4, the specific steps for inputting text features and image features into two text-image feature enhancement modules for processing are as follows: S41. For the multi-scale image features output by the YOLO encoder, use text features and image features at each scale to perform feature enhancement on the image features at each scale based on a cross-attention mechanism; S42. For the multi-scale image features output by the YOLO encoder, pool the multi-scale image features to the same scale and concatenate them, and enhance the text features based on the cross-attention mechanism.

7. The online dynamic learning model based on the YOLO object detection model according to claim 4, characterized in that, In S6, the formula for calculating the multi-class loss function BCE is: In the formula, N is the number of bounding boxes, C is the number of categories, and y i,c The true label of the i-th bounding box belonging to the c-th category is either 0 or 1, p i,c It is the probability that the i-th bounding box predicted by the model belongs to the c-th category; The formula for calculating the localization loss CIoU is: In the formula, IoU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box, and ρ 2 (b,b gt ) is the center point of the predicted bounding box, and b is the center point of the ground truth bounding box. gt The square of the Euclidean distance between them, c is the diagonal length of the smallest closure region covering the predicted box and the ground truth box, α is the weighting coefficient, and v is a parameter that measures aspect ratio consistency, defined as: Among them, w gt and h gt is the width and height of the ground truth bounding box, and w and h are the width and height of the predicted bounding box.

8. The online dynamic learning model based on the YOLO object detection model according to claim 1, characterized in that, The automatic label generation based on the large model constitutes a bimodal large model-driven annotation method framework. This framework consists of a text-driven annotation generation system and an online learning feedback loop. The specific steps of the text-driven annotation generation system and the online learning feedback loop are as follows: Step 1: Use Grounding-DINO-based target detection to achieve open vocabulary target recognition, perform open vocabulary target recognition and instance segmentation, encode text prompts into spatial attention weights, and interact with image features to obtain semantically guided target localization; Step 2: Use the Segment-Anything encoder to receive the coordinates of the image and the detection box to obtain the location-semantic encoded instance segmentation; Step 3: After obtaining the final automatically annotated image containing bounding boxes and masks, transcode the image into YOLO standard annotation information.

9. The online dynamic learning model based on the YOLO object detection model according to claim 1, characterized in that, Data augmentation based on detected high-quality targets is achieved through the following steps: Step 1: Filter out target boxes whose accuracy output by the YOLO object detection model exceeds a preset threshold, and extract high-quality local image data. Step 2: Extract high-quality local images from the original image, preserving complete information about the target object; Step 3: By using the method of stitching multiple target images or weighted summation of the target image with the non-target region of a complete image, multiple high-quality local images are combined into a new image dataset, increasing the diversity and complexity of the data; Step 4: Use the generated new image dataset for model training.

10. The online dynamic learning model based on the YOLO object detection model according to claim 9, characterized in that, The formula used to weight and sum the extracted target image with a complete image is: In the formula, I represents the original image, and b represents the cropped target image. For the generated new image, u,v, φ represents the coordinates of the image pixels, and φ represents the weight.

Citation Information

Patent Citations

  • Unmanned ship small sample target detection method based on self-distillation data enhancement

    CN118196391A

  • Multi-source passive field adaptive target detection method and system based on deep learning

    CN118864816A

  • Hydraulic power plant water-collecting well and pool semi-supervised flooding detection method based on DINO

    CN119625276A

  • Bridge disease detection method and system based on improved multi-modal visual language model

    CN119649177A

  • Capacitor appearance intelligent defect detection method based on YOLO-NAS and Transform fusion

    CN119762889A