A category-incremental object detection method based on generative replay

Abstract

The present invention belongs to the field of computer vision technology, and specifically discloses a method for category incremental target detection based on generative playback, including: obtaining a fine-tuned Stable Diffusion model; using the fine-tuned Stable Diffusion model to generate image-level playback data corresponding to old tasks and new tasks, annotating the image-level playback data with pseudo labels, and performing confidence sampling to obtain initial image-level playback data; screening the initial image-level playback data to obtain final image-level playback data; combining the final image-level playback data with the real data of the new task to obtain a complete training data set to train the detection model, and obtain a trained detection model. The present invention can improve the quality of generated data in generative category incremental target detection. It solves the problem that the model in the prior art generates low-quality data in the generative category incremental target detection method.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a category incremental target detection method based on generative playback. Background Art

[0002] Category-incremental object detection (CIOD) aims to enable the object detection model to retain the detection ability of old categories when learning new categories, avoiding "catastrophic forgetting". Traditional methods usually require access to data of all categories, but in practical scenarios, the problem of inaccessible data or limited storage makes incremental learning challenging.

[0003] To address this challenge, existing techniques mainly include regularization-based and data replay-based methods. Regularization methods mitigate the forgetting problem by imposing constraints during parameter optimization, while replay methods propose to train the model using a combination of new data and data from previous tasks. In particular, generation-based replay strategies have attracted much attention due to their simplicity and effectiveness, and they are usually implemented by saving partial data from past tasks or leveraging powerful generative models to generate image data related to previous tasks.

[0004] With the emergence of generative models such as Stable Diffusion, their ability to generate high-quality images provides new ideas for CIOD. However, since existing generative models still do not have the ability to generate complex images with precise spatial arrangements, their application in class-incremental object detection is still limited. At the same time, problems such as the domain difference between generated samples and real data, and the unreasonable selection of new and old category samples still exist, resulting in a decrease in classification and positioning performance. For this reason, an efficient solution that can effectively alleviate forgetting and adapt to new tasks is urgently needed. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a category incremental target detection method based on generative playback, which solves the problem that the model in the prior art generates low-quality data in the generative category incremental target detection method.

[0006] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a category incremental target detection method based on generative playback, comprising the following steps:

[0007] S1. Obtain target detection data, and use the target detection data to fine-tune the Stable Diffusion model to obtain a fine-tuned Stable Diffusion model;

[0008] S2. Use the fine-tuned Stable Diffusion model to generate image-level playback data corresponding to the old task and the new task. Use the model of the old task and the model of the new task to annotate the image-level playback data with pseudo labels and perform confidence sampling to obtain the initial image-level playback data.

[0009] S3, using a cross-task similarity cross-sampling module to filter the initial image-level playback data to obtain the final image-level playback data;

[0010] S4. Combine the final image-level playback data with the real data of the new task to obtain a complete training data set to train the detection model corresponding to the new task, obtain a trained detection model, and use the trained detection model to perform category incremental target detection.

[0011] The beneficial effects of the above scheme are:

[0012] (1) When the present invention is applied to the category-incremental target detection task, the forgetting of prior knowledge mainly occurs in the classification subtask. The knowledge of the regression subtask can be well transferred to the incremental task.

[0013] (2) The present invention proposes an image-level generative replay (IGR) method, which uses pseudo-labels and confidence sampling to simply and efficiently reduce the forgetting of old category knowledge without the need for Stable Diffusion to generate complex images with precise spatial arrangements. In addition, because there are no restrictions on model learning, the present invention improves model stability without affecting the generalization of the model.

[0014] (3) This paper proposes a cross-task similarity cross sampling (SCS) method to screen out more challenging confusing samples based on similarity, significantly reducing the false alarm rate of new categories and strengthening the classification ability of the model. This mechanism effectively expands the discrimination boundary between new and old categories, thereby improving the model's adaptability to new tasks. In addition, by generating new data to replay the current task, the model is prevented from confusing category knowledge with domain knowledge, reducing the impact of domain differences on the model.

[0015] (4) The present invention can achieve category-incremental target detection without complex generative model optimization or large amounts of real data storage, and is suitable for scenarios with limited data privacy or storage resources, such as autonomous driving, video surveillance, and medical image analysis.

[0016] Furthermore, S1 specifically includes:

[0017] S11, obtaining target detection data;

[0018] S12, extracting each image in the target detection data and the annotation information corresponding to each image;

[0019] S13, constructing text prompt information corresponding to each image according to the annotation information corresponding to each image;

[0020] S14, pairing each image with the text prompt information corresponding to each image to form an image-text pair;

[0021] S15. Fine-tune the Stable Diffusion model using LoRA to obtain the initial Stable Diffusion model;

[0022] S16. Based on the image-text pairs, the initial Stable Diffusion model is trained to obtain a fine-tuned Stable Diffusion model.

[0023] The beneficial effect of the above further scheme is that the fine-tuned Stable Diffusion model can better adapt to the distribution characteristics of target detection task data, alleviate the inter-domain difference problem between generated data and real data, and thus improve the practical application value of generated samples.

[0024] Furthermore, in S2, the fine-tuned Stable Diffusion model is used to generate image-level playback data corresponding to the old and new tasks, including:

[0025] Use the fine-tuned Stable Diffusion model to generate image-level playback data based on preset text prompts;

[0026] The preset text prompt is:

[0027]

[0028] in, The category name representing the old task, Represents a collection of categories.

[0029] The beneficial effect of the above further scheme is that it directly uses text prompts of preset categories to generate image data of different categories, without the need for additional annotation or preparation of real data, which greatly reduces the complexity of data preparation and saves time and resources.

[0030] Furthermore, in S2, the model of the old task and the model of the new task are used to perform pseudo-label annotation on the image-level playback data, specifically including:

[0031] Pseudo-label the image-level replay data using the model of the old task and / or the model of the new task;

[0032] Perform confidence sampling, including:

[0033] Use the detector corresponding to the model of the old task and / or the detector corresponding to the model of the new task to filter the pseudo-label annotation results;

[0034] When the number of category bounding boxes in the pseudo-label annotation result meets the preset number condition, and the confidence of the category bounding box meets the first preset confidence condition, the pseudo-label annotation result is retained, otherwise the pseudo-label annotation result is deleted;

[0035] The preset quantity conditions are:

[0036]

[0037] in, Indicates the number of category bounding boxes in the pseudo-label annotation results;

[0038] The default confidence conditions are:

[0039]

[0040] in, Indicates the pseudo-label annotation result The categories of bounding boxes, Indicates The confidence score of the bounding box, Represents the first confidence threshold.

[0041] The beneficial effects of the above further solution are: introducing confidence conditions, retaining images with high confidence category bounding boxes, effectively reducing the impact of noise data, and screening out images that are more in line with the previous stage of the task, thereby improving the model's anti-forgetting ability. And directly generating image annotations through pseudo-labels solves the pain point that Stable Diffusion does not have the ability to generate complex images with precise spatial arrangements.

[0042] Furthermore, in S3, the initial image-level playback data is screened using a cross-task similarity cross-sampling module, specifically including:

[0043] S31, using the real data of the new task to train the detector corresponding to the model of the new task;

[0044] S32, using the detector corresponding to the model of the new task to predict the initial image-level playback data of the old task, and retaining the initial image-level playback data of the old task that meets the second preset confidence condition in the prediction results, to obtain the screened initial image-level playback data of the old task;

[0045] The initial image-level playback data of the old task is the data corresponding to the old task in the initial image-level playback data;

[0046] The default confidence conditions are:

[0047]

[0048] in, The detector corresponding to the model representing the new task predicts the set of prediction boxes generated by the initial image-level playback data of the old task, The detector corresponding to the model representing the new task generates the first bounding boxes, The detector corresponding to the model representing the new task generates the first The confidence score of the bounding box, represents a second preset confidence threshold;

[0049] S33, using the detector corresponding to the model of the old task to generate a pseudo-annotation box for the filtered initial image-level playback data of the old task;

[0050] S34, calculating the minimum intersection-over-union ratio of the predicted box and the pseudo-annotated box in the filtered initial image-level playback data of the old task;

[0051] S35 , further screening the filtered old task initial image-level playback data according to the minimum intersection-union comparison to obtain the final image-level playback data.

[0052] The beneficial effect of the above further scheme is that for the model, knowledge that is easily confused is more likely to be forgotten. By cross-sampling the similarity across tasks and screening out these more challenging old task pictures, the model's memory of key knowledge can be strengthened during the incremental learning process, significantly improving the model's stability and anti-forgetting ability. The old task pictures with high similarity between the selected tasks can effectively balance the model's attention to new and old tasks, avoid the model being too biased towards new tasks, thereby reducing the false detection rate of new tasks and improving the overall performance of the model in incremental learning scenarios.

[0053] Furthermore, in S4, the final image-level playback data is combined with the real data of the new task, specifically including:

[0054] S41, removing unreliable pseudo-label data in the final image-level playback data to obtain the filtered final image-level playback data;

[0055] S42, combining the filtered final image-level playback data with the real data of the new task.

[0056] The beneficial effect of the above further scheme is: screening and eliminating unreliable pseudo labels with too large intersection-over-intersection ratios with real labels in the final training data can effectively reduce the interference of noise data on model training.

[0057] Furthermore, in S4, the loss function of the detection model corresponding to the new task is for:

[0058]

[0059] in, represents the loss function for synthetic data, and , The network module weight hyperparameters representing the synthetic data loss, represents the synthetic data RPNHead network loss, represents the loss of the synthetic data ROIHead classification network, represents the ROIHead bounding box regression network loss, represents the loss function for real data.

[0060] The beneficial effect of the above further scheme is that by weighting the loss function of the generated data in each part of the network, the contribution of the generated data in different modules can be optimized in a targeted manner, and the knowledge of the generated data playback can be obtained to the greatest extent. At the same time, the negative impact of the domain difference between the generated data and the real data on the model training can be effectively reduced through weight adjustment, thereby improving the overall performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 Schematic diagram of the flow of a category-incremental object detection method based on generative replay.

[0062] Figure 2 Forgetting curves for mean average precision and regression accuracy on the PASCAL VOC dataset. DETAILED DESCRIPTION

[0063] Now, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be understood that the embodiments shown and described in the accompanying drawings are only exemplary and are intended to explain the principles and spirit of the present invention, rather than to limit the scope of the present invention.

[0064] like Figure 1 As shown, a category-incremental target detection method based on generative playback includes the following steps:

[0065] S1. Obtain target detection data, and use the target detection data to fine-tune the Stable Diffusion model to obtain a fine-tuned Stable Diffusion model.

[0066] In this embodiment, S1 specifically includes:

[0067] S11, obtaining target detection data;

[0068] S12, extracting each image in the target detection data and the annotation information corresponding to each image;

[0069] S13, constructing text prompt information corresponding to each image according to the annotation information corresponding to each image;

[0070] S14, pairing each image with the text prompt information corresponding to each image to form an image-text pair;

[0071] S15. Fine-tune the Stable Diffusion model using LoRA to obtain the initial Stable Diffusion model;

[0072] S16. Based on the image-text pairs, the initial Stable Diffusion model is trained to obtain a fine-tuned Stable Diffusion model.

[0073] For example, the target detection data can be the PASCAL VOC 2007 dataset or the MS COCO 2017 dataset. In order to avoid data leakage during model fine-tuning, all experiments on the PASCAL VOC 2007 dataset use the Stable Diffusion model fine-tuned on the MS COCO 2017 dataset. Similarly, experiments on the MS COCO 2017 dataset use the model fine-tuned on the PASCAL VOC 2007 dataset. For the fine-tuning hyperparameters, the batch size is set to 8, and the AdamW optimizer is selected to apply a linearly decaying learning rate scheduling strategy.

[0074] Optionally, the Stable Diffusion model in this embodiment is a Stable Diffusion 1.5 model, and is initialized using the weights provided by the official website. The target detection data is uniformly adjusted to a size of 1333×800. During the training process, some data enhancement techniques are used to improve the generalization ability of the model, including random cropping and horizontal flipping. These techniques can effectively increase the diversity of data and prevent model overfitting.

[0075] S2. Use the fine-tuned Stable Diffusion model to generate image-level playback data corresponding to the old task and the new task. Use the models of the old task and the new task to pseudo-label the image-level playback data and perform confidence sampling to obtain the initial image-level playback data.

[0076] In this embodiment, in S2, the image-level playback data corresponding to the old task and the new task is generated using the fine-tuned Stable Diffusion model, specifically including:

[0077] Use the fine-tuned Stable Diffusion model to generate image-level playback data based on preset text prompts;

[0078] The preset text prompt is:

[0079]

[0080] in, The category name representing the old task, Represents a collection of categories.

[0081] In this embodiment, in S2, the model of the old task and the model of the new task are used to perform pseudo-label annotation on the image-level playback data, specifically including:

[0082] Pseudo-label the image-level replay data using the model of the old task and / or the model of the new task;

[0083] Perform confidence sampling, including:

[0084] Use the detector corresponding to the model of the old task and / or the detector corresponding to the model of the new task to filter the pseudo-label annotation results;

[0085] When the number of category bounding boxes in the pseudo-label annotation result meets the preset number condition, and the confidence of the category bounding box meets the first preset confidence condition, the pseudo-label annotation result is retained, otherwise the pseudo-label annotation result is deleted;

[0086] The preset quantity conditions are:

[0087]

[0088] in, Indicates the number of category bounding boxes in the pseudo-label annotation results;

[0089] The default confidence conditions are:

[0090]

[0091] in, Indicates the pseudo-label annotation result The categories of bounding boxes, Indicates The confidence score of the bounding box, Represents the first confidence threshold.

[0092] Optionally, the model used to process the new task in this embodiment is based on Faster R-CNN, with ResNet50 as the backbone and initialized with ImageNet pre-trained weights provided by pytorch. The hyperparameters of the training are set as follows:

[0093] Optimizer: Use Adam optimizer, which is a commonly used gradient descent optimization method with good convergence and stability;

[0094] Learning rate: The initial learning rate is set to 1×10^4, and decays to 0.1 and 0.01 at the 8th and 10th epochs;

[0095] Loss function: For the RPN network, the classification loss is calculated using the binary cross entropy loss, and the regression loss is calculated using the smooth L1 loss only for positive samples. For the ROIHead network, the cross entropy loss is used to measure the difference between the predicted category distribution and the true label. The smooth L1 loss is also used and is only applied to the positive sample candidate box.

[0096] During the training process, about Epochs: The model is usually trained for 12 epochs, and the timing of early stopping is determined based on the performance of the validation set. Forward propagation: Images are generated through Stable Diffusion, and combined with real data to be sent to the Faster R-CNN network for forward propagation, then the loss is calculated, and backpropagation is performed to update the model parameters. Early stopping strategy: If the performance of the validation set no longer improves within a few epochs, training is stopped early to prevent overfitting.

[0097] S3. Use the cross-task similarity cross-sampling module to filter the initial image-level playback data to obtain the final image-level playback data.

[0098] In this embodiment, in S3, the initial image-level playback data is screened using a cross-task similarity cross-sampling module, specifically including:

[0099] S31, using the real data of the new task to train the detector corresponding to the model of the new task;

[0100] S32, using the detector corresponding to the model of the new task to predict the initial image-level playback data of the old task, and retaining the initial image-level playback data of the old task that meets the second preset confidence condition in the prediction results, to obtain the screened initial image-level playback data of the old task;

[0101] The initial image-level playback data of the old task is the data corresponding to the old task in the initial image-level playback data;

[0102] The default confidence conditions are:

[0103]

[0104] in, The detector corresponding to the model representing the new task predicts the set of prediction boxes generated by the initial image-level playback data of the old task, The detector corresponding to the model representing the new task generates the first bounding boxes, The detector corresponding to the model representing the new task generates the first The confidence score of the bounding box, represents a second preset confidence threshold;

[0105] S33, using the detector corresponding to the model of the old task to generate a pseudo-annotation box for the filtered initial image-level playback data of the old task;

[0106] S34, calculating the minimum intersection-over-union ratio of the predicted box and the pseudo-annotated box in the filtered initial image-level playback data of the old task;

[0107] S35 , further screening the filtered old task initial image-level playback data according to the minimum intersection-union comparison to obtain the final image-level playback data.

[0108] For example, S33 can be expressed as ,in, It represents the pseudo-annotation box generated by the detector corresponding to the model of the old task for the filtered initial image-level playback data of the old task, express The pseudo-annotated box in S34 can be expressed as ,in, represents the minimum intersection-over-union ratio of the predicted box and the pseudo-annotated box, Pseudo-labeled box and prediction box In S35, the filtered old task initial image-level playback data is further filtered according to the minimum intersection-and-union ratio, which can be: setting the corresponding minimum intersection-and-union ratio threshold ,Will The data that needs to be retained is retained, and the other data does not need to be retained.

[0109] Optionally, since the old task data is mainly synthetic data and the new task data is mainly real images, the model tends to be based on domain information rather than semantic differences between different categories. Therefore, after step S35, the steps of steps S2-S4 can also be used to perform the same processing on the data of the new task.

[0110] Optionally, the data obtained by cross-task similarity cross-sampling should be mixed with unsampled generated data in a certain ratio for training. This ratio is usually 2:1, that is, the ratio of sampled data to unsampled data is 2:1, to avoid the model tending to favor data with high similarity between tasks and ignoring the original characteristics of the data.

[0111] S4. Combine the final image-level playback data with the real data of the new task to obtain a complete training data set to train the detection model corresponding to the new task, obtain a trained detection model, and use the trained detection model to perform category incremental target detection.

[0112] In this embodiment, in S4, the final image-level playback data and the real data of the new task are combined, specifically including:

[0113] S41, removing unreliable pseudo-label data in the final image-level playback data to obtain the filtered final image-level playback data;

[0114] S42, combining the filtered final image-level playback data with the real data of the new task.

[0115] In this embodiment, in S4, the loss function of the detection model corresponding to the new task is for:

[0116]

[0117] in, represents the loss function for synthetic data, and , The network module weight hyperparameters representing the synthetic data loss, represents the synthetic data RPNHead network loss, represents the loss of the synthetic data ROIHead classification network, represents the ROIHead bounding box regression network loss, represents the loss function for real data.

[0118] Alternatively, if Figure 2 As shown, Figure 2 is the forgetting curve of the average precision and regression accuracy on the PASCAL VOC dataset. Figure 2It can be seen that the average precision (AP) on the PASCAL VOC dataset is the forgetting curve under incremental conditions. oAP represents the bounding box regression accuracy (AP calculated in a class-independent manner), and mAP represents the average precision, which is given by Figure 2 It can be seen that when catastrophic forgetting occurs, the accuracy of the regression subtask does not drop significantly. It can be inferred that the forgetting of prior knowledge mainly occurs in the classification subtask, not the regression subtask.

[0119] For example, the method of removing unreliable pseudo-label data from the final image-level replay data can be: for each pseudo-label bounding box generated by the old detector, calculate its maximum intersection over union (IoU) value with the set of all known true label bounding boxes in the same image. If the maximum IoU is greater than a preset threshold (usually set to 0.5), the pseudo-label is considered unreliable and is removed from the training set. The generated data of the old task is included in the final image-level replay data after screening. And the data of the new task generated , so S42 can be expressed as , Represents the combined data, Represents the training data of the new task annotated with pseudo labels (that is, the real data of the new task).

[0120] Optionally, the training set data integration needs to be integrated in a certain ratio. Therefore, in a preferred embodiment, The ratio can be In the loss function of the detection model corresponding to the new task, the contribution of synthetic data and real data in the training process is balanced through a weighting mechanism. And because synthetic data usually consists of simple single-instance images, such data may not provide enough regression information. However, this part of knowledge will not be forgotten in large quantities, so it is necessary to calculate the loss Assign greater weight to Assign less weight.

[0121] Optionally, after step S4, the trained detection model can be applied to an actual scene to complete category-incremental target detection based on generative playback.

[0122] Optionally, after step S4, the trained detection model may be further verified under multiple category incremental learning settings of the PASCAL VOC 2007 dataset, including a 19-1 setting, a 15-5 setting, a 10-10 setting, and a 5-15 setting.

[0123] Among them, the 19-1 setting means that in the experiment, 19 categories in the PASCAL VOC 2007 dataset are first selected as old categories for the initial training of the model, and then 1 new category is added for incremental learning. The model needs to learn to detect new categories without forgetting the detection ability of old categories.

[0124] 15-5 setting: First use 15 categories as old categories for training, and then introduce 5 new categories for incremental learning.

[0125] 10-10 setting: refers to using 10 of the 20 categories as old categories for preliminary training, and then using the other 10 categories as new categories for incremental learning, in order to observe the learning ability and performance of the model when the number of new and old categories is balanced.

[0126] 5-15 setting: It means that 5 categories are first selected as old categories for training, and then the remaining 15 categories are gradually introduced as new categories for incremental learning. It is mainly used to study the learning effect and coping strategy of the model when there are fewer old categories and more new categories.

[0127] The verification results can be shown in Table 1. These tasks divide the 20 target categories into 2 tasks or multiple tasks, which are used for continuous incremental learning experiments. In the table, "Joint Training" represents the ideal performance of joint training of all tasks, which can be used as the performance upper limit of incremental learning; "Fine-tuning" represents the sequential fine-tuning method without any anti-forgetting strategy, which reflects the impact of catastrophic forgetting on the model; other methods such as ILOD-Meta, OW-DETR and BPF are the latest existing methods, which are used for performance comparison analysis with the method of the present invention. Among them, "RGR (Ours)" represents a category incremental target detection method based on generative playback proposed in this embodiment.

[0128] Table 1 Performance comparison of different methods

[0129]

[0130]

[0131] As can be seen from Table 1, the proposed implementation is superior to existing methods, especially in terms of retaining old category knowledge and learning new categories:

[0132] In the 10-10 setting, the overall mAP of this implementation is 75.8%, which is 2.9% higher than the existing best method;

[0133] In all settings, the detection performance of our implementation on old categories remains stable. For example, in the 10-10 setting, our implementation achieves an average precision of 75.4% on old categories, significantly better than the 74.5% of other methods.

[0134] This implementation also performs well in learning new categories. For example, in the 5-15 setting, the average accuracy of this implementation for new categories is 77.2%, which is a significant improvement over existing methods.

[0135] In this embodiment, the catastrophic forgetting problem is effectively alleviated through generative replay and pseudo-labeling technology. For example, compared with the traditional fine-tuning method, the comprehensive mAP of this implementation method in the 19-1 setting is improved by 65.6%, showing a strong knowledge retention ability.

[0136] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A method for class-incremental target detection based on generative playback, characterized in that: The method comprises: S1. Obtain target detection data, and use the target detection data to fine-tune the Stable Diffusion model to obtain a fine-tuned Stable Diffusion model; S2. Use the fine-tuned Stable Diffusion model to generate image-level playback data corresponding to the old task and the new task, use the model of the old task and the model of the new task to annotate the image-level playback data with pseudo labels, and perform confidence sampling to obtain initial image-level playback data; S3, using a cross-task similarity cross-sampling module to filter the initial image-level playback data to obtain final image-level playback data; S4, combining the final image-level playback data with the real data of the new task to obtain a complete training data set to train the detection model corresponding to the new task to obtain a trained detection model, and using the trained detection model to perform category incremental target detection; In S3, the use of a cross-task similarity cross-sampling module to screen the initial image-level playback data specifically includes: S31, using the real data of the new task to train the detector corresponding to the model of the new task; S32, using the detector corresponding to the model of the new task to predict the initial image-level playback data of the old task, and retaining the initial image-level playback data of the old task that meets the second preset confidence condition in the prediction results, to obtain the screened initial image-level playback data of the old task; The old task initial image-level playback data is the data corresponding to the old task in the initial image-level playback data; The preset confidence condition is: in, The detector corresponding to the model representing the new task predicts the set of prediction boxes generated by the initial image-level playback data of the old task, The detector corresponding to the model representing the new task generates the first bounding boxes, The detector corresponding to the model representing the new task generates the first The confidence score of the bounding box, represents a second preset confidence threshold; S33, using the detector corresponding to the model of the old task to generate a pseudo-annotation frame for the filtered initial image-level playback data of the old task; S34, calculating the minimum intersection-over-union ratio of the predicted box and the pseudo-annotated box in the filtered initial image-level playback data of the old task; S35 , further filtering the filtered old task initial image-level playback data according to the minimum intersection-union ratio to obtain the final image-level playback data.

2. The method according to claim 1, characterized in that The S1 specifically includes: S11, acquiring the target detection data; S12, extracting each image in the target detection data and the annotation information corresponding to each image; S13, constructing text prompt information corresponding to each image according to the annotation information corresponding to each image; S14, pairing each image with the text prompt information corresponding to each image to form an image-text pair; S15. Based on the image-text pair, fine-tune the Stable Diffusion model using LoRA to obtain the fine-tuned Stable Diffusion model.

3. The method according to claim 2, characterized in that In S2, the use of the fine-tuned StableDiffusion model to generate image-level playback data corresponding to the old task and the new task specifically includes: Generate the image-level playback data according to preset text prompts using the fine-tuned Stable Diffusion model; The preset text prompt is: in, The category name representing the old task. Represents a collection of categories.

4. The method according to claim 3, characterized in that In S2, the pseudo-labeling of the image-level playback data using the model of the old task and the model of the new task specifically includes: Using the model of the old task and / or the model of the new task to pseudo-label the image-level playback data; The confidence sampling specifically includes: Using a detector corresponding to the model of the old task and / or a detector corresponding to the model of the new task to screen the pseudo-label annotation results; When the number of category bounding boxes in the pseudo-label annotation result meets a preset number condition, and the confidence of the category bounding box meets a first preset confidence condition, retaining the pseudo-label annotation result, otherwise deleting the pseudo-label annotation result; The preset quantity conditions are: in, Indicates the number of category bounding boxes in the pseudo-label annotation results; The preset confidence condition is: in, Indicates the pseudo-label annotation result. The categories of bounding boxes, Indicates The confidence score of the bounding box, Represents the first confidence threshold.

5. The method according to claim 1, characterized in that In S4, combining the final image-level playback data with the real data of the new task specifically includes: S41, removing unreliable pseudo-label data from the final image-level playback data to obtain filtered final image-level playback data; S42: Combining the filtered final image-level playback data with the real data of the new task.

6. The method according to claim 5, characterized in that In S4, the loss function of the detection model corresponding to the new task is for: in, represents the loss function for synthetic data, and , The network module weight hyperparameters representing the synthetic data loss, represents the synthetic data RPNHead network loss, represents the loss of the synthetic data ROIHead classification network, represents the ROIHead bounding box regression network loss, represents the loss function for real data.

Images