Incremental detection method based on elastic weight disturbance isolation and prototype drift calibration

Abstract

The present application relates to an incremental detection method based on elastic weight interference isolation and prototype drift calibration, comprising: in the elastic weight interference isolation stage, detecting the interference area of the current data through the historical model and generating an interference area set (new class target mistaken as background), combining the historical and current interference information importance parameters to construct a perception score, constructing a loss function to train the model and updating the old knowledge importance parameter; in the prototype drift calibration stage, establishing the old stage class prototype based on the historical features, performing drift compensation through the projection layer and generating drift compensation features, splicing the drift compensation features with the model output features to retrain the classification head, so as to calibrate the feature distribution and complete the two-stage technical closed loop. The incremental detection method can effectively solve the problem of catastrophic forgetting.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence and computer vision, and particularly relates to an incremental detection method based on elastic weight interference isolation and prototype drift calibration. BACKGROUND

[0002] Target detection is one of the core tasks in the field of computer vision. In recent years, with the development of deep neural networks, target detection has also made significant progress. However, existing target detection methods generally rely on static data sets and can only recognize fixed classes defined during the training phase, making it difficult to adapt to dynamic scenarios where classes are constantly changing in the real world. Therefore, the incremental target detection task is proposed, which aims to continuously learn new classes without accessing the original old task data, and maintain the detection ability of old classes, supporting the long-term evolution and application of the model.

[0003] Compared with traditional incremental learning classification tasks, incremental target detection faces more complex challenges, the most fundamental of which is catastrophic forgetting, that is, the model is prone to forgetting the knowledge of the original class when learning new classes, resulting in serious degradation of detection performance. In addition, since the data in the incremental training stage usually only contains the new class labels of the current task, the old class targets learned previously are not labeled in the current data, and future class targets may also appear in the current image but are also not labeled. These unlabeled objects are mistakenly treated as background during training, which can easily lead to model misjudgment, resulting in knowledge conflicts in feature expression and parameter updating.

[0004] Although existing methods attempt to alleviate the forgetting problem through knowledge distillation or parameter regularization, they are difficult to effectively solve the cross-task interference caused by unlabeled objects. For example, treating old class targets as background weakens the ability to retain old knowledge, while future class targets as background participate in training, introducing unpredictable noise and disrupting the model's understanding of class boundaries. This knowledge conflict caused by incomplete labeling is a key difficulty in the incremental target detection task that distinguishes it from traditional incremental learning. In addition, during continuous learning, the feature distribution of old and new classes is constantly changing, which also leads to feature semantic drift, further degrading the model's performance in the classification head.

[0005] In short, existing incremental detection methods still cannot effectively solve the problem of catastrophic forgetting. SUMMARY

[0006] In order to solve the above problems in the prior art, the present application provides an incremental detection method based on elastic weight interference isolation and prototype drift calibration. The technical problems to be solved by the present application are solved by the following technical solutions:

[0007] The application provides an incremental detection method based on elastic weight disturbance isolation and prototype drift calibration, comprising:

[0008] (a) elastic weight disturbance isolation stage:

[0009] The detection model trained in the first stage The received image data set in the first stage The interference region identification is performed, the unlabeled new class target region misjudged as a background region is screened, and an interference region set is generated . is a positive integer greater than 1; The interference information importance parameter in the first stage The interference information importance parameter in the first stage

[0010] The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage

[0011] The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage

[0012] The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage The interference information importance parameter in the first stage

[0013] (b) prototype drift calibration stage:

[0014] Before acquisition Multiple regional features of each stage are used to obtain the prototype of the old stage based on Gaussian distribution modeling ;

[0015] The detection model Regarding the Image dataset of the stage The output features of the projection layer are used as input to align the detection model Regarding the Image dataset of the stage The output features of the detection model Regarding the Image dataset of the stage The output features of the projection layer are used as optimization targets, and the projection layer is trained to obtain a trained projection layer;

[0016] The trained projection layer is used to project the old stage class prototype Processing is performed to obtain the drift-compensated prototype;

[0017] Performing Gaussian sampling on the drift-compensated class prototype to obtain a plurality of drift-compensated features;

[0018] The first Group splicing features into the detection model In the classification head, the cross entropy loss function is used to classify the detection model The classification head is retrained to obtain a calibrated classification head; the The group splicing feature is obtained by combining the plurality of drift compensation features, the detection model Regarding the Image dataset of the stage The output features of are concatenated.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] (1) In the elastic weight interference isolation stage, the interference region in the current data is dynamically identified and modeled based on the historical model, and the core is to generate an interference awareness score as a weight adjustment basis by quantifying the importance parameter of the interference information relative to the old knowledge. This mechanism builds an adaptive loss function, which prioritizes the suppression of the influence of the interference region on the model decision in the training process, while strengthening the representative feature extraction of the non-interference region. In addition, this stage updates the importance parameter of the old knowledge through continuous iteration, establishing a dynamic barrier for knowledge protection, so that the model can resist new environmental interference and consolidate the representation stability of existing knowledge in the incremental learning process, ultimately significantly improving the robustness and knowledge continuity of the model in complex scenarios.

[0021] (2) In the prototype drift calibration stage, first, the prototype features of historical classes are reconstructed as a reference, and then a learnable feature projection layer is introduced to compensate for the prototype feature drift caused by changes in data distribution in the incremental training. By resampling the compensated prototype features into a Gaussian distribution representation and fusing them with the current feature vectors extracted by the model, an enhanced feature input stream is formed. This fused feature is used to re-optimize the decision boundary of the classification head, so that the model can adaptively calibrate the distribution relationship between new and old classes in the feature space, effectively alleviating the classification bias caused by data evolution, thereby improving the classification consistency and generalization ability across stages.

[0022] (3) By combining the elastic weight interference isolation stage and the prototype drift calibration stage, the elastic weight interference isolation stage is used as a defense layer to filter noise erosion of the knowledge system through the importance perception mechanism, and the prototype drift calibration stage is used as a correction layer to rebuild a stable classification space through feature reconstruction and fusion. This two-stage closed-loop optimization mechanism can have both environmental adaptability and knowledge protection in the continuous learning process, providing a solution for incremental learning systems that balances robustness and accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a flowchart of an incremental detection method based on elastic weight interference isolation and prototype drift calibration provided by an embodiment of the present application;

[0024] Figure 2 is a flowchart of the elastic weight interference isolation stage provided by an embodiment of the present application;

[0025] Figure 3 is a specific operation flowchart of the elastic weight interference isolation stage provided by an embodiment of the present application;

[0026] Figure 4 is a flowchart of the prototype drift calibration stage provided by an embodiment of the present application;

[0027] Figure 5 is a working principle diagram of an incremental detection device based on elastic weight interference isolation and prototype drift calibration provided by an embodiment of the present application;

[0028] Figure 6 is a simulation result diagram of the present application and a plurality of existing incremental detection methods under two incremental stages provided by an embodiment of the present application;

[0029] Figure 7 is a simulation result diagram of the present application and a plurality of existing incremental detection methods under multiple incremental stages provided by an embodiment of the present application;

[0030] Figure 8 is a simulation result diagram of the present application and a plurality of existing incremental detection methods under the condition of using a COCO data set provided by an embodiment of the present application;

[0031] Figure 9 is a simulation result diagram of ablation research on the incremental detection device based on elastic weight interference isolation and prototype drift calibration provided by the present application under different setting modes provided by an embodiment of the present application;

[0032] Figure 10 is a simulation result diagram of the stability of the incremental detection device based on elastic weight interference isolation and prototype drift calibration provided by the present application under different provided by an embodiment of the present application. DETAILED DESCRIPTION

[0033] The present application will be further described below in conjunction with specific embodiments, but the embodiments of the present application are not limited thereto.

[0034] In the description of the present application, the terms "first", "second" are only used for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0035] In the description of the present application, the description of the terms "example", etc. means that the specific features, structures or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms is not necessarily for the same embodiment or example. Moreover, the specific features, structures or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in the present application.

[0036] Although the present application is described in conjunction with various embodiments, it will be understood by those skilled in the art that other variations of the disclosed embodiments can be understood and effected by a skilled person in the art upon reading the description in conjunction with the drawings, the disclosure, and the appended claims. In the present application, the word "comprising" does not exclude other components or steps, and the word "a" or "an" does not exclude a plurality.

[0037] Here, before a detailed description of the incremental detection method provided by the present application is given, a brief introduction to the incremental detection model to be optimized is first given. It should be noted that the symbols appearing in the following text do not represent any specific numerical value, but are only used to represent numbering or position labels.

[0038] The incremental detection model to be optimized is a two-stage Faster R-CNN detection model, which specifically includes a backbone network for feature extraction, a region proposal network (RPN) for generating candidate target regions, a region of interest pooling layer (RoIPooling) for converting candidate regions of different sizes into fixed-size features, a fully connected layer (FC layer), and a target detection head. The target detection head includes a classification head (for class prediction) and a regression head (for bounding box refinement) .

[0039] In the working process, the target detection head can use the region of interest pooling layer to propose in the region proposal network, and then process the features through the FC layer to extract the features of a specific region. These features are then passed to the classification head and the regression head for processing, and then the detection results are output, including the classification results output by the classification head and the positioning task output by the regression head . The classification results include several target classes and background classes, and the positioning task includes several bounding box offsets.

[0040] However, the current two-stage Faster R-CNN detection model still cannot effectively solve the problem of catastrophic forgetting.

[0041] The incremental detection method based on elastic weight interference isolation and prototype drift calibration proposed in the present invention is now described in detail with reference to the accompanying drawings. This incremental detection method can effectively solve the problem of catastrophic forgetting that the current two-stage Faster R-CNN detection model cannot effectively solve. It should be noted that catastrophic forgetting can be understood as: when a model is trained using data from a new category, its performance on the new data will generally improve, but its performance on the old categories will drop sharply, as if it has "forgotten" the previously learned knowledge.

[0042] Figure 1 1 is a flow chart of an incremental detection method based on elastic weight interference isolation and prototype drift calibration provided by an embodiment of the present invention. Figure 2 is a flow chart of the elastic weight interference isolation phase provided by an embodiment of the present invention. Figure 3 This is a schematic diagram of a specific operation flow in the elastic weight interference isolation phase provided by an embodiment of the present invention. Figure 4 4 is a flow chart of the prototype drift calibration phase provided by an embodiment of the present invention.

[0043] like Figures 1-4 As shown, the method includes: (a) an elastic weight interference isolation stage and (b) a prototype drift calibration stage; wherein (a) includes steps 110 to 140, and (b) includes steps 210 to 250. The specific operations of each stage are now further described in detail.

[0044] (a) Elastic weight interference isolation stage:

[0045] Step 110: Using Detection model obtained by stage training , for the received Image dataset of the stage Identify interference areas, filter out unlabeled new target areas that are misjudged as background areas, and generate interference area sets ; is a positive integer greater than 1.

[0046] It should be noted that the Image dataset of the stage Contains at least one Image dataset of the stage For example, the image dataset of stage 1 The image dataset of the second stage includes three image categories: cat, dog and background. It not only covers The image dataset of the third stage includes cats, dogs, rabbits, elephants and other new image categories. Image dataset in stage 2 On the basis of the original image, new image categories will be added, such as tigers, lions, etc. In other words, the "old categories" and "new categories" in this article are relative concepts. "Old categories" refer to object categories that already existed in the previous stage (excluding background categories, because background is usually regarded as a fixed or general category rather than a specific object category), and "new categories" refer to object categories newly added in the current stage (also excluding background categories).

[0047] It should be noted that before executing step 110, the incremental detection method further includes: Image dataset of the stage Carry out the Secondary region candidate generation, the corresponding Group current stage area collection Specifically, through the region proposal network or selective search, the Image dataset of the stage Redefining the Group current stage area collection , among which, Image dataset of the stage Each photo in can generate hundreds to thousands of independent candidate regions, each of which is represented by a bounding box. Group current stage area collection The basic unit is no longer an image, but a collection of multiple independent candidate regions as the basic unit.

[0048] Through the Image dataset of the stage Carry out the Sub-region candidate generation can facilitate subsequent refined regularization processing and support subsequent feature drift calibration.

[0049] Here, the detection model is used For the first Image dataset of the stage When identifying interference areas, two sets of data are output, one set is the historical task pseudo label , which is used to simulate the detection model The prediction results of historical categories, the other group is the detection model The new class target is misjudged as the background area. Then, step 110 specifically includes: using the detection model For the first Image dataset of the stage Perform reasoning and generate pseudo labels for historical tasks ; Pseudo labels for historical tasks The real annotations of the new categories in the dataset are shielded to obtain the pseudo-annotated historical dataset. ; Pseudo-labeled historical dataset The area predicted as background in Image dataset of the stage Perform IoU matching on the real annotations of the new category to obtain multiple IoU values; filter the IoU values ​​greater than the preset threshold among the multiple IoU values ​​and summarize the corresponding areas as the interference area set .

[0050] It should be understood that the detection model For the first Image dataset of the stage When making inferences, Group current stage area collection Generate a pseudo label for each candidate region in the task, and summarize it as the historical task pseudo label Here, the new category real label refers to the manually labeled new category bounding box, and the historical task pseudo label The new category real annotation in the masking means deleting the new category label information in the annotation information, and does not delete the image area. If the new category label information is retained, it will cover the detection model Misjudging new targets as “background” will cause cognitive pollution. The area predicted as background in Image dataset of the stage The IoU matching of the real annotations of the new categories can be understood as calculating the pseudo-annotated historical dataset. The IoU value of each background area bounding box and the new category bounding box in the , one IoU value corresponds to a background area bounding box; obtaining an IoU value greater than the preset threshold means that the pseudo-labeled historical dataset The areas that meet the conditions are screened and summarized to obtain the interference area set .

[0051] It should be noted that in , there is no interference from old tasks, so there is no historical task pseudo label.

[0052] In one possible implementation, the interference region set The expression is:

[0053] ;

[0054] in, , represents the set of real regions corresponding to the new category, represents the true area corresponding to the new category, Indicates the number of true regions corresponding to the new category, , represents the pseudo-labeled historical dataset Detection Model The set of detected background regions, Represents a pseudo-annotated historical dataset Detection Model The detected background area, Represents a pseudo-annotated historical dataset Detection Model The number of detected background regions, Refers to the preset threshold.

[0055] Please refer to Figure 3 , for example, Image dataset of the stage Input detection model After that, the detection model The newly emerged category "Sheep" ( Figure 3 "sheep" in the image) is misidentified as background area ( Figure 3 In the "bg"), through step 110, the detection model Filter out multiple areas that are mistakenly identified as "bg" from the output results.

[0056] In step 110, the interference area set is obtained Then, go to step 120 for the next step of processing.

[0057] Step 120: Combine The importance parameter of previous knowledge at each stage Hedi Interference information importance parameter , generating the Interference perception importance score Among them, Interference information importance parameter Based on the interference area set , detection model Model parameters and detection model The loss function is calculated.

[0058] Here, The importance parameter of previous knowledge at each stage Is to use the The importance parameter of previous knowledge at each stage Hedi Interference information importance parameter The calculated The importance parameter of previous knowledge at each stage Essentially equivalent to a first interference awareness importance score . And, a second interference information importance parameter is calculated by estimating its gradient sensitivity on the set of interference regions . Further, the first interference awareness importance score is expressed as:

[0059] ;

[0060] ;

[0061] wherein, is a proportionality factor, is the probability of the set of interference regions in the current stage region set of the first group, is the probability of the set of non-interference regions in the predicted previous stage history region set.

[0062] It should be understood that, when the value of the first interference awareness importance score increases (e.g. tends to positive infinity), the second interference awareness importance score tends to 0, allowing to correct false responses, when the value of the first interference awareness importance score decreases (e.g. tends to 0), the second interference awareness importance score

[0063] tends to , strengthening the protection of the importance parameter of old knowledge. It should be noted that the set of non-interference regions is obtained by removing the set of interference regions from the predicted previous stage history region set, which is composed of the history region set of the first stage to the history region set of the previous stage. By removing the set of interference regions from the predicted previous stage history region set, the real old class target regions and background regions correctly identified by the detection model are preserved, which represent pure old knowledge and need to be protected by regularization. In addition, there is no interference at the annotation level in the current stage region set of the first group, i.e. the interference only comes from the detection model​The misjudgment of the current stage non-interference area directly inherits the first Group current stage area collection This ensures that when the parameters are regularized, the knowledge updates of the new and old tasks do not interfere with each other and avoid conflict propagation.

[0064] It should be noted that the historical region set does not come directly from historical data, but is derived from pseudo labels of historical tasks. The candidate regions corresponding to the old categories are composed.

[0065] In step 120, by The importance parameter of previous knowledge at each stage Before continuation Key historical parameters of the phase, Interference information importance parameter Block the current interference propagation and then use Dynamic adjustment The importance parameter of previous knowledge at each stage Hedi Interference information importance parameter The proportion of the parameters that need to be protected is increased, and the parameters that need to be suppressed are further weakened to generate the first Interference perception importance score , which can avoid overwriting important old parameters when learning new classes, and achieve dual optimization of new task absorption and old knowledge protection in the incremental learning process.

[0066] In step 120, the Interference perception importance score Then, proceed to step 130 for the next step of processing.

[0067] Step 130: According to Interference perception importance score Build the A loss function that suppresses interference to detect the model Train and get Detection model obtained by stage training .

[0068] Here, The loss function for suppressing interference is expressed as:

[0069] ;

[0070] ;

[0071] in, It refers to the A loss function to suppress interference, Refers to the detection model In the The current loss function of the stage, is the regularization strength adjustment factor, It is Interference perception importance score, It is Stage detection model No. model parameters, It is Stage detection model No. model parameters, It is a detection model The loss function in the region proposal network stage, It is a detection model Loss function during the region of interest pooling stage.

[0072] Here, the parameter offset penalty term The model update range can be limited to avoid catastrophic forgetting. You can control the balance between the model's learning of the current task and retaining old knowledge. Increasing it will increase the constraints on old knowledge and reduce the possibility of forgetting; Reducing it allows for more flexible learning.

[0073] Here, The calculation formula is . Interference perception importance score The difference between the expressions is only the replacement of the subscripts. The meaning of each term in the formula and the calculation process are the same. I will not repeat them here.

[0074] Here, in getting After the loss function of suppressing interference, the Image dataset of the stage , for the detection model Train until The loss value output by the loss function of the first interference suppression is less than or equal to the preset value, or when the number of training times reaches the preset upper limit, the training is stopped and the first Detection model obtained by stage training .

[0075] Step 140: Combine the non-interference area sets , detection model Corresponding model parameters and detection model The loss function, and Interference perception importance score , update to get The importance parameter of previous knowledge at each stage ; Non-interference area set The interference area is collected by From the predicted The result is obtained by removing the historical regions from the set of historical regions in the previous stage.

[0076] Here, the detection model The corresponding loss function is the first A loss function for suppressing interference, then step 140 specifically includes: using the non-interference area set , detection model Corresponding model parameters and detection model The loss function is used to calculate the importance parameter of the current old knowledge ; Parameters of importance to current old knowledge Hedi Interference perception importance score Perform weighted summation to obtain The importance parameter of previous knowledge at each stage .

[0077] It should be noted that all the importance parameters of previous knowledge are measured using the Hessian matrix.

[0078] The above is the full content of the elastic weight interference isolation phase. The elastic weight interference isolation phase can effectively alleviate catastrophic forgetting without accessing historical data, but this is not enough to prevent severe forgetting. In the incremental learning process, the update of feature extractor parameters causes the distribution of old class features to shift, which in turn causes the classification head to lose its original state. The decision boundary of the classification head is invalid, which reduces To this end, the present invention utilizes (b) the prototype drift calibration stage to alleviate the failure of the target detection head caused by feature drift.

[0079] (b) Prototype drift calibration stage:

[0080] Step 210: Before obtaining Multiple regional features of each stage are used to obtain the prototype of the old stage based on Gaussian distribution modeling .

[0081] Here, step 210 specifically includes: collecting the non-interference areas Input detection model In the output The complete regional feature set of Gejiu category, , is a positive integer, which refers to the total number of old categories; The set of all regional features of Gejiu category is obtained The means and covariances corresponding to the old categories are used to obtain the Gaussian distribution models; summary Gaussian distribution model of categories, get the prototype of the old stage class .

[0082] Here, the set of non-interference regions Input detection model In the process, after the pooling layer and the fully connected layer of the region of interest, the output is All regional feature sets of Gejiu categories , Indicates the The number of regional feature samples for each old category, , Refers to the total number of regional feature samples of the old category; furthermore, The mean corresponding to each old category can be expressed as: . No. The covariance corresponding to each old category can be expressed as: , ; Furthermore, The Gaussian distribution model can be expressed as: Finally, the old stage class prototype It can be expressed as: .

[0083] It should be noted that the old stage class prototype Stored as lightweight statistics.

[0084] Step 220: Test the model For the first Image dataset of the stage The output features of are used as the input of the projection layer to align the detection model For the first Image dataset of the stage Output features and detection model For the first Image dataset of the stage The output features of are used as optimization targets to train the projection layer and obtain the trained projection layer.

[0085] Specifically, step 220 includes: using the detection model and detection models , extract the Image dataset of the stage The features of the same image area in the image are used to obtain multiple feature pairs containing different categories; each feature pair includes a detection model Output features and a detection model Output features; from multiple feature pairs, independently filter the top features with the smallest L2 distance for each old category by category feature pairs, forming a training subset; where the L2 distance is the Euclidean distance of each feature pair; is a positive integer; in the current training stage, the training subset is used to train the projection layer obtained in the previous stage to obtain the projection layer of the current stage; using the projection layer of the current stage, the detection model is adopted For the first Image dataset of the stage The output features of are projected and Gaussian sampled in sequence to obtain multiple aligned features in one-to-one correspondence; it is determined whether the loss value corresponding to the projection layer of the current stage is less than or equal to the preset value; when the loss value corresponding to the projection layer of the current stage is greater than the preset value, the training of the next stage is continued until the loss value corresponding to the projection layer is less than or equal to the preset value, and the training is stopped to obtain the trained projection layer.

[0086] Here, the loss function corresponding to the projection layer of each training stage is The expression is the same as that of , which is specifically expressed as:

[0087] ;

[0088] ;

[0089] in, Refers to the detection model Output Features, Refers to the detection model Output Features, It refers to the projection layer. Refers to the detection model The corresponding set of multiple alignment features, is the training subset, is a learnable weight matrix in the projection layer, is a learnable bias term in the projection layer.

[0090] It should be noted that the detection model The output features are also used to input the classification head and regression header The training is performed in , where the classification head Based on the classification loss function Train and get the classification head , regression head based on the positioning loss function to train.

[0091] Step 230: process the old stage class prototype using the trained projection layer to obtain the prototype after drift compensation.

[0092] Step 240: Gaussian sampling is performed on the prototype after drift compensation to obtain a plurality of drift compensation features.

[0093] Step 250: input the first group of spliced features into the classification head of the detection model , and retrain the classification head of the detection model using the cross-entropy loss function to obtain the calibrated classification head; the first group of spliced features is obtained by splicing the plurality of drift compensation features and the output features of the detection model on the image data set of the first stage.

[0094] Here, the classification head of the detection model refers to the classification head in step 220.

[0095] Here, after completing the prototype drift calibration stage of step (b), the incremental detection method further includes:

[0096] (c) detection stage:

[0097] obtain the image to be detected;

[0098] input the image to be detected into the detection model trained in step (b) prototype drift calibration stage, and output the detection result; the detection result at least includes: all categories in the image data set of the first stage.

[0099] The above is the incremental detection method based on elastic weight interference isolation and prototype drift calibration provided by the embodiment of the application. Corresponding to the incremental detection method, the application also provides an incremental detection device based on elastic weight interference isolation and prototype drift calibration. The device can effectively alleviate the catastrophic forgetting problem that is prone to occur in the traditional two-stage Faster R-CNN detection model.

[0100] Specifically, the incremental detection device includes an elastic weight interference isolation module (IKI-EWC) and a prototype drift calibration module (PKC). Figure 5 ​​is a working principle diagram of an incremental detection device based on elastic weight interference isolation and prototype drift calibration provided by an embodiment of the present application. As shown in Figure 5 , the IKI-EWC module includes a detection model and a backbone network, a region proposal network, a fully connected layer and a fully connected layer of the detection model ; the PKC module includes a projection layer, a drift calibration unit, a target detection head training unit and a classification head retraining unit; in the IKI-EWC module, based on steps 110-140 in the elastic weight interference isolation stage (a), the detection model and the image data set of the first stage are used to train the detection model ; then, in the PKC module, based on steps 210-250 in the prototype drift calibration stage (b), the detection model and the detection model are used for calculation, respectively, to retrain the classification head, so as to obtain the classification head after feature drift compensation, and enhance the performance of the classification head. Then, the incremental detection device is used to detect the image to be recognized.

[0101] To verify the effectiveness of the incremental detection method based on elastic weight interference isolation and prototype drift calibration provided by the present application, a simulation software is used to establish an incremental detection device based on elastic weight interference isolation and prototype drift calibration, so as to verify it. Specifically, an incremental object detector based on Faster R-CNN is constructed using the backbone of ResNet-50, and the pre-training weight from ImageNet is used for initialization.

[0102] Here, the PASCAL VOC 2007 data set (hereinafter referred to as VOC data set) and the COCO 2017 data set (hereinafter referred to as COCO data set) are used for evaluation. The VOC data set is composed of 9,963 images of 20 classes, while the COCO data set includes 118,000 training images and 5,000 validation images of 80 classes. In order to evaluate, the average precision mean of the VOC data set is used with an IoU threshold of 0.5 (mAP@0.5), and the average precision mean of the COCO data set is used with an IoU threshold of 0.5 to 0.95. In each incremental setting (A-B), A represents the number of classes in the initial stage, and B represents the number of new classes introduced in each subsequent stage.

[0103] For the VOC (10-10) setting, the simulation parameter settings , , and two features are sampled for each class in the PKC module. The experiments are conducted on a single Nvidia GeForce RTX 3090 GPU with a batch size of 16, implemented using PyTorch, and trained using an SGD optimizer. In the two-stage setting, the tasks are divided into 19-1, 15-5, and 10-10, where the number of new classes added in each stage increases by 1, 5, and 10 classes, respectively. In the multi-stage setting, the tasks are divided into 10-5, 5-5, 10-2, and 15-1, where the number of classes added in each step increases by 5, 5, 2, and 1, respectively, until all 20 classes are included.

[0104] Figure 6 is a simulation result diagram of the present application and a plurality of existing incremental detection methods in the two-incremental stage provided by the embodiment of the present application. Specifically, the plurality of existing incremental detection methods include: a joint training method, a fine-tuning method, an open domain target detection (abbreviated as ), an open-world target detection framework based on Transformer (English full name: Open-World Detection Transformer, abbreviated as ), an incremental learning of object detectors with meta-learning (English full name: Incremental Learning of Object Detectors with Meta-Learning, abbreviated as ), and an augmented box replay (English full name: Augmented Box Replay, abbreviated as Faster Incremental Learning of Object Detectors (Faster ILOD), Progressive Proposal Aggregation Strategy (PPAS), Multi-View Context (MVC), Probabilistic Bounding Box (PROB), Pseudo Replay Mechanism (PseudoRM), Multi-Modality Attention (MMA), Bipartite Matching with Proposals Fusion (BPF). It should be understood that, for the sake of brevity and illustration, Figures 6-10 the English abbreviations of the methods are used in the figure. It should be noted that the symbol "-" in the figure means that the current data or module does not exist, and the symbol "√" means that the current module exists.

[0105] Based on Figure 6 It can be seen that, in the case of using VOC dataset, in the 19-1, 15-5 and 10-10 settings, the method provided by the present application improves the average AP (compared with the average AP of all categories) by 1.3%, 1.0% and 2.8% respectively than BPF, and is superior to the rehearsal-based method of ABR by 4.5%, 2.7% and 3.7%. The improvement in 1-20 and average performance (Avg) indicates that the method provided by the present application has better stability and plasticity than most other methods.

[0106] Figure 7 is the simulation result graph of the present application and a plurality of existing incremental detection methods under the multi-incremental stage provided by the embodiment of the present application. As shown in Figure 7 , in the case of using VOC dataset, the method provided by the present application improves by 3.6%, 7.0%, 1.4% and 1.7% than ABR, and is superior to BPF by 2.8%, 2.9%, 1.5% and 0.5% in the 10-5, 5-5, 10-2 and 15-1 settings.

[0107] Figure 8are simulation result figures of the present application and various existing incremental detection methods in the case of using the COCO data set provided by an embodiment of the present application. As shown in Figure 8 the method provided by the present application is always superior to the latest state-of-the-art method BPF. In the 40-40 and 70-10 settings, the method provided by the present application respectively obtains a performance gain of 1.5% and 0.7% in average AP compared with BPF. The results prove the effectiveness of the proposed method provided by the present application in selectively suppressing parameter-level task interference, which becomes particularly important as the number of classes increases.

[0108] Figure 9 are simulation result figures of ablation studies of the incremental detection device based on elastic weight interference isolation and prototype drift calibration provided by the present application in different setting modes. As shown in Figure 9 , the IKI-EWC module is subjected to ablation studies in the modes of VOC(10-10), VOC(10-5) and VOC(5-5), and the historical task pseudo-labels are used as a baseline to recalculate the parameter importance based on incremental object detection. As shown in Figure 9 , the IKI-EWC module alleviates the interference between old and new knowledge in the regularization process, effectively reduces severe forgetting while maintaining the plasticity of the model. Compared with the baseline (the first and second rows), the IKI-EWC module achieves a performance improvement of 1.1%, 5.0% and 6.1% in the settings of VOC(10-10), VOC(10-5) and VOC(5-5), respectively. Similarly, the PKC module brings an additional performance improvement of 0.5%, 1.5% and 0.4% in the settings of VOC(10-10), VOC(10-5) and VOC(5-5), respectively.

[0109] Figure 10 are simulation result figures of the stability of the incremental detection device based on elastic weight interference isolation and prototype drift calibration provided by the present application in different settings. As shown in Figure 10 , when is too large, the incremental detection device becomes more stable, but the plasticity decreases, making it more difficult to learn new tasks, and when , the best balance between plasticity and stability is achieved, and the highest average AP is achieved. The change of will cause some fluctuations in the results, with little effect, and when Figure 10 , the best performance is achieved, and as can be seen from , the change of will cause some fluctuations in the results, with little effect, and when , the best effect is achieved.

[0110] In view of the fact that the existing incremental detection method still cannot effectively solve the problem of catastrophic forgetting, the embodiment of the present application provides an incremental detection method based on elastic weight interference isolation and prototype drift calibration, which has the following technical effects:

[0111] (1) In the elastic weight interference isolation stage, the interference region in the current data is dynamically identified and modeled based on the historical model, and the core is to generate an interference awareness score as a weight adjustment basis by quantifying the importance parameter of the interference information relative to the old knowledge. This mechanism constructs an adaptive loss function, which preferentially suppresses the influence of the interference region on the model decision in the training process, while strengthening the representative feature extraction of the non-interference region. In addition, this stage continuously updates the importance parameter of the old knowledge through iteration, establishing a dynamic barrier for knowledge protection, so that the model can resist new environmental interference and consolidate the representation stability of existing knowledge in the incremental learning process, ultimately significantly improving the robustness and knowledge continuity of the model in complex scenarios.

[0112] (2) In the prototype drift calibration stage, the prototype features of historical classes are first reconstructed as a reference, and then a learnable feature projection layer is introduced to compensate for the prototype feature drift caused by changes in data distribution in the incremental training. By resampling the compensated prototype features into a Gaussian distribution representation and performing multi-modal concatenation and fusion with the current feature vectors extracted by the model, an enhanced feature input stream is formed. This fused feature is used to re-optimize the decision boundary of the classification head, so that the model can adaptively calibrate the distribution relationship of new and old classes in the feature space, effectively alleviate the classification bias caused by data evolution, and thus improve the classification consistency and generalization ability across stages.

[0113] (3) By combining the elastic weight interference isolation stage and the prototype drift calibration stage, the elastic weight interference isolation stage is used as a defense layer to filter the erosion of noise on the knowledge system through the importance perception mechanism, and the prototype drift calibration stage is used as a correction layer to reconstruct a stable classification space through feature reconstruction and fusion. This two-stage closed-loop optimization mechanism can have both environmental adaptability and knowledge protection in the continuous learning process, providing a solution for incremental learning systems that balances robustness and accuracy.

[0114] The above is a further detailed description of the present application in combination with specific preferred embodiments, and cannot be considered as limiting the specific implementation of the present application to these descriptions. For ordinary skilled persons in the technical field to which the present application belongs, without departing from the concept of the present application, a number of simple deductions or substitutions can be made, which should be considered as falling within the protection scope of the present application.

Claims

1. A method for incremental detection based on elastic weight disturbance isolation and prototype drift calibration, characterized in that, Comprising: (a) elastic weight interference isolation stage: Utilizing the first The detection model trained in the first stage , the received first The image data set of the first stage Interference region identification, screening unmarked new class target regions misjudged as background regions, generating an interference region set ; is a positive integer greater than 1; Combined with The importance parameter of previous knowledge at each stage Hedi Interference information importance parameter , generating the Interference perception importance score ; wherein, the Interference information importance parameter is based on the interference area set , the detection model Model parameters and the detection model The loss function is calculated; According to the first interference awareness importance score construct a first interference suppression loss function to train the detection model , and obtain a detection model trained in the first stage ; Combine non-interference region set , the detection model Corresponding model parameters and the detection model Loss function and the first Interference awareness importance score , update the old knowledge importance parameter of the first Stage ; The non-interference region set Is obtained by eliminating the interference region set From the predicted first Stage history region set (b) prototype drift calibration stage: Before acquisition Multiple regional features of each stage are used to obtain the prototype of the old stage based on Gaussian distribution modeling ; output features of the image data set of the first stage as input of a projection layer, to align the detection model output features of the image data set of the first stage output features of the image data set of the first stage output features of the image data set of the first stage, to align the detection model output features of the image data set of the first stage output features of the image data set of the first stage output features of the image data set of the first stage, to align the detection model output features of the image data set of the first stage output features of the image data set of the first stage output features of the image data set of the first stage, to align the detection model using the trained projection layer on the old stage class prototypes to obtain the drift compensated class prototypes; Gaussian sampling is performed on the prototype-like after the drift compensation to obtain a plurality of drift compensation features; The first Group splicing features into the detection model In the classification head, the cross entropy loss function is used to classify the detection model The classification head is retrained to obtain a calibrated classification head; the The group splicing feature is obtained by combining the plurality of drift compensation features, the detection model Regarding the Image dataset of the stage The output features of are concatenated.

2. The incremental detection method based on elastic weight disturbance rejection and prototype drift calibration of claim 1, wherein, The utilization of the first The detection model trained in the stage , the received first The image data set of the stage Interference region identification is performed to screen unmarked new class target regions misjudged as background regions, and an interference region set is generated , comprising: Utilizing the detection model To the first Stage image data set Inference, generate historical task pseudo labels ; masking real labels of new categories in the history task pseudo labels to obtain a pseudo-labeled history dataset ; The pseudo-labeled historical data set is obtained by performing the pseudo-labeling on the historical data set The region predicted as background in the first stage of image data set is matched with the new class real label in the second stage of image data set to obtain a plurality of IoU values The region predicted as background in the first stage of image data set is matched with the new class real label in the second stage of image data set to obtain a plurality of IoU values The region predicted as background in the first stage of image data set is matched with the new class real label in screening IoU values greater than a preset threshold from the plurality of IoU values, and aggregating corresponding regions into the interference region set .

3. The incremental detection method based on elastic weight disturbance rejection and prototype drift calibration of claim 1, wherein, The set of non-interference regions The detection model The corresponding model parameters and their loss function, and the Interference-aware importance score The old knowledge importance parameter of the Stage is updated to obtain It includes: Utilize the non-interference region set , the detection model Corresponding model parameters and the detection model Loss function of the current old knowledge importance parameter ; the current old knowledge importance parameter and the first interference awareness importance score are weighted summed to obtain the old knowledge importance parameter of the first stage.

4. The incremental detection method based on elastic weight disturbance rejection and prototype drift calibration of claim 1, wherein, The acquisition of the plurality of region features of the previous stage is based on the Gaussian distribution modeling to obtain the old stage class prototype , comprising:​ The non-interference region set is obtained The detection model is input In the method, the total region feature set of the first old category is output, , is a positive integer, and refers to the total number of old categories. Using the first All the region feature sets of the old category, the mean and covariance corresponding to the first Old category are calculated to obtain the first Gaussian distribution model; aggregate a Gaussian distribution model of the one category, to obtain the old stage class prototype .

5. The incremental detection method based on elastic weight disturbance rejection and prototype drift calibration of claim 1, wherein, The detection model Regarding the Image dataset of the stage The output features of the projection layer are used as input to align the detection model Regarding the Image dataset of the stage The output features of the detection model Regarding the Image dataset of the stage The output features of are used as optimization targets to train the projection layer to obtain a trained projection layer, including: Using the detection model and the detection model , extract the Image dataset of the stage The features of the same image area in the image are obtained to obtain multiple feature pairs containing different categories; each feature pair includes a detection model Output features, and a detection model Output characteristics; From the multiple feature pairs, the top pair with the smallest L2 distance for each old category is independently selected by category. feature pairs to form a training subset; wherein the L2 distance is the Euclidean distance of each feature pair; is a positive integer; In the current training stage, using the training subset, the projection layer trained in the last stage is trained to obtain the projection layer in the current stage; Using the projection layer of the current stage, the detection model is adopted The output features of the first stage image data set are sequentially subjected to projection transformation and Gaussian sampling to obtain a one-to-one corresponding plurality of alignment features; Determine whether the loss value corresponding to the projection layer in the current stage is less than or equal to a preset value; In the case where the loss value corresponding to the projection layer in the current stage is greater than the preset value, the next stage of training is continued until the loss value corresponding to the projection layer is less than or equal to the preset value, the training is stopped, and the trained projection layer is obtained.

6. The incremental detection method based on elastic weight contamination and prototype drift calibration according to claim 1, characterized in that, In the first The detection model trained in the first stage For the received first The image data set in the first stage Perform interference region identification, screen unannotated new class target regions misjudged as background regions, and generate an interference region set Before, the incremental detection method further comprises: The first image data set of the stage The first region candidate generation is performed, and the corresponding first set of current stage regions is obtained .

7. The incremental detection method based on elastic weight contamination and prototype drift calibration according to claim 6, characterized in that, The first interference awareness importance score is expressed as: ； ； wherein, is a scaling factor, is the probability that the current stage belongs to the set of non-interference regions in the first set of current stage regions in the first set of non-interference regions in the set of predicted previous sets of historical regions.

8. The incremental detection method based on elastic weight disturbance rejection and prototype drift calibration of claim 1, wherein, The said The loss function for suppressing interference is expressed as: ； ； wherein, refers to the first interference-suppressed loss function, refers to the detection model a current loss function of the first phase, is a regularization strength adjustment factor, is a first interference-aware importance score, is a first model parameter of the detection model of the first phase, is a first model parameter of the detection model of the first phase, is a first model parameter of the detection model of the first phase, is a loss function of the detection model at a region proposal network phase, is a loss function of the detection model at a region of interest pooling phase.

9. The incremental detection method based on elastic weight contamination and prototype drift calibration according to claim 1, wherein, the first image data set of the phase at least one new class not present in the first image data set of the phase at least one new class not present in the first 10. The incremental detection method based on elastic weight disturbance rejection and prototype drift calibration of claim 1, wherein, After completing step (b) prototype drift calibration stage, the incremental detection method further comprises: (c) detection stage: Obtain the image to be detected; The image to be detected is input into the detection model trained in the prototype drift calibration phase of step (b), and the detection result is output; the detection result at least includes: All categories in the image dataset of the first stage.

Images