A target detection method based on memory enhancement and feature migration

Abstract

The application discloses a target detection method based on memory enhancement and feature migration, and relates to the field of target detection in computer vision.The target detection method based on memory enhancement and feature migration has the characteristics of high detection accuracy, low calculation complexity, convenient use and the like.Innovations of the application are that target features of old tasks are stored and used as key components of target features of new tasks.Compared with a traditional target detection method, the method greatly improves target detection performance on new tasks on the premise of ensuring performance on old tasks.

Description

Technical Field

[0001] This invention relates to the field of object detection in computer vision, and in particular to transfer learning techniques in machine learning. Background Technology

[0002] With the development and maturation of deep learning technology, numerous object detection applications have emerged in people's daily lives, such as defect detection, facial recognition, autonomous driving, intelligent object recognition, and intelligent photography. These object detection applications have been welcomed and favored by users. Existing object detection applications are mainly based on deep learning technology, requiring model developers to first collect data from the target application scenario, then use this data to train the model, and finally deploy the model in the target application scenario. However, after training an existing deep learning-based object detection model on a dataset of one target scenario, if it is then trained on data from another target scenario, the model will severely lose the capabilities gained in the previous target scenario. This problem limits the model's transferability to new target scenarios. This invention primarily aims to improve the ability of deep object detection models to transfer to new task scenarios while maintaining performance on older task scenarios.

[0003] This invention uses target features from old tasks to adaptively reorganize them into target features for new tasks. Essentially, target features from old tasks are collected, and after training, representative features for each target category are obtained and stored in memory units. Then, when learning a new task, the target features for the new task are obtained through adaptive reorganization of these representative features, thus enabling the model to remember old tasks and transfer them to new tasks. Summary of the Invention

[0004] This invention addresses the problems in the prior art where object detection models lack the ability to transfer to new tasks and perform poorly on old tasks after learning new tasks, thereby ensuring the model's performance on both new and old tasks.

[0005] The technical solution adopted by this invention to solve the above-mentioned technical problems is a target detection method based on memory enhancement and feature transfer, the method comprising:

[0006] Step 1: Establish a basic detection network, which includes: a feature extractor. Classifier, bounding box regressor, and feature extractor It includes: a backbone network, a region suggestion network, and a RoI head. The input image first enters the backbone network. The output of the backbone network passes through the region suggestion network to obtain the features of each candidate region, and then passes through the RoI head. The output of the RoI head enters the classifier and the bounding box regressor, respectively. In this basic detection network, the backbone network initially extracts image features, the region suggestion network generates candidate detection boxes, the RoI head further processes the candidate region features, and the classifier and bounding box regressor read the output features of the RoI head to generate the category prediction of the target in the region and the accurate prediction of the detection box.

[0007] Step 2: Train the feature extractor using old task data

[0008] Step 3: Use the trained feature extractor Extract features for each target of the old task. i ;

[0009] Step 4: Establish memory unit O; extract features o based on each target. i Calculate the feature mean o for each type of target. c These representative feature vectors are used as the representative feature vectors of this type of target and are stored in memory unit O.

[0010] Step 5: Build a memory enhancement module and add it to the basic detection network to obtain the target detection network; the feature extractor in the basic detection network. The output of the memory enhancement module is used as the input of the memory enhancement module, and the output of the memory enhancement module is used as the input of the classifier and the bounding box regressor.

[0011] Step 6: Establish the loss function;

[0012] Step 7: Using the old task data, train the memory enhancement module established in Step 5 based on the target detection loss and region cueing network loss established in Step 6;

[0013] Step 8: Using the new task data, train the target detection network with the added memory enhancement module based on the target detection loss, region cueing network loss, and distillation loss established in Step 6;

[0014] Step 9: Use the target detection network trained in Step 8 to perform target detection on the data to be detected.

[0015] Furthermore, the calculation method for the memory enhancement module in step 5 is as follows:

[0016]

[0017] Where C represents the total number of target categories, W1 and W2 represent fully connected layers with an output dimension of d, W3, W4, and W5 all represent fully connected layers, σ(·) represents the Softmax function, and Cat(·) represents the cascade operation. The target feature represented by the input o i The output after passing through the memory enhancement module.

[0018] Furthermore, the loss function established in step 6 includes:

[0019]

[0020]

[0021]

[0022] in, Indicates the target detection loss. This indicates a region experiencing network loss. Let l represent the distillation loss, and l be the classification probability vector. * Represents the true category, [l * ≥1] indicates that the background does not participate in the calculation of the regression loss R(·), and p represents the predicted detection box annotation vector. * Here, o represents the true bounding box annotation vector, and o represents the binary classification probability vector for the target and background. * ∈{0,1} represents whether the region feature is background or target, α and β are hyperparameters, and l′ and p′ are the class vector and bounding box annotation vector of the old model prediction output, respectively. Let KL divergence be the KL divergence. This is the mean squared error loss; This represents the probability value corresponding to the true category. This represents the probability of being classified as either background or target.

[0023] The object detection method based on memory enhancement and feature transfer proposed in this invention features high detection accuracy, low computational complexity, and ease of use. The innovation lies in storing the target features from the old task and using them as a key component of the target features for the new task. Compared to traditional object detection methods, this method significantly improves object detection performance on new tasks while maintaining performance on older tasks. Attached Figure Description

[0024] Figure 1 This is an overall framework diagram of the present invention.

[0025] Figure 2 This is a diagram of the memory enhancement module of the present invention. Detailed Implementation

[0026] This invention is implemented on the PyTorch deep learning framework and mainly includes several steps: training a basic detection network, extracting target features using the basic detection network, calculating and storing target representative features, training a memory enhancement module using the target representative features, and memory transfer training on new tasks.

[0027] Basic model training steps: A basic object detection model is obtained by training using old task data;

[0028] Target feature extraction steps: Receive N pairs of target label pairs as input using a trained target detection model. To generate features for each target image i , where x i For the target image, y i Label the corresponding targets;

[0029] Representative feature calculation and memorization steps: Calculate the mean value of each target feature. c As representative features of this type, where c represents the category to which the target belongs, these representative features are stored in memory unit O;

[0030] Training steps for the memory enhancement module: Add a memory enhancement module to the basic object detection model, such as... Figure 1 As shown, N pairs of target labels are used. Training memory enhancement module, this module as Figure 2 As shown, its calculation method is as follows:

[0031]

[0032] Where C represents the total number of target categories, W1 and W2 represent fully connected layers with an output dimension of d, W3, W4, and W5 all represent fully connected layers, σ(·) represents the Softmax function, and Cat(·) represents the cascade operation. The target feature represented by the input o i The output after passing through the memory enhancement module.

[0033] Feature transfer steps on the new task: When training the model on the new task, the object detection loss is used. Regional network loss warning and distillation loss The constraints are applied, and the specific calculation method is as follows:

[0034]

[0035]

[0036]

[0037] Where l is the classification probability vector, l * Represents the true category, [l * ≥1] indicates that the background does not participate in the calculation of the regression loss R(·), and p represents the predicted detection box annotation vector. * Here, o represents the true bounding box annotation vector, and o represents the binary classification probability vector for the target and background. * ∈{0,1} represents whether the region feature is background or target, α and β are hyperparameters, and l′ and p′ are the class vector and bounding box annotation vector of the old model prediction output, respectively. Let KL divergence be the KL divergence. The mean squared error loss is used. A comparison of the specific performance of this invention is shown in Table 1.

[0038] Table 1. Comparison of detection accuracy of the present invention and the baseline method based on fine-tuning on the MS COCO validation set.

[0039] Baseline method 11.9 20.5 12.5 4.2 13.6 16.4 This invention 24.8 44.1 24.6 11.6 28.1 34.2

Claims

1. A target detection method based on memory enhancement and feature transfer, the method comprising: Step 1: Establish a basic detection network, which includes: a feature extractor. Classifier, bounding box regressor, and feature extractor It includes: a backbone network, a region suggestion network, and a RoI head. The input image first enters the backbone network. The output of the backbone network passes through the region suggestion network to obtain the features of each candidate region, and then passes through the RoI head. The output of the RoI head enters the classifier and the bounding box regressor, respectively. In this basic detection network, the backbone network initially extracts image features, the region suggestion network generates candidate detection boxes, the RoI head further processes the candidate region features, and the classifier and bounding box regressor read the output features of the RoI head to generate the category prediction of the target in the region and the accurate prediction of the detection box. Step 2: Train the feature extractor using old task data ; Step 3: Use the trained feature extractor Extract features for each target of the old task ; Step 4: Establish memory units Extract features based on each target. Calculate the feature mean of each target class. This serves as the representative feature vector for this type of target, and is used in the memory unit. These representative feature vectors are stored in the middle; Step 5: Build a memory enhancement module and add it to the basic detection network to obtain the target detection network; the feature extractor in the basic detection network. The output of the memory enhancement module is used as the input of the memory enhancement module, and the output of the memory enhancement module is used as the input of the classifier and the bounding box regressor. The calculation method for the memory enhancement module is as follows: ； in, The total number of categories representing the target. , The output dimension is represented as Fully connected layer , and Both represent fully connected layers. Represents the Softmax function. Represents cascading operations. Target features representing input The output after passing through the memory enhancement module; Step 6: Establish the loss function; Step 7: Using the old task data, train the memory enhancement module established in Step 5 based on the target detection loss and region cueing network loss established in Step 6; Step 8: Using the new task data, train the target detection network with the added memory enhancement module based on the target detection loss, region cueing network loss, and distillation loss established in Step 6; Step 9: Use the target detection network trained in Step 8 to perform target detection on the data to be detected.

2. The target detection method based on memory enhancement and feature transfer as described in claim 1, characterized in that, The loss function established in step 6 includes: ； ； ； in, Indicates the target detection loss. This indicates a region experiencing network loss. Indicates distillation loss, For classification probability vectors, Represents the real category, The background does not participate in the regression loss. The calculation, The vector representing the predicted detection bounding box. These are the actual bounding box annotation vectors. For the binary classification probability vectors of target and background, Is the representative regional feature the background or the target? and For hyperparameters, and These are the class vector and the bounding box annotation vector, respectively, predicted by the old model. Let KL divergence be the KL divergence. This is the mean squared error loss; This represents the probability value corresponding to the true category. This represents the probability of being classified as either background or target.

Images