Label selection adaptive incremental detection method and system based on border distance measurement

Abstract

This invention discloses an adaptive incremental target detection method and system based on bounding box distance metric and label selection. It uses an existing model to generate prior information about old target categories, including category and location information, for the dataset. When training the new network model, pseudo-labels obtained through a label selection algorithm are used for training. A skip attention mechanism is employed to learn and retain important features of the old categories. Finally, a multi-scale network adaptive training model is used. This method can detect targets at different scales and is suitable for incremental target detection without existing sample data.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a label selection adaptive increment detection method and system based on border distance measurement. Background Art

[0002] Object detection, as a visual task, supports a wide range of applications, including autonomous driving, medical imaging, human-computer interaction, and foreign body detection in high-speed rail contact networks. It plays an irreplaceable role in many applications. With the development of deep learning in recent years, object detection technology has made remarkable progress, but complex systems require a lot of training time to learn models, and the final detector performance depends heavily on the availability of a set of representative training examples. In practical applications, obtaining representative data is time-consuming and expensive, so it is not uncommon for data to appear in small batches for a period of time.

[0003] As time goes by, new categories of objects may appear that the model has never learned before. If the model is fine-tuned directly with new data, catastrophic forgetting will occur, and the detection performance of old categories will be severely degraded while also affecting the detection accuracy of new categories. If the method of training the model from scratch by combining all the data is adopted, it will not only be time-consuming and computationally expensive, but the old dataset may also be inaccessible due to privacy, storage and other reasons.

[0004] In recent years, some scholars have used models to filter out a batch of representative old class samples and store them, and train the model together with new data to adapt to data changes. However, as the data grows, a very large memory overhead is required, which cannot meet actual needs. Therefore, it is urgent to design an incremental detection method that can detect targets well without accessing old data. Summary of the invention

[0005] The object of the present invention is to provide a method and system for adaptive incremental detection of label selection based on border distance measurement.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] An adaptive incremental detection method for label selection based on border distance measurement, comprising:

[0008] Obtain the old category detection model, use the old model to detect the dataset to obtain the prior information of the old category, and use the label selection algorithm to combine the dataset labels to generate pseudo labels for training the detection network;

[0009] Based on the generated pseudo-labels, a feature extractor is trained using skip attention to retain the important features of the old categories and adaptively train all parameters of the network;

[0010] The trained detection network is used to detect the preset multi-scale anchor boxes to obtain the category and location information of the target.

[0011] An adaptive incremental detection system for label selection based on border distance metric, comprising:

[0012] Label selection module: Use the old model to obtain information about old category targets in the dataset, and obtain pseudo labels for training the incremental detection model through a label selection algorithm based on bounding box distance measurement;

[0013] Feature extraction module based on attention mechanism: learn and retain important feature information of old categories by skipping the attention module;

[0014] Detection network module: Using the trained detection model, for a batch of input new category images, the category and location information of all targets including new and old categories can be detected.

[0015] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned label selection adaptive incremental detection method based on border distance measurement are implemented.

[0016] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned label selection adaptive incremental detection method based on border distance measurement.

[0017] Compared with the prior art, the significant advantages of the present invention are as follows: (1) The present invention proposes a label selection adaptive incremental detection method based on bounding box distance metric, which uses pseudo labels obtained by label selection algorithm, multi-scale candidate boxes, and skip attention mechanism for feature extraction, learns and retains important features of old categories, and then uses multi-scale network adaptive training of new categories to detect targets of different scales. The above method is suitable for incremental target detection without old sample data. (2) The trained network can realize incremental detection of all targets, including category and location information, without accessing old data. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Flow chart of the method of the present invention.

[0019] Figure 2 Schematic diagram of the system of the present invention.

[0020] Figure 3 This is a flow chart of the tag selection algorithm of the present invention. DETAILED DESCRIPTION

[0021] An adaptive incremental detection method for label selection based on bounding box distance metric, including,

[0022] Obtain the old category detection model; use the old model M to detect the data set to obtain the prior information of the old category; generate pseudo labels for training the detection network by combining the data set labels with the label selection algorithm;

[0023] According to the generated pseudo-label G, a feature extractor is trained using skip attention to retain the important features of the old category;

[0024] In the first stage, the features learned by the old model M remain unchanged, and the important features of the old categories are obtained. The attention module used in the second stage retains the key information of the old categories and adaptively trains all parameters of the network.

[0025] The trained detection network is used to detect the preset multi-scale anchor boxes to obtain the category and location information of the target.

[0026] Furthermore, the process of obtaining prior information of old categories is:

[0027] Through a trained old model, setting a suitable threshold and then detecting the data set, we can obtain the prior information of the old class targets, including category and location information.

[0028] Furthermore, the label selection process is:

[0029] For the information of the old category targets obtained, this process only uses the bounding box coordinate information, and reads the true label of the new category target in the dataset at the same time. The result calculated by the label selection algorithm is compared with the set threshold, and then the bounding box information exceeding the threshold is removed until all labels are selected. The last old category label is merged with the true label of the dataset as the pseudo label G for training the incremental detection model.

[0030] G=YUY″

[0031] Y″ is the prior information of the old class target after being processed by the label selection algorithm, namely the category and bounding box coordinates.

[0032] The process of label selection algorithm is as follows Figure 3As shown in the figure, X represents a dataset sample, the dataset X only contains the label information of the new category, Y is the true label of the new target in the dataset, M is the old category detection model, Y' is the annotation information of the old category obtained by the model M, y and y' are the true bounding box coordinates of the new category target in the new data sample and the bounding box coordinates of the old category target obtained by the old model, respectively, h is a preset threshold, and the result calculated by the generalized intersection on the union (GIOU) method of measuring the bounding box distance is compared with the set threshold, and the annotations that meet the conditions are removed until all the labels to be selected are selected. Finally, Y' is merged with the true label Y of the dataset to obtain the pseudo label G for training the incremental detection model.

[0033] Furthermore, a skip attention mechanism is used to learn important features of old categories, where the skip attention mechanism is:

[0034]

[0035] The attention module Attention first performs channel-domain attention processing on the input feature F to obtain M ch (F), and then perform element-wise multiplication with feature F; during the multiplication process, the channel attention value is propagated along the spatial dimension; for such a module, it is inserted at the three levels in the feature extraction module respectively to form the final skip attention mechanism, which is used to learn and retain the key feature information of the old category.

[0036] The important features learned by the attention module are adaptively learned and trained to obtain the detection network. The loss function of the detection network is:

[0037] L loss =L local +L conf +L prob

[0038] L local =λ∑∑(2-w×h)(1-CIoU)

[0039] L conf =-∑∑I obj [C′log(C)+(1-C′)log(1-C)]-λ∑∑I noobj [C′log(C)+(1-C′)log(1-C)]

[0040] L prob =∑I obj ∑[p′(c)log(p(c))+(1-p′(c))log(1-p(c))]

[0041] Among them, λ represents the confidence, w and h are the width and height of the image, and a border distance calculation method CIoU that focuses on the aspect ratio is used to calculate the accuracy of the target position. C and C' represent the confidence of the target and the result of network prediction, respectively, and p and p' represent the category probability of the target and the result of network prediction, respectively. obj The value is 1 or 0, that is, whether there is a target in the cell. If there is a target in the cell, it is 1, otherwise it is 0; I noobj : If there is no target in the cell, its value is 1, otherwise it is 0.

[0042] The present invention also provides an adaptive incremental detection system based on tag selection, comprising:

[0043] Label selection module: Use the old model to obtain information about old category targets in the data set, and obtain pseudo labels for training through the label selection algorithm;

[0044] Feature extraction module based on attention mechanism: learn and retain important feature information of old categories by skipping the attention module;

[0045] Detection network module: Using the trained detection model, for a batch of input new category images, the category and location information of all targets including new and old categories can be detected.

[0046] The tag selection module includes:

[0047] Detection module: detects the category and bounding box information of old category targets based on the preset multi-scale anchor frames;

[0048] Filtering module: Calculate the border distance value based on the obtained border coordinates and the true label of the data set, remove the border coordinates that exceed the threshold, and finally merge them with the true label of the data set.

[0049] After obtaining the feature information of the target, the important features of the old category are retained by skipping attention. In the subsequent training process of the model, the key feature information of the old category can be obtained, so that the network can adaptively learn to detect the target.

[0050] The loss function of the detection network is:

[0051] L loss =L local +L conf +L prob

[0052] Where L local is the target bounding box loss, L conf is the target confidence loss, L prob is the target category loss, and the loss function of the detection network is the sum of the three losses.

[0053] The trained network can incrementally detect all targets, including category and location information, without accessing old data.

[0054] A computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which when executed by a computing device, cause the computing device to perform a label selection adaptive incremental detection method based on a bounding box distance metric.

[0055] A computing device includes one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing an adaptive incremental detection method for label selection based on a border distance metric.

[0056] The present invention is further described below in conjunction with the accompanying drawings. The following implementation cases are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0057] Example 1

[0058] like Figure 2 As shown, a label selection adaptive incremental detection method based on border distance measurement includes the following steps:

[0059] Step 1: Use the old model to detect the data set to obtain prior knowledge of the old categories, and then combine the real annotations of the new data to obtain the pseudo labels for training the model through the label selection algorithm.

[0060] Get a new data set as a training sample, such as Figure 1 As shown, the old model is used to obtain the annotation information of the old target, including the category and location information, including the following steps:

[0061] 11) The old category detection model is used to predict the data set, and the detection results (the category and bounding box coordinates of the old target) are obtained and saved as the label of the old category.

[0062] 12) Get the real labels of the data set, combine them with the old category annotations, and use the label selection algorithm to generate the final pseudo labels for training, including the following:

[0063] A1) Read the true label of the data set (including category information and location information) and pre-set the GIOU threshold;

[0064] A2) Obtain the information of the old category target. In this process, only the bounding box coordinate information is used. At the same time, the truth of the new category target in the data set is read. The result calculated by the label selection algorithm is compared with the set threshold, and then the bounding box information exceeding the threshold is removed until all labels are selected;

[0065] A3) Combine the remaining annotation information of all old categories with the true labels of the dataset as the pseudo labels G of the training model, calculated as follows:

[0066] G=YUY″

[0067] Where Y is the true label of the dataset, and Y″ is the prior information of the old class target after being processed by the label selection algorithm, namely the category and bounding box coordinates.

[0068] Step 2: Obtain pseudo labels according to the label selection algorithm, and then perform feature selection; including:

[0069] 21) According to the size of the feature map, a preset multi-scale is used to generate a multi-scale pre-selected box of the feature map. The multi-scale preset is a result obtained by clustering the data set as a preset value, and a multi-scale anchor box is used to adapt to targets of different sizes.

[0070] According to the pseudo labels for training obtained in step 1, for the same input image, it is randomly scaled to any of the preset scales. For different feature maps, there are pre-selected boxes of different scales. Each grid unit in the feature map corresponds to 3 pre-selected boxes, corresponding to 3 scales.

[0071] 22) Calculate the IOU value for the multi-scale pre-selected boxes and true labels of the feature map, retain the candidate boxes that exceed the threshold (generally set to 0.5), and if no condition is met, retain the candidate box with the largest threshold. Multi-scale region extraction can accurately detect objects of different scales, and the candidate boxes do not need to be learned later, which improves the speed of object detection.

[0072] 23) Use the weights of the old model to initialize the new model, use multi-scale and skip attention mechanisms for feature extraction, and learn important features of the old category during the feature extraction process; the details are as follows:

[0073] B1) Load the weights of the old model, excluding the last output layer. In the first stage of training the network, do not train the parameters of the feature extraction layer, use the feature extraction weights of the old model, and let the attention module learn the important features of the old category;

[0074] B2) The attention module first performs channel-domain attention processing on the input feature F and obtains Then perform element-wise multiplication with feature F to obtain the learning of attention to the feature channel, and then the channel attention value is propagated along the spatial dimension to obtain

[0075] B3) Such attention modules are inserted at three levels in the feature extraction module respectively, and the attention learning is propagated along three scales. Through such a skip attention mechanism, the key feature information of the old category is learned and retained.

[0076] Step 3: Use the feature extraction module and skip attention to learn the features of the target, then propagate them on the three-scale network, and finally output the prediction results of the three scales to obtain the category information and location information of the target; including:

[0077] 31) Obtain the candidate box of the data set, propagate the weights obtained by the feature selection module on the three-level network, train and learn to detect the target, obtain the position offset data on three scales, and finally obtain the final predicted border through correction and IOU calculation; the details are as follows:

[0078] C1) Propagate the learned feature extraction weights forward, train the weights on three levels, and pass the weights on a single level to the next level, which connects the features twice to obtain shallow feature information and deep semantic information;

[0079] C2) Through pyramid-type multi-level learning, the predicted results are output at three scales, and the predicted categories and bounding box coordinates are obtained by correcting the results through functions;

[0080] 32) Obtain the category information and location information predicted by the network, compare the results with the true labels of the dataset, and obtain the border loss, confidence loss and category loss of the model detection:

[0081] L local =λ∑∑(2-w×h)(1-CIoU)

[0082] L conf =-∑∑I obj [C′log(C)+(1-C′)log(1-C)]-λ∑∑I noobj [C′log(C)+(1-C′)log(1-C)]

[0083] L prob =∑I obj ∑[p′(c)log(p(c))+(1-p′(c))log(1-p(c))]

[0084] Among them, λ represents the confidence, w and h are the width and height of the picture, CIoU is used to calculate the accuracy of the target position, C and C' represent the confidence of the target and the result of network prediction, respectively, and p and p' represent the category probability of the target and the result of network prediction, respectively.

[0085] 33) When the loss function converges, the training ends. The results are used as the weights of the incremental detection network, and the target category and location information are obtained based on the detection network. When detecting targets, since the borders predicted by the network are mostly redundant, non-maximum suppression is used to remove the borders with high overlap to obtain the best position detection results.

[0086] The above method uses pseudo labels obtained by the label selection algorithm, multi-scale candidate boxes, and a skip-attention mechanism for feature extraction, learns and retains important features of old categories, and then uses a multi-scale network to adaptively train new categories. It can detect targets of different scales. The above method is suitable for incremental target detection without old sample data.

[0087] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0088] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0090] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are included in the scope of the claims of the present invention to be approved.

Claims

1. A label selection adaptive incremental detection method based on border distance metric, characterized in that: include: Obtain the old category detection model, use the old model to detect the dataset to obtain the prior information of the old category, and use the label selection algorithm to combine the dataset labels to generate pseudo labels for training the detection network; Based on the generated pseudo-labels, a feature extractor is trained using skip attention to retain the important features of the old categories and adaptively train all parameters of the network; The trained detection network is used to detect the preset multi-scale anchor boxes to obtain the category and location information of the target.

2. The label selection adaptive incremental detection method based on border distance metric according to claim 1, characterized in that: The process of obtaining prior information of old categories is: Through a trained old model, a threshold is set and then the data set is detected to obtain the prior information of the old class targets, including category and location information.

3. The label selection adaptive incremental detection method based on border distance metric according to claim 1, characterized in that: The tag selection process is: For the information of old category targets obtained, only the bounding box coordinate information is used. At the same time, the true label of the new category target in the dataset is read. The result calculated by the label selection algorithm is compared with the set threshold, and then the bounding box information exceeding the threshold is removed until all labels are selected. The last old category label is merged with the true label of the dataset as the pseudo label for training the incremental detection model.

4. The label selection adaptive incremental detection method based on border distance metric according to claim 1, characterized in that: The skip attention mechanism is used to learn the important features of the old categories, where the skip attention mechanism is: The attention module Attention first performs channel-domain attention processing on the input feature F to obtain M ch (F), and then perform element-wise multiplication with feature F; during the multiplication process, the channel attention value is propagated along the spatial dimension; it is inserted at the three levels in the feature extraction module respectively to form the final skip attention mechanism, which is used to learn and retain the key feature information of the old category.

5. The label selection adaptive incremental detection method based on border distance metric according to claim 1, characterized in that: The important features learned by the attention module are adaptively learned and trained to obtain the detection network. The loss function of the detection network is: L loss =L local +L conf +L prob L local =λ∑∑(2-w×h)(1-CIoU) L conf =-∑∑I obj [C′log(C)+(1-C′)log(1-C)]-λ∑∑I noobj [C′log(C)+(1-C′)log(1-C)] L prob =∑I obj ∑[p′(c)log(p(c))+(1-p′(c))log(1-p(c))] Among them, L local is the target bounding box loss, L conf is the target confidence loss, L prob is the target category loss, λ represents the confidence, w and h are the width and height of the image, and a border distance calculation method CIoU that focuses on the aspect ratio is used to calculate the accuracy of the target position. C and C′ represent the target confidence and the network prediction result, respectively, and p and p′ represent the target category probability and the network prediction result, respectively; I obj The value is 1 or 0, that is, whether there is a target in the cell. If there is a target in the cell, it is 1, otherwise it is 0; I noobj : If there is no target in the cell, its value is 1, otherwise it is 0.

6. A label selection adaptive incremental detection system based on border distance metric, characterized in that: include: Label selection module: Use the old model to obtain information about old category targets in the dataset, and obtain pseudo labels for training the incremental detection model through a label selection algorithm based on bounding box distance measurement; Feature extraction module based on attention mechanism: learn and retain important feature information of old categories by skipping the attention module; Detection network module: Using the trained detection model, for the input new category image, detect the category and location information of all targets including new and old categories.

7. The label selection adaptive incremental detection system based on border distance metric according to claim 6, characterized in that: The label selection module includes: Detection module: detects the category and bounding box information of old category targets based on the preset multi-scale anchor frames; Filtering module: Calculate the border distance based on the obtained border coordinates and the true label of the data set, remove the border coordinates that exceed the threshold, and finally merge them with the true label of the data set.

8. The label selection adaptive incremental detection system based on border distance metric according to claim 6, characterized in that: After obtaining the feature information of the target, the important features of the old category are retained by skipping attention. In the subsequent training process of the model, the key feature information of the old category can be obtained, so that the network can adaptively learn to detect the target; The loss function of the detection network is: L loss =L local +L conf +L prob Where L local is the target bounding box loss, L conf is the target confidence loss, L prob is the target category loss, and the loss function of the detection network is the sum of the three losses.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the label selection adaptive incremental detection method based on border distance measurement described in any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the label selection adaptive incremental detection method based on border distance measurement described in any one of claims 1 to 5 are implemented.

Images