A small sample target detection method based on meta-learning method

Abstract

The application discloses a small sample target detection method based on a meta learning method, and can realize accurate target detection under a small sample condition.The application mainly comprises the following steps: obtaining a public target detection data set, and then constructing a training set and a test set; constructing a small sample target detection network based on meta learning; and training, verifying and testing the small sample target detection algorithm based on meta learning by using the training set and the test set.Compared with current main small sample target detection algorithms, the application can overcome overfitting and underfitting of certain specific sample categories in the training process, has higher detection accuracy on all sample categories, and is a more efficient and accurate small sample target detection algorithm.

Description

TECHNICAL FIELD

[0001] The application relates to a small sample target detection method based on a meta-learning method, and is suitable for the technical field of small sample target detection in computer vision. BACKGROUND

[0002] With the development of human society, the important source of information obtained by human beings gradually becomes images and videos. A method based on a large number of image and video samples for deep learning is more and more mature, but in the industry, there is still a large demand for target detection of very few categories. It is a difficult problem to apply target recognition technology to these few categories, and the most important problems are as follows: on the one hand, there is a lack of data set, that is, a lack of data sample quantity of few categories; on the other hand, high-quality sample labeling is obtained. The labeling work of these data not only takes time but also needs certain professional technical background. Based on the above two difficulties, the application of the target detection technology based on deep learning in the industry still has great challenges. If a relatively good algorithm model can be trained and learned from a small amount of industrial target data, it will be of great benefit to the development of industrial target detection, so it is an urgent need to carry out target detection under the condition of a small amount of samples. Therefore, the target detection under the condition of a small amount of samples has very high research value and has a wide application in the fields of medical treatment, agriculture, ocean, military and the like.

[0003] Because the data set of the target category is small, simply increasing the iteration number of deep learning on the training set in expectation of obtaining a detection model with higher performance will not greatly improve the detection effect of the detection model, but will cause overfitting of the detection model on the test set, reduce the generalization ability of the model and reduce the detection precision on the test sample. In recent years, the model-independent meta-learning architecture in the meta-learning method has emerged in the field of small sample learning. The core idea is to embed the target detection network into the meta-learning model algorithm, let the detection model learn on the training set through the training method of meta-learning, and neither overfit nor underfit the sample data of the training set, and make the detection model obtain good detection effect on the new target category of the test set through a small amount of sample iteration of the trained meta-learning model.

[0004] In recent years, the meta-learning method has emerged in the field of deep neural network training. By embedding the deep neural network model into the meta-training framework, the deep neural network can learn autonomously, and the sensitivity of the deep neural network model to the sample data size can be reduced, that is, the deep neural network model based on meta-training can be retrained on a small amount of samples and converge, thereby obtaining excellent performance on new samples. Therefore, the application provides a small sample target detection algorithm based on a meta-learning method. SUMMARY

[0005] For the existing small sample target detection problem, a small sample target detection method based on meta-learning method is proposed. The main contents include: 1) constructing a meta-learning process, including a meta-training stage and a meta-testing stage. Specifically, in the meta-training stage, FSDBO is added to reduce the loss value imbalance problem between different tasks in the same batch, so that the loss value of a batch is not dominated by overfitting or underfitting tasks, thereby avoiding the bias of the trained meta-learning model. Specifically, the test stage includes a model fine-tuning stage on new small sample categories and a detection accuracy verification stage. 2) A target detection network model is constructed, which extracts various different size feature maps of the input image through a feature extraction module, and inputs them to a feature fusion module for different scale feature fusion. Finally, according to the different sizes of the instances in the image, they are automatically assigned to different feature layers to input to the classification module and the regression module for instance classification and position bounding box regression. In the meta-testing stage, a limited number of training iterations are performed on the new small sample categories in the test set, so that the model can quickly converge to a relatively optimal position in the parameter space. The model converged to the optimal performance has higher average precision mAP than the current mainstream model in the small sample target detection field, and the results show that the application can effectively improve the small sample target detection accuracy.

[0006] To achieve the above object, the application provides the following technical scheme:

[0007] A small sample target detection algorithm based on meta-learning method, characterized by comprising the following steps:

[0008] Step S1: Obtain a small sample target detection data set, and construct a training set and a test set;

[0009] Step S2: Construct a small sample target detection algorithm based on meta-learning;

[0010] Step S3: Construct an index map for each data set, and construct a data sampler according to different strategies to sample all task samples according to different task requirements;

[0011] Step S4: In the meta-training stage, FSDBO is added, and the meta-model is trained on the training set;

[0012] Step S5: In the meta-testing stage, the trained model is fine-tuned on the test set and the target detection performance is verified.

[0013] Further, the step S1 small sample target detection data set contains a data set VOC.

[0014] Further, the meta-learning architecture constructed in step S2 includes a meta-training phase and a meta-testing phase. Specifically, the testing phase includes a model fine-tuning phase on new small sample categories and a detection accuracy verification phase. The constructed target detection network includes a feature extraction module, a feature fusion module, a classification module, and a regression module.

[0015] Further, the index map of each data set is constructed in step S3, which is a dictionary {key1:value1, key2:value2, …, key n :value n}, wherein the key is all categories of each data set, and the value corresponding to each key is all image index numbers corresponding to each category in the data set. Meanwhile, the data sampler also divides the data set into a training data set D train and a test data set D tes according to different groups.

[0016] Further, step S4 specifically includes:

[0017] Step S401: Randomly divide the samples collected by the data collector from the training set into a support set S i and a query set Q i , and perform preprocessing to adjust the size of the input image and the corresponding true label to HxW using an interpolation algorithm, wherein H represents the image height and W represents the image width. Subsequently, image data enhancement processing is performed, including rotation, cropping, scaling, affine transformation, and flipping.

[0018] Step S402: Input the preprocessed support set S i to the target detection network for one forward propagation, and input the forward propagation result and the sample label to the loss functions Focal Loss and Smooth to respectively calculate the classification loss and the regression loss , and add them to obtain the total loss

[0019] Step S403: Obtain the gradient value of each parameter by one pass of chain derivation on the total loss , and update the parameters of the model by multiplying the gradient value by the inner learning rate a:

[0020]

[0021] Step S404: Analogous to step S403, use the updated parameters and the support set S i to update the latest parameters for n step times Finish Use support set S i The training process.

[0022] Step S405: Use this task Query set Q i Perform an inference and calculate the loss. This loss is then saved, completing one task in the meta-training phase. All meta-learning processes.

[0023] Step S406: Repeat n task The meta-training process for each task (steps S401-S405) yields n. task loss And for this n task loss Inputting this into FSDBO yields the final loss function for this batch. right Backpropagation is performed, and the Adam optimizer is used to perform gradient descent and parameter updates on the initial meta-model parameters according to the set learning rate β, thus completing one gradient update.

[0024]

[0025] Step S407: The specific implementation of FSDBO is as follows: within a batch of tasks, first obtain the loss {l} of all tasks in the same batch on the query set. i Let i = 1, 2, ..., N, where N is the total number of all tasks in this batch, and calculate the average of all losses. according to The standard deviation σ of the losses for all tasks in this batch was calculated. 2 :

[0026]

[0027] Step S408: Calculate the obtained variance σ 2 Multiply by the modulation factor η, for each {l i}, calculate whether it is in the interval If the value is higher than this range, then the model is considered to be within this range. Poor training performance on the task indicates underfitting; however, if the performance falls below this range, the model is considered to be poorly fitted. Overfitting occurs, so only the task loss value within the interval is retained. And on The mean value is used to obtain the meta-learning loss value for this batch. and loss value Perform backpropagation to update the meta-model parameters.

[0028] Further, the step S5 specifically comprises a fine-tuning phase and a verification phase:

[0029] Step S501: In the fine-tuning phase, the model in the training phase is first selected by the sample sampler DataSampler in the test set D test n tasks of the task n of the training set, k and n step Unless otherwise specified in a specific experiment, the default setting is consistent with that on the training set.

[0030] Step S502: Use the n x k support set samples The original model is updated according to the learning rate alpha to obtain the meta model relative to the task Gradient updated submodel theta i In the verification phase, the n classes of the updated submodel theta of the task i , respectively, the detection average precision AP of the samples in the query set of the class is calculated, and then the average precision mAP of all classes is calculated, so as to finally obtain the model precision verification result of a task .

[0031] Step S503: For a model, 100 tasks are collected in the test phase, and the above gradient update based on the support set of each task is executed, and the average precision mAP of all classes is calculated through the test query set , so as to obtain the mAP of each task, that is, Map 1-100 ={Map 1, Map2,…,Map 100}, and the average of the 100 mAPs is obtained. The average detection precision value of a model.

[0032] The beneficial effects of the present application are: the small sample target detection algorithm based on the meta learning method constructed by the present application can perform meta training on the target detection model through meta learning, so as to obtain a good target detector in a few iterations in a new class. And in the meta training phase, the variance balance optimizer FSDBO is innovatively used to balance the loss values between different tasks in the same batch, so as to avoid some tasks from causing biased training on the meta learning model due to overfitting or underfitting. Thus, the detection performance of the model on new small sample classes in the test set is improved. Specifically, the detection index mAP of the small sample target detection algorithm based on meta learning on the test set is obviously improved. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1A flow chart of a small sample target detection algorithm based on a meta-learning method in embodiment 1.

[0034] Figure 2 A structure diagram of a target detection network in embodiment 1.

[0035] Figure 3 A structure diagram of a classification module and a regression module in embodiment 1.

[0036] Figure 4 A comparison diagram of effects before and after using a small sample target detection algorithm based on a meta-learning method in embodiment 1, (a) and (c) are detection effect diagrams before using the algorithm, and (b) and (d) are detection effect diagrams after using the algorithm. DETAILED DESCRIPTION

[0037] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0038] Embodiment 1

[0039] Reference Figures 1-4 , the present embodiment provides a small sample target detection algorithm based on a meta-learning method.

[0040] Specifically, with reference to Figure 1 , the method specifically includes:

[0041] Step S1: acquiring a small sample target detection data set VOC, and constructing a training set and a test set;

[0042] More specifically: the constructed training set includes 15 class picture label pairs in the VOC; and the constructed test set includes 5 class picture label pairs in the VOC.

[0043] Step S2: constructing a small sample target detection algorithm based on meta-learning;

[0044] More specifically, the constructed meta-learning architecture includes a meta-training stage and a meta-testing stage, and specifically, the testing stage includes a model fine-tuning stage on a new small sample class and a detection accuracy verification stage. The constructed target detection network includes a feature extraction module, a feature fusion module, a classification module and a regression module, and the detailed architecture is as shown in Figure 3 .

[0045] Step S3: constructing an index graph for each data set and constructing a data sampler according to different strategies to sample all task samples according to different task requirements;

[0046] More specifically, the index graph constructed for each data set is characterized in that the index graph is a dictionary {key1:value1, key2:value2, …, key n :value n}, wherein the key is all categories of each data set, and the value corresponding to each key is all image index numbers corresponding to each category in the data set, and the data sampler also divides the data set into a training data set D train and a test data set D test according to different groups.

[0047] Step S4: in the meta-training phase, adding FSDBO to perform meta-model training on the training set.

[0048] More specifically, the meta-training phase includes the following processes:

[0049] Step S401: randomly dividing the samples collected by the data collector from the training set into a support set S i and a query set Q i , and pre-processing to adjust the size of the input image and the corresponding true label to HxW using an interpolation algorithm, wherein H represents the image height and W represents the image width, and then performing image data enhancement processing.

[0050] Step S402: inputting the pre-processed support set S i to the target detection network for one forward propagation, inputting the forward propagation result and the sample label to the loss function Focal Loss and SmoothL1 respectively to calculate the classification loss and the regression loss and adding them to obtain the total loss

[0051] Step S403: performing one pass of chain derivation on the total loss to obtain the gradient value of each parameter and multiplying the gradient value by the inner learning rate a to update the parameters of the model:

[0052]

[0053] Step S404: using the updated parameters and the support set S i to update the latest parameters step n times to complete Support set S i Process of training.

[0054] Step S405: Use this task Query set Q i Perform a reasoning and calculate the loss And save this loss, complete the meta-training phase of a task All meta-learning processes.

[0055] Step S406: Repeat n task The meta-training process of the task (step S401-step S405) to obtain n task Loss And input the n task Loss Into the FSDBO to get the final loss function of this batch Backpropagation , using the Adam optimizer to update the parameters according to the set learning rate β The gradient of the initial meta-model parameters is reduced and the parameters are updated, completing a gradient update.

[0056]

[0057] Step S407: FSDBO is specifically implemented as, in a batch of tasks, first get all tasks in the same batch on the query set loss {l i , i = 1, 2, …, N}, N is the total number of all tasks in this batch, and calculate the mean of all losses According to Calculate the standard deviation σ 2 of all task losses in this batch:

[0058]

[0059] Step S408: Multiply the obtained variance σ 2 By the modulation factor η, for each {l i}, calculate whether it is in the interval , if it is higher than the interval, it is considered that the model has poor training performance on Task underfitting phenomenon, and if it is lower than the interval, it is considered that the model has overfitting to Therefore, only the task loss value in the interval is retained And take the average of To get the meta-learning loss value of this batch And loss value Backpropagation, update the meta-model parameters.

[0060] Step S5: In the meta-test phase, the trained model is fine-tuned on the test set and the target detection performance is verified.

[0061] More specifically, the meta-test phase includes a fine-tuning phase and a verification phase:

[0062] Step S501: In the fine-tuning phase, the model in the training phase is first selected by the sample sampler DataSampler in the test set D tes of a task n is consistent with the training set, and k and n step Unless otherwise specified in a specific experiment, the default is consistent with the settings on the training set.

[0063] Step S502: Use the n x k support set samples to update the parameters of the original model according to the learning rate a to obtain the meta-model relative to this task Gradient updated submodel θ i In the verification phase, the average precision AP of the query set samples of the n classes of the updated submodel θ of the task i is calculated, and then the average precision mAP of all classes is calculated, so as to finally obtain the model precision verification result of a task .

[0064] Step S503: For a model, 100 tasks are collected in the test phase and the above gradient update based on the support set of each task is performed, and the average precision mAP of all classes is calculated through the test query set , so as to obtain the mAP of each task, i.e. Map 1-100 ={Map 1, Map2,…,Map 100}, and the average of the 100 mAPs is obtained. The average detection precision value of a model.

[0065] The pseudo code of the target detection network in Embodiment 1 is as follows:

[0066]

[0067] The preferred embodiments of the application are described in detail above. It should be understood that those skilled in the art can make many modifications and changes without creative labor based on the concept of the present application. Therefore, any technical solution obtained by logical analysis, reasoning or limited experiment based on the prior art according to the concept of the present application shall be within the protection scope determined by the claims.

Claims

1. A small sample target detection method based on a meta-learning method, characterized in that, The method comprises the following steps: Step S1: Obtain a small sample target detection data set, and construct a training set and a test set; Step S2: Construct a small sample target detection network based on meta-learning; Step S3: Construct an index map for each data set, and construct a data sampler based on different strategies, and then generate task samples according to different tasks; Step S4: Perform the meta-training phase of small sample target detection, and use the modified standard deviation balancing operator strategy to balance the weights of each loss during the training process; Step S5: Perform the meta-test phase of small sample target detection, fine-tune the trained model on the small sample test set, and verify the target detection performance; The small sample target detection data set in step S1 comprises a data set VOC; The small sample target detection network based on the meta-learning method constructed in step S2 comprises a feature extraction module, a feature fusion module, a classification module and a regression module; The step S3 of constructing the index map of each data set is a dictionary {key1:value1, key2:value2, …, key n :value n}, wherein the key is all the categories of the data set, and the value corresponding to each key is all the image index numbers corresponding to each category in the data set. Meanwhile, the data sampler also divides the data set into a training data set D train and a test data set D test according to different groups. Step S4 specifically comprises: Step S401: randomly divide the samples collected by the data collector from the training set into a support set S i and a query set Q i , and pre-process the input image and the corresponding true label to adjust the size of the input image and the corresponding true label to HxW by using an interpolation algorithm, where H represents the image height and W represents the image width, and then perform image data enhancement processing; Step S402: the preprocessed support set S i Forward propagation is performed on the n x k pictures, and the forward propagation result is compared with the sample label The input is input into the loss function Focal Loss and Smooth The classification loss And the loss of regression are calculated respectively, and the total loss is obtained by adding them Step S403: Obtain the gradient value of each parameter by applying the chain rule to the total loss and update the parameters of the model by multiplying the gradient value by the inner learning rate α: Step S404: Similar to step S403, the updated parameters and support set S i A total of n step times of updates to obtain the latest parameters Complete the task Support set S i The process of training; Step S405: Use the task query set Q i Perform one inference and compute loss and save this loss, complete the meta-training phase one task All meta-learning processes; Step S406: Repeat n task The meta-training process for each task, steps S401-S405, yields n. task loss And for this n task loss The input is fed into the variance-balanced optimizer FSDBO to obtain the final loss function for this batch. right Perform backpropagation, use the Adam optimizer to perform gradient descent and parameter update on the initial meta-model parameters according to the set learning rate β, and complete one gradient update; The variance balance optimizer FSDBO is specifically implemented as follows: in a batch of tasks, first, the loss of all tasks in the same batch on the query set is obtained i , i = 1, 2, …, N}, N is the total number of all tasks in the batch, and the mean value of all losses is calculated According to , the standard deviation v of the loss of all tasks in the batch is calculated 2 : The obtained variance σ 2 Multiply by the modulation factor η, for each {l i }, calculate whether it is in the interval If the value is higher than this range, then the model is considered to be within this range. Poor training performance on the task indicates underfitting; however, if the performance falls below this range, the model is considered to be poorly fitted. Overfitting occurs, so only the task loss value within the interval is retained. And on The mean value is used to obtain the meta-learning loss value for this batch. and loss value Perform backpropagation to update the meta-model parameters.

2. The method of claim 1, wherein the method is based on a meta-learning method. Step S5 specifically comprises a fine-tuning phase and a verification phase: Step S501: In the fine-tuning stage, the small sample target detection algorithm based on the meta-learning method converged in the meta-training stage first selects a task in the test set D test by the sample sampler DataSampler wherein the parameters of the task include n, k and n step , all of which are consistent with the settings in the meta-training stage; Step S502: use the n x k support set samples The parameter of the original model is updated according to the learning rate a to obtain the meta model relative to the task Gradient updated sub model θ i In the verification stage, the updated task Sub model θ i The n classes of the sub model θ The average precision AP of the query set sample of each class is calculated, and then the average precision mAP of all classes is calculated, so as to finally obtain the model precision verification result of a task Step S503: For a model, 100 tasks are collected in the test stage and the gradient update based on the above per-task test support set is performed, and the average precision mAP of all categories is calculated through the test query set , so as to obtain the mAP of each task, i.e., Map 1-100 ={Map1,Map2,…,Map 100} , and the average of the 100 mAPs is obtained as the average detection precision value of the model.

Images