A structured filter learning small sample target detection method

Abstract

The application discloses a small sample target detection method based on structured filter learning, and comprises the following steps: preparing a small sample target detection data set and a base class target detection data set, and building a network structure; a backbone feature extraction network uses a CSPDarknet53, and a general feature enhancement module is introduced; a small sample target detection deep neural network based on structured filter learning is built; the network is trained using the base class data set; on the small sample target detection data set, a KSVD algorithm is used to hierarchically learn a dictionary of input feature maps of the structured filter, which is used as a weight parameter of the filter; after one forward propagation process, the initialization of the parameters of the structured filter is completed; the initialized network model is used to continue training on the small sample data set; and a trained neural network is used to complete a detection task. According to the method, a small sample data set can be used to train a target detection deep neural network with good generalization performance.

Description

Technical Field

[0001] The present invention belongs to the field of small sample target detection, and in particular relates to a small sample target detection method based on structured filter learning. Background Art

[0002] Computer vision is a highly sought-after research area within artificial intelligence. In today's information-based society, with vast amounts of visual information, such as images and videos, requiring processing and application every day, research in this field has become particularly necessary and urgent. Object detection is a key branch of computer vision. The primary task of object detection is to identify one or more specific objects within a digital image and provide a confidence score for the category to which the objects belong. As one of the most fundamental problems in computer vision technology, object detection is the foundation and prerequisite for many other advanced computer vision tasks, such as image segmentation, image understanding, object tracking, and intelligent driving. Furthermore, object detection itself is widely used in various visual scenarios, such as pedestrian recognition, vehicle recognition, and face recognition. The development of object detection closely follows the progress of deep learning. The development and popularization of deep learning is inseparable from the development of parallel computing. In 2006, NVIDIA, a renowned graphics card manufacturer, launched CUDA (Compute Unified Device Architecture), a general-purpose parallel computing architecture based on its graphics cards. CUDA enabled graphics cards to handle large-scale parallel computing tasks. Subsequently, companies such as Microsoft and Apple also launched their own cross-platform parallel computing interfaces. The promotion of these tools has greatly improved the performance of deep neural networks, and the research and application of artificial intelligence has gradually become popular. Traditional object detection algorithms mostly use manually defined features, detecting images through a sliding window method and analyzing the features within the window, such as Haar features, HOG features, and LBP features. The drawback of these traditional methods is that they require a large amount of prior knowledge in the field. The manually designed features are not targeted to the distribution of the dataset, making them difficult to adapt to the ever-changing application scenarios. Since the advent of AlexNet, object detection methods based on deep convolutional neural networks have attracted much attention. Convolutional neural networks have a multi-layer convolution structure. For the input raw signal, this structure can progressively extract features at each stage and gradually obtain higher-order semantic information. This semantic information is a necessary prerequisite for further processing of the raw signal. In target detection, this semantic information is fused at multiple scales, and then the corresponding algorithm is used to further process the output prediction box coordinates, category, and confidence.

[0003] Although deep learning has surpassed humans in many fields, deep learning needs to learn a large number of samples if it wants to make achievements in a new field. In comparison, human learning is particularly simple and efficient. With only a photo, humans can identify a person in a crowd. Such a task is very difficult to complete using deep learning. However, there are a large number of similar problems in the real world. Most of the problems faced by deep learning cannot find sufficient data, such as failure data in industrial production and characteristic data of new molecules in drug research and development. The data collection itself requires high costs; there are also a considerable number of tasks that do not have the conditions to obtain a large number of samples due to ethical issues or privacy, security and other issues. In many cases, even if there are sufficient samples, it takes a lot of manpower and material resources to pre-label these data, which is time-consuming and labor-intensive. Summary of the Invention

[0004] The purpose of the present invention is to provide a small sample target detection method using a structured filter.

[0005] The technical solution for achieving the purpose of the present invention is: a small sample target detection method using a structured filter, comprising the following steps:

[0006] Step 1: Prepare a small sample target detection dataset and a base class target detection dataset;

[0007] Step 2: Build the network structure: Use CSPDarknet53 as the backbone feature extraction network and introduce a general feature enhancement module; use the structured filter S_conv to construct the convolution unit S_CBL and the Yolo detection head S_Yolo; then, based on the feature fusion network structure of YOLOv4, build a small sample target detection deep neural network with structured filter learning;

[0008] Step 3, the first stage of training, uses the base class dataset to train the network;

[0009] Step 4: In the second stage of training, the backbone feature extraction network CSPDarknet53 and the general feature enhancement module use the weights pre-trained in the base class object detection dataset in step 3. On the small sample object detection dataset, the KSVD algorithm is used to hierarchically learn the dictionary of the structured filter input feature map as the filter weight parameters. After a forward propagation process, the parameters of the structured filter are initialized.

[0010] Step 5: The third stage of training, using the initialized network model to continue training on the small sample data set;

[0011] Step 6: Use the trained neural network to complete the detection task.

[0012] Compared with the prior art, the present application has the following advantages: (1) The present application is suitable for general target detection tasks of data sets containing a small amount of samples, and a well-generalized target detection deep neural network can be trained with a small amount of sample data; (2) In the first training stage of the present application, the general feature extraction module is introduced, the feature map extracted by the backbone feature extraction network is associated with the general prototype after the channel dimension is expanded through convolution operation, the multi-level weight information of the feature map after the channel dimension is expanded is extracted as a feature, and the residual error of the feature map and the general prototype is combined to obtain a feature map containing general features, which is input into the network model in the form of residual error, and the feature enhancement is completed, and the general features in the base class samples are effectively introduced into the network model; (3) In the second training stage of the present application, the KSVD sparse representation algorithm is used to generate a sparse representation dictionary based on the backbone feature extraction network and the general feature extraction module generated in the previous stage, and the structure of the convolution filter in the subsequent network is optimized using the dictionary, and the initialization of other parts of the network is completed through an unsupervised forward propagation process, so that the convolution filter can more directly extract the features of the input feature map on the small sample data set, and the performance of the filter feature extraction is greatly enhanced; at the same time, the intensity parameter is introduced for the structured filter, and the feature expression ability of the convolution filter is effectively improved; (4) After the training of the first two stages, the network is provided with a better initialization parameter, so that the training on the small sample data set in the third stage can quickly converge and achieve good generalization performance. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a small sample target detection method flow chart of the structured filter learning of the present application.

[0014] Figure 2 is a hierarchical dictionary filter learning method flow chart of the present application based on KSVD.

[0015] Figure 3 is a schematic diagram of the present application using a structured filter to construct a convolution unit S_CBL and a Yolo detection head S_Yolo.

[0016] Figure 4 is a structure and principle schematic diagram of the general feature enhancement module of the present application.

[0017] Figure 5 is a network structure schematic diagram in the detection method of the present application. DETAILED DESCRIPTION

[0018] As shown in Figure 1 , a small sample target detection method of structured filter learning of the present application comprises the following steps:

[0019] Step 1: Prepare a small sample target detection dataset and a base class target detection dataset;

[0020] Step 2: Build the network structure. The backbone feature extraction network uses CSPDarknet53 and introduces a general feature enhancement module. The structured filter S_conv is used to construct the convolution unit S_CBL and the Yolo detection head S_Yolo. Combined with the feature fusion network structure of YOLOv4, a small sample target detection deep neural network with structured filter learning is built.

[0021] Step 3, the first stage of training, uses the base class dataset to train the network;

[0022] Step 4: In the second stage of training, the backbone feature extraction network CSPDarknet53 and the general feature enhancement module use the weights pre-trained in the base class object detection dataset in step 3. On the small sample object detection dataset, the KSVD algorithm is used to hierarchically learn the dictionary of the structured filter input feature map as the filter weight parameters. After a forward propagation process, the parameters of the structured filter are initialized.

[0023] Step 5: The third stage of training, using the initialized network model to continue training on the small sample data set;

[0024] Step 6: Use the trained neural network to complete the detection task.

[0025] Furthermore, a deep neural network for small-sample target detection based on structured filter learning is first constructed. The network introduces a universal feature extraction module. The feature map extracted by the backbone feature extraction network is used to expand the channel dimension through convolution operation and then associated with the universal prototype. The multi-level weight information in the feature map after the expanded channel dimension is extracted as features, and the residual of the feature map and the universal prototype is combined to obtain a feature map containing universal features, which is input into the network model in the form of residual to complete the feature enhancement. A structured filter containing filter parameters and intensity parameters is introduced, and the filter parameters are initialized by a hierarchical dictionary filter learning method based on KSVD. The features of the input feature map can be extracted more directly on a small-sample dataset, which greatly enhances the performance of the filter in extracting features.

[0026] The execution process of the small sample target detection method of structured filter learning is:

[0027] Step 1: Prepare a small-sample target detection dataset and a base-class target detection dataset. The small-sample target detection dataset is a target detection dataset with N categories and K samples per category, where K is less than 30. The base-class target detection dataset has more than or equal to N categories, and the number of samples in each category is much greater than K. Common base-class datasets include the PASCAL VOC dataset and the MSCOCO dataset.

[0028] Step 2: Build the network structure. The backbone feature extraction network uses CSPDarknet53 and introduces a general feature enhancement module. CSPDarknet53 has three outputs of different scales, such as Figure 5 As shown in the BackBone section. For these three outputs, three general feature enhancement modules are introduced. The structure and principle of the general feature enhancement module are as follows Figure 4 As shown, is the input feature map of the general feature enhancement module, is the output feature map of the general feature enhancement module, the general prototype V i It is an image-level feature information descriptor. First, the input feature map is expanded through the convolution channel to generate the channel expansion matrix I of X, I = W g *X+b g , For the j-th channel vector I j , get I j The position corresponds to c i The component elements and Calculate the component element sum Relative to the entire channel vector I j The weight E j,i , Then, for the j-th channel vector I j , get I j Elements and Calculate the weight of this element and relative to the channel expansion matrix I The weight E j,i and weights After summing, it is used as a residual block The weights constitute the feature descriptor V i,j ,Right now Using feature descriptor V i,j Construct feature map V, Perform maximum pooling on the feature map V on the channel to obtain V * , Finally, X and V *The residual is used as the output P to complete the construction of the general feature enhancement module. The structured filter S_conv contains the weight parameter W and the strength parameter t. The filter involved in the convolution calculation is t⊙W, where ⊙ represents element-wise multiplication. The structured filter S_conv is used to construct the convolution unit S_CBL and the Yolo detection head S_Yolo. Figure 3 As shown in the figure, the convolution unit S_CBL contains three structures: "structured filter-batch normalization (BatchNormalization)-activation", using the LeakyRELU activation function; the detection head S_Yolo is composed of the convolution unit S_CBL and 1×1 convolution conv. Combined with the feature fusion network structure of YOLOv4, a small sample target detection deep neural network with structured filter learning is built. The constructed network structure is shown in the figure. Figure 5 As shown;

[0029] Furthermore, in step 2, the feature fusion network is constructed based on the structure of SPP and PAN in YOLOv4 and a combination of S_CBL convolutional units.

[0030] Step 3: The first phase of training sets all strength parameters t to 1 and freezes them during training. The network is trained using the base dataset. This step aims to train the backbone feature extraction network and the general feature extraction module using the base dataset, thereby incorporating prior information from the base dataset, which contains a large number of samples, into the model.

[0031] Step 4, the second stage of training, the backbone feature extraction network CSPDarknet53 and the general feature enhancement module use the weights pre-trained in the base class target detection dataset in step 3, and use the KSVD algorithm to hierarchically learn the dictionary of the structured filter input feature map on the small sample target detection dataset as the weight parameter of the filter. After a forward propagation process, the parameters of the structured filter are initialized. The specific method is as follows: First, sample in the small sample target detection dataset, with no less than 2 samples for each category. Each sample is cropped according to the detection box in the grand truth, and the cropped samples are spliced ​​as the input of the algorithm. After the backbone feature extraction network and the general feature extraction module, deep image features containing general features are extracted. These deep feature maps are hierarchically learned in the subsequent network propagation process. The dictionary of its sparse representation is taken as an example of the lth structured filter S_conv. Figure 2 As shown, the input feature map of S_conv is X l , c l 、w l 、h l X l Number of channels, width and height; Y l For Xl The expanded two-dimensional matrix, W l is the convolution filter parameter of the lth layer, Y l Use KSVD to generate the dictionary matrix D l , Then D l Rearrange according to the size of the filter in this layer to get W l , Continue using X l and W l After performing the convolution calculation and activation function calculation in the network forward propagation, we get X l+1 During the process, the intensity parameter t remains at 1;

[0032] Step 5, the third phase of training, continues training on the small sample dataset using the initialized network model. Since each structured filter has a strength parameter set, the forward propagation process of the relevant convolutional unit in the network is, f(X,W,b,t)=X*(t⊙W)+b), where X is the input image or feature map, W is the initialized structured filter, t is the strength parameter, * is the convolution operation, and ⊙ represents element-wise multiplication. The strength parameter is regularized using ElasticNet, that is, the loss function during training needs to be added with λ1‖θ‖1+λ2‖θ‖2, where λ1 and λ2 are hyperparameters and θ is the set of parameters t.

[0033] Step 6: Use the trained neural network to complete the detection task.

[0034] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A small sample target detection method using structured filter learning, characterized in that: The following steps are involved: Step 1: Prepare a small sample target detection dataset and a base class target detection dataset; Step 2: Build the network structure: Use CSPDarknet53 as the backbone feature extraction network and introduce a general feature enhancement module; Use the structured filter S_conv to construct the convolution unit S_CBL and the Yolo detection head S_Yolo; then build a small sample target detection deep neural network based on the feature fusion network structure of YOLOv4; The structured filter S_conv contains the weight parameter W and the strength parameter t. The filter involved in the convolution calculation is t⊙W, where ⊙ represents element-wise multiplication. The convolution unit S_CBL uses the LeakyRELU activation function; the detection head S_Yolo consists of the convolution unit S_CBL and 1×1 convolution; Based on the structure of SPP and PAN in YOLOv4, a feature fusion network is built using a combination of S_CBL convolutional units; CSPDarknet53 has three outputs of different scales. For these three outputs, three general feature enhancement modules are introduced; In the general feature enhancement module, is the input feature map of the general feature enhancement module, is the output feature map of the general feature enhancement module, the general prototype V i It is an image-level feature information descriptor; The input feature map is expanded through the convolution channel to generate the channel expansion matrix I of X: For the j-th channel vector I j , get I j The position corresponds to c i The component elements and Calculate the component element sum Relative to the entire channel vector I j The weight E j,i , For the j-th channel vector I j , get I j Elements and Calculate the weight of this element and relative to the channel expansion matrix I The weight E j,i and weights After summing, it is used as a residual block The weights constitute the feature descriptor V i,j ,Right now Using feature descriptor V i,j Construct feature map V, Perform maximum pooling on the feature map V on the channel to obtain V * , Finally, X and V * The residual is used as the output P; Step 3, the first stage of training, uses the base class dataset to train the network; Step 4: In the second stage of training, the backbone feature extraction network CSPDarknet53 and the general feature enhancement module use the weights pre-trained in the base class object detection dataset in step 3. On the small sample object detection dataset, the KSVD algorithm is used to hierarchically learn the dictionary of the structured filter input feature map as the filter weight parameters. After a forward propagation process, the parameters of the structured filter are initialized. Step 5: The third stage of training, using the initialized network model to continue training on the small sample data set; Step 6: Use the trained neural network to complete the detection task.

2. The small sample target detection method using structured filter learning according to claim 1, characterized in that In step 1, the small sample target detection dataset is a target detection dataset required by the task, containing N categories and K samples in each category, where K is less than 30.

3. The small sample target detection method using structured filter learning according to claim 2, characterized in that In step 1, the number of base class object detection dataset categories is greater than or equal to N, and the number of samples in each category is much greater than K; the base class datasets include the PASCAL VOC dataset and the MSCOCO dataset.

4. The small sample target detection method using structured filter learning according to claim 1, characterized in that In the first stage of training in step 3, all intensity parameters t are set to 1 and frozen during the training process; the purpose of this step is to use the base class dataset to train the backbone feature extraction network and the general feature extraction module, and introduce the prior information in the base class dataset containing a large number of samples into the model.

5. The small sample target detection method using structured filter learning according to claim 1, characterized in that In step 4, the KSVD algorithm is used to hierarchically learn the dictionary of the input feature map of the structured filter to generate the weight parameters of the filter. The specific method is as follows: First, sample in the small sample target detection data set, with no less than 2 samples for each category. Each sample is cropped according to the detection box in the grand truth, and the cropped samples are spliced ​​as the input of the algorithm. After passing through the backbone feature extraction network and the general feature extraction module, deep image features containing general features are extracted. These deep feature maps are hierarchically learned in the subsequent network propagation process. The sparse representation dictionary. For the lth structured filter S_conv, the input feature map of S_conv is X l , c l 、w l 、h l X l Number of channels, width and height; Y l For X l The expanded two-dimensional matrix, W l is the convolution filter parameter of the lth layer, Y l Use KSVD to generate the dictionary matrix D l , Then D l Rearrange according to the size of the l-th layer filter to get W l , Continue using X l and W l Perform the convolution calculation and activation function calculation in the network forward propagation to obtain X l+1 ; The intensity parameter t remains at 1.

6. The small sample target detection method using structured filter learning according to claim 1, characterized in that In step 5, the forward propagation process of the convolution unit S_conv in the network is: f(X,W,b,t)=X*(t⊙W)+b), where X is the input image or feature map, W is the initialized structured filter, b is the randomly initialized bias term, t is the strength parameter, * is the convolution operation, and ⊙ represents element-wise multiplication. The regularization method of the strength parameter is ElasticNet, that is, the loss function in the training process needs to be added with λ1||θ||1+λ2||θ||2, λ1 and λ2 are hyperparameters, and θ is the set of parameters t.

Images