An object detection method based on optimized anchor box mechanism

Abstract

The invention provides a target detection method based on an optimized anchor frame mechanism, and relates to the technical field of target detection. The method obtains a feature map of the number of pyramid levels corresponding to a to-be-detected instance image through a skeleton and a neck of a target detection model; then designs anchor frame parameters according to the feature map of the number of pyramid levels to generate an anchor frame; at the same time, the feature map passes through a feature adaptation module to obtain a new feature map; the new feature map and the anchor frame are input into the head of the target detection model for training; the feature map with the smallest sum of the average values ​​of various losses of the anchor frames of each feature map is selected to train the target detection model; finally, the trained target detection model is used to perform target detection on the to-be-detected image. The method designs anchor frame parameters for the target detection model in a forward operation, and reduces the cost of manually adjusting the hyperparameters of the anchor frame. The feature map is selected and trained according to the sum of the average values ​​of each loss of each forward training, so as to achieve optimization in training.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection based on deep learning, and in particular to a target detection method based on an optimized anchor frame mechanism. Background Art

[0002] Object detection is a hot topic in computer vision and digital image processing. It is widely used in many fields such as robot navigation, intelligent video surveillance, industrial inspection, aerospace, etc. It has important practical significance to reduce the consumption of human capital through computer vision. Therefore, object detection has become a hot topic in theory and application research in recent years. It is an important branch of image processing and computer vision, and it is also the core part of intelligent monitoring system. At the same time, object detection is also a basic algorithm in the field of pan-identity recognition, which plays a vital role in subsequent tasks such as face recognition, gait recognition, crowd counting, and instance segmentation. Due to the widespread application of deep learning, object detection algorithms have developed rapidly. The task of object detection is to find all the objects of interest in the image and determine their categories and locations. It is one of the core problems in the field of computer vision. Because various objects have different appearances, shapes and postures, and are interfered by factors such as illumination and occlusion during imaging, object detection has always been the most challenging problem in the field of computer vision.

[0003] Object detection algorithms based on deep learning have become the mainstream in the field of object detection. Object detection algorithms based on deep learning are usually divided into two types, namely single-stage object detection algorithms and two-stage object detection algorithms. There is no essential difference between single-stage object detection algorithms and two-stage object detection algorithms. As the name suggests, the two-stage object detection algorithm consists of two stages: the first stage is called region proposal, which is a stage for processing images to find areas where the objects to be detected may be located; the second stage is called classification regression stage, which is a stage for classifying and predicting the positions of the areas generated in the previous stage. The single-stage object detection algorithm does not have a region proposal stage, but directly locates and classifies the objects in the image.

[0004] However, the current mainstream deep learning-based target detection algorithms, whether single-stage or two-stage, will use a mechanism, that is, the anchor box mechanism. It can be said that the anchor box mechanism is one of the cornerstones of the current target detection algorithm. The anchor box mechanism was first introduced in Faster R-CNN, which is a set of manually set bounding boxes of fixed size and aspect ratio. Faster R-CNN has achieved very good results by using anchor boxes. The emergence of the anchor box mechanism solves some problems that were difficult to solve with previous methods without anchor boxes. Although the anchor box has a profound impact on the target detection algorithm, the anchor box mechanism also has its limitations, such as: anchor boxes of different sizes and shapes must be manually set for different problems; the introduction of anchor boxes also introduces a lot of computational costs, etc. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a target detection method based on an optimized anchor frame mechanism to achieve detection of image targets in view of the deficiencies of the above-mentioned prior art.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: a target detection method based on an optimized anchor frame mechanism, comprising the following steps:

[0007] Step 1: Perform data preprocessing on the instance image to be detected with the real frame to obtain a standard image;

[0008] The data preprocessing includes: resizing the image to a specific size, data enhancement and standardization operations;

[0009] Step 2: Input the preprocessed instance image to be detected into the skeleton-feature extraction network of the target detection model to obtain a feature map;

[0010] Step 3: Input the feature map output by the skeleton part into the neck-feature pyramid part of the target detection model to obtain the feature map of the number of pyramid levels;

[0011] Step 4: Design anchor box parameters according to the feature maps of the number of pyramid levels through the guided anchor box design module, and use these parameters to generate anchor boxes;

[0012] Step 4.1: Input the feature maps of the number of pyramid levels obtained in step 3 into multiple groups of guided anchor frame modules respectively to obtain feature map anchor frame parameters; the guided anchor frame design module includes two components, namely, an anchor frame position prediction component and an anchor frame shape prediction component, which are parallel components, so each feature map is input into the two components respectively;

[0013] Step 4.1.1: Input the feature map output by the target detection model neck-feature pyramid into the position prediction component, which uses a 1x1 convolution kernel to scan the feature map to obtain a shape size and original feature Figure 1 The new feature map is then passed through a nonlinear layer using the sigmoid function as the activation function to compress the values ​​of each point in the new feature map to between (0,1). The value of each point on the new feature map is used to represent the probability that the anchor box on the original feature map exists at that point.

[0014] Step 4.1.2: Input the feature map output by the target detection model neck-feature pyramid into the shape prediction component, which contains a convolution layer with a 2-channel output. This component uses two 1x1 convolution kernels to scan the feature map to obtain a 2-channel shape and original feature map. Figure 1 The new feature map group is used to represent the width and height of the anchor box of the point on the original feature map of the feature map;

[0015] Step 4.2: The anchor box parameters obtained in step 4.1: the probability score parameter of the feature map anchor box existence and the width and height parameters of the feature map anchor box are used to generate several sets of new anchor boxes for subsequent regression prediction and classification prediction in the target detection head - classifier;

[0016] Step 5: The feature map output by the feature pyramid of the target detection model is passed through a feature adaptation module to obtain a new feature map for classification and regression;

[0017] Step 6: Input the new feature map output by the feature adaptation module and the anchor frame generated by the anchor frame parameters output by the guided anchor frame module into the head of the object detection model - the classifier, and assign several anchor frames on each feature map to the real frame of the instance to be detected as training samples, and take all anchor frames in each feature map that fall into the central area of ​​the real frame of the instance to be detected as positive samples for training;

[0018] Step 7: Collect the sum of the average values ​​of various losses of the anchor boxes of each feature map through the online feature selection mechanism; compare the sum of the average values ​​of the losses of each feature map, select the feature map with the smallest sum of loss values, and let the target detection model use the anchor box and feature map corresponding to the feature map for training to achieve the effect of optimizing feature selection;

[0019] Step 7.1: Calculate the classification loss, regression loss, and anchor box shape loss of the anchor box corresponding to each feature map of the instance to be detected, and calculate the average value of each loss value;

[0020] Step 7.2: Find the hierarchical index of the feature map with the smallest average sum of the loss values ​​of the instance to be detected;

[0021] Step 7.3: Set the losses of all anchor boxes on the feature maps of the instance to be detected that are not at the level of the minimum feature map to 0 and do not use them for training;

[0022] Step 7.4: Use the anchor box on the feature map of the level where the feature map with the smallest sum of the mean losses of the instance to be detected is located as the training sample for back propagation, so as to train the target detection model until the target detection model converges;

[0023] Step 8: Input the instance to be detected without the real frame into the trained target detection model to achieve target detection.

[0024] The beneficial effects of adopting the above technical solution are: the target detection method based on the optimized anchor frame mechanism provided by the present invention, (1) the guided anchor frame design module can automatically and intelligently design important anchor frame parameters for the target detection model in the forward operation, reducing the cost of manually adjusting the anchor frame hyperparameters and the physical video memory required for the training model. (2) It is difficult for traditional anchor frames to use the optimal feature map for a certain detection instance for training. The online feature selection mechanism of the present invention can break through the defects of traditional anchor frames and can select and train feature maps based on the effective information of the sum of the mean values ​​of each loss in each forward training, thereby achieving optimization in training. (3) The method provided by the present invention can be integrated into various single-stage target detection models, and can ensure that the detection accuracy of the integrated target detection model is increased, and can reduce some costs of the target detection model in training, such as physical costs, labor costs, etc. (4) The method provided by the present invention combines two currently popular improvements to the anchor box mechanism, namely, some relatively excellent mechanisms in the direction of no anchor box are adjusted to a certain extent, so as to combine them with the anchor box mechanism that is contrary to them, thereby obtaining a new method with advantages in two directions. This new method has the universality that can be integrated into most of the current target detection models. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A flowchart of a target detection method based on an optimized anchor frame mechanism provided by an embodiment of the present invention;

[0026] Figure 2 A schematic diagram of the model operation of a forward operation of the target detection model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0028] This embodiment integrates the target detection method based on the optimized anchor frame mechanism of the present invention into the RetinaNet target detection model to achieve target detection.

[0029] In this embodiment, a target detection method based on an optimized anchor frame mechanism is provided. Figure 1 , 2 As shown, the following steps are included:

[0030] Step 1: Perform general data preprocessing on the instance image to be detected with the real frame to obtain a standard image;

[0031] Target detection requires the use of a target detection model, and before a forward operation of the target detection model, some preprocessing of the input image is required, such as resizing or enlarging the image to a certain size, performing certain data enhancement (such as random flipping, brightness adjustment, etc.) and standardization operations. Only after the preprocessing is completed can it be input into the target detection model.

[0032] Step 2: Input the preprocessed instance image to be detected into the skeleton-feature extraction network of the target detection model to obtain a feature map;

[0033] The first part of the target detection model is its backbone part. The function of this backbone part is to extract features from the input image. It consists of several convolutional layers, pooling layers, and Relu activation layers. Common feature-specific networks include VGG and ResNet. After the instance image to be detected passes through the backbone part of the target detection model, several feature maps can be obtained.

[0034] Step 3: Input the feature map output by the skeleton part into the neck-feature pyramid part of the target detection model to obtain the feature map of the number of pyramid levels;

[0035] The second part of the target detection model is the neck part. This part is used to strengthen the feature map obtained by the image through the target detection model skeleton, so as to obtain feature maps of several pyramid levels for detecting instances of different sizes. The neck part of the target detection model consists of several upsampling layers. Common target detection model necks include feature pyramids (FPN), etc. After passing through this part, several strengthened feature maps are obtained for classification and prediction regression.

[0036] Step 4: The guided anchor frame design module is used to design anchor frame parameters according to the feature graph of the number of pyramid levels, and these parameters are used to generate the anchor frames needed for subsequent use, thus realizing the function of automatically designing anchor frames;

[0037] Step 4.1: Input the feature maps of the number of pyramid levels obtained in step 3 into multiple groups of guided anchor frame modules respectively to obtain feature map anchor frame parameters; the guided anchor frame design module includes two components, namely, an anchor frame position prediction component and an anchor frame shape prediction component, which are parallel components, so each feature map is input into the two components respectively;

[0038] Step 4.1.1: Input the feature map output by the target detection model neck-feature pyramid into the position prediction component, which uses a 1x1 convolution kernel to scan the feature map to obtain a shape size and original feature Figure 1 The new feature map is then passed through a nonlinear layer using the sigmoid function as the activation function to compress the values ​​of each point in the new feature map to between (0,1). The value of each point on the new feature map is used to represent the probability that the anchor box on the original feature map exists at that point.

[0039] Step 4.1.2: Input the feature map output by the target detection model neck-feature pyramid into the shape prediction component, which contains a convolution layer with a 2-channel output. This component uses two 1x1 convolution kernels to scan the feature map to obtain a 2-channel shape and original feature map. Figure 1 The new feature map group is used to represent the width and height of the anchor box of the point on the original feature map of the feature map;

[0040] Step 4.2: The anchor box parameters obtained in step 4.1: the probability score parameter of the feature map anchor box existence and the width and height parameters of the feature map anchor box are used to generate several sets of new anchor boxes for subsequent regression prediction and classification prediction in the target detection head - classifier;

[0041] Step 5: The feature map output by the feature pyramid of the target detection model is passed through a feature adaptation module. The feature adaptation module is a deformable convolutional layer. The feature map output by the feature pyramid is input into the feature module to obtain new feature maps for classification and regression. These new feature maps for classification and regression are more consistent with the previously designed anchor frames, and the effect of predicting classification and regression positioning will be better than using feature maps that have not passed the feature adaptation module.

[0042] Step 6: Input the new feature map output by the feature adaptation module and the anchor frame generated by the anchor frame parameters output by the guided anchor frame module into the head of the object detection model - the classifier, and assign several anchor frames on each feature map to the real frame of the instance to be detected as training samples, and take all anchor frames in each feature map that fall into the central area of ​​the real frame of the instance to be detected as positive samples for training;

[0043] In this embodiment, the target detection method is integrated into the classic target detection model RetinaNet, whose classifier contains two parallel branches, namely the classification subnetwork and the regression subnetwork, which are used for classification prediction and positioning regression respectively. Each feature map has an anchor frame specially designed by the previous guided anchor frame design module. The anchor frame on each feature map is assigned to the real frame of the instance to be detected as a training sample through a certain selection method. In this embodiment, the central area allocation method is adopted. In short, all anchor frames whose central coordinates of each feature map fall into the central part of the real frame are set as training samples.

[0044] Step 7: Collect the sum of the average values ​​of various losses of the anchor boxes of each feature map through the online feature selection mechanism; compare the sum of the average values ​​of the losses of each feature map, select the feature map with the smallest sum of loss values, and let the target detection model use the anchor box and feature map corresponding to the feature map for training to achieve the effect of optimizing feature selection;

[0045] Step 7.1: Calculate the classification loss, regression loss, and anchor box shape loss of the anchor box corresponding to each feature map of the instance to be detected, and calculate the average value of each loss value;

[0046] Step 7.2: Find the hierarchical index of the feature map with the smallest average sum of the loss values ​​of the instance to be detected;

[0047] In this embodiment, the feature map with the smallest sum of the average loss values ​​is the first layer feature map, so this training will use the feature map Figure 1 The anchor boxes on are used as training samples.

[0048] Step 7.3: Set the losses of all anchor boxes on the feature maps of the instance to be detected that are not at the level of the minimum feature map to 0 and do not use them for training;

[0049] Step 7.4: Use the anchor box on the feature map of the level where the feature map with the smallest sum of the mean losses of the instance to be detected is located as the training sample for back propagation, so as to train the target detection model until the target detection model converges;

[0050] Step 8: Input the instance to be detected without the real frame into the trained target detection model to achieve target detection.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A target detection method based on an optimized anchor box mechanism, characterized in that: The following steps are involved: Step 1: Perform data preprocessing on the instance image to be detected with the real frame to obtain a standard image; Step 2: Input the preprocessed instance image to be detected into the skeleton-feature extraction network of the target detection model to obtain a feature map; Step 3: Input the feature map output by the skeleton part into the neck-feature pyramid part of the target detection model to obtain the feature map of the number of pyramid levels; Step 4: Design anchor box parameters according to the feature maps of the number of pyramid levels through the guided anchor box design module, and use these parameters to generate anchor boxes; Step 4.1: Input the feature maps of the number of pyramid levels obtained in step 3 into multiple groups of guided anchor frame modules respectively to obtain feature map anchor frame parameters; Step 4.2: The anchor box parameters obtained in step 4.1: the probability score parameter of the feature map anchor box existence and the width and height parameters of the feature map anchor box are used to generate several sets of new anchor boxes for subsequent regression prediction and classification prediction in the target detection head - classifier; Step 5: The feature map output by the feature pyramid of the target detection model is passed through a feature adaptation module to obtain a new feature map for classification and regression; Step 6: Input the new feature map output by the feature adaptation module and the anchor frame generated by the anchor frame parameters output by the guided anchor frame module into the head of the object detection model - the classifier, and assign several anchor frames on each feature map to the real frame of the instance to be detected as training samples, and take all anchor frames in each feature map that fall into the central area of ​​the real frame of the instance to be detected as positive samples for training; Step 7: Collect the sum of the average values ​​of various losses of the anchor boxes of each feature map through the online feature selection mechanism; compare the sum of the average values ​​of the losses of each feature map, select the feature map with the smallest sum of losses, and let the target detection model use the anchor box and feature map corresponding to the feature map for training to achieve the effect of optimizing feature selection; Step 8: Input the instance to be detected without the real frame into the trained target detection model to achieve target detection.

2. The object detection method based on the optimized anchor box mechanism according to claim 1, characterized in that: The data preprocessing described in step 1 includes: adjusting the image to a specific size, and performing data enhancement and standardization operations.

3. The object detection method based on the optimized anchor box mechanism according to claim 1, characterized in that: The guided anchor frame design module includes two components, namely an anchor frame position prediction component and an anchor frame shape prediction component. The two components are parallel components, and each feature map is input into the two components respectively.

4. The object detection method based on the optimized anchor box mechanism according to claim 3, characterized in that: The position prediction component uses a 1x1 convolution kernel to scan the feature map; the anchor box shape prediction component includes a convolution layer with a 2-channel output and uses two 1x1 convolution kernels to scan the feature map.

5. The object detection method based on the optimized anchor box mechanism according to claim 4, characterized in that: The specific method of step 4.1 is: Step 4.1.1: Input the feature map output by the feature pyramid of the target detection model neck into the position prediction component to obtain a new feature map with the same shape and size as the original feature map. Then, the new feature map passes through a nonlinear layer using the sigmoid function as the activation function to compress the values ​​of each point in the new feature map to between (0, 1). The value of each point on the new feature map is used to represent the probability that the anchor box on the original feature map exists at that point. Step 4.1.2: Input the feature map output by the neck-feature pyramid of the target detection model into the shape prediction component to obtain a new feature map group with 2 channels whose shape is the same as the original feature map. The values ​​of the 2 channels in the new feature map are used to represent the width and height of the anchor box on the original feature map.

6. The object detection method based on the optimized anchor box mechanism according to claim 1, characterized in that: The specific method of step 7 is: Step 7.1: Calculate the classification loss, regression loss, and anchor box shape loss of the anchor box corresponding to each feature map of the instance to be detected, and calculate the average value of each loss value; Step 7.2: Find the hierarchical index of the feature map with the smallest average sum of the loss values ​​of the instance to be detected; Step 7.3: Set the losses of all anchor boxes on the feature maps of the instance to be detected that are not at the level of the minimum feature map to 0 and do not use them for training; Step 7.4: Use the anchor box on the feature map of the layer where the sum of the mean losses of the instances to be detected is the smallest as the training sample for back propagation, so as to train the target detection model until the target detection model converges.

Images