A target detection method based on attention region selection

Abstract

The present invention discloses a deep model compression method based on attention region selection, which belongs to the field of computer vision technology. The student network improves the ability of target detection and reduces the model size by learning the information of the selected region in the teacher network. The present invention uses a simple distillation method combining region selection and attention mechanism for target detection. On the basis of comprehensive analysis, the method of the present invention is introduced. First, the distillation region is selected according to the prediction box, and then the attention map of the region is extracted for distillation. The method of the present invention is evaluated on the Pascal VOC benchmark using the Faster R‑CNN model. Compared with the teacher, the present invention improves the detection effect and reduces the model size.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a target detection method based on attention region selection. Background Art

[0002] As one of the foundations and hot topics in the field of computer vision, object detection has attracted widespread attention and discussion. Recent studies have proposed many deep learning-based methods to handle the task of object detection. These methods often rely on the operation of complex backbone networks. Although they can greatly improve the performance of the model, they occupy a lot of memory and take a lot of computing time. Knowledge Distillation (KD) is an effective method commonly used in model compression. It imitates the capabilities of large teacher models by training compact student models to obtain excellent generalization capabilities. At present, the main research work of KD is mostly applied to classification tasks, and less to object detection tasks, because object detection is not simply to classify an image into the corresponding category, but to find the location of the object and classify the found object. Although logarithm-based knowledge, feature-based knowledge, and relationship-based knowledge have been proposed, the application of KD in object detection is far from perfect because it requires some additional operations and designs. Summary of the invention

[0003] The present invention provides a target detection method based on attention region selection, which performs target detection by using a simple distillation method combining region selection and attention mechanism. On the basis of comprehensive analysis, a deep model compression method is introduced to reduce the number of parameters of the model in target detection and speed up the inference speed.

[0004] To achieve the above objectives, the present invention adopts the following technical solutions:

[0005] A target detection method based on attention region selection comprises the following steps:

[0006] Step 1: Extract the output of the specified layer of the teacher network and the student network as the feature map to be calculated;

[0007] Step 2: Apply the attention function to calculate the attention map of the feature map;

[0008] Step 3: Apply the region selection algorithm to calculate the selected region;

[0009] Step 4: The student network and the teacher network simultaneously extract the attention map of the selected area;

[0010] Step 5: The student network acquires the capabilities of the teacher network by learning the information in this area.

[0011] In the above steps, the attention function in step 2 is:

[0012]

[0013]

[0014] Define a convolutional layer, and the activation tensor associated with this layer is Where F is the activation function of the relevant layer, its input dimension is (H, W, C), H and W represent the width and height of the feature map, C represents the number of channels, and the output is an attention map with a size of H×W, A i =A(i;:;:) is the Matlab notation, which represents the i-th channel of tensor A. i The dimensions are H×W, Indicates that the input is the activation tensor A, the superscript p indicates the pth power, and the subscript sum indicates the sum;

[0015] Step 3 specifically includes the following steps:

[0016] Step 3.1: Get the IOU value of the predicted box and the true value box

[0017] IOU (Intersection over Union) represents the similarity between the predicted area and the true area where the object exists. For each predicted box, its IOU value with each true value box is calculated and added to the IOUs;

[0018] Step 3.2: Take the largest IOU among the IOUs, maxIou, and set the fai value to 0.5

[0019] maxIou←MAX(IOUs)

[0020] Among them, MAX means taking the maximum value

[0021] Step 3.3: Filter out the prediction boxes whose IOU value is less than threshold = maxIou × fai;

[0022] Step 3.4: Perform an OR operation on all the selected prediction boxes whose IOU values ​​are greater than the threshold threshold to obtain the final mask, which is the selected area;

[0023] Mask is a matrix composed of 0 and 1, with a dimension of H×W;

[0024] The or operation represents a bitwise or operation, and all positions included in the prediction box are set to 1;

[0025] In step 4, extract the attention map of the selected area:

[0026]

[0027] Where t and s represent the activation tensors of the convolutional layers of the teacher network and the student network, respectively. Mask(mask,t) and Mask(mask,s) are the attention maps of the selected areas of the teacher network and the student network, respectively.

[0028] Step 5 specifically includes the following steps:

[0029] Step 5.1: Region selection loss function of the student network:

[0030]

[0031] mask is the mask for region selection, N p is the number of points in the selected area, f adap is the adaptation function, i and j represent the i-th row and j-th column on the two-dimensional feature map. The output dimensions of the student network and the teacher network may be different, so an adaptation layer needs to be added to stretch the outputs of the two networks to the same size;

[0032] Step 5.2 Overall loss function

[0033] Loss = L gt +λL RAT

[0034] L gt refers to the original target detection loss, λ is the RAT loss weight coefficient, which is used to balance L gt and L RAT The weight between .

[0035] Step 5.3: The student network learns the information of the teacher network by minimizing the overall loss function;

[0036] Furthermore, the network is trained to minimize the overall loss function, and the training is iterated until the loss converges.

[0037] Beneficial effects: The present invention provides a target detection method based on attention region selection, which uses a simple distillation method combining region selection and attention mechanism for target detection. Based on comprehensive analysis, a deep model compression method is introduced. First, the distillation region is selected according to the prediction box, and then the attention map of the region is extracted for distillation. The experimental results on several target detection datasets and different detection frameworks verify the effectiveness of the method of the present invention, and analyze and prove the important role of applying region selection and attention mechanism in the distillation process; and the present invention is easy to implement and can be applied to single-stage and dual-stage detection frameworks, with universality. The present invention uses the Faster R-CNN model to evaluate our method on the Pascal VOC benchmark. Compared with the teacher, the present invention improves the detection effect and reduces the model size. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is an overall flow chart of an embodiment of the present invention;

[0039] Figure 2 is a result diagram of step 2 in an embodiment of the present invention;

[0040] Figure 3 is a result diagram of step 3 in an embodiment of the present invention;

[0041] Figure 4 It is a result diagram of step 4 in the embodiment of the present invention. DETAILED DESCRIPTION

[0042] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments:

[0043] like Figure 1 As shown, a target detection method based on attention region selection includes the following steps:

[0044] Step 1.1 Extract the specified network layer

[0045] Extract the output t, s of the specified layer of the teacher network and the student network as the feature map to be calculated;

[0046] Furthermore, the teacher network is Faster R-CNN based on ResNet152-FPN, the student network is Faster R-CNN based on ResNet34-FPN, and the layer selected for distillation is the highest layer of FPN;

[0047] Step 2.1: Specify the attention function

[0048]

[0049] The dimension of feature F is (H, W, C), H and W represent the width and height of the feature map, C represents the number of channels, and the size of the attention map is H × W. The superscript p represents the pth power, and p is set to 2;

[0050] Step 2.2 Substitute the formula to calculate the attention map of the feature map

[0051] Furthermore, substitute t,s obtained in step 1.1 into the attention function specified in step 2.1, that is,

[0052] Step 3.1 Get the IOU value of the predicted box and the true value box

[0053] IOU (Intersection over Union) represents the similarity between the predicted area and the true area where the object exists. For each predicted box, its IOU value with each true value box is calculated and added to the IOUs;

[0054] Step 3.2 Take the largest IOU among the IOUs, maxIou, and set the fai value to 0.5

[0055] maxIou←MAX(IOUs)

[0056] Among them, MAX means taking the maximum value

[0057] Step 3.3: filter out the prediction boxes whose IOU values ​​are less than threshold = maxIou × fai;

[0058] Step 3.4 performs an OR operation on all the selected prediction boxes whose IOU values ​​are greater than the threshold threshold to obtain the final mask, which is the selected area;

[0059] Furthermore, mask is a matrix composed of 0s and 1s with a dimension of H×W. The or operation represents a bitwise or operation, and all positions included in the prediction box are set to 1.

[0060] Step 4.1 Calculate the attention map of the selected area

[0061]

[0062] Where t and s represent the activation tensors output by the convolutional layers of the teacher network and the student network in step 1.1, respectively. Mask(mask,t) and Mask(mask,s) are the attention maps of the selected areas of the teacher network and the student network, respectively. Mask is the mask obtained in step 3.4, i.e., the selected area.

[0063] Step 5.1 Region selection loss function of student network:

[0064]

[0065] mask is the mask for region selection, N p is the number of points in the selected area, f adap is the adaptation function, i,j represents the i-th row and j-th column on the two-dimensional feature map. The output dimensions of the student network and the teacher network may be different, so an adaptation layer needs to be added to stretch the outputs of the two networks to the same size;

[0066] Step 5.2 Overall loss function

[0067] Loss = L gt +λL RAT

[0068] L gt refers to the original target detection loss, λ is the RAT loss weight coefficient, which is used to balance L gt and L RAT The weight between

[0069] Step 5.3 The student network learns the information of the teacher network by minimizing the overall loss function.

[0070] Furthermore, the network is trained to minimize the overall loss function, and the training is iterated until the loss converges;

[0071] The method of the present invention is evaluated on the Pascal VOC benchmark using the Faster R-CNN model. As shown in Table 1, the results show that compared with the teacher, the present invention improves the detection effect and reduces the model size.

[0072] Furthermore, the evaluation indicators are mAP (mean Average Precision) and the memory size occupied by the model;

[0073] To further calculate mAP, we first need to calculate p precision and r recall;

[0074]

[0075] Among them, TP (True Positive) is a sample that is judged to be detected correctly, that is, the target classification in the predicted bounding box is correct and the overlap between the bounding box and the true bounding box is greater than the threshold.

[0076] FP (False Positive) is a sample that is judged to be detected incorrectly, that is, the target classification in the predicted bounding box is wrong or the overlap between the bounding box and the true bounding box is less than the threshold.

[0077] FN (False Negative) is a sample that has not been detected, that is, a missed sample.

[0078] AP (Average Precision) is the area of ​​the precision-recall curve. It is calculated as follows:

[0079]

[0080] mAP is the average value of AP of all categories. The calculation method is as follows:

[0081]

[0082] Where N is the total number of categories, and i represents a certain category.

[0083] Table 1 Comparative evaluation results of the above methods

[0084]

[0085] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention is described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A target detection method based on attention region selection, characterized in that: The following steps are involved: Step 1: Extract the output of the specified layer of the teacher network and the student network as the feature map to be calculated; Step 2: Apply the attention function to calculate the attention map of the feature map, and the attention function is: Define a convolutional layer, and the activation tensor associated with this layer is Where F is the activation function of the relevant layer, its input dimension is (H, W, C), H, W represent the width and height of the feature map, C represents the number of channels, and the output is an attention map with a size of H×W, A i =A(i;: ； :) is the Matlab notation, which represents the i-th channel of tensor A. i The dimensions are H×W, Indicates that the input is the activation tensor A, the superscript p indicates the pth power, and the subscript sum indicates the sum; Step 3: Apply the region selection algorithm to calculate the selected region, which specifically includes the following steps: Step 3.1: Get the IOU value of the predicted box and the true value box. For each predicted box, calculate its IOU value with each true value box and add it to IOUs. Step 3.2: Take the largest IOU among the IOUs, maxIou, and set the fai value to 0.5; maxIou←MAX(IOUs) Step 3.3: Filter out the prediction boxes whose IOU value is less than threshold = maxIou × fai; Step 3.4: Perform an OR operation on all the selected prediction boxes whose IOU values ​​are greater than the threshold threshold to obtain the final mask, which is the selected area; Step 4: The student network and the teacher network simultaneously extract the attention map of the selected area; Step 5: The student network acquires the capabilities of the teacher network by learning the information in this area.

2. The target detection method based on attention region selection according to claim 1, characterized in that: The teacher network in step 1 is Faster R-CNN based on ResNet152-FPN, and the student network is Faster R-CNN based on ResNet34-FPN.

3. The target detection method based on attention region selection according to claim 2, characterized in that: The layer selected for distillation is the highest layer of FPN.

4. The target detection method based on attention region selection according to claim 1, characterized in that: Mask is a matrix composed of 0 and 1, with a dimension of H×W.

5. The target detection method based on attention region selection according to claim 1, characterized in that: The OR operation represents a bitwise OR operation, and all positions included in the prediction box are set to 1.

6. The target detection method based on attention region selection according to claim 1, characterized in that: In step 4, extract the attention map of the selected area: Mask(mask,t) = mask dot product Mask(mask,s) = mask dot product Among them, t and s represent the activation tensors of the convolutional layers of the teacher network and the student network, respectively, and Mask(mask,t) and Mask(mask,s) are the attention maps of the selected areas of the teacher network and the student network, respectively.

7. The target detection method based on attention region selection according to claim 1, characterized in that: Step 5 specifically includes the following steps: Step 5.1: Region selection loss function of the student network: mask is the mask for region selection, N p is the number of points in the selected area, f adap is the adaptation function, i and j represent the i-th row and j-th column on the two-dimensional feature map. The output dimensions of the student network and the teacher network may be different, so an adaptation layer needs to be added to stretch the outputs of the two networks to the same size; Step 5.2 Overall loss function Loss=L gt +λL RAT L gt refers to the original target detection loss, λ is the RAT loss weight coefficient, which is used to balance L gt and L RAT The weight between Step 5.3: The student network learns the information of the teacher network by minimizing the overall loss function Loss.

8. The target detection method based on attention region selection according to claim 7, characterized in that: Step 5.3 trains the network and minimizes the overall loss function, iterating the training until the loss converges.

Images