A lightweight object detection network and detection method based on OSA blocks

Abstract

This invention provides a lightweight object detection network and method based on OSA blocks. The target detection network consists of a Focus layer and multiple OSA blocks in series, forming the backbone of the detection network. The neck of the network is formed by fusing high-order features with low-order features through CBL (convolution + batch normalization + leaky activation function) blocks and upsampling operations, creating an FPN (feature pyramid network)-like structure. The head of the network is composed of CBL blocks combined with a channel attention mechanism. The detection method involves preprocessing an image and inputting it into the target detection network. The target detection network predicts the target's location coordinates and category information. Finally, a filter is applied to retain the predicted rectangles with the highest confidence, eliminating the remaining redundant rectangles. This improves computational efficiency and reduces storage space overhead, thus facilitating industrial applications.

Description

Technical Field

[0001] The present invention relates to target detection technology in the field of computer vision, and in particular to a lightweight target detection network and detection method based on OSA blocks. Background Art

[0002] With the emergence of deep learning technology, deep learning-based object detection has become a mainstream research direction in the field of object detection in recent years. Compared with object detection methods based on traditional image processing, deep learning-based object detection methods have significant advantages in accuracy, efficiency, and stability.

[0003] Currently, mainstream object detection networks are categorized into single-stage and two-stage networks. While two-stage networks offer high accuracy, they are less efficient and less practical than single-stage networks. Furthermore, single-stage networks have two detection modes: anchor-based and anchor-free. The anchor-based detection mode pre-places anchor boxes of various scales at each point in the prediction feature map. Object center, width, and height are then predicted based on these anchor boxes. Dense anchor boxes can effectively improve object detection rates and size fit, but detecting a small number of objects in a large feature map generates a large number of redundant boxes, severely impacting detection efficiency. Furthermore, the definition of anchor box size directly impacts the detection network's object localization accuracy. The anchor-free detection mode, on the other hand, does not require additional hyperparameters to aid detection. Instead, each point in the feature map is predicted to contain only a single bounding box, avoiding significant computational overhead and resulting in higher detection efficiency. However, due to the lack of pre-defined object sizes similar to those used for anchor boxes, the detection rate is slightly lower than that of the anchor-based detection mode.

[0004] Furthermore, most current mainstream object detection networks typically stack a large number of parameters in the backbone to more easily acquire more expressive high-level features. From an industrial application perspective, this structural design, which prioritizes improving detection accuracy at the expense of efficiency and storage space, is relatively impractical. Summary of the Invention

[0005] The purpose of the present invention is to provide a lightweight target detection network and detection method based on OSA blocks, which can improve computational efficiency and reduce storage space overhead, thereby facilitating industrial applications.

[0006] To address the aforementioned technical issues, the present invention provides the following technical solution: The network's macroscopic structure is similar to that of mainstream object detection networks, consisting of three components: the backbone, the neck, and the head. The backbone reduces the scale of the original image and converts low-order features into high-order features; the neck performs a series of fusions on the high-order features to further enhance their expressiveness; and the head uses these enhanced features to predict the object's location and category.

[0007] The backbone of the network in the present invention consists of a Focus layer and four OSA blocks connected in series. The Focus layer directly receives the preprocessed image, splits and merges the image of size n×n×3, and generates a new low-order feature map of size (n / 2)×(n / 2)×12. The feature map then passes through several OSA blocks in series, and the width and height of the feature map output by each OSA block are both 1 / 2 of the input feature map. Each OSA block contains the same number of convolutional layers, and the convolutional layers of the same OSA block have the same hyperparameters. The outputs of all convolutional layers in the backbone are batch normalized and activated by the mish function.

[0008] The neck receives the feature maps output by each OSA block in the backbone. It resizes the width and height of each subsequent OSA block to twice its original value through a CBL block and an upsampling layer. The result is then concatenated and fused with the output feature map of the previous OSA block, channel by channel, to form an FPN. It then selects one or more enhanced feature maps of various scales and outputs them to the final head.

[0009] The head uses independent CBL blocks and channel attention machines to assign different weights to each channel of the input enhanced feature maps of various scales, thereby preserving the dominant role of valid channels in subsequent detection and minimizing the negative impact of redundant channels. Three parallel convolutional layers with a kernel size of 1x1 are then connected to predict the confidence of the object category, center position offset, and width and height, respectively.

[0010] Since the head part predicts a target rectangular box for each point on the feature map, a filter needs to be used at the end of the detection method to filter out a large number of redundant boxes and only retain the rectangular boxes that predict the target position and category most accurately.

[0011] As an optional implementation method, all activation functions in the network can be replaced by ReLU functions;

[0012] As an optional implementation method, the upsampling operation in the neck part can use nearest neighbor interpolation, bilinear interpolation or deconvolution layer;

[0013] As an optional implementation method, the neck part does not perform any feature fusion, and directly inputs the output feature map of the last OSA block into the head part;

[0014] As an optional implementation method, the neck part fuses the output features of the specified OSA block and inputs the final fused feature map into the head part;

[0015] As an optional implementation method, the neck part fuses the output features of the specified OSA block and inputs the feature map obtained by each fusion into the head part;

[0016] As an optional implementation method, the two convolutional layers in the head part used to predict the target center position offset and width and height can be replaced by a convolutional layer with an output channel of 4;

[0017] As an optional implementation method, for feature maps of only one size input to the head part for prediction, the maximum pooling layer combined with the confidence threshold is used as a filter to filter out the redundant boxes predicted;

[0018] As an optional implementation method, for feature maps of various sizes input to the head part for prediction, non-maximum suppression or improved non-maximum suppression methods such as soft-nms and diou-nms are used in combination with confidence thresholds as filters to filter out redundant boxes predicted. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is the structural diagram of the backbone part;

[0020] Figure 2 This is the structural diagram of the OSA block;

[0021] Figure 3 This is the structural diagram of the neck part;

[0022] Figure 4 This is the structural diagram of the head part;

[0023] Figure 5 This is a flowchart of target detection.

[0024] Specific implementation form

[0025] The following is a further description of an implementation of the present invention with reference to the accompanying drawings.

[0026] The present invention provides a lightweight target detection network and detection method based on OSA blocks. The complete process is shown in Figure 5 The specific implementation steps are as follows:

[0027] 1) Preprocess the original color image, adjust the image size to 640x640, and shrink all pixel values in the image to between 0 and 1 through linear scaling.

[0028] 2) The pre-processed image is input to the Focus layer at the entrance of the backbone part of the network. The Focus layer divides the input image into 320x320 blocks, and each block is 2x2 in size. Pixels at the same position in all blocks are extracted and merged to obtain four feature maps of shape 320x320x3. These feature maps are spliced by channel, and the fused feature map shape is 320x320x12. Without losing information, the width and height of the feature map are compressed. The structural diagram of the backbone part is shown in Figure 1 .

[0029] 3) The above features are then fed into four serially connected OSA blocks. Each OSA block contains a max pooling layer and five small modules consisting of 3x3 convolutional layers, batch normalization, and the mish activation function. The mathematical expression of the mish function is as follows:

[0030] mish(x)=x*tanh(ln(1+e x ))

[0031]

[0032] The width and height of the high-order feature maps output by the four OSA blocks are 160x160, 80x80, 40x40, and 20x20, respectively. See the structure diagram of the OSA block for details. Figure 2 .

[0033] 4) This implementation fuses the high-order feature maps output by the last three OSA blocks. The last three OSA blocks are input into the neck. The high-order feature map output by the last OSA block is magnified by 2 times in width and height through the CBL block and bilinear interpolation operation. The number of channels is the same as the high-order features to be fused, and it is spliced with the high-order feature map output by the third OSA block by channel. The spliced feature map is again subjected to the CBL block and bilinear interpolation operation, and then spliced with the high-order feature map output by the second OSA block by channel. Since this implementation only uses the high-order feature map obtained by the final fusion for subsequent predictions, only the fused high-order features with a width and height of 80x80 are input into the head part. See the structural diagram of the neck part. Figure 4 .

[0034] 5) The high-order feature map of the input head is sequentially passed through the CBL block and the channel attention mechanism. The channel attention mechanism uses an average pooling layer to calculate the mean of the feature map on each channel. The calculated mean is then passed through a 1x1 convolution and a hard-swish activation function to obtain the weight of each channel of the high-order feature map. The mathematical expression of the hard-swish activation function is as follows:

[0035]

[0036] Relu(x)=min(max(0,x),6)

[0037] The obtained channel weights are multiplied by the high-order feature map to eliminate redundant channels in the high-order feature map. The remaining valid channels are then passed to three convolutional layers with a kernel size of 1x1. These three convolutional layers predict the target category confidence, target center coordinate offset, and target width and height, respectively, with each point in the feature map as the target center.

[0038] 6) For the feature map with a width and height of 80x80, the head part inputs 80x80 predicted target information, most of which is invalid redundant information. In order to remove this redundant information, the target category confidence in the predicted target information is input into a maximum pooling layer with a kernel size of 3x3, and compared point by point with the new target category confidence output by the pooling layer. All predicted information corresponding to the points where the confidence changes are regarded as invalid redundant information and filtered out. For the remaining points, a confidence threshold of 0.3 is added for secondary filtering. For all points with a confidence greater than 0.3, the predicted target center offset, target width and height, and the scaling factor of the feature map relative to the original image are combined with the coordinates of the point to obtain the coordinates of the upper left and lower right corners of the final enclosing rectangular box of the target in the original image. The conversion formula is as follows:

[0039] top_left_x:[scale_w*(cx+offset_cx-0.5*w)]

[0040] top_left_y:[scale_h*(cy+offset_cy-0.5*h)]

[0041] bottom_right_x:[scale_w*(cx+offset_cx+0.5*w)]

[0042] bottom_right_y:[scale_h*(cy+offset_cy+0.5*h)]

[0043] In the above formula, scale_w and scale_h are the width and height scaling factors of the feature map relative to the original image; cx and cy are the two-dimensional coordinates of the current point on the feature map; offset_cx and offset_cy are the offsets of the two-dimensional coordinates of the target center on the feature map relative to the current point; w and h represent the width and height of the target on the feature map.

[0044] This network achieves high detection accuracy and efficiency. When the four OSA blocks output feature maps with 64, 96, 128, and 160 channels, the model size is less than 10MB, and detection on a single image takes less than 10ms. Compared to current open-source mainstream object detection methods, this lightweight object detection network demonstrates excellent practicality for industrial applications.

[0045] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A lightweight object detection network based on OSA blocks, consisting of three parts: backbone, neck and head, characterized by: The backbone part is used to reduce the scale of the original image and convert the original image from low-order features to high-order features; The backbone part consists of a Focus layer and four OSA blocks connected in series. The Focus layer directly receives the preprocessed image, splits and merges the image of size n×n×3, and generates a new low-order feature map of size (n / 2)×(n / 2)×12. The feature map then passes through several OSA blocks in series. The width and height of the feature map output by each OSA block are both 1 / 2 of the input feature map. Each OSA block contains the same number of convolutional layers, and the convolutional layers of the same OSA block have the same hyperparameters. The outputs of all convolutional layers in the backbone are batch normalized and activated by the mish function. The neck part performs a series of fusions on high-order features to further enhance the feature expression of high-order features; The neck part receives the feature maps output by each OSA block in the backbone, adjusts the width and height of the output feature map of the next OSA block to twice the original through the CBL block and the upsampling layer, and then splices and fuses it with the output feature map of the previous OSA block by channel to form FPN. Then, one or more enhanced feature maps of various scales are selected and output to the final head part; The head part is responsible for predicting the target location and category using the enhanced features.

2. The detection method of the lightweight object detection network based on the OSA block according to claim 1, characterized in that: The head part uses independent CBL blocks and channel attention machines to assign different weights to each channel of the enhanced feature map for various scales of input enhancement feature maps, thereby retaining the dominant role of the effective channel in subsequent detection and reducing the negative impact of redundant channels. Subsequently, three parallel convolutional layers with a kernel size of 1x1 are connected to predict the confidence, center position offset and width and height of the target category respectively. Since the head part predicts a target rectangular box for each point on the feature map, a filter needs to be used at the end of the detection method to filter out a large number of redundant boxes, retaining only the rectangular box with the most accurate prediction of the target position and category.

3. The detection method of the lightweight object detection network based on OSA blocks according to claim 1, characterized in that: The upsampling operation in the neck part can use nearest neighbor interpolation, bilinear interpolation or deconvolution layer.

4. The lightweight object detection network and detection method based on OSA blocks according to claim 1, characterized in that: The neck part fuses the output features of the specified OSA block and inputs the final fusion feature map into the head part.

5. The lightweight object detection network and detection method based on OSA blocks according to claim 1, characterized in that: The neck part fuses the output features of the specified OSA block and inputs the feature map obtained by each fusion into the head part.

6. The lightweight object detection network and detection method based on OSA blocks according to claim 1, characterized in that: The two convolutional layers in the head part used to predict the target center position offset and width and height are replaced by a convolutional layer with an output channel of 4.

7. The lightweight object detection network and detection method based on OSA blocks according to claim 1, characterized in that: For feature maps of only one size input to the head part for prediction, the maximum pooling layer combined with the confidence threshold is used as a filter to filter out the redundant boxes predicted; for feature maps of multiple sizes input to the head part for prediction, non-maximum suppression or soft-nms, diou-nms improved non-maximum suppression method combined with the confidence threshold is used as a filter to filter out the redundant boxes predicted.

Images