A logo detection method fusing MobileNetV2

Abstract

The application discloses a logo detection method fusing MobileNetV2, wherein through analyzing the connection mode between a YOLOv5 backbone network and a neck network, it is found that the backbone network of YOLOv5 is connected with the neck network at the convolution layers of 8 times, 16 times and 32 times down-sampling of an input image, then a series of up-sampling and down-sampling operations are performed to fuse the feature maps output by the three convolution layers, and three feature maps are output to a head prediction network for use. In the network structure of MobileNetV2, the input image is also subjected to 8 times, 16 times and 32 times down-sampling, and the network can also output three kinds of down-sampling feature maps with different multiples. Therefore, according to the connection mode between the backbone network and the neck network of YOLOv5, the three down-sampling feature maps in MobileNetV2 are input to the neck network of YOLOv5, so that the network is lightened.

Description

Technical Field

[0001] This invention relates to the field of image detection, and more particularly to a logo detection method that integrates MobileNetV2. Background Technology

[0002] With the continuous development of the computer industry, image classification and object detection technologies have shown their advantages in many fields such as school education, urban transportation, medical and health care, and consumer entertainment. In the consumer field, the data of product logo images has grown on a large scale. Researchers have carried out a lot of related visual processing on the massive product data in e-commerce platforms, providing a diverse application environment for logo classification and detection technologies. With the rapid development of computer hardware, high-precision object detection methods have been realized.

[0003] Girshick et al. (Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]Proceedings of the IEEE conference on computer vision and pattern recognition.2014:580-587) proposed the object detection algorithm R-CNN. This algorithm combines candidate regions and convolutional neural networks. The image is sent to the network after being cropped and deformed, which leads to information loss and changes in positional information, thus affecting the detection accuracy. Moreover, the R-CNN network needs to repeatedly perform convolution operations on the deformed regions in the image, which makes feature calculation very time-consuming.

[0004] To address the slow detection speed of R-CNN, He et al. (He K, Zhang X, Ren S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE transactions on pattern analysis and machine intelligence, 2015, 37(9):1904-1916) proposed SPP-Net. This algorithm avoids the drawback of R-CNN repeatedly using CNN to calculate features, does not require image cropping, and reduces the loss of low-level image information. However, it divides regression and classification into multiple stages, which makes the network occupy a lot of hardware resources.

[0005] To address the problems of SPP-Net, Girshick et al. (Girshick R. Fast-CNN [C] / / Proceedings of the IEEE international conference on computer vision. 2015:1440-1448) borrowed the idea of ​​pyramid pooling from SPP-Net and proposed the Fast R-CNN network. This network uses ROI pooling to transform features into ROI features, which are then input into fully connected layers for target classification. This network reduces hardware caching and significantly improves detection speed. However, it cannot accurately align the feature map with the original pixels and cannot meet the requirements of real-time detection. Although the R-CNN series of networks have certain advantages in detection accuracy, these networks have low detection efficiency and slow speed.

[0006] To address these issues, Redmond et al. (Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection[C] Proceedings of the IEEE conference on computer vision and pattern recognition. 2016:779-788) proposed the YOLO detection model, which uses regression to replace candidate regions, resulting in a significant improvement in detection speed.

[0007] The model uses the bounding box with the highest confidence as the final output, but it is not ideal for detecting adjacent groups or small target groups, and the detection accuracy is low. Summary of the Invention

[0008] This invention overcomes the shortcomings of the prior art and provides a logo detection method that integrates MobileNetV2.

[0009] To achieve the above objectives, the technical solution adopted by this invention is as follows: a logo detection method integrating MobileNetV2, comprising the following steps: S1: Replacing the backbone network of the YOLOv5 network with the Bottleneck based on MobileNetV2 to construct a MobileNetV2-YOLOv5 network; S2: Inputting a logo image, first passing it through a CBR to downsample the feature map once, the length and width of the output feature map are 1 / 4 of the network input size; then passing it through two Bottleneck-2, the length and width of the obtained feature map are respectively 1 / 4 of the network input size. 1 / 4; After passing through 3 Bottleneck-2, the length and width of the output feature map are 1 / 8 of the network input size; After passing through 4 Bottleneck-2 and 5 Bottleneck-1, the length and width of the output feature map are 1 / 16 of the network input size; After passing through 3 Bottleneck-2 and 1 Bottleneck-1, the length and width of the output feature map are 1 / 32 of the network input size; S3: Input the 8, 16, and 32 times downsampled feature maps into the neck network of YOLOv5 for feature information fusion, perform detection, and output the detection results.

[0010] In a preferred embodiment of the present invention, the CBR includes a regular convolutional layer, a batch normalization layer, and a ReLU activation function. The stride of the regular convolutional layer is 2, and the kernel size is 3*3.

[0011] In a preferred embodiment of the present invention, only the first layer of Bottleneck in each Bottleneck-2 sequence has a stride of 2, while the stride of the remaining Bottlenecks is 1.

[0012] In a preferred embodiment of the present invention, an RB module is incorporated before detection to enhance the semantic information of the image.

[0013] In a preferred embodiment of the present invention, the detection results include the coordinates of the detection box, the confidence level of the logo, and the category.

[0014] In a preferred embodiment of the present invention, the MobileNetV2-YOLOv5 network is trained using the PyTorch framework.

[0015] In a preferred embodiment of the present invention, two Shortcut connections are used in the RB module, enabling the network to acquire feature information from earlier layers and enhancing feature reuse.

[0016] This invention addresses the shortcomings of the prior art and has the following beneficial effects:

[0017] This invention leverages the lightweight network advantages of the MobileNetV2 algorithm, using it as the backbone network to improve the YOLOv5 object detection algorithm in a lightweight manner. This not only ensures that the overall network structure is not destroyed, but also accelerates the model with less time and resource costs. The results were verified in sports brand logo detection and showed that the detection results were superior. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a diagram illustrating the operation process of Residual Block and Inverted Residual Block;

[0020] Figure 2 This refers to the convolution process of MobileNetV2 at different strides;

[0021] Figure 3 This is a diagram of the YOLOv5 network structure;

[0022] Figure 4 This is a structural diagram of the reasoning module of the present invention;

[0023] Figure 5 This is a diagram of the MobileNetV2-YOLOv5 network architecture of the present invention;

[0024] Figure 6 This is a diagram showing the detection results of a preferred embodiment. Detailed Implementation

[0025] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0026] This invention analyzes the connection between the YOLOv5 backbone and neck network, discovering that the YOLOv5 backbone connects to the neck network at convolutional layers that downsample the input image by 8x, 16x, and 32x, respectively. Then, a series of upsampling and downsampling operations are performed to fuse the feature maps output from these three convolutional layers, resulting in three feature maps for use by the head prediction network. The MobileNetV2 network structure also downsamples the input image by 8x, 16x, and 32x, and can also output three different downsampled feature maps. Therefore, based on the connection between the YOLOv5 backbone and neck network, the three downsampled feature maps from MobileNetV2 are input into the YOLOv5 neck network, thereby achieving network lightweighting.

[0027] like Figure 5 In one embodiment shown, the backbone of the YOLOv5 network is replaced by Bottleneck from MobileNetV2 to construct a MobileNetV2-YOLOv5 network. The input logo image first passes through a CBR (Continuous Batch Normalization) layer, which includes a regular convolutional layer, batch normalization, and a ReLU activation function. The stride in the regular convolutional layer is 2, and the kernel size is 3*3. This downsamples the feature map once, and the output feature map's length and width are each 1 / 4 of the network input size. Then, it passes through two Bottleneck-2 sequences, each containing only... The first Bottleneck layer has a stride of 2, while the remaining Bottleneck layers have a stride of 1. The resulting feature map has dimensions that are 1 / 4 of the network input size. After three Bottleneck-2 layers, the output feature map has dimensions that are 1 / 8 of the network input size. After four Bottleneck-2 layers and five Bottleneck-1 layers, the output feature map has dimensions that are 1 / 16 of the network input size. After three Bottleneck-2 layers and one Bottleneck-1 layer, the output feature map has dimensions that are 1 / 32 of the network input size. The 8x, 16x, and 32x downsampled feature maps are input into the YOLOv5 neck network for feature information fusion. An RB module is incorporated before detection to enhance the semantic information of the image. Finally, prediction is performed, and the detection results are output, including the bounding box coordinates, the confidence score of the sports brand logo, and the category.

[0028] Experiment: A network model framework was constructed using YOLOv5 network fused with MobileNetv2. Collected sports brand logo data was used as training data for model training. The specific experimental steps are as follows:

[0029] (1) Dataset expansion. This paper will expand the dataset by performing random cropping, rotation and mirroring operations, etc., in order to enhance the diversity and variety of samples, improve the generalization ability of the model and prevent overfitting during training.

[0030] (2) Divide the dataset. Divide the training set, validation set, and test set into 85%, 5%, and 10% respectively.

[0031] (3) Model building and training. A network model was built by fusing YOLOv5 network with MobileNetv2 and trained using the PyTorch framework.

[0032] (4) To evaluate the model's performance, the trained model was used to detect sports brand logos, and the results are as follows: Figure 6 As shown.

[0033] The research process of this invention is as follows:

[0034] MobileNetV2 is a convolutional neural network architecture proposed based on MobileNetV1. This algorithm retains the depth-separable manipulation method of MobileNetV1 and adds a width multiplier α and a resolution multiplier β to concentrate the necessary feature information within a relatively narrow dimension. Furthermore, this algorithm borrows from the Residual Block in ResNet and designs an Inverted Residual Block method (such as...). Figure 1 (As shown); Structurally, both use convolution kernels of size 1×1→3×3→1×1 for convolution operations, and the Residual Block operation is as follows: Figure 1 As shown in (a), the feature maps undergo dimensionality reduction, convolution, and then dimensionality increase. The entire Residual Block model is large at both ends and small in the middle, resembling an hourglass structure. The Inverted Residual Block operation is as follows: Figure 1 As shown in (b), the feature map undergoes an hourglass-like operation: dimensionality reduction-convolution-dimensionality increase. Conversely, the inverted residual block performs a spindle-like operation: dimensionality increase-convolution-dimensionality reduction. After the second 1×1 convolution in the inverted residual block, a Linear activation function is used instead of ReLU6 to reduce feature loss and maintain the model's expressive power. This structure greatly alleviates the gradient vanishing or exploding problem that occurs as convolutional neural networks deepen, enabling the training of deep convolutional neural networks and improving accuracy.

[0035] MobileNetV2 has different operations for convolutions of different lengths (e.g.) Figure 2 As shown, when the stride is 1, Skip Connection is used during the convolution process; when the stride is 2, Skip Connection is not used during the convolution process.

[0036] Table 1 Overall Network Structure of MobilenetV2

[0037] Input Operator t c n s <![CDATA[224 2 ×3]]> conv2d - 32 1 2 <![CDATA[112 2 ×3]]> bottleneck 1 16 1 1 <![CDATA[112 2 ×3]]> bottleneck 6 24 2 2 <![CDATA[56 2 ×24]]> bottleneck 6 32 3 2 <![CDATA[28 2 ×32]]> bottleneck 6 64 4 1 <![CDATA[28 2 ×64]]> bottleneck 6 96 3 1 <![CDATA[14 2 ×96]]> bottleneck 6 160 3 2 <![CDATA[7 2 ×160]]> bottleneck 6 320 1 1 <![CDATA[7 2 ×320]]> conv2d 1×1 - 1280 1 1 <![CDATA[7 2 ×1280]]> avgpool 7×7 - - 1 - 1×1×k conv2d 1×1 - k - -

[0038] MobileNetV2 mainly consists of a 54-layer structure, as shown in Table 1. Initially, it uses general convolution to extract basic features, and then uses Inverted Residual Blocks for processing. The feature map size becomes smaller and smaller, but its number of channels increases continuously. In addition, the number of channels used inside each Inverted Residual Block is also expanded to 6 times the original size.

[0039] In Table 1, the channel "expansion" factor is represented by t, the number of output channels is represented by c, n represents the number of repetitions, s is the step size, and Inverted Residual Block indicates the operation mode of Bottleneck.

[0040] Table 2 shows the changes in feature map size and channels during the Bottleneck operation. The channel "expansion" factor is still t in Table 1. Based on the data, the Inverted Residual Block processing first uses a 1*1 standard convolution to increase its channel count by a factor of t, then uses a 3*3 depthwise separable convolution, and finally activates the feature maps using a linear activation function and outputs a 1*1 standard convolution, while simultaneously reducing the dimensionality of the feature maps.

[0041] Table 2 Bottleneck processing procedure

[0042]

[0043] The core idea of ​​YOLO series neural networks is to divide the input image into n×n grids, using the center point of each grid as the center of the anchor box, generating two initial prior boxes with different aspect ratios. Then, the prior boxes are bound to the ground truth boxes to calculate the bounding box regression loss, classification loss, and confidence loss. The detection boxes output by the network are subjected to non-maximum suppression (NMS) to remove detection boxes that exceed a given confidence threshold and intersection-over-union (IoU) threshold, and the detection box results are output.

[0044] YOLOv5 optimizes the core ideas of the YOLO series of neural networks. In image data preprocessing, it employs a Mosaic approach for data augmentation and adaptive image scaling. The network uses a CrossStage Partial (CSP) structure, Spatial Pyramid Pooling (SPP), and PathAggregation Network (PAN) structures, and adopts the GIOU Loss function. The YOLOv5 network model structure is as follows: Figure 3 As shown.

[0045] Image data preprocessing mainly involved data augmentation and adaptive image scaling. Mosaic is an effective data augmentation method that merges four input images of different sizes into a single image for training. This data augmentation method effectively improves the network's ability to detect targets outside the background. In actual image detection, the aspect ratios of images acquired by different devices vary greatly. Before being output to the network, these images need to be scaled and padded to the size of the network's input image, such as 640*640 or 416*416, with varying sizes of zero-pixel values ​​padded on both sides. The more pixels padded, the more information redundancy there is, affecting the network's inference speed. YOLOv5 adopts an adaptive image scaling method, adding a minimum number of zero-pixel values ​​to the original image, thereby greatly improving the network's inference speed.

[0046] In the backbone network, CSP divides the input feature map into two parts along the channel direction. The first part does not pass through residual units, while the second part does. Finally, the two outputs are concatenated along the channel direction. This structural design reduces the number of parameters while maintaining accuracy. The Spatial Pyramid Pooling structure uses four different sizes of convolutional kernels (1*1, 5*5, 9*9, and 13*13) for max pooling. By padding the feature map, the output feature map size is made consistent with the input feature map size. Finally, the feature maps from convolutional kernels of different sizes are concatenated along the channel direction as the output. SPP enhances the network's ability to extract information from the feature map by fusing overall and local detail information, thereby improving the accuracy of the network's detection results.

[0047] In PAN, bottom-up downsampling and top-down upsampling combine the feature information of adjacent layers. The bottom-up path makes it easier for lower-layer feature information to be transmitted to the top layer, improving the network's ability to preserve the details of the feature map and effectively addressing the problem of small sports brand logos being difficult to detect. YOLOv5's prediction layer uses the GIoU loss function for bounding box regression, with values ​​in the range of [-1, 1]. The calculation method is shown in Formula 1.

[0048]

[0049] In Formula 1, A c IoU represents the area of ​​the smallest closure rectangle between the detection box and the truth box, IoU represents the intersection-union ratio of the detection box and the truth box, and U represents the area of ​​the union of the detection box and the truth box.

[0050] Reasoning module:

[0051] The Transformer approach is widely used in natural language processing. It employs parallel computation to reduce the computational burden of sequence processing tasks, improving computational efficiency. Furthermore, the Attention mechanism allows for better model interpretation. This invention proposes a ReasoningBlock module. The architecture of the Reasoning layer is as follows: Figure 4 As shown, this structure consists of a Flatten layer, a position encoder, a multi-head attention structure, and Norm, linear, ReLU, and Rearrange layers.

[0052] First, the input tensor is reshaped into a sequence through the Flatten layer. However, this sequence cannot obtain position information. Therefore, sine and cosine encoding is used to obtain relative position encoding in order to obtain regional position information, as shown in Equations 2 and 3.

[0053]

[0054]

[0055] Where i represents the position of the region in the sequence, j represents the feature depth index, and the values ​​obtained after processing by sine and cosine functions are embedded in pairs into the convolutional features and input to the Multi-Head Attention layer. The output of Multi-Head Attention is normalized and then input to the linear layer. To handle internal covariate shift, this invention uses a layer normalization method. In the RB module, two Shortcut connections are used, enabling the network to acquire feature information from earlier layers and enhancing feature reuse. Rearrange is the last sub-layer of the RB module, and its function is to transform the normalized output of the second layer into the shape expected by the detection head.

[0056] In summary, this invention leverages the advantages of the lightweight network of the MobileNetV2 algorithm as its backbone network to improve the YOLOv5 object detection algorithm in a lightweight manner. This not only ensures that the overall network structure is not damaged, but also accelerates the model with less time and resource costs. The results were verified in the detection of sports brand logos and showed that the detection results were superior.

[0057] Based on the preferred embodiments of the present invention described above, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A logo detection method of fusing MobileNetV2, characterized in that, The method comprises the following steps: S1: replacing the backbone network of the YOLOv5 network with a MobileNetV2-based bottleneck to construct a MobileNetV2-YOLOv5 network, wherein the MobileNetV2-YOLOv5 network is trained by using collected sports brand logo data as training data; S2: inputting a logo image, first passing through a CBR to perform one time of down-sampling on a feature map, and outputting a feature map with a length and a width of 1 / 4 of the network input size, wherein the CBR comprises an ordinary convolution layer, a batch normalization layer and a Relu activation function, the ordinary convolution layer has a stride of 2 and a convolution kernel size of 3*3; then passing through two Bottleneck-2s to obtain a feature map with a length and a width of 1 / 4 of the network input size; after passing through three Bottleneck-2s, outputting a feature map with a length and a width of 1 / 8 of the network input size; after passing through four Bottleneck-2s and five Bottleneck-1s, obtaining an output feature map with a length and a width of 1 / 16 of the network input size; after passing through three Bottleneck-2s and one Bottleneck-1, outputting a feature map with a length and a width of 1 / 32 of the network input size; S3: inputting the 8, 16 and 32 times down-sampled feature maps into the neck network of the YOLOv5 to fuse feature information, performing detection and outputting a detection result, wherein an RB module is fused before detection to enhance image semantic information. 2.The logo detection method of fusing MobileNetV2 according to claim 1, characterized in that: In each Bottleneck-2 sequence, only the first layer Bottleneck has a stride of 2, and the remaining Bottleneck has a stride of 1. 3.The logo detection method of fusing MobileNetV2 according to claim 1, characterized in that: The detection result comprises a detection frame coordinate, a logo confidence and a category.

4. The logo detection method of claim 1, wherein: The MobileNetV2-YOLOv5 network is trained through a Pytorch framework.

5. The logo detection method of claim 1, wherein: In the RB module, two Shortcut connections are used, so that the network can obtain feature information of previous layers and strengthen feature reuse.

Images