Small ship detection method based on convolution block attention mechanism and explicit visual center

Abstract

The invention discloses a small ship detection method based on a convolution block attention mechanism and an explicit visual center. The small ship detection method comprises the following steps: acquiring a water surface ship image; constructing a backbone network model, and inputting the water surface ship image into a backbone network for feature extraction to obtain a semantic information output feature map; a learnable up-sampling operation based on point sampling is carried out on the feature map through a feature pyramid network module, and an up-sampling operation is carried out based on the offset of the feature map to obtain a high-resolution feature map; performing explicit visual center post-processing on the high-resolution feature map to capture a long-distance dependency relationship between pixels and restore edge information to obtain a final feature map; and carrying out splicing and down-sampling on the final feature map again to generate a deep feature map with low resolution and high channel number, and combining the deep feature map with the final feature map to carry out target detection to obtain a central point coordinate of the small ship, a width value and a height value of a detection frame, and category and confidence information of the target.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and artificial intelligence detection technology, and in particular to a small ship detection method based on a convolutional block attention mechanism and an explicit visual center. Background Art

[0002] In the field of computer vision, there are a large number of mature solutions for target detection, such as MobileNet, Faster R-CNN, and You Only Look Once (YOLO) series. Among them, the YOLO series is widely used in industrial sites to achieve target detection tasks for specific objects due to its simple structure, easy deployment, and balanced performance in detection accuracy and inference speed. Generally, the YOLO network consists of three parts, in which the backbone network downsamples the incoming image to generate a feature map with semantic features; the feature pyramid network resamples the feature map and fuses the image information of the low-dimensional feature map to obtain better detection results; the detection head receives feature maps of different dimensions and resolutions in the feature pyramid, completes the calculation of the loss function during training and the output of the detection results during testing, including the location and category of the detected object. However, in the task of small target detection, the YOLO series has a dilemma. When the model parameters are low, the detection accuracy will be seriously reduced under high detection accuracy requirements, and the high-parameter model has application scenario restrictions. Under limited hardware conditions, there are problems such as difficulty in deployment and slow inference speed.

[0003] In recent years, feature extraction based on attention mechanism is considered to be an effective method to capture the relationship between pixels. Among them, the self-attention mechanism is widely used in the field of natural language processing to capture the relationship between characters in the whole text. However, in the field of computer vision, the self-attention mechanism will bring unacceptable large matrix operations to the reasoning process. Therefore, the attention mechanism in the field of computer vision attempts to extract the relationship between pixels at a low reasoning cost. Classic works in this field include SENet and CBAM (convolutional block attention mechanism). However, in practical applications, the attention mechanism is usually cascaded to the output of a module, the computational load of the model is still high, and the cascade operation increases the length of the shortest path in the neural network, bringing additional risk of feature loss. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention discloses a small ship detection method based on a convolutional block attention mechanism and an explicit visual center, which specifically includes the following steps:

[0005] Acquire images of surface vessels;

[0006] Constructing a backbone network model, wherein the backbone network model includes a convolution block, a combined convolution block with a convolution block attention mechanism, and a feature pyramid network module;

[0007] The surface ship image is input into the backbone network for feature extraction, wherein the combined convolution block performs feature map processing on the surface ship image, separates the feature map of the surface ship image with a low number of channels through a gradient path, extracts the convolution block attention on the feature map, and fuses it with the unprocessed low number of channels feature map to obtain a semantic information output feature map;

[0008] Through the feature pyramid network module, the feature map is subjected to a learnable upsampling operation based on point sampling, and the upsampling operation is performed based on the feature map offset to obtain a high-resolution feature map. Then, the high-resolution feature map is subjected to explicit visual center post-processing to capture the long-distance dependency between pixels and restore the edge information to obtain the final feature map.

[0009] The final feature map is spliced ​​and downsampled again to generate a deep feature map with low resolution and high number of channels. The deep feature map is combined with the final feature map for target detection to obtain the center point coordinates of the small ship, the width and height values ​​of the detection frame, and the target category and confidence information.

[0010] Furthermore, when the convolution block is combined to process the feature map of the surface ship image: the convolution block attention mechanism is used to extract attention in the space and channel dimensions of the image respectively:

[0011] CA(·)=σ(PConv(AvgPool C (·)+MaxPool C (·)))

[0012] SA(·)=σ(Conv 7×7 (Concat(AvgPool HW (·),MaxPool HW (·))))

[0013]

[0014] Where AvgPool and MaxPool represent average pooling and maximum pooling operations respectively, PConv is the point convolution operation, σ is the Sigmoid activation function, It is an element-level multiplication operation between two vectors. Before the feature map enters the bottleneck, the convolution block divides the feature map into two parts with equal number of channels. Only one part will be processed by the bottleneck, and the other part will directly participate in the splicing operation. The part participating in the bottleneck processing is marked as X. in2 ;

[0015] Xout =PConv(Concat(X in1 ,CBAM(Bottleneck(X in2 ))))

[0016] Bottleneck represents the bottleneck operation. After CBAM attention processing, X in1 With X in2 The information gradient difference increases, thereby performing attention extraction.

[0017] Furthermore, when performing a learnable upsampling operation based on point sampling on the feature map: for an input feature map X with a shape of (B, C, H, W) in In the offset value generation branch, the feature map dimension is changed to (B, C / s) by pixel confusion. 2 ,sH,sW), respectively input two point convolutions to transform the channel dimension to 2g;

[0018]

[0019] PS stands for pixel confusion operation, a point convolution operation is performed after the Sigmoid activation function and then the element multiplication operation;

[0020] For the obtained feature map, pixel anti-aliasing is performed and the initial sampling value based on bilinear interpolation is added to prevent the generated offset value from degenerating into neighbor interpolation, and the final feature map offset Offset (X in ), sort out and obtain

[0021]

[0022] Where PU represents the pixel anti-aliasing operation, and INIT_POS is the bilinear interpolation initial sampling value of the feature map;

[0023] The original input feature map X in Reorganize in the channel dimension to obtain a feature map of shape (B, g, C / g, H, W), and then perform grid sampling combined with the feature map offset to obtain the upsampled feature map:

[0024] X out =GSample(X in ,PU(Offset(X in )))

[0025] Among them, GSample represents grid sampling, and the upsampling strategy retains the semantic information in the feature map to reduce the loss.

[0026] Furthermore, when performing explicit visual center post-processing on the high-resolution feature map: for the input feature map X in,Through 7×7 convolution kernel, batch regularization and activation function preliminary processing, the explicit visual center network is divided into two parts: a lightweight multi-layer perceptron and a learnable visual center, which respectively capture the long-distance dependencies between pixels and restore the edge information;

[0027] The lightweight multi-layer perceptron performs layer regularization and depth-wise separable convolution on the input feature map, performs channel weighting in the channel dimension, performs attention extraction in the channel dimension, and performs residual connection between the result and the input feature map before regularization.

[0028]

[0029] Where LN is layer normalization, DConv is depth-separable convolution, and T1 is a set of vectors that are related to the input feature map X in Perform element-wise multiplication on The layer is regularized again and passed into the channel multi-layer perceptron to extract the dependency relationship between pixels in different channels. The results are channel-weighted and residually connected to obtain the output of the lightweight multi-layer perceptron.

[0030] CMLP(·)=PConv2(GELU(PConv1(·)))

[0031]

[0032] Where GELU is the GELU activation function, T2 is the output result of the channel multi-layer perceptron, and the channel weighting is performed to obtain the long-distance dependency relationship LMLP (X in );

[0033] After the learnable visual center performs inverse residual connection and convolution on the incoming feature map, the learnable encoding dictionary B = {b1, b2, ..., b K} and weighting factor S = {s1,s2,…,s K} performs encoding operation, where K is the number of encoded words in the dictionary; for each vector with C channels in the feature map and encoding dictionary b k , the location-related information can be obtained by calculating the weighting factor After that, the position-related information is summed according to the encoding dictionary, batch normalization and activation function processing are performed to obtain the relative position information of the encoding dictionary for the global feature map. Finally, the average value of K relative position information is calculated to obtain the eigenvalue of the completed encoding.

[0034] After the encoded feature value e is processed by the fully connected layer and the Sigmoid activation function, it is element-wise multiplied with the feature map of the learnable visual center, and then the result is residually connected with the feature map to restore the edge information lost in the feature map:

[0035]

[0036] Among them, FC is the fully connected layer processing, (·) + It is a negative filter, which removes the negative values ​​in the feature map and finally obtains the feature map LVC (X in );

[0037] Finally, the output feature maps of the two parts are concatenated and point-convolved to obtain the output feature map of the explicit visual center.

[0038] EVC(X in )=PConv(Concat(LVC(X in ),LMLP(X in ))

[0039] This can capture long-distance dependencies between pixels and restore edge information.

[0040] Due to the adoption of the above technical scheme, the present invention provides a small ship detection method based on the convolutional block attention mechanism and the explicit visual center. The method is carried out to address the characteristics of small target detection difficulty, poor detection accuracy, and large amount of computation required for the detection model. Based on the existing neural network of YOLO, many aspects of optimization and improvement are performed. By combining the gradient path design idea, more effective attention extraction is achieved than cascade output, avoiding the problem of feature map information loss caused by the increase in the shortest path of the neural network, and the parameter increase brought by the design is much smaller than that of the cascade output.

[0041] The present invention adopts learnable upsampling based on point sampling to replace the nearest neighbor interpolation and bilinear interpolation based on image information. For the offset generation network, in the initial stage of training, the offset generation strategy can be regarded as bilinear interpolation with random initialization values. However, during the back propagation of the loss function, the convolution kernel will be updated, so that the upsampling strategy tends to retain the semantic information in the feature map to reduce the loss, thereby realizing upsampling for ship type detection.

[0042] The present invention uses a post-processing module based on explicit visual center to process the upsampled feature map, wherein the lightweight perceptron uses an operation similar to the convolutional block attention to capture the dependencies between two-dimensional pixels in the feature map and between the same pixels in different channels in the plane and channel dimensions, respectively, to achieve long-distance dependency capture similar to the self-attention mechanism; the learnable visual center can focus on the edge information of different channels through encoding operations to enhance the edge information expression of the feature map. After the feature map is processed by the explicit visual center, the edge information of the small ship target is restored, and the detection accuracy of the small ship target is significantly improved. Overall, the present invention proposes a small ship detection method based on the convolutional block attention mechanism and the explicit visual center. The method has a strong small ship detection capability, and the network requires small computing resources, short inference time, and fast detection speed. Under the condition of low parameter quantity, it has a detection effect comparable to the best small target detection method at that time. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 It is a flow chart of the small ship target detection method of the present invention;

[0045] Figure 2 This is an example diagram of a small and medium-sized ship target of the present invention;

[0046] Figure 3 is the visual feature map before explicit visual center processing in the present invention

[0047] Figure 4 This is the visualization feature map after explicit visual center processing in the present invention.

[0048] Figure 5 This is the visual feature map after the comprehensive detection head input in the present invention

[0049] Figure 6 Schematic diagram of the detection result of the ship target in the figure in the present invention DETAILED DESCRIPTION

[0050] In order to make the technical solutions and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention:

[0051] like Figure 1A small ship detection method based on convolutional block attention mechanism and explicit visual center is shown, which specifically includes the following steps:

[0052] The method performs downsampling through convolution blocks and convolution blocks based on the convolution block attention mechanism, extracts image information in the feature map as semantic information, and then performs resampling processing through the feature pyramid network. After processing by the upsampling module based on point sampling, the feature map is spliced ​​with the shallow layer and the convolution transformation channel is performed. After the last step of upsampling is completed, it is passed to the post-processing module based on the explicit visual center. The detection capability of small ships is improved by capturing the long-distance dependency relationship between pixels and restoring edge information. Finally, the feature maps of different resolutions and channel numbers are passed to the detection head, and the detection results are integrated to obtain the ship detection frame and ship type detected in the input image. The specific steps include the following:

[0053] For the incoming image to be detected Figure 2 ,like Figure 1 As shown in the figure, the convolutional block without integrated attention mechanism aggregates and transforms the image information, reduces the resolution of the feature map, and increases the number of channels to improve the ability to express semantic information; while the convolutional block combined with convolutional block attention mechanism (CBAM) will extract attention from the feature map;

[0054] The convolution block combined with the convolution block attention mechanism splits the incoming feature map on the channel to obtain two feature maps with the same resolution, denoted as X in1 With X in2 , where X in1 Without performing any calculation, it directly participates in the final splicing operation, while X in2 The channel is transformed into the bottleneck module and then passed into the convolution block attention mechanism for processing

[0055] CA(·)=σ(PConv(AvgPool C (·)+MaxPool C (·)))

[0056] SA(·)=σ(Conv 7×7 (Concat(AvgPool HW (·),MaxPool HW (·))))

[0057]

[0058] Processed X in2 With X in1 By concatenating and point convolution, we can get a feature map X with rich information gradients. out ;

[0059] Xout =PConv(Concat(X in1 ,CBAM(Bottleneck(X in2 ))))

[0060] In the feature pyramid network, the convolution block behaves differently in the upsampling path and the downsampling path. In the downsampling path, the convolution block simultaneously reduces the feature map resolution and increases the number of feature map channels; in the upsampling path, the convolution block only reduces the number of channels of the incoming feature map without changing the feature map resolution. The resolution improvement of the feature map is completed by the upsampling module. For the upsampling module, the learnable upsampling module based on point sampling used in the present invention generates a feature map displacement offset through a neural network, and completes the upsampling according to the guidance of the offset. For an input feature map X with a shape of (B, C, H, W) in , the processing in the offset generation branch is as follows:

[0061]

[0062] X in By pixel confusion, the feature map dimension is changed to (B, C / s 2 ,sH,sW), and then pass two point convolutions respectively to transform the channel dimension to 2g. One point convolution operation is activated by the Sigmoid function and then element-wise multiplied with the output of the other point convolution. After pixel anti-aliasing, it is added to the initial sampling value based on bilinear interpolation to obtain the final feature map offset Offset (X in ).

[0063] For the original input feature map, it is reorganized in the channel dimension to obtain a feature map of shape (B, g, C / g, H, W), and the upsampled feature map is obtained after grid sampling combined with the feature map offset.

[0064] X out =GSample(X in ,PU(Offset(X in )))

[0065] Furthermore, the feature maps obtained after all upsampling are visualized by EigenCAM, and the results are as follows Figure 3 As shown, it is obvious that the feature map pays less attention to small ships, so it also needs to be processed by explicit visual center to capture the information of small ships by restoring the lost edge information and capturing the long-distance dependency between pixels.

[0066] For the input feature map X in,After preliminary processing by 7×7 convolution kernel, batch regularization and activation function, the ,explicit visual center network is divided into a lightweight multi-layer perceptron and a learnable ,visual center to capture long-distance dependencies between pixels and ,restore edge information.

[0067] The lightweight multi-layer perceptron performs layer regularization and depth-separable convolution on the input feature map, then performs channel weighting in the channel dimension, and performs residual connection with the input feature map before regularization to obtain a feature map that captures the dependency relationship between pixels in the same channel.

[0068]

[0069] right The layer is normalized again, and then the layer is passed into the channel multi-layer perceptron to extract the dependency relationship between pixels in different channels. The result is channel weighted and residual connected to obtain the output feature map LMLP (X in )

[0070] CMLP(·)=PConv2(GELU(PConv1(·)))

[0071]

[0072] After the learnable visual center performs inverse residual connection and convolution on the incoming feature map, the learnable encoding dictionary B = {b1, b2, ..., b K} and weighting factor S = {s1,s2,…,s K} performs encoding operation, for each vector with C channels in the feature map and encoding dictionary b k , the location-related information can be obtained by calculating the weighting factor After that, the position-related information is summed according to the encoding dictionary, batch normalization and activation function processing are performed to obtain the relative position information of the encoding dictionary for the global feature map. Finally, the average value of K relative position information is calculated to obtain the eigenvalue of the completed encoding. After the encoded feature value e is processed by the fully connected layer and the Sigmoid activation function, it is element-wise multiplied with the feature map of the learnable visual center. The result is then residually connected to the feature map and the negative values ​​in the feature map are filtered, that is, the edge information LVC(X) lost in the feature map is restored. in ),

[0073]

[0074] Finally, the output feature maps of the two parts are concatenated and point-convolved to obtain the output feature map EVC (X in ),like Figure 4 As shown, the processed feature map focuses on small ships, proving the effectiveness of the explicit visual center;

[0075] EVC(X in )=PConv(Concat(LVC(X in ),LMLP(X in ))

[0076] After the explicit visual center is processed, the feature maps are spliced ​​and downsampled again to generate deep feature maps with low resolution and high number of channels. The feature maps at each level are combined for target detection. Figure 5 It can be seen that the detection of the feature maps of each layer can filter out irrelevant background information and make the detection head focus on small ships. The feature map generated by the explicit visual center post-processing and the deep feature map are passed to different detection heads, and the results of each detection head are combined to obtain the center point coordinates of the detection object (C x ,C y ), the width and height of the detection box (w, h), and the category and confidence level of the target (cls, pred).

[0077] The original input image is Figure 1 The results after the treatment of the present invention are as follows Figure 6 As shown, it can be seen that the present invention can effectively and accurately detect small ships.

[0078] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A small ship detection method based on convolutional block attention mechanism and explicit visual center, characterized in that include: Acquire images of surface vessels; Constructing a backbone network model, wherein the backbone network model includes a convolution block, a combined convolution block with a convolution block attention mechanism, and a feature pyramid network module; The surface ship image is input into the backbone network for feature extraction, wherein the combined convolution block performs feature map processing on the surface ship image, separates the feature map of the surface ship image with a low number of channels through a gradient path, extracts the convolution block attention on the feature map, and fuses it with the unprocessed low number of channels feature map to obtain a semantic information output feature map; Through the feature pyramid network module, the feature map is subjected to a learnable upsampling operation based on point sampling, and the upsampling operation is performed based on the feature map offset to obtain a high-resolution feature map. Then, the high-resolution feature map is subjected to explicit visual center post-processing to capture the long-distance dependency between pixels and restore the edge information to obtain the final feature map. The final feature map is spliced ​​and downsampled again to generate a deep feature map with low resolution and high number of channels. The deep feature map is combined with the final feature map for target detection to obtain the center point coordinates of the small ship, the width and height values ​​of the detection frame, and the target category and confidence information.

2. According to claim 1, a small ship detection method based on convolutional block attention mechanism and explicit visual center is characterized in that: When the convolution block is combined to process the feature map of the surface ship image: the convolution block attention mechanism is used to extract attention in the image space and channel dimensions respectively: CA(·)=σ(PConv(AvgPool C (·)+MaxPool C (·))) SA(·)=σ(Conv 7×7 (Concat(AvgPool HW (·),MaxPool HW (·)))) Where AvgPool and MaxPool represent average pooling and maximum pooling operations respectively, PConv is the point convolution operation, σ is the Sigmoid activation function, It is an element-level multiplication operation between two vectors. Before the feature map enters the bottleneck, the convolution block divides the feature map into two parts with equal number of channels. Only one part will be processed by the bottleneck, and the other part will directly participate in the splicing operation. The part participating in the bottleneck processing is marked as X. in2 ; X out =PConv(Concat(X in1 ,CBAM(Bottleneck(X in2 )))) Bottleneck represents the bottleneck operation. After CBAM attention processing, X in1 With X in2 The information gradient difference increases, thereby performing attention extraction.

3. According to claim 1, a small ship detection method based on convolutional block attention mechanism and explicit visual center is characterized in that: When performing a learnable upsampling operation based on point sampling on the feature map: for an input feature map X with a shape of (B, C, H, W) in In the offset value generation branch, the feature map dimension is changed to (B, C / s) by pixel confusion. 2 ,sH,sW), respectively input two point convolutions to transform the channel dimension to 2g; PS stands for pixel confusion operation, a point convolution operation is performed after the Sigmoid activation function and then the element multiplication operation; For the obtained feature map, pixel anti-aliasing is performed and the initial sampling value based on bilinear interpolation is added to prevent the generated offset value from degenerating into neighbor interpolation, and the final feature map offset Offset (X in ), sort out and obtain Where PU represents the pixel anti-aliasing operation, and INIT_POS is the bilinear interpolation initial sampling value of the feature map; The original input feature map X in Reorganize in the channel dimension to obtain a feature map of shape (B, g, C / g, H, W), and then perform grid sampling combined with the feature map offset to obtain the upsampled feature map: X out =GSample(X in ,PU(Offset(X in ))) Among them, GSample represents grid sampling, and the upsampling strategy retains the semantic information in the feature map to reduce the loss.

4. According to claim 1, a small ship detection method based on convolutional block attention mechanism and explicit visual center is characterized in that: When performing explicit visual center post-processing on high-resolution feature maps: For the input feature map X in ,Through 7×7 convolution kernel, batch regularization and activation function preliminary processing, the explicit visual center network is divided into two parts: a lightweight multi-layer perceptron and a learnable visual center, which respectively capture the long-distance dependencies between pixels and restore the edge information; The lightweight multi-layer perceptron performs layer regularization and depth-wise separable convolution on the input feature map, performs channel weighting in the channel dimension, performs attention extraction in the channel dimension, and performs residual connection between the result and the input feature map before regularization. Where LN is layer normalization, DConv is depth-separable convolution, and T1 is a set of vectors that are related to the input feature map X in Perform element-wise multiplication on Perform layer regularization again, input the channel multi-layer perceptron, extract the dependency relationship between pixels in different channels, perform channel weighting and residual connection on the results, and obtain the output of the lightweight multi-layer perceptron; CMLP(·)=PConv2(GELU(OConv1(·))) Where GELU is the GELU activation function, T2 is the output result of the channel multi-layer perceptron, and the channel weighting is performed to obtain the long-distance dependency relationship LMLP (X in ); The learnable visual center performs inverse residual connection and convolution processing on the input feature map, and uses the learnable encoding dictionary B = {b1, b2, ..., b K } and weighting factor S = {s1,s2,…,s K } performs encoding operation, where K is the number of encoded words in the dictionary, and for each vector with C channels in the feature map and encoding dictionary b k , the location-related information can be obtained by calculating the weighting factor After that, the position-related information is summed according to the encoding dictionary, batch normalization and activation function processing are performed to obtain the relative position information of the encoding dictionary for the global feature map. Finally, the k relative position information is averaged to obtain the eigenvalue of the completed encoding. After the encoded feature value e is processed by the fully connected layer and the Sigmoid activation function, it is element-wise multiplied with the feature map of the learnable visual center, and then the result is residually connected with the feature map to restore the edge information lost in the feature map: Among them, FC is the fully connected layer processing, (·) + It is a negative filter, which removes the negative values ​​in the feature map and finally obtains the feature map LVC (X in ); Finally, the output feature maps of the two parts are concatenated and point-convolved to obtain the output feature map of the explicit visual center. EVC(X in )=PConv(Concat(LVC(X in ),LMLP(X in )) This captures long-distance dependencies between pixels and restores edge information.

Images