Infrared ship target detection method based on bar convolution enhanced YOLOv5

Abstract

The invention relates to the technical field of computer vision and image processing, in particular to an infrared ship target detection method based on bar convolution enhanced YOLOv5. The method includes the following steps that a long-wave infrared thermal imaging sensor is adopted to collect images of ships on the sea, self-adaptive histogram equalization processing is carried out on the images, and size normalization is carried out on the processed images; performing multi-scale feature extraction on the processed image through a trunk part of the convolutional neural network; fusing the multi-scale features based on a path aggregation network to obtain three-scale feature maps respectively corresponding to small, medium and large target detection requirements; strip-shaped feature enhancement is carried out on the fused feature maps of the three scales; and detecting the enhanced features by using a detection head to obtain a finally identified target frame.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and image processing, and relates to a technology for identifying large-aspect-ratio ship targets in infrared images, in particular to an infrared ship target detection method based on bar convolution enhanced YOLOv5. BACKGROUND

[0002] Infrared imaging technology plays an important role in sea surface monitoring, sea defense early warning and other fields due to its all-weather operation, strong anti-interference ability and other characteristics. Accurate detection and identification of ships, as the main targets on the sea surface, is of great significance. However, existing infrared ship target detection technologies face the following challenges:

[0003] 1. Weak target features: Infrared images reflect the thermal radiation distribution of objects, lack color and fine texture information, and sea clutter (such as waves, reflections) can easily interfere with the target. Detection mainly relies on the outline and thermal radiation features of the target. Traditional convolutional networks tend to weaken feature expression in deep features, small target features disappear, and background noise overwhelms the target.

[0004] 2. Large aspect ratio problem: Ships often present a narrow geometric shape (large aspect ratio) as shown in Figure 4 (a). Existing mainstream detection algorithms (such as YOLOv5, YOLOv8, etc.) mainly use square convolution kernels (such as 3x3, 5x5) for feature extraction. Its receptive field features are: insufficient receptive field in the ship direction (horizontal or overhead angle), absorbing a lot of background in the vertical direction of the ship, and insufficient feature extraction for long strip targets (such as ships, piers, boats). Square convolution kernels often introduce a lot of background noise (such as surrounding seawater) when processing narrow targets, resulting in impure feature expression, which in turn affects detection accuracy and positioning accuracy, leading to inaccurate positioning, high miss detection rate and high false detection rate.

[0005] 3. Limitations of existing models: Although Swin Transformer and other models introduce global attention mechanisms, they have large computational complexity and slow inference speed, making it difficult to meet the real-time requirements of infrared detection systems. Traditional YOLO series CNN networks, while fast, are limited by square receptive fields and cannot effectively capture the bar structure features of ships.

[0006] Therefore, there is a need for a detection method that can maintain the efficiency of the YOLO series algorithm while effectively extracting the infrared ship's narrow features. SUMMARY

[0007] The purpose of the present application is to solve the problem that the square convolution kernel is difficult to adapt to the long and narrow characteristics of infrared warships and the large background noise interference in the prior art, and an improved single-stage target detection network architecture is proposed, the core of which is to design a strip convolution feature enhancement module and embed it innovatively in the front end of the detection head (Prediction Head) of the network. The method uses large-size strip convolution kernel decomposition technology to explicitly enhance the network's perception ability of long-distance features in the horizontal and vertical directions of infrared warships.

[0008] The technical scheme adopted by the present application to achieve the above purpose is:

[0009] An infrared warship target detection method based on strip convolution enhanced YOLOv5, comprising the following steps:

[0010] 1) A long-wave infrared thermal imaging sensor is used to collect sea ship images, which are subjected to adaptive histogram equalization processing, and the processed pictures are subjected to size normalization;

[0011] 2) The processed images are subjected to multi-scale feature extraction by the backbone part of the convolutional neural network YOLOv5;

[0012] 3) The multi-scale features are fused based on a path aggregation network to obtain three scale feature maps corresponding to small, medium and large target detection requirements respectively;

[0013] 4) The three scale feature maps after fusion are subjected to strip feature enhancement respectively;

[0014] 5) The enhanced features are detected using the detection head to obtain the final recognized target bounding box.

[0015] In step 2), the feature information of the image is extracted layer by layer by a lightweight backbone network based on the CSP architecture composed of multiple convolution modules and C3 modules stacked alternately, specifically comprising the following steps:

[0016] 2.1) The input image is subjected to preliminary feature mapping and down-sampling by a first layer of 6x6 convolution with a step size of 2 to obtain a preliminary feature layer P1

[0017] 2.2) The preliminary feature layer is sequentially subjected to four times of 3x3 convolution with a step size of 2 to gradually reduce the resolution of the feature map, and generate P2 feature layer with a size of 1 / 4 of the input image, P3 feature layer with a size of 1 / 8 of the input image, P4 feature layer with a size of 1 / 16 of the input image and P5 feature layer with a size of 1 / 32 of the input image, respectively;

[0018] 2.3) After each down-sampling, a C3 module containing different numbers of bottleneck layers is connected;

[0019] 2.4) Introducing a fast spatial pyramid pooling module at the end of the backbone network, fusing local features of different scales through parallel connection of multiple 5x5 max pooling layers to obtain high-level feature maps containing rich semantic information.

[0020] In step 3), P3, P4 and P5 feature layers output by the backbone network are fused through top-down and bottom-up stages, specifically including the following steps:

[0021] 3.1) Top-down path: After reducing the dimension of deep feature map P5 through 1x1 convolution, the size is enlarged through upsampling operation, and the channel is spliced with P4 feature map, and then fused through C3 module to generate an intermediate feature layer containing rich semantic information; then, the intermediate feature layer is upsampled again and spliced and fused with shallow feature map P3 to obtain P3 / 8 scale feature map fused with deep semantic and shallow details;

[0022] 3.2) Bottom-up path: P3 / 8 feature map is down-sampled by 3x3 convolution with a step of 2, spliced with the intermediate feature layer, and fused with C3 to generate P4 / 16 scale feature map; then P4 / 16 scale feature map is down-sampled and spliced and fused with the feature map after reducing the dimension of P5 to generate P5 / 32 scale feature map, and the feature maps P3 / 8, P4 / 16 and P5 / 32 obtained after fusion correspond to the detection requirements of small, medium and large targets respectively.

[0023] Step 4) includes the following steps:

[0024] 4.1) The fused features are sequentially processed by 3x3 convolution, batch normalization BatchNorm and LeakyReLU activation function;

[0025] 4.2) Using a residual structure with an internal core containing a strip block as a strip attention unit, using the strip attention unit to process the preprocessed fusion features;

[0026] 4.3) The features processed by the strip attention unit are used as attention weights or enhanced features, which are added to the original fusion features through residual connection to form the final enhanced feature map.

[0027] The strip attention unit performs the following steps:

[0028] (1) Using a 5x5 depth separable convolution to capture local context information of features;

[0029] (2) using a pair of orthogonal bar depth convolutions to process the captured local context information, obtaining a feature map matching the ship shape, wherein the orthogonal bar depth convolutions are respectively a horizontal bar convolution with a kernel size of 1*19 and a vertical bar convolution with a size of 19*1;

[0030] (3) using a 1*1 point convolution to interact and fuse the information between channels.

[0031] The step 5) is specifically:

[0032] The enhanced features are detected by a 1*1 convolution layer of the detection head, and the target bounding box coordinates, confidence and class probability are obtained, and the bounding box with a confidence greater than a specified threshold is taken as the final detection result.

[0033] An infrared ship target detection system based on bar convolution enhanced YOLOv5, comprising:

[0034] An image acquisition and preprocessing module is used to collect sea ship images by using a long-wave infrared thermal imaging sensor, to perform adaptive histogram equalization processing on the images, and to perform size normalization on the processed images.

[0035] A backbone feature extraction module is used to perform multi-scale feature extraction on the processed images by the backbone part of the convolutional neural network YOLOv5.

[0036] A multi-scale feature fusion module is used to fuse multi-scale features based on a path aggregation network, to obtain three scale feature maps corresponding to small, medium and large target detection requirements respectively.

[0037] A bar feature enhancement module is used to perform bar feature enhancement on the three scale feature maps after fusion respectively.

[0038] A target detection module is used to detect the enhanced features using a detection head to obtain the final recognized target bounding box.

[0039] The present application has the following advantages and benefits:

[0040] 1. The present application significantly improves the detection accuracy and positioning accuracy of large aspect ratio ship targets.

[0041] 2. The present application realizes large receptive field feature extraction while maintaining real-time performance, and has high computational efficiency.

[0042] 3. The present application enhances the adaptability to infrared dim target and low contrast environment.

[0043] 4. The present application realizes effective fusion and targeted enhancement of multi-scale features. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 Data preprocessing flowchart of the present application;

[0045] Figure 2 Network structure diagram of the present application;

[0046] Figure 3 Bar regression enhancement module schematic diagram of the present application;

[0047] Figure 4 Convolution kernel receptive field comparison schematic diagram. DETAILED DESCRIPTION

[0048] The present application will be further described in detail below in combination with the accompanying drawings and embodiments.

[0049] The core innovation points of the present application include:

[0050] 1. Detection head pre-enhancement architecture: A dedicated regression enhancement module is inserted between the multi-scale feature fusion network (Neck) and the final prediction layer. The module reshapes the features in an "anisotropic" manner without changing the number of feature map channels, so that the features entering the regression branch focus more on the axial structure of the target.

[0051] 2. Orthogonal bar convolution decomposition mechanism: Abandoning traditional large-size square convolution, a pair of orthogonal bar convolutions (horizontal direction 1xK and vertical direction Kx1) are used to approximate large receptive field. This design not only accurately captures the linear profile of the ship, but also greatly reduces the parameter amount.

[0052] 3. Lightweight attention residual structure: Layer scale and residual connection are introduced in the bar convolution module, and a lightweight attention mechanism is constructed, which effectively alleviates the problem of small target feature disappearing in the deep network in the infrared image.

[0053] As shown in the figure, the method of the present application mainly includes the following steps: Figure 2

[0054] ​1) Image Acquisition and Preprocessing: A long-wave infrared (LWIR) thermal imaging sensor was used as the acquisition terminal. LWIR has high sensitivity for detecting low-temperature targets and strong smoke penetration capability, making it suitable for all-weather operations in complex sea conditions. The sensor output data is uncompressed, high dynamic range raw data. Compared with ordinary 8-bit images, high bit depth data retains subtle temperature difference information of the sea surface, providing a data foundation for extracting weak targets from strong clutter. Targeted enhancement strategies were implemented to address the characteristics of infrared ship images, such as low contrast, blurred edges, and high noise. Adaptive histogram equalization was used, dividing the image into 16×16 sub-blocks and calculating histograms within local areas. This effectively addresses local differences in infrared images, such as extremely cold skies and large temperature variations on the sea surface. The processed images were then normalized in size before being input into the network.

[0055] As shown in Figure 1, the infrared image preprocessing specifically includes: (1) high dynamic range RAW data reading; (2) adaptive histogram equalization (CLAHE) to improve the visibility of weak contrast ships; and (3) normalization and size scaling (e.g., 640×640). This step aims to enhance the effective contrast of the infrared image and conditionally adjust the input data to fit the YOLOv5 input format.

[0056] 2) Backbone Feature Extraction: The image is input into the convolutional neural network backbone, which extracts feature information from the preprocessed infrared image layer by layer, from low-level texture to high-level semantics; for example... Figure 2 As shown, in the specific implementation, a lightweight backbone network based on the CSP (Cross Stage Partial) architecture is adopted. This network consists of a series of convolutional modules (Conv) and C3 modules stacked alternately. First, the input image undergoes preliminary feature mapping and downsampling (P1) through a first-layer 6×6 convolution (stride of 2). Subsequently, the network progressively reduces the feature map resolution through four 3×3 convolution operations with a stride of 2, generating feature layers P2 / 4, P3 / 8, P4 / 16, and P5 / 32 respectively. After each downsampling, a C3 module containing different numbers of bottleneck layers is connected (specifically, the number of repetitions is configured to be 3, 6, 9, and 3 times). The cross-stage connection structure enhances the gradient propagation efficiency and feature reuse capability while deepening the network depth. Finally, a fast spatial pyramid pooling module (SPPF) is introduced at the end of the backbone network (layer 9). By connecting multiple 5×5 max pooling layers in parallel, local features at different scales are fused, and the final output is a high-level feature map containing rich semantic information, which is then used for feature fusion in the subsequent neck network.

[0057] 3) Multi-scale feature fusion: A multi-scale feature fusion module based on path aggregation network (PANet) is constructed (i.e., the Neck part), aiming to enhance the detection ability of the model for infrared ship targets of different scales. First, three feature layers of different scales output by the backbone network are received, namely P3 (the 4th layer, the size is 1 / 8 of the input image), P4 (the 6th layer, the size is 1 / 16), and P5 (the 9th layer, the size is 1 / 32). The fusion process includes two stages: top-down and bottom-up:

[0058] Top-down path: After the deep feature map P5 is reduced in dimension by 1x1 convolution, it is enlarged in size by upsample operation, concatenated with the P4 feature map, and then fused by the C3 module to generate an intermediate feature layer containing rich semantic information. Then, the intermediate feature layer is upsampled again and concatenated and fused with the shallow feature map P3 to obtain a P3 / 8 scale feature map that fuses deep semantic and shallow details.

[0059] Bottom-up path: The P3 / 8 feature map is downsampled by 3x3 convolution with a step of 2, concatenated and C3 fused with the intermediate feature layer generated by the top-down path to generate a P4 / 16 scale feature map. Then, it is downsampled again and concatenated and fused with the reduced P5 feature map to finally generate a P5 / 32 scale feature map.

[0060] Through the above bidirectional fusion mechanism, the network outputs three scale feature maps (layers 17, 20, and 23), which correspond to the detection needs of small, medium, and large targets, effectively solving the problem of large scale changes and susceptibility to background interference of infrared ship targets, and providing feature input for the subsequent detection head (Detect Head) with strong semantic information and accurate positioning information.

[0061] 4) Strip feature enhancement: As shown in Figure 3 , between the multi-scale feature fusion module (Neck) and the final detection head (Detect Head), a strip regression enhancement module (StripRegHead) is introduced for the geometric characteristics of infrared ships. This step independently processes the three scale feature maps (P3, P4, P5) after fusion, aiming to enhance the network's perception ability for slender targets. The specific processing procedure is as follows:

[0062] Preprocessing and local feature extraction: First, the input feature map is passed through a preprocessing layer containing 3x3 convolution, batch normalization (BatchNorm), and LeakyReLU activation function to align the feature space and extract preliminary local texture information.

[0063] Strip attention mechanism construction: Subsequently, the feature map enters the strip attention unit (AttentionHead). The unit adopts a residual structure, and the internal core contains a strip block (StripBlock). In StripBlock, the feature stream passes through: 1) a 5x5 depthwise convolution (Depthwise Conv) for capturing local context information in a slightly larger range; 2) a pair of orthogonal strip depthwise convolutions, which are a horizontal strip convolution with a convolution kernel size of 1x19 and a vertical strip convolution with a convolution kernel size of 19x1. As shown in FIGS. Figure 4 (b), this asymmetric large convolution kernel design can construct an elongated strip receptive field, establishing long-range spatial dependencies in the horizontal and vertical directions. Compared with the square receptive field in (a), this design can accurately match the morphological characteristics of elongated targets such as ships, capturing the target body while significantly reducing the intake of background noise on the upper and lower sides of the sea surface; this asymmetric large convolution kernel design can establish long-range spatial dependencies in the horizontal and vertical directions, accurately match the morphological characteristics of elongated targets such as ships; 3) a 1x1 point convolution (Pointwise Conv) for information interaction and fusion between channels. Figure 4

[0064] Feature reorganization and output: The features processed by the strip convolution are used as attention weights or enhancement features, which are added to the original input features through a residual connection (Shortcut) to form the final enhanced feature map. This feature map retains rich long-range context information, which is then sent to the 1x1 convolution layer of the detection head to generate the final target bounding box coordinates, confidence, and class probability.

[0065] Figure 4 The present invention and the receptive field of the convolution kernel are compared with the prior art, wherein, Figure 4 (a) shows the receptive field diagram when a standard square convolution kernel (such as 3x3) is used in the prior art. As can be seen, for an elongated ship target, the square receptive field inevitably introduces a large amount of sea surface background clutter (red shaded area in the figure) on the upper and lower sides of the target while covering the target, resulting in dilution of the effective features.

[0066] Figure 4 (b) shows the receptive field diagram when a strip convolution kernel (such as 1xK) is used in the present invention. The horizontal strip receptive field (green solid line box) is highly consistent with the geometric shape of the ship, which can extract features closely along the target contour, and maximally exclude the interference of background noise, thereby improving the signal-to-noise ratio of feature extraction.​

Claims

1. A method for infrared ship target detection based on strip convolution enhancement YOLOv5, characterized in that, Includes the following steps: 1) Long-wave infrared thermal imaging sensors are used to acquire images of ships at sea, adaptive histogram equalization is applied to them, and the size of the processed images is normalized. 2) Multi-scale feature extraction is performed on the processed image using the backbone of the YOLOv5 convolutional neural network; 3) Based on the path aggregation network, multi-scale features are fused to obtain feature maps of three scales corresponding to the detection requirements of small, medium and large targets respectively; 4) Perform bar feature enhancement on the feature maps of the three scales after fusion; 5) Use the detection head to detect the enhanced features to obtain the final identified target bounding box.

2. The infrared ship target detection method based on strip convolution enhancement YOLOv5 according to claim 1, characterized in that, Step 2) involves extracting image feature information layer by layer using a lightweight backbone network based on a CSP architecture, which consists of multiple convolutional modules and C3 modules stacked alternately. Specifically, this includes the following steps: 2.1) The input image is initially mapped and downsampled by a 6×6 convolution with a stride of 2 in the first layer to obtain the initial feature layer P1; 2.2) The initial feature layer is passed through four 3×3 convolutions with a stride of 2 to gradually reduce the feature map resolution, generating a P2 feature layer with a size of 1 / 4 of the input image, a P3 feature layer with a size of 1 / 8 of the input image, a P4 feature layer with a size of 1 / 16 of the input image, and a P5 feature layer with a size of 1 / 32 of the input image. 2.3) After each downsampling, connect to the C3 module, which contains a different number of bottleneck layers; 2.4) A fast spatial pyramid pooling module is introduced at the end of the backbone network. By connecting multiple 5×5 max pooling layers in parallel, local features of different scales are fused to obtain a high-level feature map containing rich semantic information.

3. The infrared ship target detection method based on strip convolution enhancement YOLOv5 according to claim 1, characterized in that, In step 3), the P3, P4, and P5 feature layers output by the backbone network are fused through two stages: top-down and bottom-up. Specifically, this includes the following steps: 3.1) Top-down path: After the deep feature map P5 is reduced in dimension by 1×1 convolution, its size is enlarged by upsampling, and it is concatenated with the P4 feature map through channels. Then it is fused by the C3 module to generate an intermediate feature layer containing rich semantic information. Subsequently, the intermediate feature layer is upsampled again and concatenated and fused with the shallow feature map P3 to obtain a P3 / 8 scale feature map that integrates deep semantics and shallow details. 3.2) Bottom-up path: The P3 / 8 feature map is downsampled by a 3×3 convolution with a stride of 2, concatenated with the intermediate feature layer, and then fused with C3 to generate a P4 / 16 scale feature map; then the P4 / 16 scale feature map is downsampled and concatenated with the feature map after dimensionality reduction of P5 to generate a P5 / 32 scale feature map. The feature maps P3 / 8, P4 / 16, and P5 / 32 obtained by fusion correspond to the detection requirements of small, medium, and large targets, respectively.

4. The infrared ship target detection method based on strip convolution enhancement YOLOv5 according to claim 1, characterized in that, Step 4) includes the following steps: 4.1) The fused features are sequentially processed by 3×3 convolution, BatchNorm normalization, and LeakyReLU activation function; 4.2) A residual structure with an internal core containing a strip block is used as a strip attention unit, and the preprocessed fused features are processed using this strip attention unit; 4.3) The features processed by the bar attention unit are used as attention weights or enhanced features, and added to the original fused features through residual connections to form the final enhanced feature map.

5. The infrared ship target detection method based on strip convolution enhancement YOLOv5 according to claim 4, characterized in that, The bar-shaped attention unit performs the following steps: (1) Use a 5×5 depthwise separable convolution to capture the local contextual information of the features; (2) The captured local context information is processed by a pair of orthogonal bar depth convolutions to obtain a feature map matching the ship shape, wherein the orthogonal bar depth convolutions are a horizontal bar convolution with a kernel size of 1×19 and a vertical bar convolution with a kernel size of 19×1. (3) Use a 1×1 point convolution to interact and fuse information between channels.

6. The infrared ship target detection method based on strip convolution enhancement YOLOv5 according to claim 1, characterized in that, Step 5) specifically involves: Enhanced features are detected using a 1×1 convolutional layer in the detection head. The target bounding box coordinates, confidence score, and class probability are used, and the bounding box with a confidence score greater than a specified threshold is taken as the final detection result.

7. An infrared ship target detection system based on strip convolution enhancement YOLOv5, characterized in that, include: The image acquisition and preprocessing module is used to acquire images of ships at sea using a long-wave infrared thermal imaging sensor, perform adaptive histogram equalization on the images, and normalize the size of the processed images. The backbone feature extraction module is used to extract multi-scale features from the processed image through the backbone of the YOLOv5 convolutional neural network. The multi-scale feature fusion module is used to fuse multi-scale features based on the path aggregation network to obtain feature maps at three scales corresponding to the detection requirements of small, medium and large targets, respectively. The bar feature enhancement module is used to enhance the bar features of the three scales of the fused feature maps respectively; The target detection module is used to detect the enhanced features using a detection head to obtain the final identified target bounding box.

Images