A method and system for detecting small ship targets in thermal infrared images based on hierarchical feature attention network

Abstract

The application discloses a kind of based on hierarchical feature attention network's micro ship target detection method and system in thermal infrared image, belong to satellite remote sensing image processing and application technical field.The first three-channel infrared ship target detection image dataset based on SDGSAT-1 pixel-level annotation is constructed by the application.The dataset covers more scenes and contains rich heat information, and is more suitable for micro ship detection task in thermal infrared image.At the same time, a network model based on hierarchical feature attention mechanism is constructed, which generates feature maps of different scales through multi-level detail enhancement processing, extracts multi-level long-distance information through multi-level large kernel attention mechanism, and generates a prediction target probability map through multi-level fusion of different scale feature maps, realizing different scale feature fusion and interaction, and accurately locating the ship.In the ship target detection task, the detection accuracy, false alarm rate and intersection over union (IoU) of the application are significantly better than those of the existing model.

Description

Technical Field

[0001] This invention relates to the field of satellite remote sensing image processing and application technology, specifically to a method and system for detecting small ship targets in thermal infrared images based on hierarchical feature attention networks. Background Technology

[0002] Accurate and reliable ship target detection is crucial for ensuring sound economic development and maintaining maritime transport safety. Remote sensing imagery can acquire a wide range of information, offering a potential solution for space-based ship detection. However, different Earth observation satellites often carry sensors with different spectral settings, and the image sizes are relatively large. Within a single image, the complex background environment, the diversity of ship appearances and surrounding targets, and the low image resolution make detecting even small ship targets a challenging task. Therefore, developing intelligent ship detection methods based on remote sensing imagery is of great significance for sustainable ocean management.

[0003] Currently, images used for ship detection are mainly concentrated in the visible light and infrared bands. Visible light images are easily affected by weather changes. In SAR images, the constant variation in natural factors such as weather and wind speed can lead to uneven sea level, which is not conducive to SAR image application. The infrared band, however, can penetrate the atmosphere more effectively. Especially in marine environments, ships typically generate significant thermal differences, so thermal infrared images may be easier to identify ships and are less affected by the surrounding environment. Using satellite infrared bands to identify ships is a typical infrared small target detection (IRSTD) task. Traditional IRSTD methods include filter-based background methods (see “Infrared small target detection via low-rank tensor completion with top-hat regularization,” IEEE Transactions on Geoscience and Remote Sensing, vol. 58, no. 2, pp. 1004-1016, 2019); local contrast-based methods (see “A local contrast method for infrared small-target detection utilizing a tri-layer window,” IEEE Geoscience and Remote Sensing Letters, vol. 17, no. 10, pp. 1822-1826, 2020); and local rank-based methods (see “Infrared small target detection via non-convex rank approximation minimization joint l 2,1norm,” Remote Sensing, vol. 10, no. 11, pp. 1821, 2018), which have made some progress. Because it requires manual design of parameters and extraction thresholds, its performance is limited in remote sensing images with complex environments, exhibiting weak generalization ability.With the rapid development of deep learning, Hou et al. used fully connected layers in skip connections to enhance the contrast between targets and backgrounds in infrared images (see, "ISTDU-Net: Infrared Small-Target Detection U-Net," IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1-5, 2022). To achieve interaction between low-level and high-level features, Li et al. proposed a Dense Nested Attention Network (DNANet) and introduced Cascaded Channel and Spatial Attention Module (CSAM) to enhance multi-level features (see, "Dense nested attention network for infrared small target detection," IEEE Transactions on Image Processing, vol. 32, no., pp. 1745-1758, 2023). While convolutional neural networks (CNNs) are indeed effective in extracting local information, the extraction of long-range information is crucial in the context of complex structures in remote sensing images. Wu et al. proposed the NUDT-SIRST-Sea space-based infrared tiny ship detection dataset, introducing a hybrid encoder structure of VisionTransformer (ViT) and CNN, and constructing a multi-level TransUNet (MTU-Net) to extract features at each level (see, "MTU-Net: Multilevel TransUNet for space-based infrared tiny ship detection," IEEE Transactions on Geoscience and Remote Sensing, vol.61, pp.1-15, 2023). However, when extracting multi-level global information, the relationships between features at each level were not utilized.

[0004] To date, scholars at home and abroad have proposed many deep learning-based ship detection methods, but there are still some obvious shortcomings: (1) The model does not fully utilize the feature information of different scales in the global remote sensing image, which is not conducive to segmenting small target objects from the image. (2) Due to the limited quantity and quality of the dataset, the detection accuracy of the model will also be affected by the selection of different satellite bands. (3) Due to the complex background environment of the image and the diversity of surrounding targets, there are often missed detections and false detections during detection. Summary of the Invention

[0005] To address the need for rich dataset information in the task of detecting small vessels in thermal infrared images, and the problems of information loss and feature extraction in target detection models, this invention provides a method and system for detecting small vessels in thermal infrared images based on a hierarchical feature attention network. This invention constructs a pixel-level annotated three-channel infrared vessel target detection dataset, providing rich thermal information and richer, more accurate data support for feature extraction in subsequent deep learning models. The constructed network model based on the hierarchical feature attention mechanism plays a crucial role in the input, extraction, aggregation, and target localization of multi-scale features, achieving full extraction and efficient utilization of multi-level features, thereby realizing high-precision extraction and localization of vessel outlines.

[0006] To achieve the above technical objectives, the present invention adopts the following technical solution:

[0007] A method for detecting small ship targets in thermal infrared images based on hierarchical feature attention networks includes the following steps:

[0008] S1: Collect and preprocess the detection area image data of the three thermal infrared bands of SDGSAT-1, and establish a three-channel infrared ship target detection dataset with pixel-level annotation.

[0009] S2: Construct a network model based on a hierarchical feature attention mechanism. Use this network model for ship target recognition in thermal infrared images. The recognition process of this network model consists of four stages: input stage, feature extraction stage, prediction stage, and localization stage, specifically including:

[0010] S21: In the input stage, the obtained three-channel infrared ship target dataset is subjected to multi-level detail enhancement processing to generate feature maps of different scales;

[0011] S22: In the feature extraction stage, multi-level long-distance information is extracted from the feature maps at multiple scales, and the long-distance information is fused into the feature maps at different scales to generate feature maps at different scales containing long-distance information.

[0012] S23: In the prediction stage, feature maps of different scales containing long-distance information are fused to generate a probability map of the predicted target.

[0013] S24: During the positioning phase, the generated predicted target probability map is used to locate the ship target using an eight-connected aggregation algorithm.

[0014] Preferably, for step S1, the detection area image data of the three thermal infrared bands of SDGSAT-1 are acquired and preprocessed to establish a pixel-level annotated three-channel infrared ship target detection dataset, including the following steps:

[0015] S11: Simultaneously acquire thermal infrared image data of three thermal infrared bands from SDGSAT-1, combine the three thermal infrared bands to form a three-channel thermal infrared image dataset.

[0016] S12: The acquired three-channel thermal infrared image dataset is calibrated by radiometric calibration to generate a three-channel thermal infrared image dataset with amplitude and brightness values.

[0017] S13: Use the SAM model to annotate the ship targets in the thermal infrared image dataset to obtain a pixel-level annotated three-channel infrared ship target detection dataset. Each image in the dataset is a three-channel image composed of three thermal infrared images.

[0018] Preferably, in step S21, during the input stage, the obtained three-channel infrared ship target dataset undergoes multi-level detail enhancement processing to generate feature maps at different scales, specifically as follows:

[0019] S211: Construct a multi-level detail enhancement pattern, with each level employing a single-input, dual-output processing structure;

[0020] S212: In each different level, the first output path does not process the input image and directly sends it to the next level; the second output path first obtains images of different sizes by downsampling, and then extracts features from the input images of different sizes by using different numbers of residual blocks to obtain local features of different scales.

[0021] S213: Fuse local features of different scales into images of different sizes, and generate feature maps of different scales at different levels.

[0022] Preferably, the second output path adopts a dual-branch structure, and each branch structure includes a different number of residual blocks for extracting semantic information and detail information of the image respectively.

[0023] Preferably, in step S22, during the feature extraction stage, a multi-level encoder-large kernel attention mechanism-decoder structure is adopted to extract multi-level long-range information from multi-scale feature maps, and then the long-range information is fused into feature maps of different scales to generate feature maps of different scales containing long-range information, specifically:

[0024] S221: A multi-level encoder encodes feature maps at different scales to generate multi-level features;

[0025] S222: The multi-level features output by the multi-level encoder are fused with the feature maps of different scales obtained in the input stage according to the corresponding levels to obtain a new multi-scale feature map;

[0026] S223: Extracts multi-level long-distance information from multi-scale feature maps through a multi-level large-kernel attention mechanism;

[0027] S224: Multi-level long-distance information is aggregated into feature maps of different scales through feature concatenation operations, and then sent to multi-level decoders to decode and restore feature maps of different scales. These feature maps contain long-distance information.

[0028] Preferably, for step S223, the large kernel attention mechanism at each level includes the following processing operations:

[0029] S2231: Reconstructing multi-scale feature maps using convolutional layers with strides equal to kernel size unifies the dimensionality of the feature maps. The calculation expression is:

[0030] V i =Conv(E i ;s=ks=RF i )

[0031] Among them, V i E represents the feature map at the i-th scale of the reconstructed output; i Let RF represent the i-th scale feature map of the input. i represents the reduction factor of the i-th level feature; different reduction factors are used for feature maps of different scales; Conv represents standard convolution; s and ks represent stride and kernel size, respectively.

[0032] S2232: The reconstructed multi-scale feature maps first undergo batch standardization, then enter the large kernel attention mechanism, where pointwise convolution, large kernel convolution, and then pointwise convolution are performed sequentially to extract multi-level long-distance information. The large kernel convolution is decomposed into three parts, and the large kernel convolution LKA calculation expression is:

[0033]

[0034] Where I represents the input of a large kernel convolution; DW represents a depthwise convolution with a kernel size of (2d-1)×(2d-1); DDW represents a depthwise convolution with a kernel size of... The depthwise convolution; PW represents a pointwise convolution with a kernel size of 1×1; d is the convolution dilation rate; K is a preset conventional parameter;

[0035] S2233: Multi-level long-distance information is transmitted to a multilayer perceptron for information enhancement. The multilayer perceptron includes 1x1 point convolution, 3x3 depth convolution, Gaussian error linear unit, and 1x1 point convolution.

[0036] Preferably, in step S23, during the prediction stage, feature maps of different scales containing long-range information are fused to generate a predicted target probability map, specifically as follows:

[0037] S231: Upsample the information of feature maps at different scales to generate feature maps of different scales with uniform dimensions;

[0038] S232: An attention mechanism is used to assign different weights to feature maps of different scales. The weighted feature maps are then summed element-wise to generate the final predicted target probability map. The formula for calculating the weight α is as follows:

[0039] β=Conv(Conv(GAP(D);ks=1);ks=1)

[0040] γ=Conv(Conv(GMP(D);ks=1);ks=1)

[0041] α=γ+β

[0042] Where D represents feature maps of different scales; Conv represents standard convolution; ks=1 indicates that the convolution kernel size is 1x1; GAP represents global average pooling; GMP represents global max pooling; β and γ represent adaptive parameters used for feature fusion.

[0043] Preferably, for step S24, in the positioning stage, the ship target is located using the generated predicted target probability map and an eight-connected aggregation algorithm, specifically as follows:

[0044] S241: Binarize the predicted target probability map using a preset threshold to generate a binary map;

[0045] S242: Use the 8-connected aggregation algorithm to count the number of pixels contained in each ship target in the binary image to complete the ship target localization.

[0046] Preferably, based on the determination of the ship target, the centroid coordinates of the ship target are calculated and determined by a weighted centroid positioning algorithm. The weighted centroid positioning algorithm is as follows: first, based on the located ship target, the horizontal and vertical coordinate information of all the pixels contained therein is obtained; then, the pixel values ​​of all the pixels of the ship target are normalized to obtain the weights; finally, the centroid of the ship target is calculated based on the weights and the horizontal and vertical coordinate information.

[0047] Preferably, the constructed network model based on hierarchical feature attention mechanism is evaluated using a deep supervision strategy. By adding a loss function to each layer of the network model, the network model can learn discriminative features at different scales, aggregate the loss values ​​of the final outputs of different layers, and then adjust the network model parameters to reduce the loss value, thereby improving the model performance.

[0048] The present invention also provides a system for detecting small ship targets in thermal infrared images based on hierarchical feature attention networks. The system uses the method for detecting small ship targets in thermal infrared images based on hierarchical feature attention networks as described above. The system includes a thermal infrared image data processing module and a ship target recognition module.

[0049] The thermal infrared image data processing module preprocesses the image data of the three thermal infrared bands of SDGSAT-1 and outputs a three-channel infrared ship target detection dataset with pixel-level annotations.

[0050] The ship target recognition module is used to construct a network model based on a hierarchical feature attention mechanism to recognize ship targets in thermal infrared images. The ship target recognition module includes a multi-level detail enhancement module, a multi-level large kernel attention module, a multi-level feature fusion module, and a ship target localization module.

[0051] The multi-level detail enhancement module is applied in the input stage to perform multi-level detail enhancement processing on the input three-channel infrared ship target dataset to generate feature maps at different scales.

[0052] The multi-level large kernel attention module is applied in the feature extraction stage. By utilizing the multi-level large kernel attention mechanism, it extracts multi-level long-distance information from the input multi-scale feature map, and then fuses the long-distance information into feature maps of different scales to generate feature maps of different scales containing long-distance information.

[0053] The multi-level feature fusion module is used in the prediction stage to generate a prediction target probability map by fusing input feature maps of different scales that contain long-distance information.

[0054] The ship target positioning module is used in the positioning phase to locate the ship target using an eight-connected aggregation algorithm based on the generated predicted target probability map, and to locate the ship's centroid coordinates using a weighted centroid positioning algorithm.

[0055] Compared with the prior art, the beneficial effects of the present invention are:

[0056] 1) This invention combines SDGSAT-1 thermal infrared image data to construct the first pixel-level annotated three-channel infrared ship target detection image dataset based on SDGSAT-1. Compared to other mainstream datasets, this dataset has the highest imaging swath, smallest target size, largest image size, lowest average target-to-background ratio, most complex observation scenarios, and the most bands, making it more suitable for detecting small ship targets in thermal infrared images. The three thermal infrared bands of this dataset can be displayed simultaneously in the image, providing rich thermal information and strong support for subsequent deep learning models to fully extract thermal information.

[0057] 2) This invention constructs a network model based on a hierarchical feature attention mechanism. In the input stage of the network model, multi-level detail enhancement processing is used to fuse local features at different scales with images of different sizes to generate feature maps at different scales. This enhances the representation of detail information at different scales, effectively compensates for useful information that may be lost during downsampling, and provides richer and more accurate data support for the next step of feature extraction. In the feature extraction stage, a multi-level large kernel attention mechanism is used to extract long-distance information at multiple levels. This multi-level large kernel attention mechanism can capture and utilize global information at multiple levels while reducing computational complexity, making up for the shortcomings of convolutional neural network models that only focus on local features. This allows the model to learn the relationship between the target and the background at different scales, effectively suppressing the interference of complex background environments and the diversity of surrounding targets on ship detection. In the prediction stage, feature maps at different scales are fused to generate the final predicted target probability map, realizing feature fusion and interaction at different scales. This can significantly improve segmentation accuracy and enhance the comprehensive representation ability of detail and global information, effectively improving the model's adaptability to complex scenes. Finally, in the localization stage, the eight-connected aggregation algorithm and the weighted centroid localization algorithm are used to achieve accurate extraction and localization of ship shape. Compared with existing convolutional neural network models, the network model proposed in this invention exhibits superior performance in ship target detection tasks, with key indicators such as detection accuracy (Pd), false alarm rate (Fa), and intersection-over-union ratio (IoU) significantly outperforming existing network models. Attached Figure Description

[0058] Figure 1 This is a flowchart of a method for detecting small ship targets in thermal infrared images based on a hierarchical feature attention network, according to an embodiment of the present invention.

[0059] Figure 2 This is a three-channel image of the SDG-IRSTD dataset in Embodiment 2 of the present invention;

[0060] Figure 3 This is a single-channel image of the NUDT-SIRST-Sea dataset from Embodiment 2 of the present invention;

[0061] Figure 4 This is a bar chart showing the target size distribution of the six datasets in Embodiment 2 of the present invention;

[0062] Figure 5 This is a bar chart showing the target quantity distribution of the six datasets in Embodiment 2 of the present invention;

[0063] Figure 6 This is a bar chart showing the target-to-background ratio distribution of six datasets in Embodiment 2 of the present invention;

[0064] Figure 7 This is an overall structural diagram of a system for detecting small ships in thermal infrared images based on a hierarchical feature attention network, according to an embodiment of the present invention. Detailed Implementation

[0065] The following will refer to the appendices in the embodiments of the present invention. Figure 1 -Appendix Figure 7 The technical solutions in the embodiments of the present invention are clearly and completely described herein. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0066] Example 1

[0067] Combination Figure 1 As shown, this embodiment of the invention provides a method for detecting small ship targets in thermal infrared images based on hierarchical feature attention networks, including the following steps:

[0068] S1: Collect and preprocess the detection area image data of the three thermal infrared bands of SDGSAT-1, and establish a three-channel infrared ship target detection dataset with pixel-level annotation.

[0069] S2: Construct a network model based on a hierarchical feature attention mechanism. Use this network model to identify small targets on ships in thermal infrared images. The identification process of this network model consists of four stages: input stage, feature extraction stage, prediction stage, and localization stage, specifically including:

[0070] S21: In the input stage, the obtained three-channel infrared ship target dataset is subjected to multi-level detail enhancement processing to generate feature maps of different scales;

[0071] S22: In the feature extraction stage, multi-level long-distance information is extracted from the feature maps at multiple scales, and the long-distance information is fused into the feature maps at different scales to generate feature maps at different scales containing long-distance information.

[0072] S23: In the prediction stage, feature maps of different scales containing long-distance information are fused to generate a probability map of the predicted target.

[0073] S24: During the positioning phase, the generated predicted target probability map is used to locate the ship target using an eight-connected aggregation algorithm.

[0074] Example 2

[0075] This embodiment, based on Embodiment 1, provides a detailed description of the implementation method for generating a three-channel infrared ship target detection dataset with pixel-level annotations in step S1.

[0076] In this embodiment, for step S1, the detection area image data of the three thermal infrared bands of SDGSAT-1 are collected and preprocessed to establish a pixel-level annotated three-channel infrared ship target detection dataset, which specifically includes the following steps:

[0077] S11: Simultaneously acquire thermal infrared image data from SDGSAT-1 across three thermal infrared bands, combine the three thermal infrared bands to form a three-channel thermal infrared image dataset, facilitating subsequent dataset production. SDGSAT-1 is my country's Sustainable Development-1 satellite, capable of simultaneously carrying three payloads: thermal infrared, low-light, and multispectral imagers. SDGSAT-1 has three thermal infrared detection bands, operating synchronously. The spatial resolution of the three thermal infrared bands acquired by SDGSAT-1 is 30m, capable of resolving temperature differences as small as 0.2 degrees Celsius over a wide dynamic range, which is helpful for Infrared Small Target Detection (IRSTD) missions.

[0078] S12: The acquired three-channel thermal infrared image dataset is calibrated using radiometric calibration to generate a three-channel thermal infrared image dataset with amplitude and brightness values. Because the thermal infrared image data acquired by the SDGSAT-1 sensor is raw, unprocessed pixel value, it cannot directly reflect any physical meaning (i.e., it is dimensionless). Therefore, radiometric calibration is needed to convert the obtained pixel values ​​into amplitude and brightness values ​​with actual physical meaning. The calculation expression is as follows:

[0079] L = GAIN·DN + BIAS

[0080] Where L represents the radiance value; GAIN and BIAS represent the gain coefficient and bias coefficient of the corresponding band, respectively.

[0081] S13: The SAM model was used to annotate ship targets in the thermal infrared image dataset using a semi-automatic annotation method. The SAM model is a large model tool that supports multiple segmentation types. The SAM model can automatically identify ship targets and generate corresponding masks and bounding boxes through point selection or bounding box selection. However, the SAM model may produce annotation errors when processing some images. Therefore, the problematic data was manually refined to ensure the quality of the dataset, ultimately resulting in a pixel-level annotated three-channel infrared ship target detection dataset. Each image in the dataset is a three-channel image composed of three thermal infrared images.

[0082] The following will further illustrate the implementation method for generating a three-channel infrared ship target detection dataset with pixel-level annotations through specific examples, and compare it with ship target detection datasets obtained by other conventional methods to demonstrate the technical effects of the present invention:

[0083] The acquired thermal infrared imagery data comes from the thermal infrared (TIS) band images of SDGSAT-1. SDGSAT-1, also known as Sustainable Development-1 satellite, is my country's first scientific satellite launched to meet the data coverage requirements for sustainable development applications. SDGSAT-1 carries three payloads: thermal infrared, low-light, and multispectral imager, enabling all-weather, multi-payload collaborative observation. SDGSAT-1 has a revisit period of approximately 11 days, a spatial resolution of 30m in the TIS band, an imaging swath width of up to 300km, and three detection bands. Its specific parameters are shown in Table 1.

[0084] Table 1

[0085] band wavelength Gain bias TIS-1 8.0-10.5μm 0.003947 0.167126 TIS-2 10.3-11.3μm 0.003946 0.124622 TIS-3 11.5-12.5μm 0.005329 0.222530

[0086] Of the three bands in the table, TIS-1 (8.0–10.5 μm) is primarily used for monitoring surface temperature and heat distribution. This spectral range is highly sensitive to surface radiation and can accurately reflect changes in surface temperature. The TIS-2 (10.3–11.3 μm) spectral range has high penetration capability for water vapor and clouds in the atmosphere, making it suitable for monitoring atmospheric temperature and humidity. The TIS-3 (11.5–12.5 μm) spectral range has high sensitivity to greenhouse gases such as carbon dioxide in the Earth's surface and atmosphere, and can be used to monitor and study greenhouse gas concentrations. The combined application of long-wave infrared bands enables thermal infrared imaging to accurately capture thermal radiation information.

[0087] In the specific implementation process, the three synchronous TIS bands in the acquired thermal infrared image data are combined, and then radiometric calibration and ship target labeling are performed based on the values ​​in Table 1. Finally, a pixel-level annotated three-channel infrared ship target detection image dataset is established, referred to as the SDG-IRSTD dataset. The SDG-IRSTD dataset in this embodiment contains 329 images acquired from SDGSAT-1, containing 3492 ship targets. Each image is 1024x1024 in size, and each image consists of TIS1-TIS3.

[0088] The basic parameters of the SDG-IRSTD dataset are compared with those of four existing publicly available datasets (NUST-SIRST, IRSTD-1k, NUDT-SIRST, and NUDT-SIRST-Sea). NUST-SIRST (see Wang et al., 2019, Miss detection vs. false alarm: adversarial learning for small objects definition in infrared images) contains 428 real and synthetic infrared images with resolutions between 256x256 and 512x512 pixels; IRSTD-1k (see Zhang et al., 2022, ISNet: shapematters for infrared small target detection) contains 1000 images with a resolution of 512x512 for infrared small target detection; and NUDT-SIRST (see Li et al., 2022, Dense nested attention network for infrared small target) contains 1000 images with a resolution of 512x512. The dataset for detecting small infrared targets (MTU-Net) contains 1327 images with a resolution of 256x256; the dataset for detecting small ships (NUDT-SIRST-Sea, see Wu et al., 2023, MTU-Net: multilevel TransUNet for space-based infrared tiny ship detection) contains 17598 infrared images with a resolution of 1024x1024. The comparison results are shown in Table 2.

[0089] Table 2

[0090]

[0091] As shown in Table 2, compared to the other five mainstream datasets, SDG-IRSTD has the highest imaging swath, the smallest target size, the largest image size, and the most bands, making it more suitable for IRSTD tasks. Most other mainstream infrared small target detection datasets are single-channel images, meaning they only contain information from one band.

[0092] Combination Figure 2 and 3 As shown in the comparison of the two images, it can be seen that the SDG-IRSTD three-channel image dataset proposed in this invention allows for simultaneous display of the three thermal infrared bands, resulting in richer thermal information in the images and thus stronger target brightness. Multiple thermal infrared bands help detect the thermal characteristics of small targets such as ships, and the increased number of bands also facilitates ship detection.

[0093] Combination Figure 4 As shown, the target size in the SDG-IRSTD dataset is concentrated between 10 and 30, accounting for 69% of the total targets. The other four mainstream datasets account for 67% (NUDT-SIRST-Sea), 63% (NUDT-SIRST), 65% (SIRST), 50% (NUST-SIRST), and 48% (IRSTD-1k), respectively. Combined with Table 2, it can be seen that the average target size in the SDG-IRSTD dataset is 23, the smallest average size. The average target size of the other four mainstream datasets is approximately 1.3 to 2.2 times that of the SDG-IRSTD dataset. Based on the above analysis, the targets in the SDG-IRSTD dataset are generally smaller than those in other datasets, which is more conducive to the model's detection of small space-based ship targets.

[0094] Combination Figure 5 As shown, compared to other datasets, the SDG-IRSTD dataset exhibits a more uniform distribution of ships in its images. The number of targets in other datasets ranges from 0 to 105, with almost no instances of densely packed or multiple targets. In contrast, the SDG-IRSTD dataset shows more scenes with densely packed ships and a more uniform target distribution, which is crucial for model learning.

[0095] Combination Figure 6As shown, approximately 92% of the targets in the SDG-IRSTD dataset have a target-to-background ratio between 0 and 0.005%, with an average target-to-background ratio of 0.02654%. Compared to other mainstream datasets, the target sizes in the SDG-IRSTD dataset are relatively much smaller, while the overall image size is larger, indicating that the SDG-IRSTD dataset has the lowest average target-to-background ratio. Furthermore, the SDG-IRSTD dataset has a higher background ratio and more complex environmental information, covering a wider range of application scenarios. The SDG-IRSTD dataset encompasses various scenarios, such as offshore scenes, nearshore scenes, scenes with dense cloud interference, and scenes with weak targets. This is because water has a high specific heat capacity; as heat is absorbed, the sea surface appears warm-toned in thermal infrared images, resulting in a smaller difference in radiation between the ship and the sea surface, thus causing the target to appear relatively weak. This lowest average target-to-background ratio and complex environmental information undoubtedly increase the difficulty of ship detection, posing a greater challenge to accurate ship detection. However, under such complex dataset conditions, the ship target detection method in this application accurately identifies ship targets, which indirectly proves that the method has better performance than other methods.

[0096] Therefore, this invention establishes the first pixel-level annotated three-channel infrared ship target detection image dataset based on SDGSAT-1 by combining SDGSAT-1 thermal infrared image data. Compared with other mainstream datasets, SDG-IRSTD has the highest imaging swath, the smallest target size, the largest image size, and the most bands, covering a wider range of application scenarios and making it more suitable for infrared small target ship detection tasks. The three thermal infrared bands can not only be displayed synchronously in the image but also provide sufficient thermal information, which helps subsequent deep learning models to fully extract thermal information and perform deep learning.

[0097] Example 3

[0098] Based on Example 1, this embodiment takes the construction of a 5-layer feature attention mechanism network model as an example. For step S2, it describes in detail the implementation method of constructing a network model based on the hierarchical feature attention mechanism and using the network model to identify small targets of ships in thermal infrared images.

[0099] In this embodiment, for step S21, during the input stage, the obtained three-channel infrared ship target dataset undergoes multi-level detail enhancement processing to generate feature maps at different scales, specifically as follows:

[0100] S211: Construct a multi-level detail enhancement pattern, with each level employing a single-input, dual-output processing structure;

[0101] S212: In each different level, the first output path does not process the input image and directly sends it to the next level; the second output path first obtains images of different sizes by downsampling, and then extracts features from the input images of different sizes by using different numbers of residual blocks to obtain local features of different scales.

[0102] S213: Fuse local features of different scales into images of different sizes, and generate feature maps of different scales at different levels.

[0103] Specifically, the second output path adopts a dual-branch structure, with each branch structure including a different number of residual blocks. The two branches are used to extract semantic information and detail information of the image, respectively, to enrich the features of the corresponding level.

[0104] Specifically: given the input image X∈R C×H×W Where C, H, and W represent the number of channels, height, and width of the input image, respectively. The input image of the i-th level can be represented as... At each level, the input images at different scales are processed by extracting semantic and detail information of different levels from the residual blocks of varying numbers. These are then fused element-wise into the images of different sizes to generate feature maps of different sizes, where the feature map at the i-th scale is denoted as D. i The generated multi-scale feature maps are used for the next step of feature extraction.

[0105] A residual block is a neural network structure that uses skip connections to directly pass input to deeper layers, enabling the network to learn residuals more easily. These skip connections allow gradients to be directly propagated back to shallower layers, effectively mitigating the vanishing gradient problem in deep networks. Furthermore, residual blocks are used here to extract multi-scale features and enhance the preservation of detailed information.

[0106] This design enhances the detail information of images, which is crucial for subsequent target segmentation. In previous IRSTD tasks, the traditional approach was to directly feed the input image into the backbone network to extract multi-level features. This method resulted in insufficient utilization of the input image information. To reduce computation and extract high-level features, the backbone network often used pooling layers multiple times to sample the feature map. While this reduced the feature map size, it inevitably led to the loss of image information that is crucial for segmentation. To address this issue, this invention performs multi-level detail enhancement processing on the input image dataset, generating feature maps at different scales. A dual-branch structure is used to supplement low-frequency and high-frequency information at different scales. This approach fully mines and utilizes image information and effectively compensates for useful information that may be lost during downsampling, providing richer and more accurate data support for subsequent feature extraction.

[0107] In this embodiment, for step S22, during the feature extraction stage, a multi-level encoder-large kernel attention mechanism-decoder structure is adopted to extract multi-level long-range information from multi-scale feature maps, and then the long-range information is fused into feature maps of different scales to generate feature maps of different scales containing long-range information. The main steps include:

[0108] S221: The multi-level encoder encodes feature maps at different scales to generate multi-level features, enriching the feature representation. These multi-level features include high-resolution low-level features and low-resolution high-level features, enabling high-precision image segmentation.

[0109] S222: The multi-level features output by the multi-level encoder are fused with the feature maps of different scales obtained in the input stage according to their hierarchical correspondence to obtain a new multi-scale feature map. Specifically, given an input image represented as X∈R C×H×W The fused i-th scale feature map is represented as Where i = 1, 2, 3, 4, 5.

[0110] S223: A multi-level large-kernel attention mechanism extracts long-distance information from multi-scale feature maps. Each level of the large-kernel attention mechanism mainly includes batch normalization (BN), a large-kernel attention module, and a multilayer perceptron (MLP). The large-kernel attention mechanism utilizes convolutions with large kernels to extract long-distance information. Long-distance information refers to the dependencies between points that are far apart in the feature map. Each level of the large-kernel attention mechanism includes the following processing operations:

[0111] S2231: Multi-scale feature maps are reconstructed using convolutional layers with strides equal to the kernel size to unify the dimensionality of the feature maps, facilitating subsequent fusion. In this embodiment, the specific formula is as follows:

[0112] V i =Conv(E i ;s=ks=RF i ), i = 1, 2, 3, 4, 5

[0113] Among them, V i Represents the feature map at the i-th scale after reconstruction; RF i represents the reduction factor of the i-th level feature; different reduction factors are used for feature maps of different scales; Conv represents standard convolution; s and ks represent stride and kernel size, respectively.

[0114] S2232: The reconstructed multi-scale feature maps first undergo batch standardization, then enter the large kernel attention mechanism, where pointwise convolution, large kernel convolution, and then pointwise convolution are performed sequentially to extract multi-level long-distance information. In the large kernel attention mechanism, the large kernel convolutional attention models long-distance relationships, and the multi-level long-distance information output by the attention mechanism is represented as T. i Its calculation formula is expressed as:

[0115] T i =PW(LKA(GELU(PW(BN(V) i )))))+V i i = 1, 2, 3, 4, 5

[0116] Where PW represents point-wise convolution; LKA represents large kernel convolution; BN represents batch normalization; and GELU represents Gaussian Error Linear Unit. In this embodiment, the large kernel convolution is decomposed into three parts, and the specific calculation formula for the large kernel convolution LKA is expressed as follows:

[0117]

[0118] Where I represents the input of a large kernel convolution; DW represents a depthwise convolution with a kernel size of (2d-1)×(2d-1); DDW represents a depthwise convolution with a kernel size of... The depthwise convolution; PW represents a pointwise convolution with a kernel size of 1×1; d is the convolution dilation rate; K is a preset conventional parameter.

[0119] S2233: The multi-level long-range information is fed into a multilayer perceptron for information enhancement. In this embodiment, the multilayer perceptron (MLP) includes 1x1 point convolutions, 3x3 depthwise convolutions, Gaussian error linear units, and 1x1 point convolutions. The multi-level long-range information Oi enhanced by the multilayer perceptron is specifically expressed by the following formula:

[0120] O i =BN(MLP(T) i ))+T i

[0121] Among them, O i To represent multi-level long-range information after information enhancement by a multilayer perceptron; BN represents batch normalization; MLP represents the multilayer perceptron information enhancement operation; Ti represents the input multi-level long-range information.

[0122] S224: Multi-level long-distance information is aggregated into feature maps of different scales through feature concatenation operations, and then sent to multi-level decoders to decode and restore feature maps of different scales. These feature maps contain long-distance information.

[0123] This invention combines a large kernel attention mechanism with multi-scale features. Feature maps of different scales are processed through a multi-level large kernel attention mechanism to extract multi-level long-distance information. The multi-level large kernel attention mechanism can capture and utilize global information at multiple levels while reducing computational complexity, and can better utilize local and long-distance information to improve the segmentation accuracy of the model.

[0124] In this embodiment, for step S23, during the prediction stage, feature maps of different scales containing long-range information are fused to generate a predicted target probability map. The specific steps are as follows:

[0125] S231: To maintain dimensionality consistency before fusion, information is upsampled from feature maps of different scales to generate feature maps of different scales with uniform dimensionality.

[0126] S232: An attention mechanism is used to assign different weights to feature maps of different scales. The weighted feature maps are then summed element-wise to generate the final predicted target probability map. In practice, the specific formula is as follows:

[0127]

[0128] in, D represents the feature map at the i-th scale after upsampling; i Represents the feature map at the i-th scale; Upsample represents the upsampling operation; sf represents the upsampling rate; RF i α represents the reduction factor of the i-th level feature; P represents the output predicted target probability map; i The weights represent the feature map corresponding to the i-th scale; the sigmoid is the activation function that maps the input data to between 0 and 1, thus obtaining the probability that each pixel is the predicted target.

[0129] Weight α i Generated through a self-attention mechanism, α is used in the specific implementation process. i It can be generated through a convolutional layer with a 1x1 kernel. The formula for calculating the weight α is:

[0130] β=Conv(Conv(GAP(D);ks=1);ks=1)

[0131] γ=Conv(Conv(GMP(D);ks=1);ks=1)

[0132] α=γ+β

[0133] Where D represents feature maps of different scales; Conv represents standard convolution; ks=1 indicates that the convolution kernel size is 1x1; GAP represents global average pooling; GMP represents global max pooling; β and γ represent adaptive parameters used for feature fusion.

[0134] Compared to traditional methods that rely solely on the final output of the decoder as the final probability likelihood map, this invention fuses feature maps of different scales output by the decoder to generate the final predicted target probability map. This achieves feature fusion and interaction at different scales, which can significantly improve segmentation accuracy and enhance the comprehensive representation of detailed and global information, effectively improving the model's adaptability to complex scenes.

[0135] In this embodiment, for step S24, during the positioning stage, the generated predicted target probability map is used to locate the ship target using an eight-connected aggregation algorithm. Specifically:

[0136] S241: The predicted target probability map is binarized using a preset threshold to generate a binary map. In the specific implementation process, a suitable threshold is first preset. Each pixel in the predicted target probability map is traversed. If the probability value of a pixel is greater than or equal to the threshold, it is assigned a value of 1 (representing the foreground, i.e., possibly a ship); if it is less than the threshold, it is assigned a value of 0 (representing the background), thereby converting the probability map into a binary map containing only 0 and 1 values.

[0137] S242: Use the 8-connected aggregation algorithm to count the number of pixels contained in each ship target in the binary image, thus completing the ship target localization. The specific formula is as follows:

[0138] N8(pixel1)∩N8(pixel2)≠φ

[0139] Here, N8(pixel1) and N8(pixel2) represent the sets of eight neighbors of pixel1 and pixel2, respectively. Specifically, if two pixels intersect within their eight neighbors, pixels1 and pixel2 can be considered adjacent pixels; if adjacent pixels have the same pixel value, pixels1 and pixel2 can be considered pixels of the same target. After obtaining the pixels corresponding to the target, the location of the ship target can be completed.

[0140] Furthermore, unlike the traditional method of obtaining the centroid by averaging the coordinates of each pixel of the target, this invention further calculates the centroid coordinates of the ship target using a weighted centroid localization algorithm. The weighted centroid localization algorithm specifically involves: first, obtaining the horizontal and vertical coordinates of all pixels within the located ship target; then, normalizing the pixel values ​​of all pixels to obtain weights; and finally, calculating the centroid of the ship target based on the weights and the horizontal and vertical coordinate information. The calculation formula is as follows:

[0141] c = (w T x,w T y)

[0142]

[0143] Where x and y represent the horizontal and vertical coordinates of the pixels contained in the ship target, respectively, v represents the pixel value of the pixel in the ship target, w represents the weight of the pixel, and c represents the centroid of the ship target.

[0144] After HFA-Net generates the predicted target probability map, a threshold is used to binarize the probability map, and an 8-connected aggregation algorithm is used to count the pixels contained in each target. If two pixels intersect within their eight neighborhoods, then these two pixels can be considered adjacent points. If the pixel values ​​of pixels are equal, then these two pixels can be considered pixels of the same target. After obtaining the pixels corresponding to the target, the target can be located, and the centroid coordinates of the target can be calculated using a weighted centroid localization algorithm.

[0145] In this embodiment, the hierarchical feature attention network model of the present invention employs a deep supervision strategy to evaluate the model. A loss function is added to each layer of the network model. By utilizing deep supervision, the network model can learn discriminative features at different scales. By measuring the similarity between the model's predicted target probability map and the ground truth, the segmentation head is allowed to consider label information at each level. In specific implementation, the overall loss function is expressed as follows:

[0146]

[0147] Where L represents the final total loss value; L i This represents the output loss value of the i-th level decoder.

[0148] This paper selects focal loss (L) F Dice loss L D The loss function for each decoder stage is calculated using the following formula:

[0149] L F = -α(1-p)γ log(p)

[0150]

[0151] L i =L F +L D

[0152] Where GT represents the ground truth map; P represents the probability map of the predicted target; α and γ represent the hyperparameters used to control the weights; ε represents the smoothing factor to avoid oscillations during training; and p represents the detection probability corresponding to any pixel.

[0153] This invention uses a deep supervision strategy to aggregate the loss values ​​of the final outputs of different network layers, and then uses backpropagation to pass the loss values ​​to each layer of the model, guiding the model to adjust parameters to reduce the loss, making the detection results closer to the true values, improving model performance, enhancing robustness, and making ship detection more accurate.

[0154] Example 4

[0155] Combination Figure 7 As shown, this embodiment of the invention also provides a system for detecting small ship targets in thermal infrared images based on hierarchical feature attention networks. This system uses the method for detecting small ship targets in thermal infrared images based on hierarchical feature attention networks described in Embodiment 3. The system includes a thermal infrared image data processing module and a ship target recognition module.

[0156] The thermal infrared image data processing module preprocesses the image data of the three thermal infrared bands of SDGSAT-1 and outputs a three-channel infrared ship target detection dataset with pixel-level annotations.

[0157] The ship target recognition module is used to construct a network model based on a hierarchical feature attention mechanism to identify small ship targets in thermal infrared images. The ship target recognition module includes a multi-level detail enhancement module (MLDEM), a multi-level large kernel attention module (MLLKAM), a multi-level feature fusion module (MLFFM), and a ship target localization module.

[0158] The Multi-Level Detail Enhancement Module (MLDEM) is applied in the input stage to perform multi-level detail enhancement processing on the input three-channel infrared ship target dataset to generate feature maps at different scales.

[0159] The Multilevel Large Kernel Attention Module (MLLKAM) is used in the feature extraction stage. By utilizing a multi-level large kernel attention mechanism, it extracts multi-level long-distance information from the input multi-scale feature map, and then fuses the long-distance information into feature maps of different scales to generate feature maps of different scales containing long-distance information.

[0160] The Multi-Level Feature Fusion Module (MLFFM) is used in the prediction stage to generate a probability map of the predicted target by fusing input feature maps of different scales that contain long-range information.

[0161] The ship target positioning module is used in the positioning phase to locate the ship target using an eight-connected aggregation algorithm based on the generated predicted target probability map, and to locate the ship's centroid coordinates using a weighted centroid positioning algorithm.

[0162] In this embodiment, the multi-level detail enhancement module consists of multiple detail enhancement modules (DEMs) connected in parallel and introduced into the input stage of the network model. Each detail enhancement module is a single-input, dual-output structure. One output path does not process the image but instead feeds it into the next-level detail enhancement module. The other output path uses a dual-branch structure to supplement local features at different scales to enhance the model's detail information. Each branch structure includes a different number of residual blocks, used to extract semantic and detail information from the image respectively to enrich the features of the corresponding stage.

[0163] In this embodiment, the multi-level large kernel attention module is a symmetrical, multi-level encoder-large kernel attention mechanism-decoder. In the multi-level large kernel attention module, the multi-level features output from the multi-level encoder are first fused with feature maps of different scales obtained in the input stage according to their corresponding levels, resulting in new multi-scale feature maps. Then, the multi-scale feature maps enter the multi-level large kernel attention module to extract multi-level long-range information. The multi-level large kernel attention module consists of multiple large kernel units (LKABlocks) connected in parallel. Feature maps of different scales are respectively entered into multiple large kernel units (LKABlocks) to extract multi-level long-range information, comprehensively capturing global information at multiple levels. Each large kernel unit (LKABlock) includes a batch normalization (BN) mechanism, a large kernel attention mechanism (Attention module), and a multilayer perceptron (MLP). In the large kernel unit, the multi-scale feature maps are first reconstructed, then long-range information is extracted through the large kernel attention mechanism, and finally, information enhancement is performed through the multilayer perceptron.

[0164] Finally, the multi-level long-distance information is aggregated into feature maps of different scales through feature concatenation operations, and then enters a multi-level decoder to decode and restore the feature maps of different scales. At this time, the feature maps contain long-distance information.

[0165] In this embodiment, the multi-level feature fusion module upsamples the input feature maps at different scales to generate feature maps of different scales with uniform dimensions; then, it uses an attention mechanism to assign different weights to the feature maps at different scales, and performs element-wise addition of the weighted feature maps to generate the final prediction target probability map.

[0166] In this embodiment, the ship target localization module uses the generated predicted target probability map and employs an eight-connected aggregation algorithm to locate the ship target. Based on this, the weighted centroid localization algorithm is used to locate the ship's centroid coordinates.

[0167] The detection method based on hierarchical feature attention mechanism network model will be compared with the detection method of existing publicly available convolutional neural network model. The detection results of each method will be compared and analyzed. The specific results are shown in Table 3.

[0168] Table 3

[0169] method <![CDATA[IoU(×10 -2 )]]> <![CDATA[P d (×10 -2 )]]> <![CDATA[F a (×10 -6 )]]> ACM 46.50 88.97 26.91 ALC-Net 46.09 76.21 46.34 ResU-Net 50.42 84.31 13.10 DNANet 59.07 76.72 8.71 ISTDU-Net 56.07 89.66 36.19 MTU-Net 63.26 90.34 10.54 HFA-Net 65.61 94.31 7.25

[0170] Among them, ACM (see Dai et al., 2021, Asymmetric contextual modulation for infrared small target detection.) mainly uses an asymmetric contextual modulation mechanism to achieve small target recognition; ALC-Net (see Dai et al., 2021, Attentional local contrast networks for infrared small target detection.) mainly uses attention mechanisms and local contrast enhancement to achieve small target recognition; ResU-Net (see Diakogiannis et al., 2020, ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data.) mainly uses a deep learning model combining U-Net and Residual Network (ResNet) to achieve small target recognition; DNANet (see Li et al., 2022, Dense nested attention network for infrared small target detection.) mainly uses a densely nested attention mechanism to achieve small target recognition; ISTDU-Net (see Hou et al., 2022, ISTDU-net: infrared small-target detection) U-Net primarily utilizes U-Net and infrared image characteristics to detect and locate small infrared targets; MTU-Net (see Wu et al., 2023, MTU-Net: multilevel TransUNet for space-based infrared tiny ship detection.) mainly achieves ship target recognition through a multi-level self-attention U-shaped network; HFA-Net (Hierarchy feature attention network) is the network model proposed in this application based on a hierarchical feature attention mechanism. Through a hierarchical feature attention mechanism, it effectively captures and utilizes multi-scale features and contextual information in the image to complete the detection task. IoU is the intersection-over-union ratio, representing the degree of overlap between the predicted bounding box and the ground truth bounding box; Pd is the detection probability, indicating the probability of correctly detecting the target; Fa is the false alarm rate, representing the probability of incorrectly detecting a target.

[0171] As can be observed from the quantitative experimental results in Table 3, the detection performance of the HFA-Net model of this invention is significantly improved compared with other methods. The IoU and P of HFA-Net are also shown. d F a The percentages were 65.61%, 94.31%, and 7.25 × 10⁻⁶, respectively. -6 From a suboptimal comparison perspective, the HFA-Net model, compared to MTU-Net, performs better in terms of IoU and P. d The above increased by 2.35% and 3.97% respectively, and F a It decreased by 3.29 × 10 -6 The detection performance has been significantly improved. This is because the HFA-Net model fully extracts and utilizes multi-level features. In the input stage of the network model, the Multi-Level Detail Enhancement Module (MLDEM) fuses local features at different scales with images of different sizes output from the corresponding stages of the backbone network, generating feature maps at different scales and enhancing the representation of detail information at different scales. In the feature extraction stage, the Multi-Level Large Kernel Attention Module (MLLKAM) extracts multi-level long-distance information, enabling the model to learn the relationship between the target and the background at different scales. In the final prediction stage, this invention uses the Multi-Level Feature Fusion Module (MLFFM) to learn the impact of different scales on the probability likelihood map.

[0172] Furthermore, to further demonstrate and analyze the impact of various modules introduced in the HFA-Net model on ship segmentation performance, this invention conducted experiments on the SDG-IRSTD dataset using different parameter configurations, and analyzed the impact of each module (MLDEM, MLLKAM, MLFFM) on accuracy based on the results of different network configurations. Specific results are shown in Table 4.

[0173] Table 4

[0174] method <![CDATA[IoU(×10 -2 )]]> <![CDATA[P d (×10 -2 )]]> <![CDATA[F a (×10 -6 )]]> MLDEM-free 64.16 92.06 7.63 No MLLKAM 63.98 93.97 11.77 No MLFFM 64.51 92.93 10.05 HFA-Net 65.61 94.31 7.25

[0175] As shown in Table 4, removing any module reduces the overall performance of the HFA-Net model to varying degrees. Removing MLLKAM has the greatest impact on the IoU metric, causing a 1.63% decrease in the HFA-Net model's IoU. Other metrics also show varying degrees of decline, with P... d Decreased by 0.34%, F a Increased by 4.52 × 10 -6This is because MLLKAM can extract and effectively utilize multi-level long-range information from remote sensing images, compensating for the shortcomings of convolutional neural network models that only focus on local features. It effectively suppresses background interference for ship detection, fully demonstrating the importance of multi-scale long-range information for infrared ship detection. Removing other modules also affects the performance of HFA-Net to varying degrees, further proving the reliability of MLDEM, MLLKAM, and MLFFM in the IRSTD task.

[0176] The experiments above demonstrate that this invention, combining all bands of SDGSAT-1TIS, establishes a pixel-level annotated three-channel infrared ship target detection dataset (SDG-IRSTD). Compared with other mainstream datasets, SDG-IRSTD exhibits the highest imaging swath, smallest target size, largest image size, smallest average target-to-background ratio, most complex observation scenarios, and the largest number of bands, making it more suitable for detecting small ship targets in thermal infrared imagery. This dataset can fully extract thermal information from remote sensing images, providing support for deep learning networks. Furthermore, the proposed HFA-Net generates feature maps at different scales using MLDEM, then employs MLLKAM to extract long-range information from multi-level features, and finally uses MLFFM to achieve feature fusion and interaction at different scales. Compared to the MTU-Net method, HFA-Net achieves better IoU and P... d The above increased by 2.35% and 3.97% respectively, and F a It decreased by 3.29 × 10 -6 The experimental results verified the superiority of the method of the present invention. The method can simultaneously obtain the overall shape profile of the ship while achieving precise target positioning, providing a reliable method for sustainable development goals and subsequent research in the field of marine management.

[0177] The above description is merely an embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the scope of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting small ship targets in thermal infrared images based on hierarchical feature attention networks, characterized in that, Includes the following steps: S1: Collect and preprocess the detection area image data of the three thermal infrared bands of SDGSAT-1, and establish a three-channel infrared ship target detection dataset with pixel-level annotation. S2: Construct a network model based on a hierarchical feature attention mechanism. Use this network model for ship target recognition in thermal infrared images. The recognition process of this network model consists of four stages: input stage, feature extraction stage, prediction stage, and localization stage, specifically including: S21: In the input stage, the obtained three-channel infrared ship target dataset is subjected to multi-level detail enhancement processing to generate feature maps of different scales; S22: In the feature extraction stage, multi-level long-distance information is extracted from the feature maps at multiple scales, and the long-distance information is fused into the feature maps at different scales to generate feature maps at different scales containing long-distance information. S23: In the prediction stage, feature maps of different scales containing long-distance information are fused to generate a probability map of the predicted target. S24: During the positioning phase, the generated predicted target probability map is used to locate the ship target using an eight-connected aggregation algorithm; For step S21, in the input stage, the obtained three-channel infrared ship target dataset undergoes multi-level detail enhancement processing to generate feature maps at different scales, specifically: S211: Construct a multi-level detail enhancement pattern, with each level employing a single-input, dual-output processing structure; S212: In each different level, the first output path does not process the input image and directly sends it to the next level; the second output path first obtains images of different sizes by downsampling, and then extracts features from the input images of different sizes by using different numbers of residual blocks to obtain local features of different scales. S213: Fuse local features at different scales into images of different sizes, and generate feature maps of different scales at different levels; The second output path adopts a dual-branch structure, with each branch structure including a different number of residual blocks, which are used to extract semantic information and detail information of the image respectively.

2. The method for detecting small ship targets in thermal infrared images based on hierarchical feature attention networks according to claim 1, characterized in that, For step S1, image data of the detection area in the three thermal infrared bands of SDGSAT-1 are collected and preprocessed to establish a pixel-level annotated three-channel infrared ship target detection dataset, including the following steps: S11: Simultaneously acquire thermal infrared image data of three thermal infrared bands from SDGSAT-1, combine the three thermal infrared bands to form a three-channel thermal infrared image dataset. S12: The acquired three-channel thermal infrared image dataset is calibrated by radiometric calibration to generate a three-channel thermal infrared image dataset with amplitude and brightness values. S13: Use the SAM model to annotate the ship targets in the thermal infrared image dataset to obtain a pixel-level annotated three-channel infrared ship target detection dataset. Each image in the dataset is a three-channel image composed of three thermal infrared images.

3. The method for detecting small ship targets in thermal infrared images based on hierarchical feature attention networks according to claim 1, characterized in that, For step S22, in the feature extraction stage, a multi-level encoder-large kernel attention mechanism-decoder structure is adopted to extract multi-level long-range information from multi-scale feature maps, and then the long-range information is fused into feature maps of different scales to generate feature maps of different scales containing long-range information, specifically: S221: A multi-level encoder encodes feature maps at different scales to generate multi-level features; S222: The multi-level features output by the multi-level encoder are fused with the feature maps of different scales obtained in the input stage according to the corresponding levels to obtain a new multi-scale feature map; S223: Extracts multi-level long-distance information from multi-scale feature maps through a multi-level large-kernel attention mechanism; S224: Multi-level long-distance information is aggregated into feature maps of different scales through feature concatenation operations, and then sent to multi-level decoders to decode and restore feature maps of different scales. These feature maps contain long-distance information.

4. The method for detecting small ship targets in thermal infrared images based on hierarchical feature attention networks according to claim 3, characterized in that, For step S223, each level of the big-kernel attention mechanism includes the following processing operations: S2231: Reconstructing multi-scale feature maps using convolutional layers with strides equal to kernel size unifies the dimensionality of the feature maps. The calculation expression is: in, This represents the feature map at the i-th scale of the reconstructed output. This represents the i-th scale feature map of the input. This represents the reduction factor for the i-th level feature; different reduction factors are used for feature maps of different scales. This represents a standard convolution; s and ks represent the stride and kernel size, respectively. S2232: The reconstructed multi-scale feature maps first undergo batch standardization, then enter the large kernel attention mechanism, where pointwise convolution, large kernel convolution, and then pointwise convolution are performed sequentially to extract multi-level long-distance information. The large kernel convolution is decomposed into three parts, and the large kernel convolution LKA calculation expression is: Where I represents the input of a large kernel convolution; DW represents a depthwise convolution with a kernel size of (2d-1)×(2d-1); DDW represents a depthwise convolution with a kernel size of... The depthwise convolution; PW represents a pointwise convolution with a kernel size of 1×1; d is the convolution dilation rate; K is a preset conventional parameter; S2233: Multi-level long-distance information is transmitted to a multilayer perceptron for information enhancement. The multilayer perceptron includes 1x1 point convolution, 3x3 depth convolution, Gaussian error linear unit, and 1x1 point convolution.

5. The method for detecting small ship targets in thermal infrared images based on hierarchical feature attention networks according to claim 1, characterized in that, For step S23, in the prediction stage, feature maps of different scales containing long-range information are fused to generate a predicted target probability map, specifically as follows: S231: Upsample the information of feature maps at different scales to generate feature maps of different scales with uniform dimensions; S232: An attention mechanism is used to assign different weights to feature maps of different scales. The weighted feature maps are then summed element-wise to generate the final predicted target probability map. The calculation formula is: Where D represents feature maps of different scales; Conv represents standard convolution; ks=1 indicates that the kernel size is 1x1; GAP represents global average pooling; GMP represents global max pooling; , This represents the adaptive parameters used for feature fusion.

6. The method for detecting small ship targets in thermal infrared images based on hierarchical feature attention networks according to claim 1, characterized in that, For step S24, in the positioning phase, the generated predicted target probability map is used to locate the ship target using an eight-connected aggregation algorithm, specifically as follows: S241: Binarize the predicted target probability map using a preset threshold to generate a binary map; S242: Use the 8-connected aggregation algorithm to count the number of pixels contained in each ship target in the binary image to complete the ship target localization.

7. The method for detecting small ship targets in thermal infrared images based on hierarchical feature attention networks according to claim 1, characterized in that, Based on the determination of the ship target, the weighted centroid positioning algorithm is used to calculate and determine the centroid coordinates of the ship target. The weighted centroid positioning algorithm is as follows: First, based on the located ship target, obtain the horizontal and vertical coordinate information of all the pixels contained in it; then, normalize the pixel values ​​of all the pixels of the ship target to obtain the weights; finally, calculate the centroid of the ship target based on the weights and the horizontal and vertical coordinate information.

8. A system for detecting small ship targets in thermal infrared images based on hierarchical feature attention networks, characterized in that, The method for detecting small ship targets in thermal infrared images based on hierarchical feature attention network according to any one of claims 1-7 includes a thermal infrared image data processing module and a ship target recognition module. The thermal infrared image data processing module preprocesses the image data of the three thermal infrared bands of SDGSAT-1 and outputs a three-channel infrared ship target detection dataset with pixel-level annotations. The ship target recognition module is used to construct a network model based on a hierarchical feature attention mechanism to recognize ship targets in thermal infrared images. The ship target recognition module includes a multi-level detail enhancement module, a multi-level large kernel attention module, a multi-level feature fusion module, and a ship target localization module. The multi-level detail enhancement module is applied in the input stage to perform multi-level detail enhancement processing on the input three-channel infrared ship target dataset to generate feature maps at different scales. The multi-level large kernel attention module is applied in the feature extraction stage. By utilizing the multi-level large kernel attention mechanism, it extracts multi-level long-distance information from the input multi-scale feature map, and then fuses the long-distance information into feature maps of different scales to generate feature maps of different scales containing long-distance information. The multi-level feature fusion module is used in the prediction stage to generate a prediction target probability map by fusing input feature maps of different scales that contain long-distance information. The ship target positioning module is used in the positioning phase to locate the ship target using an eight-connected aggregation algorithm based on the generated predicted target probability map, and to locate the ship's centroid coordinates using a weighted centroid positioning algorithm.

Images