Infrared iceberg target detection method based on global feature and multi-resolution semantic feature interaction network

Abstract

The invention discloses an infrared iceberg target detection method based on a global feature and multi-resolution semantic feature interaction network, and relates to the technical field of target detection. The objective of the invention is to solve the problem of low ice mountain target detection accuracy in the prior art. The method comprises the following steps: acquiring an infrared iceberg image, and acquiring a training set by using the infrared iceberg image; the training set comprises an infrared iceberg image and a corresponding label; the label is an iceberg target contour in the infrared iceberg image; and training a GMSINet network by using the training set, and taking the trained GMSINet network as an iceberg target detection model. And obtaining a to-be-detected infrared iceberg image, and inputting the to-be-detected infrared iceberg image into the iceberg target detection model to obtain the position of the sea surface iceberg target. The method is used for iceberg target detection.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to an infrared iceberg target detection method based on a global feature and a multi-resolution semantic feature interactive network. Background Art

[0002] In recent years, with global climate change and the attention paid by various countries to polar resources, activities in polar waters have become more frequent. The growth and calving of icebergs have also become increasingly active. They move randomly on the water surface, posing a serious threat to navigation safety. Therefore, timely sensing of sea ice status and detection of sea ice targets can help ships make timely risk avoidance decisions and ensure navigation safety, which is of great significance for polar operations, cargo navigation, and autonomous ships crossing iceberg areas.

[0003] At present, ice navigation mainly uses remote sensing to obtain ice maps and evaluate the safety of distant routes through expert analysis, including synthetic aperture radar (SAR), airborne radar, scatterometer and other technologies. Although these methods provide high-resolution sea ice information and provide important support for the macro-planning of ship routes, these methods require the establishment of reliable communications, and the quality of remote sensing imaging is significantly affected by clouds and extreme weather, which will lead to low accuracy in iceberg target detection. In the increasingly frequent navigation activities, the need for low-cost and high-real-time decision-making makes shipborne detection systems an effective means of floating ice detection. X-band navigation radar is a common shipborne detection system, but the information obtained by navigation radar is limited. Ice and other targets are similar in radar images, and it is difficult to distinguish the target type, which will lead to low accuracy in iceberg target detection. At present, the floating ice targets in ship driving mainly rely on experienced crew members to visually judge, and the crew's visual inspection will have subjective influences, resulting in inaccurate and inefficient iceberg target detection. Therefore, it is necessary to automatically and accurately detect floating ice in the channel and get rid of the dependence on experienced crew members.

[0004] Shipboard cameras can provide rich sea ice image information. In recent years, some sea ice detection and segmentation methods based on shipboard visible optical equipment have been proposed. They use machine learning methods to automatically segment sea ice images, thereby realizing sea ice detection and classification. However, low visibility scenes such as fog, rain and snow, which are common in the Arctic, will interfere with vision-based iceberg detection methods, making it difficult to detect iceberg targets in low visibility weather. Infrared target detection technology is widely used in military, industrial, transportation and other fields due to its significant advantages. Infrared sensors have strong adaptability to extreme weather and can avoid the problem of ice and water confusion when optical cameras detect remote ice sources. It is an effective alternative method for detecting irregularly moving floating ice on routes. The earlier the danger is discovered, the faster the avoidance decision can be made. Therefore, the distance between floating ice and the sensor is usually very long. In addition, the size and shape of sea ice vary, occupying a very small proportion in the infrared image, and sea ice targets are easily disturbed by sea wave clutter. Compared with other weak targets, sea ice has lower contrast and signal-to-clutter ratio with the background, resulting in low accuracy of iceberg target detection. The above problems have brought great challenges to infrared floating ice detection. At present, there are some methods that separate targets and backgrounds based on image characteristics, such as contrast measurement, grayscale, gradient and intensity measurement, and low-rank sparse decomposition methods. Traditional algorithms have good performance in scenes with uniform backgrounds and simple clutter, but iceberg detection scenes are often complex and changeable, and it is difficult to keep consistent with the background conditions preset by traditional methods, resulting in significantly reduced detection performance. Some deep learning-based methods design related small target feature highlighting modules to maintain small target information through feature interaction and attention module methods. However, it is not sufficient to detect small targets by simply obtaining target contrast information. Therefore, some methods add detailed features to the network by designing related edge information enhancement modules. Although the current methods have supplemented the target features to a certain extent, the brightness of the ice target and the background is extremely close, which makes it difficult for the existing methods to distinguish between the ice target and the background, resulting in low accuracy in iceberg target detection. Summary of the invention

[0005] The purpose of the present invention is to solve the problem of low accuracy of existing iceberg target detection, and propose an infrared iceberg target detection method based on an interactive network of global features and multi-resolution semantic features.

[0006] The infrared iceberg target detection method based on the interactive network of global features and multi-resolution semantic features is as follows:

[0007] Acquire an infrared iceberg image to be detected, input the infrared iceberg image to be detected into an iceberg target detection model, and obtain the position of the iceberg target on the sea surface;

[0008] The iceberg target detection model is obtained by:

[0009] Step 1: Obtain infrared iceberg images and use the infrared iceberg images to obtain training sets;

[0010] The training set includes: infrared iceberg images and corresponding labels; the labels are the outlines of iceberg targets in the infrared iceberg images;

[0011] Step 2: Use the training set to train the GMSINet network, and use the trained GMSINet network as the iceberg target detection model.

[0012] Furthermore, the GMSINet network includes: a dual-branch encoder module, a multi-resolution feature interaction module MFI and a feature fusion decoder module;

[0013] The dual-branch encoder module is used to extract the global feature set of infrared iceberg images and local feature set The global and local features of the infrared iceberg image are and Input to the feature fusion decoder module, the local features of the infrared iceberg image Send to the multi-resolution feature interaction module MFI;

[0014] The dual-branch encoder module includes: a global branch encoder submodule and a local branch encoder submodule;

[0015] The global branch encoder submodule is used to obtain the global feature set of the infrared iceberg image And the global features of the infrared iceberg image are input into the feature fusion decoder module;

[0016] The local branch encoding submodule is used to obtain the local feature set of the infrared iceberg image. The local features of the infrared iceberg image and Input to the feature fusion decoder module, the local features of the infrared iceberg image Send to the multi-resolution feature interaction module MFI;

[0017] The multi-resolution feature interaction module MFI is used to analyze the local features of infrared icebergs. Perform feature enhancement to obtain the enhanced local feature set Will Send to the feature fusion decoder module;

[0018] The feature fusion decoder module uses the global feature set Local features and Enhanced local feature set Get infrared iceberg target detection results.

[0019] Furthermore, the global branch encoder submodule is a swim-transformer network;

[0020] The input of the global branch encoder submodule is the infrared iceberg image, and the output is the global feature

[0021] Further, the local branch encoding submodule includes: a first stage unit, a second stage unit, a third stage unit, a fourth stage unit, a fifth stage unit and a sixth stage unit;

[0022] The input of the first stage unit is the infrared iceberg image, and the output is the local feature

[0023] Among them, H i1 It is a local feature Height, W i1 It is a local feature Width, C i1 It is a local feature The number of channels;

[0024] The first stage unit includes: a first convolution layer, a second convolution layer, a first summation layer, a third convolution layer, a fourth convolution layer, a second summation layer, a fifth convolution layer, a sixth convolution layer, and a third summation layer;

[0025] All convolutional layers of the first stage units are 3*3 convolutional layers;

[0026] The input of the first convolution layer is the infrared iceberg image; the output of the first convolution layer is the input of the second convolution layer; the input of the first summation layer is the output of the second convolution layer and the infrared iceberg image; the input of the third convolution layer is the output of the first summation layer, and the input of the fourth convolution layer is the output of the third convolution layer; the input of the second summation layer is the output of the first summation layer and the output of the fourth convolution layer; the input of the fifth convolution layer is the output of the second summation layer; the output of the fifth convolution layer is the input of the sixth convolution layer; the input of the third summation layer is the output of the sixth convolution layer and the output of the second summation layer;

[0027] The structure of the stage 2 unit, the stage 3 unit, the stage 4 unit, the stage 5 unit, the stage 6 unit and the stage 1 unit are the same;

[0028] The input of the second stage unit is the local feature Output is local features

[0029] Among them, H i2 It is a local feature Height, W i2It is a local feature Width, C i2 It is a local feature The number of channels;

[0030] The input of the third stage unit is the local feature Output is local features

[0031] Among them, H i3 It is a local feature Height, W i3 It is a local feature Width, C i3 It is a local feature The number of channels;

[0032] The input of the fourth stage unit is the local feature Output is local features

[0033] Among them, H i4 It is a local feature Height, W i4 It is a local feature Width, C i4 It is a local feature The number of channels;

[0034] The input of the fifth stage unit is the local feature Output is local features

[0035] Among them, H i5 It is a local feature Height, W i5 It is a local feature Width, C i5 It is a local feature The number of channels;

[0036] The input of the sixth stage unit is the local feature Output is local features

[0037] Among them, H i6 It is a local feature Height, W i6 It is a local feature Width, C i6 It is a local feature The number of channels.

[0038] Furthermore, the multi-resolution feature interaction module MFI includes: a first convolution unit, a linear embedding unit, a channel attention unit, a first de-redundancy unit, a spatial attention unit, a second de-redundancy unit, and a multi-resolution feature interaction output unit;

[0039] The first convolution unit includes: a 7th convolution layer, an 8th convolution layer, a 9th convolution layer and a 10th convolution layer;

[0040] All convolutional layers in the first convolutional unit are 3*3 convolutional layers;

[0041] The input of the 7th convolutional layer is the local feature And input the output of the 7th convolutional layer into the linear embedding unit;

[0042] The input of the 8th convolutional layer is the local feature And input the output of the 8th convolutional layer into the linear embedding unit;

[0043] The input of the 9th convolutional layer is the local feature And input the output of the 9th convolutional layer into the linear embedding unit;

[0044] The input of the 10th convolutional layer is the local feature And input the output of the 10th convolutional layer into the linear embedding unit;

[0045] The linear embedding unit is used to obtain local features The corresponding plate sequence T i' , specifically:

[0046]

[0047] Among them, Conv is a 3*3 convolution operation. is the i'th local feature of the input MFI, Aver is the average pooling operation, rearrange is the reshape operation, Conv 1×1 It is a 1*1 convolution operation, T i' It is a local feature The corresponding plate sequence, i' is the number of the local feature of the input MFI, which takes 2, 3, 4 or 5;

[0048] The average pooling operation divides each feature map output by the first convolution unit into n p f ×p f Non-overlapping plates;

[0049]

[0050] Among them, P is a constant, f is the feature map number output by the first convolution unit, and p fis the size of the plate segmented by the f-th feature map output by the first convolutional unit;

[0051] The input of the channel attention unit is The corresponding plate sequence T i' , the output is CA(T i' );

[0052] The first de-redundancy unit is used to i' ) performs redundancy removal processing to obtain the output of the first redundancy removal unit

[0053] The input of the spatial attention unit is The output is SA(T i CCA );

[0054] The second redundancy removal unit is used to i CCA ) performs redundancy removal processing to obtain the output of the second redundancy removal unit

[0055]

[0056] The multi-resolution feature interactive output unit utilizes Get enhanced local features

[0057] Furthermore, the first de-redundancy unit is used to i' ) performs redundancy removal processing to obtain the output of the first redundancy removal unit Specifically:

[0058]

[0059] in, It is CA(T i' ) is the output after being processed by the first redundancy removal unit, ⊙ is the point-by-point multiplication operation in the channel dimension, It is a point-by-point addition operation, and Sigmoid() is the Sigmoid function.

[0060] Furthermore, the second redundancy removal unit is used to SA(T i CCA ) performs redundancy removal processing to obtain the output of the second redundancy removal unit Specifically:

[0061]

[0062] in, yes The output after being processed by the second redundancy removal unit, It is a point-by-point multiplication operation of the spatial dimensions.

[0063] Furthermore, the multi-resolution feature interactive output unit utilizes Get enhanced local features Specifically:

[0064]

[0065] in, It is a local feature The enhanced local features obtained after enhancement, up() is the upsampling operation, and LN() is the batch normalization operation.

[0066] Furthermore, the feature fusion decoder module utilizes the global feature set Local features and Enhanced local feature set F S ={F S 2 ,F S 3 ,F S 4 ,F S 5}Get the infrared iceberg target detection results, specifically:

[0067] D1. Using global feature sets Local features and Enhanced local feature set Get feature fusion results:

[0068]

[0069] Among them, Fea i is the i-th feature fusion result, i∈[1,6], i is the label of the feature fusion result, and BN() is the batch normalization operation;

[0070] D2. Use the feature fusion result Fea1 to obtain the infrared iceberg target detection result Output:

[0071] Output=Sigmoid(Conv(ReLU(Conv(Fea1))))

[0072] Among them, ReLU() is the ReLU activation function, and Conv is a 3*3 convolution.

[0073] Furthermore, the loss function of the GMSINet network is:

[0074]

[0075] Output i =Sigmoid(Conv(Fea i )),i=1,2,3,4,5,6

[0076] Among them, loss mix is the overall loss function of the GMSINet network, loss softIoU (Output i ,GT i ) is the Soft-IoU loss function, m and n are the horizontal and vertical coordinates of the pixel, Output i The pixel value at the (m,n)th point, GT i It is the label map of the infrared iceberg image corresponding to the label downsampled to the preset resolution. It's GT i The pixel value at the (m,n)th point.

[0077] The beneficial effects of the present invention are:

[0078] The present invention proposes an infrared iceberg target detection method (GMSINet) based on a global feature and multi-resolution semantic feature interaction network. The present invention uses a dual-branch encoder to obtain complementary features. The designed global branch encoder uses the advantages of the Swim transformer structure in global information capture and long-range dependency establishment to separately obtain global features in iceberg infrared images and restore features submerged by low contrast. The multi-resolution feature interaction module proposed in the present invention uses the correlation within and between feature maps to increase the interaction of information between feature maps at different resolutions. At the same time, the de-redundancy unit further improves the transmission of useful information, filters out redundant information, and effectively suppresses the interference of clutter in complex backgrounds, so that iceberg targets and backgrounds can be distinguished more accurately, and the accuracy of iceberg target detection is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 It is a structural diagram of the present invention;

[0080] Figure 2 This is the structure diagram of the multi-resolution feature interaction module;

[0081] Figure 3(a) is a diagram showing the connection structure of the channel cross attention unit and the first redundancy removal unit;

[0082] Figure 3(b) is a connection diagram of the spatial cross attention unit and the second de-redundancy unit;

[0083] Figure 4A visual comparison chart of GMSINet's detection performance on the SFIIT dataset and other methods;

[0084] Figure 5 Visual comparison of GMSINet’s detection performance on the SIRST dataset with other methods. DETAILED DESCRIPTION

[0085] Specific implementation method 1: The specific process of the infrared iceberg target detection method based on the global feature and multi-resolution semantic feature interactive network in this implementation method is as follows:

[0086] Acquire an infrared iceberg image to be detected, input the infrared iceberg image to be detected into an iceberg target detection model, and obtain the position of the iceberg target on the sea surface;

[0087] The iceberg target detection model is obtained by:

[0088] Step 1: Obtain infrared iceberg images and use the infrared iceberg images to obtain training sets;

[0089] The training set includes: infrared iceberg images and corresponding labels; the labels are the outlines of iceberg targets in the infrared iceberg images;

[0090] The size of the infrared iceberg image is 512×512×3;

[0091] Step 2: Use the training set to train the GMSINet network, and use the trained GMSINet network as the iceberg target detection model;

[0092] like Figure 1 As shown, the GMSINet network includes: a dual-branch encoder module, a multi-resolution feature interaction module MFI and a feature fusion decoder module;

[0093] The dual-branch encoder module is used to extract the global feature set of infrared iceberg images and local feature set The global and local features of the infrared iceberg image are and Input to the feature fusion decoder module, the local features of the infrared iceberg image Send to the multi-resolution feature interaction module MFI;

[0094] The dual-branch encoder module includes: a global branch encoder submodule and a local branch encoder submodule;

[0095] The global branch encoder submodule is used to obtain the global feature set of the infrared iceberg image And the global features of the infrared iceberg image are input into the feature fusion decoder module;

[0096] The local branch encoding submodule is used to obtain the local feature set of the infrared iceberg image. The local features of the infrared iceberg image and Input to the decoder module, the local features of the infrared iceberg image Send to the multi-resolution feature interaction module MFI;

[0097] The multi-resolution feature interaction module MFI is used to analyze the local features of infrared icebergs. Perform feature enhancement to obtain the enhanced feature set Will Send to the feature fusion decoder module;

[0098] The feature fusion decoder module uses the global feature set Local features and Enhanced local feature set Get infrared iceberg target detection results.

[0099] Specific implementation method 2: The global branch encoder submodule is used to obtain the global feature set of the infrared iceberg image Specifically:

[0100] A1. Construct the global branch encoder submodule;

[0101] The global branch encoder submodule is a swim-transformer network;

[0102] The swim-transformer network includes: a segmentation unit, a first encoding unit, a second encoding unit, a third encoding unit, and a fourth encoding unit;

[0103] The input of the segmentation unit is an infrared iceberg image, and the size of the input infrared iceberg image is 512×512×3;

[0104] The first encoding unit includes: a first encoding subunit and a second encoding subunit; the output of the first encoding subunit is the input of the second encoding subunit; the first encoding subunit and the second encoding subunit have the same structure;

[0105] The first encoding unit outputs a global feature

[0106] Among them, H l1 =128 is the global feature Height, C l1 =64 is the global feature The number of channels, W l1 =128 is the global feature Width;

[0107] The first encoding subunit includes: a linear embedding layer, a first swim-transformer layer;

[0108] The input of the linear embedding layer is the output of the segmentation unit, the output of the linear embedding layer is the input of the first swim-transformer layer, and the first swim-transforme layer is the output of the current encoding subunit;

[0109] The first swim-transformer layer includes: a first normalization sublayer, a window-based multi-head self-attention (W-MSA) sublayer, a first summation sublayer, a second normalization sublayer, a first fully connected sublayer, a second summation sublayer, a third normalization sublayer, a moving window-based multi-head self-attention (SW-MSA) sublayer, a third summation sublayer, a fourth normalization layer, a second fully connected sublayer, and a fourth summation sublayer;

[0110] The input of the first normalization sublayer is the output of the linear embedding layer; the output of the first normalization sublayer is the input of the W-MSA sublayer, and the input of the first summation sublayer is the output of the W-MSA sublayer and the output of the linear embedding layer; the output of the first summation sublayer is the input of the second normalization sublayer; the output of the second normalization sublayer is the input of the first fully connected sublayer; the input of the second summation sublayer is the output of the first fully connected sublayer and the output of the first summation sublayer; the input of the third normalization sublayer is the output of the second summation sublayer, the input of the SW-MSA sublayer is the output of the third normalization sublayer, and the input of the third summation sublayer is the output of the second summation sublayer and the output of the SW-MSA sublayer; the input of the fourth normalization sublayer is the output of the third summation sublayer; the output of the fourth normalization sublayer is the input of the second fully connected layer, and the input of the fourth summation sublayer is the output of the third summation sublayer and the output of the second fully connected sublayer;

[0111] The second encoding unit includes: a third encoding subunit and a fourth encoding subunit; the third encoding subunit and the fourth encoding subunit have the same structure; the output of the third encoding subunit is the input of the fourth encoding subunit; the second encoding unit outputs the global feature

[0112] Among them, H l2 =64 is the global feature Height, C l2 =128 is the global feature The number of channels, W l2 =64 is the global feature Width;

[0113] The third encoding subunit includes: a first plate merging layer and a second swim-transformer layer;

[0114] The input of the first plate merging layer is the output of the first encoding unit, and the input of the second swim-transformer layer is the output of the first plate merging layer;

[0115] The second swim-transformer layer has the same structure as the first swim-transformer layer;

[0116] The third encoding unit includes: a fifth encoding subunit, a sixth encoding subunit, a seventh encoding subunit, an eighth encoding subunit, a ninth encoding subunit, and a tenth encoding subunit; the output of the fifth encoding subunit is the input of the sixth encoding subunit; the output of the sixth encoding subunit is the input of the seventh encoding subunit; the output of the seventh encoding subunit is the input of the eighth encoding subunit; the output of the eighth encoding subunit is the input of the ninth encoding subunit; the output of the ninth encoding subunit is the input of the tenth encoding subunit; the output of the tenth encoding subunit is the output of the third encoding unit; the third encoding unit outputs the global feature

[0117] Among them, H l3 =32 is the global feature Height, C l3 =256 is the global feature The number of channels, W l3 =32 is the global feature Width;

[0118] The fifth encoding subunit, the sixth encoding subunit, the seventh encoding subunit, the eighth encoding subunit, the ninth encoding subunit and the tenth encoding subunit all have the same structure;

[0119] The fifth encoding subunit includes: a second plate merging layer and a third swim-transformer layer;

[0120] The input of the second block merging layer is the output of the second encoding unit, and the input of the third swim-transformer layer is the output of the second block merging layer;

[0121] The third swim-transformer layer has the same structure as the first swim-transformer layer;

[0122] The fourth encoding unit includes: an eleventh encoding subunit and a twelfth encoding subunit; the input of the eleventh encoding subunit is the output of the third encoding unit, and the output of the eleventh encoding subunit is the input of the twelfth encoding subunit; the fourth encoding unit outputs the global feature

[0123] The eleventh encoding subunit and the twelfth encoding subunit have the same structure;

[0124] Among them, H l4 =16 is the global feature Height, C l4 =512 is the global feature The number of channels, W l4 =16 is the global feature Width;

[0125] The eleventh encoding subunit includes: a third plate merging layer and a fourth swim-transformer layer;

[0126] The input of the third block merging layer is the output of the third encoding layer, and the input of the fourth swim-transformer layer is the output of the third block merging layer;

[0127] The fourth swim-transformer layer has the same structure as the first swim-transformer layer.

[0128] A2. Using the global branch encoding submodule’s output global features Global feature set constituting infrared iceberg images

[0129] In this implementation, the window-based multi-head self-attention (W-MSA) calculates the self-attention within a fixed window, and the moving window-based multi-head self-attention (SW-MSA) realizes the pixel interaction between different windows by moving window calculation, which reduces the computational complexity to a certain extent, while expanding the computational receptive field, enhancing the long-distance information interaction in the image, and helping the model to better obtain global information.

[0130] Specific implementation method three: The local branch encoding submodule is used to obtain the local feature set of the infrared iceberg image Specifically:

[0131] B1, construct local branch encoding submodule;

[0132] The local branch encoding submodule includes: a first stage unit, a second stage unit, a third stage unit, a fourth stage unit, a fifth stage unit, and a sixth stage unit;

[0133] The first stage unit includes: a 3*3 first convolution layer, a 3*3 second convolution layer, a 1st summation layer, a 3*3 third convolution layer, a 3*3 fourth convolution layer, a 2nd summation layer, a 3*3 fifth convolution layer, a 3*3 sixth convolution layer, and a 3rd summation layer;

[0134] The input of the first stage unit is the infrared iceberg image, and the output is the local feature

[0135] Among them, H i1 =512 is a local feature Height, W i1 =512 is a local feature Width, C i1 =16 is a local feature The number of channels;

[0136] The input of the first convolution layer is the infrared iceberg image; the output of the first convolution layer is the input of the second convolution layer; the input of the first summation layer is the output of the second convolution layer and the infrared iceberg image; the input of the third convolution layer is the output of the first summation layer, and the input of the fourth convolution layer is the output of the third convolution layer; the input of the second summation layer is the output of the first summation layer and the output of the fourth convolution layer; the input of the fifth convolution layer is the output of the second summation layer; the output of the fifth convolution layer is the input of the sixth convolution layer; the input of the third summation layer is the output of the sixth convolution layer and the output of the second summation layer;

[0137] The structure of the stage 2 unit, the stage 3 unit, the stage 4 unit, the stage 5 unit, the stage 6 unit and the stage 1 unit are the same;

[0138] The input of the second stage unit is the local feature Output is local features

[0139] Among them, H i2 =256 is a local feature Height, W i2 =256 is a local feature Width, C i2 =32 is a local feature The number of channels;

[0140] The input of the third stage unit is the local feature Output is local features

[0141] Among them, H i3 =128 is a local feature Height, W i3 =128 is a local feature Width, C i3 =64 is a local feature The number of channels;

[0142] The input of the fourth stage unit is the local feature Output is local features

[0143] Among them, H i4 =64 is a local feature Height, W i4 =64 is a local feature Width, C i4 =128 is a local feature The number of channels;

[0144] The input of the fifth stage unit is the local feature Output is local features

[0145] Among them, H i5 =32 is a local feature Height, W i5 =32 is a local feature Width, C i5 =256 is a local feature The number of channels;

[0146] The input of the sixth stage unit is the local feature Output is local features

[0147] Among them, H i6 =16 is a local feature Height, W i6 =16 is a local feature Width, C i6 =512 is a local feature The number of channels;

[0148] A2. Using the local branch encoding submodule’s output local features Local feature set constituting infrared iceberg image

[0149] In this embodiment, the local feature extraction only retains the sub-sampling between each stage, which is conducive to retaining the effective feature information of small-sized targets and enriching the feature representation of the target.

[0150] Specific implementation method 4: The multi-resolution feature interaction module MFI is used to analyze the local features of infrared icebergs Perform feature enhancement to obtain the enhanced local feature set Specifically:

[0151] C1. Figure 2 As shown, a multi-resolution feature interaction module MFI is constructed;

[0152] The multi-resolution feature interaction module MFI includes: a first convolution unit, a linear embedding unit, a channel attention unit, a first de-redundancy unit, a spatial attention unit, a second de-redundancy unit, and a multi-resolution feature interaction output unit;

[0153] The first convolution unit includes: a 3*3 7th convolution layer, a 3*3 8th convolution layer, a 3*3 9th convolution layer, and a 3*3 10th convolution layer;

[0154] The input of the 7th convolutional layer is the local feature The output of the 7th convolutional layer is the input to the linear embedding unit;

[0155] The input of the 8th convolutional layer is the local feature The output of the 8th convolutional layer is the input to the linear embedding unit;

[0156] The input of the 9th convolutional layer is the local feature The output of the 9th convolutional layer is the input of the linear embedding unit;

[0157] The input of the 10th convolutional layer is the local feature The output of the 10th convolutional layer is the input to the linear embedding unit;

[0158] The linear embedding unit includes: an average pooling layer, a format conversion layer, and an 11th convolutional layer;

[0159] The average pooling layer is used to divide each feature map output by the first convolution unit into n p f ×p f Non-overlapping plates;

[0160]

[0161] Among them, P = 8 is a constant, f is the feature map number output by the first convolution unit, and p f is the size of the plate segmented by the f-th feature map output by the first convolutional unit;

[0162] The format conversion layer is used to reshape the segmentation result of each feature map output by the average pooling layer, so as to expand the segmentation result into a 2D plate sequence to obtain The corresponding plate sequence;

[0163] Where i' is the number of the local feature input to MFI, which can be 2, 3, 4 or 5;

[0164] The 11th convolutional layer is a 1*1 convolutional layer, which is used to The corresponding plate sequence is mapped to obtain the mapped The corresponding plate sequence;

[0165] The linear embedding unit has the following specific processing formula:

[0166]

[0167] Among them, Conv is a 3*3 convolution operation. is the i'th local feature of the input MFI, Aver is the average pooling operation, rearrange is the reshape operation, Conv 1×1 It is a 1*1 convolution operation, T i' It is a local feature The corresponding plate sequence;

[0168] The channel attention unit is used to input The corresponding plate sequence T i' , and then the output CA(T i' ) is input to a first redundancy removal unit;

[0169] The first de-redundancy unit includes: a 12th convolution layer, a first activation function layer, a first channel dimension point-by-point multiplication layer, and a 4th summation layer;

[0170] The 12th convolutional layer is a 3*3 convolutional layer, and the input is CA(T i' ), and input the output of the 12th convolutional layer into the first activation function layer;

[0171] The first activation function layer is a sigmoid function, and the output of the first activation function layer is input to the first channel dimension point-by-point multiplication layer;

[0172] The first channel dimension point-by-point multiplication layer is used to multiply the output of the first activation function layer and T i' Multiply in the channel dimension to obtain the first channel product;

[0173] The fourth summation layer is used to multiply the first channel product with T i' Perform point-by-point addition to obtain a fourth summation result; the fourth summation result is the output of the first redundancy removal unit;

[0174] The first redundancy removal unit is specifically:

[0175]

[0176] Among them, CA(T i' ) is T i The output after processing by the channel cross attention unit, It is CA(T i' ) is the output after being processed by the first redundancy removal unit, ⊙ is the point-by-point multiplication operation in the channel dimension, It is a point-by-point addition operation;

[0177] The spatial attention unit is used to input the output of the first de-redundancy unit and to convert the output of the spatial attention unit into input to the second redundancy removal unit;

[0178] The second de-redundancy unit includes: a 13th convolution layer, a second activation function layer, a second channel dimension point-by-point multiplication layer, and a 5th summation layer;

[0179] The 13th convolutional layer is a 3*3 convolutional layer, and the input is And the output of the 13th convolutional layer is input to the second activation function layer;

[0180] The second activation function layer is a sigmoid function, and the output of the second activation function layer is input to the second channel dimension point-by-point multiplication layer;

[0181] The second channel dimension point-by-point multiplication layer is used to sum the output of the second activation function layer Multiply in the channel dimension to obtain the first channel product;

[0182] The fifth summation layer is used to multiply the first channel product with Perform point-by-point addition to obtain the fifth summation result; the fifth summation result is the output of the second redundancy removal unit;

[0183] The second redundancy removal unit is specifically:

[0184]

[0185] in, yes The output after processing by the spatial attention unit, yes The output after being processed by the second redundancy removal unit, It is a point-by-point multiplication operation of the spatial dimension;

[0186] The multi-resolution feature interactive output unit includes: a fifth normalization layer, an upsampling layer, and a sixth summation layer;

[0187] The input of the fifth layer is the output of the first redundancy removal unit. The output of the fifth layer is the input of the upsampling layer;

[0188] The upsampling layer is used to perform an upsampling operation on the output of the fifth layer normalized to the first layer;

[0189] The sixth summation layer is used to And the upsampling layer output is added to obtain Enhanced feature map

[0190] The multi-resolution feature interactive output unit is specifically:

[0191]

[0192] in, It is a local feature The enhanced local features, up(*) is the upsampling operation, and LN() is the layer normalization operation.

[0193] C2. The enhanced local feature maps are combined into an enhanced local feature set

[0194] In this embodiment, the four sequences of the output linear embedding unit are respectively input into the channel cross attention module and the spatial attention cross module. The structures are shown in Figure 3(a) and Figure 3(b). i' When performing calculations, the four input sequences are connected along the channel dimension and also participate in the calculation of some keys, values, and queries. This strategy allows the network to interact with feature maps at other resolutions, while paying attention to the features between the current resolution and other resolutions, realizing the cross-utilization of multi-resolution features, and obtaining the dependencies within and between sequences, which helps to obtain target feature information more comprehensively. The forms of channel cross-attention and spatial cross-attention are:

[0195]

[0196]

[0197] Among them, Q, K, V are the matrices of query, key and value respectively. and are proportional factors, h c =4. To solve the problem of clutter caused by complex background, this implementation mode sets a de-redundancy unit. Finally, the present invention normalizes the output application layer of the de-redundancy unit and upsamples the four outputs to a structure corresponding to the input. At the end of the multi-resolution feature interaction module, we add a jump connection to add the input feature map to the enhanced feature map, highlighting the emphasized features and suppressing irrelevant clutter background information.

[0198] Specific implementation method 5: The feature fusion decoder module uses a global feature set Local features and Enhanced local feature set Get the infrared iceberg target detection results, specifically:

[0199] D1. Obtain feature fusion results:

[0200]

[0201] Among them, Fea i is the i-th feature fusion result, i∈[1,6], i is the label of the feature fusion result, and BN(*) is the batch normalization operation.

[0202] D 2. Use the feature fusion result Fea1 to obtain the infrared iceberg target detection result (iceberg target contour) Output:

[0203] Output=Sigmoid(Conv(ReLU(Conv(Fea1))))

[0204] Among them, ReLU() is the ReLU activation function, and Conv is a 3*3 convolution;

[0205] Specific implementation method 6: The loss function of the GMSINet network is:

[0206]

[0207] Output i =Sigmoid(Conv(Fea i )),i=1,2,3,4,5,6

[0208] Among them, loss mix is the overall loss function of the GMSINet network, loss softIoU (Output i ,GT i ) is the Soft-IoU loss function, m and n are the horizontal and vertical coordinates of the pixel, Output i The pixel value at the (m,n)th point, GT i is a label map with labels downsampled to a preset resolution, It's GT i The pixel value at the (m,n)th point.

[0209] Example: In order to verify the beneficial effects of the present invention, the present invention was verified on a self-constructed data set and a public data set:

[0210] According to Table 1 and Table 2, the present invention has outstanding superiority over other methods in terms of indicators. In the infrared iceberg target dataset SFIIT constructed by ourselves (taking photos of the sea surface with icebergs and cropping the photos to the same size to form the dataset SFIIT), GMSINet has achieved the best results in IoU, nIoU, Pd and F1 indicators, which are much higher than other methods. In the comparative experiment of the public dataset SIRST, the evaluation indicators IoU, nIoU, Prec and F1 of the present invention also rank first among all indicators. Through qualitative analysis, it can be seen that GMSINet has outstanding advantages in both infrared iceberg target detection and infrared small target detection.

[0211] Table 1

[0212]

[0213] Table 2

[0214]

[0215] according to Figure 4 and Figure 5 From the visual comparison, it can be seen that GMSINet can effectively restore some target structures submerged by low-contrast characteristics by focusing on global features, can more completely divide low-contrast target areas, and effectively avoid the interference of background clutter on detection.

[0216] In this embodiment, the F output by the local branch encoder L i It contains six feature maps at different resolutions, among which the first layer feature F L 1 After only one stage of simple processing by ResNet, it has full resolution and contains a lot of detailed information in the image. We directly input it into the decoder without processing. L 6 After the maximum number of convolution operations, the target information is gradually discarded as the network deepens. It contains the global features and contrast information of the entire image, and there is little semantic information of the target. We also do not perform enhancement processing and directly send it to the decoder. We use the Soft-IoU loss function loss softIoU For training, we calculate the loss of each layer output on the six scales to reduce false positive predictions. We introduce a combined loss formula loss mix , the output of each layer is combined to calculate the total loss.

Claims

1. Infrared iceberg target detection method based on global feature and multi-resolution semantic feature interaction network, characterized by The specific process of the method is: Acquire an infrared iceberg image to be detected, input the infrared iceberg image to be detected into an iceberg target detection model, and obtain the position of the iceberg target on the sea surface; The iceberg target detection model is obtained by: Step 1: Obtain infrared iceberg images and use the infrared iceberg images to obtain training sets; The training set includes: infrared iceberg images and corresponding labels; the labels are the outlines of iceberg targets in the infrared iceberg images; Step 2: Use the training set to train the GMSINet network, and use the trained GMSINet network as the iceberg target detection model.

2. The infrared iceberg target detection method based on the global feature and multi-resolution semantic feature interactive network according to claim 1 is characterized in that: The GMSINet network includes: a dual-branch encoder module, a multi-resolution feature interaction module MFI and a feature fusion decoder module; The dual-branch encoder module is used to extract the global feature set of infrared iceberg images and local feature set The global and local features of the infrared iceberg image are and Input to the feature fusion decoder module, the local features of the infrared iceberg image Send to the multi-resolution feature interaction module MFI; The dual-branch encoder module includes: a global branch encoder submodule and a local branch encoder submodule; The global branch encoder submodule is used to obtain the global feature set of the infrared iceberg image And the global features of the infrared iceberg image are input into the feature fusion decoder module; The local branch encoding submodule is used to obtain the local feature set of the infrared iceberg image. The local features of the infrared iceberg image and Input to the feature fusion decoder module, the local features of the infrared iceberg image Send to the multi-resolution feature interaction module MFI; The multi-resolution feature interaction module MFI is used to analyze the local features of infrared icebergs. Perform feature enhancement to obtain the enhanced local feature set Will Send to the feature fusion decoder module; The feature fusion decoder module uses the global feature set Local features and Enhanced local feature set Get infrared iceberg target detection results.

3. The infrared iceberg target detection method based on the global feature and multi-resolution semantic feature interactive network according to claim 2 is characterized in that: The global branch encoder submodule is a swim-transformer network; The input of the global branch encoder submodule is the infrared iceberg image, and the output is the global feature 4. The infrared iceberg target detection method based on the global feature and multi-resolution semantic feature interactive network according to claim 3 is characterized by: The local branch encoding submodule includes: a first stage unit, a second stage unit, a third stage unit, a fourth stage unit, a fifth stage unit and a sixth stage unit; The input of the first stage unit is the infrared iceberg image, and the output is the local feature Among them, H i1 It is a local feature Height, W i1 It is a local feature Width, C i1 It is a local feature The number of channels; The first stage unit includes: a first convolution layer, a second convolution layer, a first summation layer, a third convolution layer, a fourth convolution layer, a second summation layer, a fifth convolution layer, a sixth convolution layer, and a third summation layer; All convolutional layers of the first stage units are 3*3 convolutional layers; The input of the first convolution layer is the infrared iceberg image; the output of the first convolution layer is the input of the second convolution layer; the input of the first summation layer is the output of the second convolution layer and the infrared iceberg image; the input of the third convolution layer is the output of the first summation layer, and the input of the fourth convolution layer is the output of the third convolution layer; the input of the second summation layer is the output of the first summation layer and the output of the fourth convolution layer; the input of the fifth convolution layer is the output of the second summation layer; the output of the fifth convolution layer is the input of the sixth convolution layer; the input of the third summation layer is the output of the sixth convolution layer and the output of the second summation layer; The structure of the stage 2 unit, the stage 3 unit, the stage 4 unit, the stage 5 unit, the stage 6 unit and the stage 1 unit are the same; The input of the second stage unit is the local feature Output is local features Among them, H i2 It is a local feature Height, W i2 It is a local feature Width, C i2 It is a local feature The number of channels; The input of the third stage unit is the local feature Output is local features Among them, H i3 It is a local feature Height, W i3 It is a local feature Width, C i3 It is a local feature The number of channels; The input of the fourth stage unit is the local feature Output is local features Among them, H i4 It is a local feature Height, W i4 It is a local feature Width, C i4 It is a local feature The number of channels; The input of the fifth stage unit is the local feature Output is local features Among them, H i5 It is a local feature Height, W i5 It is a local feature Width, C i5 It is a local feature The number of channels; The input of the sixth stage unit is the local feature Output is local features Among them, H i6 It is a local feature Height, W i6 It is a local feature Width, C i6 It is a local feature The number of channels.

5. The infrared iceberg target detection method based on the global feature and multi-resolution semantic feature interactive network according to claim 4 is characterized in that: The multi-resolution feature interaction module MFI includes: a first convolution unit, a linear embedding unit, a channel attention unit, a first de-redundancy unit, a spatial attention unit, a second de-redundancy unit, and a multi-resolution feature interaction output unit; The first convolution unit includes: a 7th convolution layer, an 8th convolution layer, a 9th convolution layer and a 10th convolution layer; All convolutional layers in the first convolutional unit are 3*3 convolutional layers; The input of the 7th convolutional layer is the local feature And input the output of the 7th convolutional layer into the linear embedding unit; The input of the 8th convolutional layer is the local feature And input the output of the 8th convolutional layer into the linear embedding unit; The input of the 9th convolutional layer is the local feature And input the output of the 9th convolutional layer into the linear embedding unit; The input of the 10th convolutional layer is the local feature And input the output of the 10th convolutional layer into the linear embedding unit; The linear embedding unit is used to obtain local features The corresponding plate sequence T i' , specifically: Among them, Conv is a 3*3 convolution operation. is the i'th local feature of the input MFI, Aver is the average pooling operation, rearrange is the reshape operation, Conv 1×1 It is a 1*1 convolution operation, T i' It is a local feature The corresponding plate sequence, i' is the number of the local feature of the input MFI, which takes 2, 3, 4 or 5; The average pooling operation divides each feature map output by the first convolution unit into n p f ×p f Non-overlapping plates; Among them, P is a constant, f is the feature map number output by the first convolution unit, and p f is the size of the plate segmented by the f-th feature map output by the first convolutional unit; The input of the channel attention unit is The corresponding plate sequence T i' , the output is CA(T i' ); The first de-redundancy unit is used to i' ) performs redundancy removal processing to obtain the output of the first redundancy removal unit The input of the spatial attention unit is The output is SA(T i CCA ); The second redundancy removal unit is used to i CCA ) performs redundancy removal processing to obtain the output of the second redundancy removal unit The multi-resolution feature interactive output unit utilizes Get enhanced local features 6. The infrared iceberg target detection method based on the global feature and multi-resolution semantic feature interactive network according to claim 5 is characterized in that: The first de-redundancy unit is used to i' ) performs redundancy removal processing to obtain the output of the first redundancy removal unit Specifically: in, It is CA(T i' ) is the output after being processed by the first redundancy removal unit, ⊙ is the point-by-point multiplication operation in the channel dimension, It is a point-by-point addition operation, and Sigmoid() is the Sigmoid function.

7. The infrared iceberg target detection method based on the global feature and multi-resolution semantic feature interactive network according to claim 6 is characterized by: The second redundancy removal unit is used to i CCA ) performs redundancy removal processing to obtain the output of the second redundancy removal unit Specifically: in, yes The output after being processed by the second redundancy removal unit, It is a point-by-point multiplication operation of the spatial dimensions.

8. The infrared iceberg target detection method based on the global feature and multi-resolution semantic feature interactive network according to claim 7 is characterized in that: The multi-resolution feature interactive output unit utilizes Get enhanced local features Specifically: in, It is a local feature The enhanced local features obtained after enhancement, up() is the upsampling operation, and LN() is the batch normalization operation.

9. The infrared iceberg target detection method based on the global feature and multi-resolution semantic feature interactive network according to claim 8 is characterized in that: The feature fusion decoder module uses the global feature set Local features and Enhanced local feature set Get the infrared iceberg target detection results, specifically: D1. Using global feature sets Local features and Enhanced local feature set Get feature fusion results: Among them, Fea i is the i-th feature fusion result, i∈[1,6], i is the label of the feature fusion result, and BN() is the batch normalization operation; D2. Use the feature fusion result Fea1 to obtain the infrared iceberg target detection result Output: Output=Sigmoid(Conv(ReLU(Conv(Fea1)))) Among them, ReLU() is the ReLU activation function, and Conv is a 3*3 convolution.

10. The infrared iceberg target detection method based on the global feature and multi-resolution semantic feature interactive network according to claim 9 is characterized in that: The loss function of the GMSINet network is: Output i =Sigmoid(Conv(Fea i )),i=1,2,3,4,5,6 Among them, loss mix is the overall loss function of the GMSINet network, loss softIoU (Output i ,GT i ) is the Soft-IoU loss function, m and n are the horizontal and vertical coordinates of the pixel, Output i The pixel value at the (m,n)th point, GT i It is the label map of the infrared iceberg image corresponding to the label downsampled to the preset resolution, GT imn It's GT i The pixel value at the (m,n)th point.

Images