Infrared iceberg target detection method based on global feature and multi-resolution semantic feature interaction network

Abstract

The application discloses an infrared iceberg target detection method based on a global feature and a multi-resolution semantic feature interaction network and relates to the technical field of target detection.The application is aimed at solving the problem of low accuracy of existing iceberg target detection.The application comprises the following steps: acquiring an infrared iceberg image, and acquiring a training set by using the infrared iceberg image;the training set comprises the infrared iceberg image and corresponding labels;the labels are the outlines of iceberg targets in the infrared iceberg image;training a GMSINet network by using the training set, and taking the trained GMSINet network as an iceberg target detection model;acquiring a to-be-detected infrared iceberg image, inputting the to-be-detected infrared iceberg image into the iceberg target detection model, and obtaining the position of the sea surface iceberg target.The application is used for iceberg target detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and particularly relates to an infrared iceberg target detection method based on a global feature and multi-resolution semantic feature interaction network. BACKGROUND

[0002] In recent years, with global climate change and the attention of countries to polar resources, activities in polar waters are becoming more frequent. Icebergs grow and break up gradually, and they move randomly on the water surface, which seriously threatens the safety of navigation. Therefore, timely sensing the state of sea ice and detecting sea ice targets can help ships make risk avoidance decisions in time and ensure navigation safety, which is of great significance for polar operations, cargo shipping and autonomous ship crossing iceberg areas.

[0003] Currently, ice navigation mainly uses remote sensing means to obtain ice charts, and assesses the safety of distant routes through expert analysis, including synthetic aperture radar (SAR), airborne radar, scatterometer and other technologies. Although these means provide high-resolution sea ice information and provide important support for the macro planning of ship routes, these methods require reliable communication, and the quality of remote sensing imaging is significantly affected by clouds and extreme weather, which can lead to low accuracy of iceberg target detection. In the increasingly frequent navigation activities, the need for low-cost and real-time decision-making makes shipborne detection systems an effective means of ice exploration. X-band navigation radar is a common shipborne detection system, but the information obtained by the navigation radar is limited, and ice and other targets are similar on the radar image, making it difficult to distinguish target types, which can lead to low accuracy of iceberg target detection. At present, the ice targets in ship driving mainly rely on the visual judgment of experienced crew, but the visual judgment of crew can be subjective, leading to inaccurate and inefficient detection of iceberg targets. Therefore, it is necessary to automatically and accurately detect ice in the channel and get rid of the dependence on experienced crew.

[0004] Shipborne cameras can provide rich sea ice image information. In recent years, some sea ice detection and segmentation methods based on shipborne visible light devices have been proposed. They use machine learning methods to automatically divide sea ice images, thereby realizing sea ice detection and classification. However, the low-visibility scenes such as fog, rain and snow in the Arctic region often interfere with visual-based iceberg detection methods, making it difficult to detect iceberg targets in low-visibility weather. Infrared target detection technology has been widely used in military, industry, transportation and other fields due to its obvious advantages. Infrared sensors have strong adaptability to extreme weather and can avoid the problem of ice-water confusion when optical cameras detect remote ice sources. It is an effective alternative method to detect irregularly moving icebergs on the route. The earlier the danger is discovered, the faster the avoidance decision can be made. Therefore, the distance between the icebergs and the sensor is usually very long. In addition, the size and shape of sea ice are different, and they occupy a very small proportion in the infrared image. Moreover, sea ice targets are easily disturbed by sea wave clutter. Compared with other weak targets, sea ice has lower contrast and signal-to-clutter ratio with the background, resulting in low accuracy of iceberg target detection. These problems bring great challenges to infrared ice detection. Some methods are designed to separate targets and backgrounds based on image characteristics, such as contrast measurement, gray level, gradient and intensity measurement, and low-rank sparse decomposition method. Traditional algorithms perform well in scenes with uniform background and simple clutter, but iceberg detection scenes are often complex and variable, making it difficult to maintain consistency with the preset background conditions of traditional methods, resulting in a significant decrease in detection performance. Some deep learning-based methods design related small target feature highlighting modules to maintain small target information through feature interaction and attention modules. However, obtaining only target contrast information to detect small targets is not sufficient, so some methods design related edge information enhancement modules to add detailed features to the network. Although the current methods supplement the target features to some extent, the brightness of the ice target and the background is extremely close, making it difficult for existing methods to distinguish between ice targets and backgrounds, resulting in low accuracy of iceberg target detection. SUMMARY

[0005] The purpose of the present application is to solve the problem of low accuracy of existing iceberg target detection, and an infrared iceberg target detection method based on global feature and multi-resolution semantic feature interaction network is proposed.

[0006] The infrared iceberg target detection method based on global feature and multi-resolution semantic feature interaction network is specifically:

[0007] An infrared iceberg image to be detected is obtained, and the infrared iceberg image to be detected is input into an iceberg target detection model to obtain the position of the sea ice target.

[0008] The iceberg target detection model is obtained by the following method:

[0009] Step 1: Obtain infrared iceberg images and use them to obtain a training set;

[0010] The training set includes: infrared iceberg images and corresponding labels; the labels are the outlines of iceberg targets in the infrared iceberg images.

[0011] Step 2: Train the GMSINet network using the training set, and use the trained GMSINet network as the iceberg target detection model.

[0012] Furthermore, the GMSINet network includes: a dual-branch encoder module, a multi-resolution feature interaction module (MFI), and a feature fusion decoder module;

[0013] The dual-branch encoder module is used to extract the global feature set of infrared iceberg images. and local feature sets And the global and local features of the infrared iceberg image. and The input is fed into the feature fusion decoder module, which converts the local features of the infrared iceberg image into... Send to the Multi-Resolution Feature Interaction Module (MFI);

[0014] The dual-branch encoder module includes: a global branch encoder submodule and a local branch encoder submodule;

[0015] The global branch encoder submodule is used to acquire the global feature set of the infrared iceberg image. The global features of the infrared iceberg image are then input into the feature fusion decoder module.

[0016] The local branch coding submodule is used to obtain the local feature set of the infrared iceberg image. Local features of infrared iceberg images and The input is fed into the feature fusion decoder module, which converts the local features of the infrared iceberg image into... Send to the Multi-Resolution Feature Interaction Module (MFI);

[0017] The Multi-Resolution Feature Interaction (MFI) module is used for local features of infrared icebergs. Perform feature enhancement to obtain the enhanced local feature set. Will Send to the feature fusion decoder module;

[0018] The feature fusion decoder module utilizes a global feature set. Local features and Enhanced local feature set Obtain infrared iceberg target detection results.

[0019] Further, the global branch encoder submodule is a swim-transformer network;

[0020] The input of the global branch encoder submodule is an infrared iceberg image, and the output is global features

[0021] Further, the local branch encoding submodule comprises a first stage unit, a second stage unit, a third stage unit, a fourth stage unit, a fifth stage unit and a sixth stage unit;

[0022] The input of the first stage unit is an infrared iceberg image, and the output is local features

[0023] Wherein, H i1 is the height of the local features , W i1 is the width of the local features , and C i1 is the number of channels of the local features ;

[0024] The first stage unit comprises a first convolutional layer, a second convolutional layer, a first summation layer, a third convolutional layer, a fourth convolutional layer, a second summation layer, a fifth convolutional layer, a sixth convolutional layer and a third summation layer.

[0025] All convolutional layers of the first stage unit are 3*3 convolutional layers.

[0026] The input of the first convolutional layer is the infrared iceberg image; the output of the first convolutional layer is the input of the second convolutional layer; the input of the first summation layer is the output of the second convolutional layer and the infrared iceberg image; the input of the third convolutional layer is the output of the first summation layer, and the input of the fourth convolutional layer is the output of the third convolutional layer; the input of the second summation layer is the output of the first summation layer and the output of the fourth convolutional layer; the input of the fifth convolutional layer is the output of the second summation layer; the output of the fifth convolutional layer is the input of the sixth convolutional layer; and the input of the third summation layer is the output of the sixth convolutional layer and the output of the second summation layer.

[0027] The second stage unit, the third stage unit, the fourth stage unit, the fifth stage unit, the sixth stage unit and the first stage unit have the same structure.

[0028] The input of the second stage unit is the local features , and the output is the local features

[0029] Wherein, H i2 is the height of the local features , W i2Local features width, C i2 Local features The number of channels;

[0030] The input to the third stage unit is local features. The output is local features

[0031] Among them, H i3 Local features Height, W i3 Local features width, C i3 Local features The number of channels;

[0032] The input to the fourth stage unit is local features. The output is local features

[0033] Among them, H i4 Local features Height, W i4 Local features width, C i4 Local features The number of channels;

[0034] The input to the fifth stage unit is local features. The output is local features

[0035] Among them, H i5 Local features Height, W i5 Local features width, C i5 Local features The number of channels;

[0036] The input to the sixth stage unit is local features. The output is local features

[0037] Among them, H i6 Local features Height, W i6 Local features width, C i6 Local features The number of channels.

[0038] Further, the multi-resolution feature interaction module MFI comprises: a first convolution unit, a linear embedding unit, a channel attention unit, a first redundancy removal unit, a spatial attention unit, a second redundancy removal unit, and a multi-resolution feature interaction output unit;

[0039] The first convolution unit comprises a 7th convolution layer, an 8th convolution layer, a 9th convolution layer, and a 10th convolution layer.

[0040] All the convolution layers in the first convolution unit are 3*3 convolution layers.

[0041] The input of the 7th convolution layer is the local feature The output of the 7th convolution layer is input to the linear embedding unit.

[0042] The input of the 8th convolution layer is the local feature The output of the 8th convolution layer is input to the linear embedding unit.

[0043] The input of the 9th convolution layer is the local feature The output of the 9th convolution layer is input to the linear embedding unit.

[0044] The input of the 10th convolution layer is the local feature The output of the 10th convolution layer is input to the linear embedding unit.

[0045] The linear embedding unit is used to obtain the local feature The corresponding patch sequence T i' , specifically:

[0046]

[0047] wherein Conv is a 3*3 convolution operation, is the ith local feature input to the MFI, Aver is an average pooling operation, rearrange is a reshape operation, Conv 1×1 is a 1*1 convolution operation, T i' is the local feature The corresponding patch sequence, i' is the number of the local feature input to the MFI, and is 2, 3, 4, or 5.

[0048] The average pooling operation divides each feature map output by the first convolution unit into n non-overlapping patches of p f ×p f ;

[0049]

[0050] wherein P is a constant, f is the number of the feature map output by the first convolution unit, and p fis the size of the divided patch of the fth feature map output by the first convolution unit;

[0051] The input of the channel attention unit is The corresponding patch sequence T i' , and the output is CA(T i' );

[0052] The first de-redundancy unit is configured to perform de-redundancy processing on CA(T i' ) to obtain the output of the first de-redundancy unit

[0053] The input of the spatial attention unit is The output is SA(T i CCA );

[0054] The second de-redundancy unit is configured to perform de-redundancy processing on SA(T i CCA ) to obtain the output of the second de-redundancy unit

[0055]

[0056] The multi-resolution feature interaction output unit obtains enhanced local features

[0057] Further, the first de-redundancy unit is configured to perform de-redundancy processing on CA(T i' ) to obtain the output of the first de-redundancy unit Specifically,

[0058]

[0059] wherein, is the output of CA(T i' ) after processing by the first de-redundancy unit, is a point-by-point multiplication operation in the channel dimension, is a point-by-point addition operation, and Sigmoid() is a Sigmoid function.

[0060] Further, the second de-redundancy unit is configured to perform de-redundancy processing on SA(T i CCA ) to obtain the output of the second de-redundancy unit Specifically,

[0061]

[0062] wherein, is the output of after processing by the second de-redundancy unit,​ is a point-wise multiplication operation in spatial dimension.

[0063] Further, the multi-resolution feature interaction output unit utilizes to obtain enhanced local features Specifically,

[0064]

[0065] wherein, is a local feature enhanced local features obtained after enhancement, up() is an up-sampling operation, and LN() is a batch normalization operation.

[0066] Further, the feature fusion decoder module utilizes a global feature set local features and enhanced local feature set F S ={F S 2 ,F S 3 ,F S 4 ,F S 5 to obtain an infrared iceberg target detection result, specifically,

[0067] D1, utilizes a global feature set local features and enhanced local feature set to obtain a feature fusion result:

[0068]

[0069] wherein, Fea i is the i-th feature fusion result, i∈[1, 6], i is the label of the feature fusion result, and BN() is a batch normalization operation.

[0070] D2, utilizes the feature fusion result Fea1 to obtain an infrared iceberg target detection result Output:

[0071] Output = Sigmoid(Conv(ReLU(Conv(Fea1)))

[0072] wherein, ReLU() is a ReLU activation function, and Conv is a 3*3 convolution.

[0073] Further, the loss function of the GMSINet network is:

[0074]

[0075] Output i =Sigmoid(Conv(Fea i )),i=1,2,3,4,5,6

[0076] Wherein, loss mix is the overall loss function of the GMSINet network, loss softIoU (Output i ,GT i ) is the Soft-IoU loss function, m, n are the horizontal and vertical coordinates of the pixels respectively, is the pixel value of Output i at the (m, n) point, GT i is the label image corresponding to the infrared iceberg image, which is down-sampled to the preset resolution, is the pixel value of GT i at the (m, n) point.

[0077] The beneficial effects of the present application are:

[0078] The present application proposes an infrared iceberg target detection method (GMSINet) based on a global feature and multi-resolution semantic feature interaction network, which uses a double-branch encoder to obtain complementary features, and a global branch encoder designed by the present application uses the advantages of the Swim transformer structure in global information capture and long-range dependency establishment to obtain global features in the infrared image of the iceberg alone, and restore the features submerged by low contrast. The multi-resolution feature interaction module proposed by the present application utilizes the correlation within and between feature maps, and increases the interaction of information between feature maps at different resolutions. At the same time, the de-redundancy unit further improves the transmission of useful information, filters out redundant information, effectively suppresses the interference of clutter in the complex background, so as to more accurately distinguish the iceberg target and the background, and improves the accuracy of the iceberg target detection. BRIEF DESCRIPTION OF DRAWINGS

[0079] Figure 1 is the structural diagram of the present application;

[0080] Figure 2 is the structural diagram of the multi-resolution feature interaction module;

[0081] Fig. 3(a) is a connection structure diagram of the channel cross-attention unit and the first de-redundancy unit;

[0082] Fig. 3(b) is a connection structure diagram of the spatial cross-attention unit and the second de-redundancy unit;

[0083] Figure 4Fig. 3 shows a visualization comparison chart of detection performance of GMSINet on SFIIT dataset with other methods;

[0084] Figure 5 Fig. 4 shows a visualization comparison chart of detection performance of GMSINet on SIRST dataset with other methods. DETAILED DESCRIPTION

[0085] Embodiment one: the specific process of the infrared iceberg target detection method based on the global feature and multi-resolution semantic feature interaction network is as follows:

[0086] An infrared iceberg image to be detected is obtained, and the infrared iceberg image to be detected is input into an iceberg target detection model to obtain the position of the sea surface iceberg target;

[0087] The iceberg target detection model is obtained by the following method:

[0088] Step one, an infrared iceberg image is obtained, and a training set is obtained by using the infrared iceberg image;

[0089] The training set includes: an infrared iceberg image and a corresponding label; the label is the outline of the iceberg target in the infrared iceberg image;

[0090] The size of the infrared iceberg image is 512*512*3;

[0091] Step two, the GMSINet network is trained by using the training set, and the trained GMSINet network is used as the iceberg target detection model;

[0092] As shown in the figure, the GMSINet network includes: a double-branch encoder module, a multi-resolution feature interaction module MFI, and a feature fusion decoder module; Figure 1 The double-branch encoder module is used to extract a global feature set and a local feature set of the infrared iceberg image, and the global feature and the local feature of the infrared iceberg image are input into the feature fusion decoder module, and the local feature of the infrared iceberg image is sent to the multi-resolution feature interaction module MFI; The double-branch encoder module includes a global branch encoder submodule and a local branch encoder submodule;

[0094] The global branch encoder submodule is used to obtain the global feature set

[0095] of the infrared iceberg image, and the global feature of the infrared iceberg image is input into the feature fusion decoder module; The local branch encoder submodule is used to obtain the local feature set

[0096] of the infrared iceberg image, and the local feature of the infrared iceberg image is sent to the multi-resolution feature interaction module MFI.

[0096] The local branch encoding submodule is configured to obtain a local feature set of the infrared iceberg image The local feature of the infrared iceberg image is input into a decoder module And The local feature of the infrared iceberg image is input into a decoder module The local feature of the infrared iceberg image is sent to a multi-resolution feature interaction module MFI.

[0097] The multi-resolution feature interaction module MFI is configured to perform feature enhancement on the local feature of the infrared iceberg Obtain an enhanced feature set The enhanced feature set is sent to a feature fusion decoder module The enhanced feature set is sent to a feature fusion decoder module

[0098] The feature fusion decoder module utilizes the global feature set The local feature And The enhanced local feature set Obtain an infrared iceberg target detection result.

[0099] Specific implementation method two: the global branch encoder submodule is configured to obtain a global feature set of the infrared iceberg image Specifically:

[0100] A1, a global branch encoder submodule is constructed;

[0101] The global branch encoder submodule is a swim-transformer network;

[0102] The swim-transformer network includes a segmentation unit, a first encoding unit, a second encoding unit, a third encoding unit, and a fourth encoding unit.

[0103] The input of the segmentation unit is an infrared iceberg image, and the size of the input infrared iceberg image is 512x512x3;

[0104] The first encoding unit includes a first encoding subunit and a second encoding subunit; the output of the first encoding subunit is the input of the second encoding subunit; the first encoding subunit and the second encoding subunit have the same structure.

[0105] The first encoding unit outputs global features

[0106] Wherein, H l1 =128 is the height of the global feature , C l1 =64 is the number of channels of the global feature , and W l1 =128 is the width of the global feature The width;

[0107] The first coding subunit includes: a linear embedding layer and a first swim-transformer layer;

[0108] The input of the linear embedding layer is the output of the segmentation unit, and the output of the linear embedding layer is the input of the first swim-transformer layer, which is the output of the current coding subunit.

[0109] The first swim-transformer layer includes: a first normalization sublayer, a window-based multi-head self-attention (W-MSA) sublayer, a first summation sublayer, a second normalization sublayer, a first fully connected sublayer, a second summation sublayer, a third normalization sublayer, a moving window-based multi-head self-attention (SW-MSA) sublayer, a third summation sublayer, a fourth normalization layer, a second fully connected sublayer, and a fourth summation sublayer;

[0110] The input to the first normalized sublayer is the output of the linear embedding layer; the output of the first normalized sublayer is the input of the W-MSA sublayer; the input of the first summation sublayer is the output of the W-MSA sublayer and the output of the linear embedding layer; the output of the first summation sublayer is the input of the second sub-normalized sublayer; the output of the second normalized sublayer is the input of the first fully connected sublayer; the input of the second summation sublayer is the output of the first fully connected sublayer and the output of the first summation sublayer; the input of the third normalized sublayer is the output of the second summation sublayer; the input of the SW-MSA sublayer is the output of the third normalized sublayer; the input of the third summation sublayer is the output of the second summation sublayer and the output of the SW-MSA sublayer; the input of the fourth normalized sublayer is the output of the third summation sublayer; the output of the fourth normalized sublayer is the input of the second fully connected layer; the input of the fourth summation sublayer is the output of the third summation sublayer and the output of the second fully connected sublayer;

[0111] The second coding unit includes a third coding subunit and a fourth coding subunit; the third and fourth coding subunits have the same structure; the output of the third coding subunit is the input of the fourth coding subunit; the second coding unit outputs global features.

[0112] Among them, H l2 =64 is a global feature Height, C l2 =128 is a global feature The number of channels, W l2 =64 is a global feature The width;

[0113] The third encoding subunit includes: a first block merging layer and a second swim-transformer layer;

[0114] The input to the first block merging layer is the output of the first coding unit, and the input to the second swim-transformer layer is the output of the first block merging layer;

[0115] The second swim-transformer layer has the same structure as the first swim-transformer layer;

[0116] The third coding unit includes: a fifth coding subunit, a sixth coding subunit, a seventh coding subunit, an eighth coding subunit, a ninth coding subunit, and a tenth coding subunit; the output of the fifth coding subunit is the input of the sixth coding subunit; the output of the sixth coding subunit is the input of the seventh coding subunit; the output of the seventh coding subunit is the input of the eighth coding subunit; the output of the eighth coding subunit is the input of the ninth coding subunit; the output of the ninth coding subunit is the input of the tenth coding subunit; the output of the tenth coding subunit is the output of the third coding unit; the third coding unit outputs global features.

[0117] Among them, H l3 =32 is a global feature Height, C l3 =256 is a global feature The number of channels, W l3 =32 is a global feature The width;

[0118] The fifth, sixth, seventh, eighth, ninth, and tenth coding subunits all have the same structure;

[0119] The fifth encoding subunit includes: a second module merging layer and a third swim-transformer layer;

[0120] The input to the second block merging layer is the output of the second coding unit, and the input to the third swim-transformer layer is the output of the second block merging layer;

[0121] The third swim-transformer layer has the same structure as the first swim-transformer layer;

[0122] The fourth coding unit includes an eleventh coding subunit and a twelfth coding subunit; the input of the eleventh coding subunit is the output of the third coding unit, and the output of the eleventh coding subunit is the input of the twelfth coding subunit; the fourth coding unit outputs global features.

[0123] The eleventh encoding subunit and the twelfth encoding subunit are the same in structure;

[0124] H l4 = 16 is the height of the global feature C l4 = 512 is the number of channels of the global feature W l4 = 16 is the width of the global feature ;

[0125] The eleventh encoding subunit comprises a third patch merging layer and a fourth swim-transformer layer.

[0126] The input of the third patch merging layer is the output of the third encoding layer, and the input of the fourth swim-transformer layer is the output of the third patch merging layer.

[0127] The fourth swim-transformer layer is the same in structure as the first swim-transformer layer.

[0128] A2, the output global feature of the global branch encoding submodule constitutes a global feature set of the infrared iceberg image

[0129] In the embodiment, the window-based multi-head self-attention (W-MSA) calculates the self-attention within a fixed window, and the window-based multi-head self-attention (SW-MSA) realizes the pixel interaction between different windows in a moving window manner, which to some extent reduces the calculation complexity, expands the calculation receptive field, enhances the long-distance information interaction in the image, and helps the model to better obtain the global information.

[0130] Specific implementation method three: the local branch encoding submodule is used to obtain a local feature set of the infrared iceberg image Specifically,

[0131] B1, a local branch encoding submodule is constructed;

[0132] The local branch encoding submodule comprises a first stage unit, a second stage unit, a third stage unit, a fourth stage unit, a fifth stage unit, and a sixth stage unit.

[0133] The first stage unit comprises a first 3*3 convolutional layer, a second 3*3 convolutional layer, a first summation layer, a third 3*3 convolutional layer, a fourth 3*3 convolutional layer, a second summation layer, a fifth 3*3 convolutional layer, a sixth 3*3 convolutional layer, and a third summation layer.

[0134] The input of the first stage unit is an infrared iceberg image, and the output is a local feature

[0135] wherein, H i1 = 512 is the height of the local feature , W i1 = 512 is the width of the local feature , and C i1 = 16 is the number of channels of the local feature ;

[0136] The input of the first convolutional layer is an infrared iceberg image; the output of the first convolutional layer is the input of the second convolutional layer; the input of the first summing layer is the output of the second convolutional layer and the infrared iceberg image; the input of the third convolutional layer is the output of the first summing layer, and the input of the fourth convolutional layer is the output of the third convolutional layer; the input of the second summing layer is the output of the first summing layer and the output of the fourth convolutional layer; the input of the fifth convolutional layer is the output of the second summing layer; the output of the fifth convolutional layer is the input of the sixth convolutional layer; and the input of the third summing layer is the output of the sixth convolutional layer and the output of the second summing layer

[0137] The second stage unit, the third stage unit, the fourth stage unit, the fifth stage unit, the sixth stage unit and the first stage unit have the same structure

[0138] The input of the second stage unit is a local feature , and the output is a local feature

[0139] wherein, H i2 = 256 is the height of the local feature , W i2 = 256 is the width of the local feature , and C i2 = 32 is the number of channels of the local feature ;

[0140] The input of the third stage unit is a local feature , and the output is a local feature

[0141] wherein, H i3 = 128 is the height of the local feature , W i3 = 128 is the width of the local feature , and C i3 = 64 is the number of channels of the local feature ;

[0142] The input of the fourth stage unit is a local feature The output is local features

[0143] Among them, H i4 =64 is a local feature Height, W i4 =64 is a local feature width, C i4 =128 is a local feature The number of channels;

[0144] The input to the fifth stage unit is local features. The output is local features

[0145] Among them, H i5 =32 is a local feature Height, W i5 =32 is a local feature width, C i5 =256 is a local feature The number of channels;

[0146] The input to the sixth stage unit is local features. The output is local features

[0147] Among them, H i6 =16 is a local feature Height, W i6 =16 is a local feature width, C i6 =512 is a local feature The number of channels;

[0148] A2. Utilizing the local features output by the local branch coding submodule Local feature sets that make up infrared iceberg images

[0149] In this embodiment, local feature extraction only retains the sub-sampling between each stage, which is beneficial for preserving the effective feature information of small-sized targets and enriching the feature representation of the targets.

[0150] Specific Implementation Method Four: The Multi-Resolution Feature Interaction Module (MFI) is used for local features of infrared icebergs. Perform feature enhancement to obtain the enhanced local feature set. Specifically:

[0151] C1, such as Figure 2 As shown, a multi-resolution feature interaction module (MFI) is constructed.

[0152] The Multi-Resolution Feature Interaction Module (MFI) includes: a first convolutional unit, a linear embedding unit, a channel attention unit, a first redundancy removal unit, a spatial attention unit, a second redundancy removal unit, and a multi-resolution feature interaction output unit.

[0153] The first convolutional unit includes: a 3*3 7th convolutional layer, a 3*3 8th convolutional layer, a 3*3 9th convolutional layer, and a 3*3 10th convolutional layer;

[0154] The input to the 7th convolutional layer is local features. The output of the 7th convolutional layer is the input of the linear embedding unit;

[0155] The input to the 8th convolutional layer is local features. The output of the 8th convolutional layer is the input of the linear embedding unit;

[0156] The input to the 9th convolutional layer is local features. The output of the 9th convolutional layer is the input of the linear embedding unit;

[0157] The input to the 10th convolutional layer is local features. The output of the 10th convolutional layer is the input of the linear embedding unit;

[0158] The linear embedding unit includes: an average pooling layer, a format transformation layer, and an 11th convolutional layer;

[0159] The average pooling layer is used to segment each feature map output by the first convolutional unit into n p-values. f ×p f Non-overlapping plates;

[0160]

[0161] Where P = 8 is a constant, f is the feature index number output by the first convolutional unit, and p f It is the size of the segment segmented from the f-th feature map output by the first convolutional unit;

[0162] The format transformation layer is used to reshape the segmentation results of each feature map output by the average pooling layer, thereby unfolding the segmentation results into a 2D plate sequence, obtaining... The corresponding sector sequence;

[0163] Where i' is the number of the local feature of the input MFI, which can be 2, 3, 4 or 5;

[0164] The 11th convolutional layer is a 1*1 convolutional layer used for... Map the corresponding sector sequence to obtain the mapped... The corresponding sector sequence;

[0165] The linear embedding unit specifically processes the formula as follows:

[0166]

[0167] wherein Conv is a 3*3 convolution operation, is the i'th local feature of the input MFI, Aver is an average pooling operation, rearrange is a reshape operation, Conv 1×1 is a 1*1 convolution operation, T i' is a local feature corresponding to the plate sequence;

[0168] The channel attention unit is used for input corresponding to the plate sequence T i' , and then the output CA(T i' ) of the channel attention unit is input to the first redundancy reduction unit;

[0169] The first redundancy reduction unit comprises a 12th convolution layer, a first activation function layer, a first channel dimension point-by-point multiplication layer, and a 4th summation layer.

[0170] The 12th convolution layer is a 3*3 convolution layer, and the input is CA(T i' ), and the output of the 12th convolution layer is input to the first activation function layer.

[0171] The first activation function layer is a sigmoid function, and the output of the first activation function layer is input to the first channel dimension point-by-point multiplication layer.

[0172] The first channel dimension point-by-point multiplication layer is used for multiplying the output of the first activation function layer and T i' in the channel dimension to obtain a first channel product.

[0173] The 4th summation layer is used for point-by-point addition of the first channel product and T i' to obtain a 4th summation result; the 4th summation result is the output of the first redundancy reduction unit.

[0174] The first redundancy reduction unit specifically comprises:

[0175]

[0176] wherein CA(T i' ) is the output of T i processed by the channel cross attention unit, is the output of CA(T i' ) processed by the first redundancy reduction unit, and is a point-by-point multiplication operation in the channel dimension, is a point-wise addition operation;

[0177] The spatial attention unit is used for inputting the output of the first de-redundancy unit, and outputting the output of the spatial attention unit to the second de-redundancy unit;

[0178] The second de-redundancy unit comprises a 13th convolutional layer, a second activation function layer, a second channel dimension point-wise multiplication layer, and a 5th summation layer.

[0179] The 13th convolutional layer is a 3*3 convolutional layer, and the input is and the output of the 13th convolutional layer is input to the second activation function layer.

[0180] The second activation function layer is a sigmoid function, and the output of the second activation function layer is input to the second channel dimension point-wise multiplication layer.

[0181] The second channel dimension point-wise multiplication layer is used for multiplying the output of the second activation function layer and in the channel dimension to obtain a first channel product.

[0182] The 5th summation layer is used for point-wise adding the first channel product and to obtain a 5th summation result; the 5th summation result is the output of the second de-redundancy unit.

[0183] The second de-redundancy unit specifically comprises:

[0184]

[0185] wherein, is the output after the spatial attention unit processing, is the output after the second de-redundancy unit processing, is a point-wise multiplication operation in the spatial dimension. The multi-resolution feature interaction output unit comprises a fifth layer normalization layer, an up-sampling layer, and a 6th summation layer.

[0186] The input of the fifth layer normalization layer is the output of the first de-redundancy unit The output of the fifth layer normalization layer is the input of the up-sampling layer.

[0187] The up-sampling layer is used for performing an up-sampling operation on the output of the fifth layer normalization layer.

[0188] The 6th summation layer is used for performing an addition operation on and the output of the up-sampling layer to obtain

[0189] an enhanced feature map ​​

[0190] The multi-resolution feature interaction output unit is specifically:

[0191]

[0192] Among them, is the local feature enhanced local feature, up(*) is the up-sampling operation, and LN() is the layer normalization operation.

[0193] C2, the enhanced local feature map is grouped into an enhanced local feature set

[0194] In this embodiment, the four sequences output by the linear embedding unit are respectively input into the channel cross attention module and the spatial cross attention module, and the structure is shown in Figures 3(a) and 3(b). When calculating each T i' , the four input sequences are connected along the channel dimension, and also participate in the calculation of part of the keys, values and queries. This strategy can make the network interact with feature maps at other resolutions, while paying attention to the features between the current resolution and other resolutions, achieving cross utilization of multi-resolution features, while obtaining intra-sequence and inter-sequence dependencies, which helps to more comprehensively obtain target feature information. The forms of channel cross attention and spatial cross attention are:

[0195]

[0196]

[0197] Among them, Q, K, and V are query, key, and value matrices, and are scale factors, h c = 4. To solve the problem of clutter caused by complex background, the redundancy removal unit is set in this embodiment. Finally, the output of the redundancy removal unit is applied to the layer normalization, and the four outputs are up-sampled to the structure corresponding to the input. At the end of the multi-resolution feature interaction module, a jump connection is added to add the input feature map to the enhanced feature map, highlighting the emphasized features and suppressing irrelevant clutter background information.

[0198] Specifically, the feature fusion decoder module utilizes the global feature set local feature and enhanced local feature set to obtain an infrared iceberg target detection result, specifically:

[0199] D1, obtain a feature fusion result:

[0200]

[0201] wherein Fea i is the i-th feature fusion result, i [1, 6], i is the label of the feature fusion result, and BN(*) is a batch normalization operation.

[0202] D2, obtain an infrared iceberg target detection result (iceberg target contour) Output using the feature fusion result Fea1:

[0203] Output = Sigmoid(Conv(ReLU(Conv(Fea1)))

[0204] wherein ReLU() is a ReLU activation function, and Conv is a 3*3 convolution;

[0205] Specific implementation method six: the loss function of the GMSINet network is:

[0206]

[0207] Output i = Sigmoid(Conv(Fea i )), i = 1, 2, 3, 4, 5, 6

[0208] wherein loss mix is the overall loss function of the GMSINet network, loss softIoU (Output i , GT i ) is a Soft-IoU loss function, m and n are respectively the horizontal and vertical coordinates of a pixel, is the pixel value of Output i at the (m, n) point, GT i is a label map obtained by downsampling a label to a preset resolution, is the pixel value of GT i at the (m, n) point.

[0209] Embodiment: In order to verify the beneficial effects of the present application, the present application is verified on a self-constructed data set and a public data set:

[0210] According to Table 1 and Table 2, the application has outstanding advantages in indicators compared with other methods. In the infrared iceberg target data set SFIIT we built ourselves (take pictures of the sea surface with icebergs, and crop the pictures to the same size to form the data set SFIIT), GMSINet achieves the optimal results in IoU, nIoU, Pd and F1 indicators, which are much higher than those of other methods. In the comparative experiment of the public data set SIRST, the evaluation indicators IoU, nIoU, Prec and F1 of the application are in the first place in all indicators. Through qualitative analysis, it can be seen that GMSINet has outstanding advantages in infrared iceberg target detection and infrared small target detection.

[0211] Table 1

[0212]

[0213] Table 2

[0214]

[0215] According to Figure 4 and Figure 5 visual comparison, it can be seen that GMSINet can effectively restore the partial target structure submerged by low contrast characteristics by focusing on global features, can more completely divide the low contrast target area, and effectively avoids the interference brought by background clutter for detection.

[0216] In this embodiment, the F L i contains six feature maps at different resolutions, of which the first layer feature F L 1 only undergoes simple processing of one stage of ResNet, has full resolution, and contains a large amount of detailed information in the image, which is directly input into the decoder without processing. The last layer feature map F L 6 undergoes the most number of convolution operations, and target information is gradually discarded as the network deepens, which contains global features and contrast information of the whole image, and the semantic information of the target therein is little, which is also directly sent into the decoder without enhancement processing. We use the Soft-IoU loss function loss softIoU for training, and calculate the loss of each layer output on the six scale outputs to reduce false positive predictions. We introduce a combined loss formula loss mix to calculate the total loss jointly with the output of each layer.

Claims

1. A method for detecting an infrared iceberg target based on a global feature and a multi-resolution semantic feature interaction network, characterized in that The method specifically comprises the following steps: An infrared iceberg image to be detected is acquired, and the infrared iceberg image to be detected is input into an iceberg target detection model to obtain the position of the iceberg target on the sea surface; The iceberg target detection model is obtained by the following method: Step 1: An infrared iceberg image is acquired, and a training set is acquired by using the infrared iceberg image; The training set comprises an infrared iceberg image and a corresponding label; the label is the contour of the iceberg target in the infrared iceberg image; Step 2: The GMSINet network is trained by using the training set, and the trained GMSINet network is used as the iceberg target detection model; The GMSINet network comprises a double-branch encoder module, a multi-resolution feature interaction module MFI and a feature fusion decoder module; The dual-branch encoder module is used to extract the global feature set of infrared iceberg images. and local feature sets And the global and local features of the infrared iceberg image. and The input is fed into the feature fusion decoder module, which converts the local features of the infrared iceberg image into... Send to the Multi-Resolution Feature Interaction Module (MFI); The double-branch encoder module comprises a global branch encoder submodule and a local branch encoder submodule; The global branch encoder submodule is configured to obtain a global feature set of the infrared iceberg image and input the global feature of the infrared iceberg image into the feature fusion decoder module The local branch encoding submodule is used for acquiring a local feature set of the infrared iceberg image The local features of the infrared iceberg image are And Input to the feature fusion decoder module, the local features of the infrared iceberg image are Sent to the multi-resolution feature interaction module MFI; The multi-resolution feature interaction module MFI is used for local features of the infrared iceberg Feature enhancement is performed to obtain an enhanced local feature set Will be sent to the feature fusion decoder module The feature fusion decoder module is used for decoding the enhanced local feature set The feature fusion decoder module utilizes a global feature set Local features And An enhanced local feature set Obtain an infrared iceberg target detection result.

2. The method of claim 1, wherein the method comprises: The global branch encoder submodule is a swim-transformer network; The input of the global branch encoder submodule is an infrared iceberg image, and the output is a global feature 3. The method of claim 2, wherein the method further comprises: The local branch encoder submodule comprises a first-stage unit, a second-stage unit, a third-stage unit, a fourth-stage unit, a fifth-stage unit and a sixth-stage unit; The input of the first stage unit is an infrared iceberg image, and the output is a local feature where H i1 is the height of the local feature , W i1 is the width of the local feature , and C i1 is the number of channels of the local feature . The first-stage unit comprises a first convolutional layer, a second convolutional layer, a first summation layer, a third convolutional layer, a fourth convolutional layer, a second summation layer, a fifth convolutional layer, a sixth convolutional layer and a third summation layer; All the convolutional layers in the first-stage unit are 3*3 convolutional layers; The input of the first convolutional layer is the infrared iceberg image; the output of the first convolutional layer is the input of the second convolutional layer; the input of the first summation layer is the output of the second convolutional layer and the infrared iceberg image; the input of the third convolutional layer is the output of the first summation layer, and the input of the fourth convolutional layer is the output of the third convolutional layer; the input of the second summation layer is the output of the first summation layer and the output of the fourth convolutional layer; the input of the fifth convolutional layer is the output of the second summation layer; the output of the fifth convolutional layer is the input of the sixth convolutional layer; and the input of the third summation layer is the output of the sixth convolutional layer and the output of the second summation layer; The second-stage unit, the third-stage unit, the fourth-stage unit, the fifth-stage unit, the sixth-stage unit and the first-stage unit have the same structure; The input of the 2nd stage unit is local features The output is local features where H i2 is the height of the local feature , W i2 is the width of the local feature , and C i2 is the number of channels of the local feature . The input of the 3rd stage unit is local features The output is local features where H i3 is the height of the local feature , W i3 is the width of the local feature , and C i3 is the number of channels of the local feature . The input of the 4th stage unit is local features The output is local features where H i4 is the height of the local feature , W i4 is the width of the local feature , and C i4 is the number of channels of the local feature . The input to the 5th stage unit is local features The output is local features where H i5 is the height of the local feature , W i5 is the width of the local feature , and C i5 is the number of channels of the local feature . The input to the 6th stage unit is local features The output is local features where H i6 is the height of the local feature , W i6 is the width of the local feature , and C i6 is the number of channels of the local feature .

4. The method of claim 3, wherein the method further comprises: The multi-resolution feature interaction module MFI comprises a first convolutional unit, a linear embedding unit, a channel attention unit, a first de-redundancy unit, a spatial attention unit, a second de-redundancy unit and a multi-resolution feature interaction output unit; The first convolutional unit comprises a seventh convolutional layer, an eighth convolutional layer, a ninth convolutional layer and a tenth convolutional layer; All the convolutional layers in the first convolutional unit are 3*3 convolutional layers; input of the 7th convolutional layer is local feature and input the output of the 7th convolutional layer to a linear embedding unit input of the 8th convolutional layer is local feature and input the output of the 8th convolutional layer to a linear embedding unit; input of the 9th convolutional layer is local feature and input the output of the 9th convolutional layer to a linear embedding unit; input of the 10th convolutional layer is local feature and input the output of the 10th convolutional layer to a linear embedding unit; The linear embedding unit is used to obtain local features The corresponding plate sequence T i' Specifically, wherein Conv is a 3*3 convolution operation, is the i'th local feature of the input MFI, Aver is an average pooling operation, rearrange is a reshape operation, Conv 1×1 is a 1*1 convolution operation, T i' is a local feature corresponding to the plate sequence, i' is the number of local features of the input MFI, taking 2, 3, 4 or 5; The average pooling operation divides each feature map output by the first convolution unit into n p f ×p f non-overlapping patches; wherein P is a constant, f is the feature map index of the first convolution unit output, p f is the size of the patch divided from the fth feature map of the first convolution unit output; The input of the channel attention unit is The corresponding plate sequence T i' , and the output is CA(T i' ); The first de-redundancy unit is configured to perform de-redundancy processing on the CA(T i' ) to obtain an output of the first de-redundancy unit The input of the spatial attention unit is The output is The second de-redundancy unit is configured to perform de-redundancy processing on the first de-redundancy unit output to obtain a second de-redundancy unit output ​ The multi-resolution feature interaction output unit utilizes acquiring the enhanced local feature 5. The method of claim 4, wherein the infrared iceberg detection method based on the global feature and multi-resolution semantic feature interaction network is characterized in that: The first de-redundancy unit is configured to perform de-redundancy processing on the CA(T i' ) to obtain an output of the first de-redundancy unit Specifically: wherein, is the output of the first de-redundancy unit processing, i' ) is the output of the first de-redundancy unit processing, is a point-wise multiplication operation in the channel dimension, is a point-wise addition operation, and Sigmoid() is a Sigmoid function.

6. The method of claim 5, wherein the infrared iceberg detection method based on the global feature and multi-resolution semantic feature interaction network is characterized in that: The second redundancy-removing unit is configured to perform redundancy-removing processing on the first redundancy-removing unit output to obtain a second redundancy-removing unit output The second redundancy-removing unit is configured to perform redundancy-removing processing on the first redundancy-removing unit output to obtain a second redundancy-removing unit output Specifically, wherein is the output after processing by the second de-redundancy unit, is a point-wise multiplication operation in the spatial dimension.

7. The method of claim 6, wherein the infrared iceberg detection method based on the global feature and multi-resolution semantic feature interaction network is characterized in that: The multi-resolution feature interaction output unit utilizes obtaining the enhanced local feature Specifically: wherein, is a local feature enhanced local feature obtained after enhancement, up() is an up-sampling operation, LN() is a batch normalization operation.

8. The method of claim 7, wherein the infrared iceberg detection method based on the global feature and multi-resolution semantic feature interaction network is characterized in that: The feature fusion decoder module utilizes a global feature set Local features And An enhanced local feature set An infrared iceberg target detection result is obtained, specifically: D1, using global feature set local features and enhanced local feature set obtaining a feature fusion result: wherein Fea i is the i-th feature fusion result, i∈[1,6], i is the label of the feature fusion result, BN() is a batch normalization operation; D2: An infrared iceberg target detection result Output is acquired by using the feature fusion result Fea1: Output = Sigmoid(Conv(ReLU(Conv(Fea1))) wherein ReLU() is a ReLU activation function, and Conv is a 3*3 convolution.

9. The method of claim 8, wherein the infrared iceberg detection method based on the global feature and multi-resolution semantic feature interaction network is characterized in that: The loss function of the GMSINet network is: Output i = Sigmoid(Conv(Fea i )), i = 1, 2, 3, 4, 5, 6 wherein, loss mix is the overall loss function of the GMSI Net network, loss softIoU (Output i , GT i ) is the Soft-IoU loss function, m, n are the horizontal and vertical coordinates of the pixel, respectively, is the pixel value of Output i at the (m, n) point, GT i is the label image corresponding to the infrared iceberg image, which is down-sampled to a preset resolution, is the pixel value of GT i at the (m, n) point.

Images