A method for detecting marine targets based on background information suppression

Abstract

The present invention discloses a method for detecting marine targets based on background information suppression, the core steps of which include: position encoding and window division of feature maps, construction of foreground-background information extraction channels, feature screening based on multi-dimensional information, and dynamic upsampling of high-dimensional feature maps. The innovation of the present invention lies in: designing an aerial image target detection algorithm model based on background information suppression, which can separate foreground information from background information. In addition, the present invention also designs a feature enrichment module based on multi-dimensional information, which can filter out large-area background information in aerial images and improve the accuracy of aerial image target detection. Experimental results show that the algorithm performs well in identifying and locating targets, especially when facing targets similar to ships, complex port distributions, and complex background information such as waves, showing high recognition accuracy, with mAP reaching 90.54% and AP50 reaching 90.60%.

Description

Technical Field

[0001] The present invention relates to the field of marine target detection in aerial images of vertical take-off and landing aircraft, and specifically to a marine target detection method based on background information suppression for electric vertical take-off and landing aircraft, especially capable of realizing target recognition and positioning when the surrounding environment of the marine target is complex. Background Art

[0002] In recent years, with the rapid development of drone camera technology, how to accurately detect marine ship targets from complex aerial images has become a research hotspot. The small size of ships relative to the dominant background in the image makes the detection of marine targets extremely challenging. In addition, the dynamic changes in the marine environment, such as waves, sea clutter, and cargo accumulation in ports, increase the difficulty of detection. The presence of these complex backgrounds not only interferes with the feature extraction of ship targets, but may also lead to misidentification, missed detection, and inaccurate target positioning.

[0003] Maritime target detection plays an important role in maritime safety supervision, maritime traffic management, marine environmental protection and other fields. Therefore, there is an urgent need to study a target detection method that can identify targets from complex backgrounds. Summary of the invention

[0004] In view of the difficult problems in the above-mentioned prior art, the present invention proposes a method for detecting marine targets based on background information suppression for aerial image processing technology of electric vertical take-off and landing aircraft in complex marine environments. The method consists of three core parts: a main structure feature extraction network, a background information suppression module, and a feature enrichment module based on multi-dimensional information, which can improve the accuracy and efficiency of target detection. In particular, the present invention can adapt to data captured by visible light cameras with different parameters for analysis and processing. When facing targets similar to ships, complex port distributions, and complex background information such as waves, this algorithm shows high detection accuracy and positioning accuracy, with mAP reaching 90.54% and AP50 reaching 90.60%.

[0005] The present invention achieves the above-mentioned object through the following technical solutions:

[0006] A method for detecting marine targets based on background information suppression comprises the following steps: (1) extracting features from a backbone network, specifically: performing three sets of convolution preprocessing on an input image, and then performing feature extraction in four stages, wherein the four stages respectively comprise a convolution module and a CSP layer, wherein the CSP layer comprises a convolution module, a depthwise separable convolution module, and a residual connection; the main purpose is to perform feature extraction on the input image to facilitate subsequent feature fusion;

[0007] (2) Construct a background information suppression module. Specifically, first, the feature maps C3, C4, and C5 of the three dimensions output by the backbone network are divided into windows. There are three branches in total. The window size of the first branch is 5, the window size of the second branch is 25, and the third branch is composed of a depth-wise separable convolution. Finally, the output results of the three branches are merged and ReLU activation is performed. The background information suppression module divides the image into PxP patches and uses a convolutional layer to map each patch to the embedding space:

[0008] E = Conv2d(I; W proj ,b proj )

[0009] Where I is the input image, W proj and b proj is the weight and bias of the projection convolution layer, and E is the embedded patch; then the patch is passed to the transformer module to calculate the score of each patch, and the multi-head self-attention mechanism:

[0010]

[0011] Where Q, K, V are query, key, and value matrices, d k is the dimension of the key;

[0012] The feed-forward network of the background information suppression module consists of a convolutional layer and a GELU activation function:

[0013] FFN(x)=GELU(Linear(x)W1)W2

[0014] Where W1, W2 are the weight matrices of the feedforward network;

[0015] The depth-wise separable convolution consists of a 3X3 convolution module and a 1X1 convolution module:

[0016]

[0017] Where G is the number of groups, W is g and b g are the weights and biases of the g-th group.

[0018] (3) Construct a multi-dimensional feature enrichment module. Specifically, first perform an average pooling operation on the features of the two dimensions C3 and C4 output by the backbone network, and reduce the size to half of the original size. Then, subtract the corresponding pooling results during the upsampling process of P4 and P5 of the pyramid network. The multi-dimensional feature enrichment module first performs average pooling of low-dimensional feature maps:

[0019] pooled_feature3,pooled_feature4=MeanPooling2d(C3,C4)

[0020] Then perform background feature filtering:

[0021] Ffeatrue4=P4-pooled_featrue3

[0022] Ffeatrue5=P5-pooled_featrue4

[0023] Where Ffeatrue represents the feature map after filtering.

[0024] The initialization of the bias matrix initializes the weights through normal distribution:

[0025] W~N(μ,σ 2 )

[0026] Where μ is the mean and σ is the standard deviation. The offset Δ is calculated by convolution:

[0027] Δ=Conv2d(I;W offset ,b offset )

[0028] Where I is the input feature map, W offset and b offset are the weights and biases of the offset convolutional layer, respectively.

[0029] After generating the offset matrix, the initial position P is used to calculate the offset coordinates:

[0030]

[0031] Where P is a two-dimensional grid that defines the initial offset position of each pixel. Scale is the scaling factor of the current feature map relative to the image. After obtaining the offset coordinates, grid sampling is performed:

[0032] I′=GridSample(I,C)

[0033] Where I' is the sampled feature map and the offset Δ is further processed by a new convolutional layer and batch normalization:

[0034] Δ″=ReLU(BatchNorm2d(Conv2d(Δ′)))+Δ′

[0035] Preferably, the window division of the feature map of the background information suppression module has two branches, the window size of one branch is set to 5 pixels, and the window size of the other branch is set to 25 pixels.

[0036] Preferably, the multi-dimensional information fusion module can be applied to any pyramid network fusion structure, wherein the sizes of the feature maps after average pooling of C3 and C4 are 50×50 and 25×25 pixels respectively.

[0037] The present invention is beneficial in that:

[0038] (1) Since the aerial images are taken at a long distance and the features of the targets at sea are blurred, the algorithm can correctly identify the ship targets when facing similar targets such as ships, cargo piles, and containers; when facing complex backgrounds such as ports, the algorithm can extract the ship targets and their positioning information from the complex background information.

[0039] (2) Applicable to any aircraft equipped with a visible light camera. By collecting and processing marine information, the camera can identify and locate specific targets.

[0040] (3) The background information suppression module is used in the module design process. By dividing the window sizes into different sizes, the network model can distinguish between background information and foreground information.

[0041] (4) In the module design process, a multi-dimensional feature enrichment module is used to filter out most of the invalid background information by filtering out the average pooling results in the low-dimensional feature map. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 is a flow chart of the planning method of the present invention;

[0044] Figure 2 The overall network architecture of the model of the present invention;

[0045] Figure 3 It is the multi-dimensional feature enrichment module of the present invention.

[0046] Figure 4 It is the background information suppression module of the present invention.

[0047] Figure 5 Visualization results of several feature maps, where a is RetinaNet, b is H2rbox-v2, c is RTMDet, and d is our proposed method.

[0048] Figure 6The following are the recognition results of several methods on the HRSC2016 dataset, where a is the correct result, b is our method, c is RTMDet, d is redet, and the dotted rectangle represents recognition errors or missed recognitions. DETAILED DESCRIPTION

[0049] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the following content is only one of the technical routes of the present invention and not all embodiments. The parameters and formulas visualized do not indicate or imply specific operations, and therefore cannot be understood as limitations on the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0050] like Figure 1 As shown, a method for detecting marine targets based on background information suppression is implemented as follows: Figure 2 The overall network architecture of the model of the present invention):

[0051] (1) Extract backbone network features

[0052] Specifically, the input image is preprocessed with three sets of convolutions, and then four stages of feature extraction are performed. These four stages include a convolution module and a CSP layer. The CSP layer consists of a convolution module, a depth-separable convolution module, and a residual connection. The main purpose is to perform feature extraction on the input image to facilitate subsequent feature fusion.

[0053] (2) Construct a background information suppression module, such as Figure 4 As shown,

[0054] Specifically, first, the feature maps C3, C4, and C5 of the three dimensions output by the backbone network are divided into windows. There are three branches in total. The window size of the first branch is 5, the window size of the second branch is 25, and the third branch is composed of a depth-separable convolution. Finally, the output results of the three branches are merged and ReLU activation is performed; the background information module is to divide the image into PxP patches and use the convolution layer to map each patch to the embedding space:

[0055] E = Conv2d(I; W proj ,b proj )

[0056] Where I is the input image, W proj and b projis the weight and bias of the projection convolution layer, and E is the embedded patch. Then the patch is passed to the transformer module to calculate the score of each patch, and the multi-head self-attention mechanism:

[0057]

[0058] Where Q, K, V are query, key, and value matrices, d k is the dimension of the key.

[0059] The feed-forward network of the background information suppression module consists of a convolutional layer and a GELU activation function:

[0060] FFN(x)=GELU(Linear(x)W1)W2

[0061] Where W1, W2 are the weight matrices of the feedforward network.

[0062] The depth-wise separable convolution consists of a 3X3 convolution module and a 1X1 convolution module:

[0063]

[0064] Where G is the number of groups, W is g and b g are the weights and biases of the g-th group.

[0065] (3) Construct multi-dimensional feature enrichment modules, such as Figure 3 As shown,

[0066] Specifically, firstly, average pooling is performed on the features of the two dimensions C3 and C4 output by the backbone network, and the size is reduced to half of the original size. Then, the corresponding pooling results are subtracted during the upsampling process of P4 and P5 of the pyramid network. The multi-dimensional feature enrichment module first performs average pooling of low-dimensional feature maps:

[0067] pooled_feature3,pooled_feature4=MeanPooling2d(C3,C4)

[0068] Then perform background feature filtering:

[0069] Ffeatrue4=P4-pooled_featrue3

[0070] Ffeatrue5=P5-pooled_featrue4

[0071] Where Ffeatrue represents the feature map after filtering.

[0072] The initialization of the bias matrix initializes the weights through normal distribution:

[0073] W~N(μ,σ 2 )

[0074] Where μ is the mean and σ is the standard deviation. The offset Δ is calculated by convolution:

[0075] Δ=Conv 2d(I;W offset ,b offset )

[0076] Where I is the input feature map, W offset and b offset are the weights and biases of the offset convolutional layer, respectively.

[0077] After generating the offset matrix, the initial position P is used to calculate the offset coordinates:

[0078]

[0079] Where P is a two-dimensional grid that defines the initial offset position of each pixel, and scale is the scaling factor of the current feature map relative to the image. After obtaining the offset coordinates, grid sampling is performed:

[0080] I′=GridSample(I,C)

[0081] Where I' is the sampled feature map. The offset Δ is further processed by a new convolutional layer and batch normalization:

[0082] Δ″=ReLU(BatchNorm2d(Conv2d(Δ′)))+Δ′

[0083] Figure 5 Results of visualizing several feature maps, where a is RetinaNet, b is H2rbox-v2, c is RTMDet, and d is our proposed method; Figure 6 The following are the recognition results of several methods on the HRSC2016 dataset, where a is the correct result, b is our method, c is RTMDet, d is redet, and the dotted rectangle represents recognition errors or missed recognitions.

[0084] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. The present embodiment is exemplary and non-restrictive, and the scope of the present invention is defined by the appended claims rather than the above description, and therefore it is intended that all changes that fall within the meaning and scope of the claims are included in the present invention, and any reference numerals in the claims should not be considered as limiting the claims involved.

[0085] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting marine targets based on background information suppression, characterized in that: The following steps are involved: (1) Perform backbone network feature extraction, specifically: perform three sets of convolution preprocessing on the input image, and then perform four stages of feature extraction. The four stages each contain a convolution module and a CSP layer. The CSP layer consists of a convolution module, a depth-wise separable convolution module, and a residual connection. (2) Constructing a background information suppression module, specifically: first, the three-dimensional feature maps C3, C4, and C5 output by the backbone network are divided into three branches. The first branch window size is 5, the second branch window size is 25, and the third branch is composed of a depthwise separable convolution. Finally, the output results of the three branches are merged and ReLU activation operation is performed; (3) Construct a multi-dimensional feature enrichment module. Specifically, first, perform an average pooling operation on the features of the two dimensions C3 and C4 output by the backbone network, reducing the size to half of the original size. Then, subtract the corresponding pooling results during the upsampling process of P4 and P5 of the pyramid network.

2. The method for detecting marine targets based on background information suppression according to claim 1, characterized in that: Each stage contains a convolution-normalization-activation layer, followed by a CSP module, which includes a convolution module and channel attention. Each time the CSP module is calculated, the size of the feature map is reduced by half.

3. The method for detecting marine targets based on background information suppression according to claim 1, characterized in that: The background information suppression module divides the image into P×P patches and uses a convolutional layer to map each patch to the embedding space: E=Conv2d(I;W proj ,b proj ) Where I is the input image, W proj and b proj is the weight and bias of the projection convolution layer, and E is the embedded patch; then the patch is passed to the transformer module to calculate the score of each patch, and the multi-head self-attention mechanism: Where Q, K, V are query, key, and value matrices, d k is the dimension of the key.

4. The method for detecting marine targets based on background information suppression according to claim 1, characterized in that: The feed-forward network of the background information suppression module consists of a convolutional layer and a GELU activation function: FFN(x)=GELU(Linear(x)W1)W2 Where W1, W2 are the weight matrices of the feedforward network.

5. The method for detecting marine targets based on background information suppression according to claim 1, characterized in that: The depth-wise separable convolution consists of a 3X3 convolution module and a 1X1 convolution module. composition: Where G is the number of groups, W is g and b g are the weights and biases of the g-th group.

6. The method for detecting marine targets based on background information suppression according to claim 1, characterized in that: The multi-dimensional feature enrichment module first performs average pooling of low-dimensional feature maps: pooled_feature3,pooled_feature4=MeanPooling2d(C3,C4) Then perform background feature filtering: Ffeatrue4=P4-pooled_featrue3 Ffeatrue5=P5-pooled_featrue4 Where Ffeatrue represents the feature map after filtering.

7. The method for detecting marine targets based on background information suppression according to claim 1, characterized in that: The offset matrix is ​​initialized based on the multi-dimensional feature enrichment module and the weights are initialized through the normal distribution: W~N(μ,σ 2 ) Where μ is the mean, σ is the standard deviation, and the offset Δ is calculated by convolution: Δ=Conv2d(I;W offset ,b offset ) Where I is the input feature map, W offset and b offset are the weights and biases of the offset convolutional layer, respectively.

8. The method for detecting marine targets based on background information suppression according to claim 1, characterized in that: After generating the offset matrix, the initial position P is used to calculate the offset coordinates: Where P is a two-dimensional grid that defines the initial offset position of each pixel. Scale is the scaling factor of the current feature map relative to the image. After obtaining the offset coordinates, grid sampling is performed: I′=GridSample(I,C) Where I' is the sampled feature map; the offset Δ is further processed by a new convolutional layer and batch normalization: Δ″=ReLU(BatchNorm2d(Conv2d(Δ′)))+Δ′.

Images