Method for estimating ship height based on single SAR image

Abstract

The invention discloses a method for estimating the height of a ship based on a single SAR image, and relates to the technical field of synthetic aperture radar image interpretation, and the method comprises the steps: processing the obtained SAR images of different sea areas, obtaining a plurality of SAR ship slices, calculating the amplitude and phase of each SAR ship slice, and obtaining a plurality of SAR amplitude images and SAR phase images; the SAR amplitude image is input into an amplitude feature extraction module, amplitude feature extraction is carried out, and a multi-size amplitude feature graph is obtained; the SAR phase image is input into a phase feature extraction module, phase feature extraction is carried out, and multi-size phase features are obtained; inputting the obtained multi-size amplitude feature map and the multi-size phase feature into a cross-size feature fusion layer, and performing cross-size feature fusion to obtain a feature map; and inputting the obtained feature map into a full connection layer, and calculating through an activation function to obtain the height of the ship.

Description

Technical Field

[0001] The present invention relates to the technical field of synthetic aperture radar image interpretation, and specifically to a method for estimating the height of a ship based on a single SAR image. Background Art

[0002] Synthetic Aperture Radar (SAR) is an active microwave sensor that can work all day and all weather, and has been widely applied to the field of marine remote sensing. Height estimation is crucial for understanding the geometric relationships in a scene. Height information is considered beneficial for various challenging remote sensing tasks, such as semantic labeling and change detection, etc. This is because height information can provide a rich representation of the target and its surrounding environment. Ships are the main carriers of maritime transportation and military operations, and ship height estimation is an important research direction for future marine applications. On the one hand, ship height can provide important background information in the 3D reconstruction of ships, which is crucial for obtaining a true and accurate ship model. On the other hand, the superstructure and cabin layout of ships are often closely related to the ship height, and different types of ships usually have different ship height characteristics, which can provide important clues for identifying ship types.

[0003] Traditional target height estimation methods utilize the principle of stereo matching, such as photogrammetry, radiometry, SAR interferometry, etc. These methods require at least two SAR images from different directions, with high data acquisition costs and computational complexities. Estimating the target height using a single SAR image has attracted the attention of many researchers because it only requires one SAR image to estimate the target height, greatly reducing the cost and data acquisition difficulty. Estimating the target height from a single SAR image is mainly achieved using deep learning methods. Deep learning has a powerful non-linear expression ability and can mine the hidden 3D features in the 2D SAR image, thereby establishing a non-linear mapping relationship between the 2D SAR image and the target height. However, currently, the SAR target height estimation methods based on deep learning are mainly applicable to building targets and not applicable to ship targets. This is because there are significant differences between ship targets and building targets. Firstly, ships are small in size in the SAR image. Secondly, ships are in a moving state on the sea surface, and their scattering characteristics in the SAR image are more complex. Thirdly, the superstructure of ships has a complex structure and uneven heights. For the above reasons, it is very difficult to estimate the ship height from the SAR image. Summary of the Invention

[0004] In order to solve the above problems, based on the deep learning theory, the present invention proposes a method for estimating the height of a ship based on a single SAR image, including: Step 1: Process the acquired SAR images of different sea areas to obtain multiple SAR ship slices, calculate the amplitude and phase of each SAR ship slice, and obtain multiple SAR amplitude images and SAR phase images; Step 2: Input the SAR amplitude image into the amplitude feature extraction module for amplitude feature extraction to obtain multi-scale amplitude feature maps; Step 3: Input the SAR phase image into the phase feature extraction module for phase feature extraction to obtain multi-scale phase features; Step 4: Input the multi-scale amplitude feature maps and multi-scale phase features obtained in Step 2 and Step 3 into the cross-scale feature fusion layer for cross-scale feature fusion to obtain the feature map FM; Step 5: Input the feature map FM obtained in Step 4 into the fully connected layer and perform activation function calculation to obtain the ship height.

[0005] In a preferred embodiment, in Step 4, the cross-scale feature fusion layer linearly fuses the amplitude feature maps of different sizes with the corresponding-size phase features respectively to obtain linearly fused feature maps of different sizes; spatially aligns the linearly fused feature maps of different sizes, and then performs channel fusion to obtain the cross-scale fusion feature map.

[0006] In a preferred embodiment, in Step 4, the amplitude feature extraction module includes a plurality of residual blocks and a plurality of pooling layers connected in series; the residual blocks are used to extract the high-dimensional semantic features of the SAR amplitude image to obtain high-dimensional feature maps; the pooling layers are used to perform downsampling on the high-dimensional feature maps.

[0007] In a preferred embodiment, in Step 4, the phase feature extraction module includes a plurality of dilated convolutional layers and a plurality of attention layers connected in series. The dilated convolutional layers are used to capture the context information at different positions in the SAR phase image to obtain phase features of different sizes related to elevation; the attention layers use channel attention to adjust the weights of different channels of the phase features.

[0008] In a preferred embodiment, the ship height at each position is the height from this position to the ship's waterline.

[0009] In a preferred embodiment, in Step 1, screen the acquired SAR images of different sea areas, delete the images without ship targets, crop them according to the positions of the ships in the screened SAR images to obtain multiple SAR ship slices, calculate the amplitude and phase of each SAR ship slice respectively, and save the calculation results as SAR amplitude images and SAR phase images in TIFF format.

[0010] Compared with the prior art, the present invention has the following beneficial technical effects: Based on the deep learning model SAR2ShipHeight, the present invention estimates the height of ships using SAR images. Compared with existing methods, the proposed method extracts the amplitude and phase features of SAR images through the SAR2ShipHeight model and performs multi-scale fusion of the amplitude features and phase features, achieving ship height estimation based on a single SAR image, effectively improving the estimation accuracy of ship height. The ship height estimation accuracy of the proposed method is 80.96%, achieving accurate estimation of ship height and providing prior knowledge and important references for the 3D reconstruction of SAR ships. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic flow chart of the method for estimating the height of ships based on a single SAR image of the present invention; Figure 2 It is a schematic diagram of the structure of the SAR2ShipHeight model of the present invention; Figure 3 It is a schematic diagram of the structure of the residual module of the present invention; Figure 4 It is a schematic diagram of the ship height profile and the waterline of the present invention; Figure 5 It is Example 1 of the SAR image of the present invention and the corresponding ship height estimation result; Figure 6 It is Example 2 of the SAR image of the present invention and the corresponding ship height estimation result. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention. Embodiment 1

[0013] As Figure 1 shown, it is a schematic flow chart of the method for estimating the height of ships based on a single SAR image of the present invention, including the following steps: Step 1, process the obtained SAR images of different sea areas to obtain multiple SAR ship slices, calculate the amplitude and phase of each SAR ship slice, and obtain multiple SAR amplitude images and SAR phase images.

[0014] First, screen the obtained SAR images of different sea areas and delete the images without ship targets. Then, crop the SAR images according to the positions of the ships in the screened SAR images to obtain multiple SAR ship slices. Calculate the amplitude and phase of each SAR ship slice respectively, and save the calculation results as a SAR amplitude image and a SAR phase image in TIFF format. Name the SAR amplitude image "amplitude" and the SAR phase image "phase".

[0015] Step 2: Input the SAR amplitude image "amplitude" into the amplitude feature extraction module to perform amplitude feature extraction and obtain a multi-scale amplitude feature map.

[0016] As Figure 2 shown, the amplitude feature extraction module includes 4 residual blocks (R1, R2, R3, R4) and 4 pooling layers (P1, P2, P3, P4) connected in series, which are used to extract spatial features such as the texture, edges, and shape of the ship.

[0017] After the SAR amplitude image is input into the amplitude feature extraction module, it successively enters the residual blocks and pooling layers of different depths.

[0018] The residual block processes the input SAR amplitude image using operations such as convolution and batch normalization to extract the high-dimensional semantic features of the ship SAR amplitude image and obtain a high-dimensional feature map.

[0019] The pooling layer downsamples the high-dimensional feature map extracted by the residual block using a pooling kernel with a size of 3×3 and a stride of 2 to remove redundant features and obtain an amplitude feature map.

[0020] The amplitude feature map output by P1 is denoted as FA1, with a size of 1 / 2 of the SAR amplitude image and 64 channels; The amplitude feature map output by P2 is denoted as FA2, with a size of 1 / 4 of the SAR amplitude image and 128 channels; The amplitude feature map output by P3 is denoted as FA3, with a size of 1 / 8 of the SAR amplitude image and 256 channels; The amplitude feature map output by P4 is denoted as FA4, with a size of 1 / 16 of the SAR amplitude image and 512 channels; After the above steps, a multi-scale amplitude feature map, namely FA1, FA2, FA3, and FA4, is obtained.

[0021] As Figure 2As shown in the figure, the residual block includes a main branch and a shortcut connection branch. Among them, the number of input channels of the first convolutional layer of the main branch of R1 is 1, and the number of output channels is 64. The number of input channels and output channels of the second convolutional layer are both 64. The number of input channels of the first convolutional layer of the main branch of R2 is 64, and the number of output channels is 128. The number of input channels and output channels of the second convolutional layer are both 128. The number of input channels of the first convolutional layer of the main branch of R3 is 128, and the number of output channels is 256. The number of input channels and output channels of the second convolutional layer are both 256. The number of input channels of the first convolutional layer of the main branch of R4 is 256, and the number of output channels is 512. The number of input channels and output channels of the second convolutional layer are both 512.

[0022] As Figure 3 shown in the figure, the convolution kernel size of the main branch of the residual module is 3×3, the stride is 1, and the activation function is the ReLU function. The size of the convolution kernel of the shortcut connection is 1×1, and the stride is 1. The results of the main branch and the shortcut connection branch are added together, and then passed through an activation function calculation to obtain the output result of the residual module. P1, P2, P3, and P4 all use the maximum pooling method.

[0023] Step 3: Input the SAR phase image phase into the phase feature extraction module for phase feature extraction to obtain multi-scale phase features.

[0024] As Figure 2 shown in the figure, the phase feature extraction module includes 4 dilated convolutional layers (C1, C2, C3, C4) and 4 attention layers (A1, A2, A3, A4) connected in series. Each dilated convolutional layer performs dilated convolution, batch normalization, and ReLU activation function processing on the input image.

[0025] After the SAR phase image is input into the phase feature extraction module, a dilated convolutional layer with a convolution kernel size of 3×3 is first used for feature extraction to capture the context information of different positions of the ship and obtain phase features related to elevation. The dilated convolutional layer can increase the receptive field, help the model extract global features, suppress noise, and ensure global phase continuity.

[0026] The attention layer uses channel attention to assign weights to the features of different channels, highlight the key channels, and enhance the feature extraction ability of the model. The convolution kernel sizes of the 4 dilated convolutional layers are all 3×3, and the stride is 2. By setting the stride, downsampling of the phase features is achieved to obtain phase features of different sizes. The dilation rates of C1, C2, C3, and C4 are 1, 2, 4, and 6 respectively.

[0027] The phase features output by A1 are denoted as FP1, with a size of 1 / 2 of the SAR phase image and 64 channels. The phase feature output by A2 is denoted as FP2, with a size that is 1 / 4 of the SAR phase image and 128 channels; The phase feature output by A3 is denoted as FP3, with a size that is 1 / 8 of the SAR phase image and 256 channels; The phase feature output by A4 is denoted as FP4, with a size that is 1 / 16 of the SAR phase image and 512 channels; After the above steps, multi - size phase features are obtained, namely FP1, FP2, FP3, FP4.

[0028] Step 4: Input the multi - size amplitude feature maps and multi - size phase features obtained in Step 2 and Step 3 into the cross - size feature fusion layer for cross - size feature fusion to obtain FM.

[0029] The cross - size feature fusion layer first linearly fuses the amplitude feature maps and phase features of different sizes, so that the output multi - size feature maps contain both local phase change details and retain the physical association characteristics of global elevation differences, that is, FA1 and FP1 are linearly fused to obtain the linearly fused feature map F1, FA2 and FP2 are linearly fused to obtain the linearly fused feature map F2, FA3 and FP3 are linearly fused to obtain the linearly fused feature map F3, and FA4 and FP4 are linearly fused to obtain the linearly fused feature map F4; Then, upsampling is performed on F4, F3, and F2 respectively to spatially align the feature maps of different sizes to obtain feature maps with the same size as F1. The upsampled F4 obtains the spatially aligned linearly fused feature map FF4, the upsampled F3 obtains the spatially aligned linearly fused feature map FF3, and the upsampled F2 obtains the spatially aligned linearly fused feature map FF2; Finally, F1 is channel - fused with FF4, FF3, and FF2 to obtain the result of the cross - size feature fusion module, that is, the cross - size fused feature map FM.

[0030] Step 5: Input the feature map FM obtained in Step 4 into the fully - connected layer to further estimate the ship height through the activation function.

[0031] The superstructure of the ship is complex and the height at each position is different. Therefore, the ship height profile is used to represent the ship height. As Figure 2 shown in the schematic diagram of the ship height profile, the ship height profile consists of the ship height at each position, and the ship height at each position refers to the height from this position to the ship's waterline.

[0032] Input the feature map FM obtained in Step 4 into the fully - connected layer. First, flatten FM to convert it into a one - dimensional vector, then perform linear combination calculations on the elements of the one - dimensional vector and perform activation function processing, and finally obtain the ship height.

[0033] Among them, the number of nodes in the fully - connected layer is 1024, and the activation function is the classic SoftPlus function.

[0034] The calculation formula of the SoftPlus function is as follows: ; h represents the height of the ship before the activation function calculation is performed.

[0035] As Figure 5 and Figure 6 shown, the SAR image input is on the left, the ship height estimation result is on the right, the red solid line is the true value of the ship height, and the blue solid line is the ship height prediction value of the SAR2ShipHeight model. It can be seen from the experimental results that the shape of the ship height contour predicted by the SAR2ShipHeight model is very close to the true value, and the details of the ship height contour are retained at some positions with large height changes. The ship height in the middle of the ship is closest to the true value with a small error. The position where the ship height differs most from the true value is the bow of the ship, where the contour shape changes greatly. The experimental results prove that the SAR2ShipHeight proposed by the present invention can accurately estimate the ship height using a single SAR image.

[0036] Adopt the evaluation index calculation method to calculate the error and accuracy of the ship height estimation result. Table 1 gives the ship height estimation accuracy of the ship height estimation model SAR2ShipHeight of the present invention and other deep learning algorithms.

[0037] Table 1 Ship height estimation accuracy of different algorithms

[0038] It can be seen from Table 1 that the ship height estimation accuracy obtained by the ship height estimation model SAR2ShipHeight of the present invention is significantly higher than that obtained by using other deep learning algorithms.

[0039] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any reference signs in the claims should not be regarded as limiting the claimed rights.

Claims

1. A method for estimating the height of a ship based on a single SAR image, characterized in that, Including: Step 1: Process the obtained SAR images of different sea areas to obtain multiple SAR ship slices, calculate the amplitude and phase of each SAR ship slice, and obtain multiple SAR amplitude images and SAR phase images; Step 2: Input the SAR amplitude image into the amplitude feature extraction module to perform amplitude feature extraction and obtain a multi-size amplitude feature map; Step 3: Input the SAR phase image into the phase feature extraction module to perform phase feature extraction and obtain multi-size phase features; Step 4: Input the multi-size amplitude feature map and multi-size phase features obtained in Step 2 and Step 3 into the cross-size feature fusion layer to perform cross-size feature fusion and obtain a feature map; Step 5: Input the feature map obtained in Step 4 into the fully connected layer and calculate through the activation function to obtain the ship height.

2. The method for estimating the height of a ship based on a single SAR image according to claim 1, wherein In Step 4, the cross-size feature fusion layer linearly fuses the amplitude feature maps of different sizes with the corresponding-size phase features respectively to obtain multiple linearly fused feature maps of different sizes; the linearly fused feature maps of multiple sizes other than the largest-size linearly fused feature map are respectively upsampled to obtain spatially aligned linearly fused feature maps, and then channel fusion is performed with the largest-size linearly fused feature map to obtain a cross-size fused feature map.

3. The method for estimating the height of a ship based on a single SAR image according to claim 1, wherein, In Step 2, the amplitude feature extraction module includes a plurality of residual blocks and a plurality of pooling layers connected in series; the residual blocks are used to extract high-dimensional semantic features of the SAR amplitude image to obtain a high-dimensional feature map; the pooling layers are used to perform downsampling on the high-dimensional feature map.

4. The method for estimating the height of a ship based on a single SAR image according to claim 1, wherein In Step 3, the phase feature extraction module includes a plurality of dilated convolutional layers and a plurality of attention layers connected in series. The dilated convolutional layers are used to capture context information at different positions in the SAR phase image to obtain phase features of different sizes related to elevation; The attention layer uses channel attention to adjust the weights of different channels of the phase features.

5. The method for estimating the height of a ship based on a single SAR image according to claim 1, wherein, The ship height at each position is the height from the position to the ship's waterline.

6. The method for estimating the height of a ship based on a single SAR image according to claim 1, wherein In Step 1, screen the obtained SAR images of different sea areas, delete the images without ship targets, crop them according to the positions of the ships in the screened SAR images to obtain multiple SAR ship slices, calculate the amplitude and phase of each SAR ship slice respectively, and save the calculation results as SAR amplitude images and SAR phase images in TIFF format.

Images