A method for estimating ship height based on a single SAR image

Abstract

The present invention discloses a method for estimating the height of a ship based on a single SAR image, and relates to the technical field of synthetic aperture radar image interpretation. SAR images acquired in different sea areas are processed to obtain multiple SAR ship slices, and the amplitude and phase of each SAR ship slice are calculated to obtain multiple SAR amplitude images and SAR phase images; the SAR amplitude image is input into an amplitude feature extraction module to perform amplitude feature extraction to obtain a multi-scale amplitude feature map; the SAR phase image is input into a phase feature extraction module to perform phase feature extraction to obtain a multi-scale phase feature; the obtained multi-scale amplitude feature map and multi-scale phase feature are input into a cross-scale feature fusion layer to perform cross-scale feature fusion to obtain a feature map; the obtained feature map is input into a fully connected layer, and the ship height is calculated through an activation function.

Description

Technical Field

[0001] The present invention relates to the technical field of synthetic aperture radar image interpretation, in particular to a method for estimating the height of a ship based on a single SAR image. Background Art

[0002] Synthetic Aperture Radar (SAR) is an active microwave sensor capable of operating around the clock and in all weather conditions, and has been widely used in ocean remote sensing. Height estimation is crucial for understanding geometric relationships in a scene. Height information is considered beneficial for a variety of challenging remote sensing tasks, such as semantic labeling and change detection. This is because height information can provide a rich representation of the target and its surroundings. Ships are the primary vehicles for maritime transportation and military operations, and ship height estimation is an important research direction for future marine applications. On the one hand, ship height can provide important contextual information in three-dimensional ship reconstruction, which is crucial for obtaining a realistic and accurate ship model. On the other hand, a ship's superstructure and cabin layout are often closely related to the ship's height. Different types of ships typically have different ship height characteristics, which can provide important clues for identifying the ship type.

[0003] Traditional methods for estimating target height rely on stereo matching, such as photogrammetry, radiometry, and SAR interferometry. These methods require at least two SAR images from different directions, resulting in high data acquisition costs and computational complexity. Using a single SAR image for target height estimation has attracted considerable research attention, as it only requires a single SAR image, significantly reducing costs and data acquisition. Estimating target height from a single SAR image is primarily accomplished using deep learning methods. Deep learning has powerful nonlinear representation capabilities and can exploit hidden three-dimensional features in two-dimensional SAR images, thereby establishing a nonlinear mapping relationship between the two-dimensional SAR image and target height. However, current deep learning-based SAR target height estimation methods are primarily applicable to buildings and not to vessels. This is due to significant differences between vessels and buildings. First, ships are smaller in SAR images; second, they are in motion on the sea, resulting in more complex scattering patterns in SAR images; and third, ships' superstructures are complex and have varying heights. These factors make estimating ship height from SAR images very difficult. Summary of the Invention

[0004] To solve the above problems, the present invention proposes a method for estimating the height of a ship based on a single SAR image based on deep learning theory, comprising:

[0005] Step 1: Process the acquired SAR images of different sea areas to obtain multiple SAR ship slices, calculate the amplitude and phase of each SAR ship slice, and obtain multiple SAR amplitude images and SAR phase images;

[0006] Step 2: Input the SAR amplitude image into the amplitude feature extraction module to extract the amplitude features and obtain a multi-scale amplitude feature map;

[0007] Step 3: Input the SAR phase image into the phase feature extraction module to extract the phase features and obtain multi-size phase features;

[0008] Step 4: Input the multi-scale amplitude feature map and multi-scale phase feature obtained in steps 2 and 3 into the cross-scale feature fusion layer to perform cross-scale feature fusion to obtain the feature map FM;

[0009] In step 5, the feature map FM obtained in step 4 is input into the fully connected layer and the activation function is calculated to obtain the ship height.

[0010] In a preferred embodiment, in step 4, the cross-scale feature fusion layer linearly fuses the amplitude feature maps of different sizes with the phase features of corresponding sizes to obtain linear fusion feature maps of different sizes; the linear fusion feature maps of different sizes are spatially aligned, and then channel fusion is performed to obtain a cross-scale fusion feature map.

[0011] In a preferred embodiment, in step 4, the amplitude feature extraction module includes multiple residual blocks and multiple pooling layers connected in series; the residual blocks are used to extract high-dimensional semantic features of the SAR amplitude image to obtain a high-dimensional feature map; and the pooling layer is used to downsample the high-dimensional feature map.

[0012] In a preferred embodiment, in step 4, the phase feature extraction module includes multiple hole convolution layers and multiple attention layers connected in series, wherein the hole convolution layer is used to capture contextual information at different positions in the SAR phase image to obtain phase features of different sizes related to elevation; the attention layer uses channel attention to adjust the weights of different channels of the phase feature.

[0013] In a preferred embodiment, the height of the vessel at each location is the height from the location to the waterline of the vessel.

[0014] In a preferred embodiment, in step 1, the acquired SAR images of different sea areas are screened, and images without ship targets are deleted. The screened SAR images are cropped according to the locations of the ships to obtain multiple SAR ship slices. The amplitude and phase of each SAR ship slice are calculated respectively, and the calculation results are saved as SAR amplitude images and SAR phase images in TIFF format.

[0015] Compared with the prior art, the present invention has the following beneficial technical effects:

[0016] Based on the deep learning model SAR2ShipHeight, the present invention uses SAR images to estimate ship height. Compared with existing methods, the proposed method extracts the amplitude and phase features of SAR images through the SAR2ShipHeight model and performs multi-scale fusion of amplitude and phase features, realizing ship height estimation based on a single SAR image, effectively improving the estimation accuracy of ship height. The ship height estimation accuracy of the proposed method is 80.96%, achieving accurate estimation of ship height, and providing prior knowledge and important reference for SAR ship three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Schematic diagram of the flow of the method for estimating the height of a vessel based on a single SAR image according to the present invention;

[0018] Figure 2 SAR2ShipHeight model structure diagram of the present invention;

[0019] Figure 3 Schematic diagram of the structure of the residual module of the present invention;

[0020] Figure 4 Schematic diagram of the ship height profile and waterline of the present invention;

[0021] Figure 5 This is Example 1 of the SAR image and the corresponding ship height estimation result of the present invention;

[0022] Figure 6 This is Example 2 of the SAR image and the corresponding ship height estimation result of the present invention. DETAILED DESCRIPTION

[0023] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention. Example 1

[0024] like Figure 1 FIG. 1 is a flow chart of a method for estimating the height of a vessel based on a single SAR image according to the present invention, comprising the following steps:

[0025] Step 1: Process the acquired SAR images of different sea areas to obtain multiple SAR ship slices, calculate the amplitude and phase of each SAR ship slice, and obtain multiple SAR amplitude images and SAR phase images.

[0026] First, we screened the acquired SAR images from different sea areas and deleted those without any ships. We then cropped the selected SAR images based on the locations of the ships, generating multiple SAR ship slices. We calculated the amplitude and phase of each SAR ship slice and saved the results as SAR amplitude and phase images in TIFF format. We named the SAR amplitude image "amplitude" and the SAR phase image "phase."

[0027] Step 2: Input the SAR amplitude image amplitude into the amplitude feature extraction module to extract the amplitude features and obtain a multi-scale amplitude feature map.

[0028] like Figure 2 As shown in the figure, the amplitude feature extraction module includes four residual blocks (R1, R2, R3, R4) and four pooling layers (P1, P2, P3, P4) connected in series, which are used to extract spatial features such as texture, edge, and shape of the ship.

[0029] After the SAR amplitude image is input into the amplitude feature extraction module, it enters the residual blocks and pooling layers of different depths in sequence.

[0030] The residual block processes the input SAR amplitude image using convolution, batch normalization and other operations to extract the high-dimensional semantic features of the ship SAR amplitude image and obtain a high-dimensional feature map.

[0031] The pooling layer uses a pooling kernel with a size of 3×3 and a stride of 2 to downsample the high-dimensional feature map extracted by the residual block to remove redundant features and obtain an amplitude feature map.

[0032] The amplitude feature map output by P1 is denoted as FA1, which has a size of 1 / 2 of the SAR amplitude image and has 64 channels;

[0033] The amplitude feature map output by P2 is denoted as FA2, which has a size of 1 / 4 of the SAR amplitude image and has 128 channels;

[0034] The amplitude feature map output by P3 is denoted as FA3, which has a size of 1 / 8 of the SAR amplitude image and has 256 channels;

[0035] The amplitude feature map output by P4 is denoted as FA4, which has a size of 1 / 16 of the SAR amplitude image and has 512 channels;

[0036] After the above steps, multi-scale amplitude feature maps, namely FA1, FA2, FA3, and FA4, are obtained.

[0037] like Figure 2 As shown in Figure 2, the residual block includes a main branch and a shortcut connection branch, where the number of input channels of the first convolution layer of the main branch of R1 is 1, the number of output channels is 64, and the number of input channels and output channels of the second convolution layer are both 64; the number of input channels of the first convolution layer of the main branch of R2 is 64, the number of output channels is 128, and the number of input channels and output channels of the second convolution layer are both 128; the number of input channels of the first convolution layer of the main branch of R3 is 128, the number of output channels is 256, and the number of input channels and output channels of the second convolution layer are both 256; the number of input channels of the first convolution layer of the main branch of R4 is 256, the number of output channels is 512, and the number of input channels and output channels of the second convolution layer are both 512.

[0038] like Figure 3 As shown in the figure, the convolution kernel size of the main branch of the residual module is 3×3, the stride is 1, and the activation function is the ReLU function; the convolution kernel size of the shortcut connection is 1×1, and the stride is 1. The output of the residual module is obtained by adding the results of the main branch and the shortcut connection branch, and then calculating the activation function. P1, P2, P3, and P4 all use the maximum pooling method.

[0039] Step 3: Input the SAR phase image phase into the phase feature extraction module to extract the phase features and obtain multi-size phase features.

[0040] like Figure 2 As shown in Figure 1, the phase feature extraction module consists of four dilated convolutional layers (C1, C2, C3, C4) and four attention layers (A1, A2, A3, A4) connected in series. Each dilated convolutional layer performs dilated convolution, batch normalization, and ReLU activation on the input image.

[0041] After the SAR phase image is input to the phase feature extraction module, it first uses a dilated convolution layer with a kernel size of 3×3 to extract features. This layer captures contextual information at different ship positions and generates phase features related to elevation. The dilated convolution layer increases the receptive field, helping the model extract global features, suppress noise, and ensure global phase continuity.

[0042] The attention layer uses channel attention to assign weights to features across different channels, highlighting key channels and enhancing the model's feature extraction capabilities. The convolution kernel size of the four dilated convolutional layers is 3×3, with a stride of 2. This stride setting allows for downsampling of phase features, resulting in phase features of varying sizes. The dilation rates for C1, C2, C3, and C4 are 1, 2, 4, and 6, respectively.

[0043] The phase feature output by A1 is recorded as FP1, which has a size of 1 / 2 of the SAR phase image and has 64 channels;

[0044] The phase feature output by A2 is recorded as FP2, which is 1 / 4 of the SAR phase image and has 128 channels;

[0045] The phase feature output by A3 is recorded as FP3, which is 1 / 8 of the SAR phase image and has 256 channels;

[0046] The phase feature output by A4 is recorded as FP4, which has a size of 1 / 16 of the SAR phase image and has 512 channels. After the above steps, multi-size phase features, namely FP1, FP2, FP3, and FP4, are obtained.

[0047] In step 4, the multi-scale amplitude feature map and multi-scale phase feature obtained in steps 2 and 3 are input into the cross-scale feature fusion layer to perform cross-scale feature fusion to obtain FM.

[0048] The cross-scale feature fusion layer first linearly fuses the amplitude feature maps of different sizes with the phase features, so that the output multi-scale feature maps contain both local phase change details and retain the physical correlation characteristics of the global elevation difference, that is, FA1 and FP1 are linearly fused to obtain the linear fusion feature map F1, FA2 and FP2 are linearly fused to obtain the linear fusion feature map F2, FA3 and FP3 are linearly fused to obtain the linear fusion feature map F3, and FA4 and FP4 are linearly fused to obtain the linear fusion feature map F4; then F4, F3 and F2 are upsampled respectively, and the feature maps of different sizes are spatially aligned to obtain a feature map with the same size as F1, F4 is upsampled to obtain the spatially aligned linear fusion feature map FF4, F3 is upsampled to obtain the spatially aligned linear fusion feature map FF3, and F2 is upsampled to obtain the spatially aligned linear fusion feature map FF2; finally, F1 is channel-fused with FF4, FF3, and FF2 to obtain the result of the cross-scale feature fusion module, that is, the cross-scale fusion feature map FM.

[0049] In step 5, the feature map FM obtained in step 4 is input into the fully connected layer, and the ship height is further estimated through the activation function.

[0050] The superstructure of a ship is complex and the height of each position is different, so the height of the ship is represented by the ship height outline. Figure 2 The ship height profile is shown as a schematic diagram. The ship height profile consists of the ship height at each position. The ship height at each position refers to the height from the position to the waterline of the ship.

[0051] The feature map FM obtained in step 4 is input into the fully connected layer. First, FM is flattened and converted into a one-dimensional vector. Then, the elements of the one-dimensional vector are linearly combined and activated. Finally, the ship height is obtained.

[0052] Among them, the number of nodes in the fully connected layer is 1024, and the activation function is the classic SoftPlus function.

[0053] The calculation formula of the SoftPlus function is:

[0054] ;

[0055] h represents the height of the ship before the activation function is calculated.

[0056] like Figure 5 and Figure 6 As shown in the figure, the left side is the input SAR image, the right side is the ship height estimation result, the red solid line is the actual ship height, and the blue solid line is the ship height prediction value of the SAR2ShipHeight model. According to the experimental results, it can be seen that the shape of the ship height contour predicted by the SAR2ShipHeight model is very close to the actual value, and the details of the ship height contour are retained in certain locations with large height changes. The ship height in the middle of the ship is closest to the actual value, with a small error. The location where the ship height differs most from the actual value is the bow, where the contour shape varies greatly. The experimental results prove that the SAR2ShipHeight proposed in this invention can accurately estimate the ship height using a single SAR image.

[0057] The error and accuracy of the ship height estimation results are calculated using the evaluation index calculation method. Table 1 shows the ship height estimation accuracy of the ship height estimation model SAR2ShipHeight of the present invention and other deep learning algorithms.

[0058] Table 1 Ship height estimation accuracy of different algorithms

[0059]

[0060] It can be seen from Table 1 that the ship height estimation accuracy obtained by the ship height estimation model SAR2ShipHeight of the present invention is significantly higher than that obtained by using other deep learning algorithms.

[0061] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. The embodiments should therefore be considered illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be encompassed therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A method for estimating the height of a ship based on a single SAR image, characterized in that: include: Step 1: Process the acquired SAR images of different sea areas to obtain multiple SAR ship slices, calculate the amplitude and phase of each SAR ship slice, name the SAR amplitude image as amplitude, and name the SAR phase image as phase, thereby obtaining multiple SAR amplitude images and SAR phase images; Step 2: Input the SAR amplitude image amplitude into an amplitude feature extraction module, which includes multiple residual blocks and multiple pooling layers connected in series; the residual blocks are used to extract high-dimensional semantic features of the SAR amplitude image amplitude to obtain a high-dimensional feature map; the pooling layers are used to downsample the high-dimensional feature map to obtain multi-scale amplitude feature maps; Step 3: Input the SAR phase image phase into the phase feature extraction module for phase feature extraction. The phase feature extraction module includes multiple hole convolution layers and multiple attention layers connected in series. The hole convolution layer is used to capture contextual information at different positions in the SAR phase image phase and obtain phase features of different sizes related to elevation. The attention layer uses channel attention to adjust the weights of different channels of the phase feature, thereby obtaining multi-scale phase features; Step 4: Input the multi-scale amplitude feature map and multi-scale phase feature obtained in steps 2 and 3 into the cross-scale feature fusion layer to perform cross-scale feature fusion to obtain a feature map; The cross-scale feature fusion layer linearly fuses the amplitude feature maps of different sizes with the phase features of the corresponding sizes to obtain multiple linear fusion feature maps of different sizes; the linear fusion feature maps of multiple sizes other than the linear fusion feature map of the largest size are upsampled to obtain spatially aligned linear fusion feature maps, which are then channel-fused with the linear fusion feature map of the largest size to obtain a cross-scale fusion feature map; In step 5, the feature map obtained in step 4 is input into the fully connected layer and the ship height is obtained through activation function calculation.

2. The method for estimating the height of a ship based on a single SAR image according to claim 1, wherein: The height of the vessel at each location is the height from the location to the vessel's waterline.

3. The method for estimating the height of a ship based on a single SAR image according to claim 1, wherein: In step 1, the acquired SAR images of different sea areas are screened, images without ship targets are deleted, and the screened SAR images are cropped according to the locations of the ships to obtain multiple SAR ship slices. The amplitude and phase of each SAR ship slice are calculated respectively, and the calculation results are saved as SAR amplitude images and SAR phase images in TIFF format.

Images