A ship classification method based on feature-level fusion of HRRP and SAR data

Abstract

A ship classification method based on feature-level fusion of HRRP and SAR data includes the following steps: inputting HRRP and SAR detection data of the same target at the same time, preprocessing each of them, and dividing them into training and test sets; constructing a SAR image feature separation module to reduce the correlation between features between samples and increase the sample feature distance; constructing a SAR image feature aggregation module to aggregate similar features in the separated sample features, reduce the intra-class distance, increase the inter-class distance, and enhance classification performance; constructing a one-dimensional range image feature extraction module based on an attention mechanism to extract ship detail features from HRRP; constructing an HRRP and SAR data feature fusion classification module to fuse the features of HRRP and SAR data to classify the target; conducting supervised training on the established multi-source feature fusion classification model to obtain parameters suitable for the model; and sending the ship target data to be classified into the trained multi-source feature fusion classification model for classification to obtain classification results. The present invention improves the accuracy and robustness of ship classification.

Description

Technical Field

[0001] The present invention belongs to the field of radar technology, and in particular relates to a ship classification method based on feature-level fusion of HRRP and SAR data. Background Art

[0002] Synthetic aperture radar (SAR) and one-dimensional range profiles are both high-resolution radar data. SAR is an active Earth observation system with the advantages of all-day, all-weather coverage and a wide detection range. It can obtain high-resolution radar images similar to optical ones even in obscured conditions and low visibility. SAR images can reflect the geometric and scattering characteristics of targets, playing an important role in the identification and classification of civilian fishing vessels and military ships. One-dimensional range profiles are obtained by high-resolution radar. When the target size is much larger than the radar resolution unit, the radar echo from the target forms a one-dimensional range profile. Radar one-dimensional range profiles have the advantages of small data size, good real-time performance, easy processing, and strong anti-interference capabilities. One-dimensional range profiles reflect the geometric structural characteristics of the target in the range direction, including the target size and the location of the scattering center. One-dimensional range profiles are considered the most promising target recognition method and have become a research hotspot in recent years.

[0003] There are two main types of ship classification technologies for synthetic aperture radar imagery. One uses traditional methods to classify target ships, primarily extracting geometric features and then applying various machine learning classifiers such as support vector machines (SVMs) and logistic regression (LR) to complete the classification. The other type relies on deep learning classification methods. Deep learning utilizes nonlinear network structures to effectively extract features, eliminating the need for manually designed feature extraction methods and possessing excellent feature extraction and learning capabilities, thus enabling ship classification.

[0004] Ship classification using one-dimensional range profile data falls into two categories: one-dimensional range profile classification based on traditional methods and classification algorithms based on deep neural networks. Traditional one-dimensional range profile classification algorithms primarily consist of two parts: dimensionality reduction and transformation. Dimensionality reduction involves performing a dimensionality reduction mapping on the high-dimensional one-dimensional range profile signal to obtain classifiable features. Transformation projects the one-dimensional range profile signal into the frequency domain to extract spectral features for identification and classification. A one-dimensional range profile recognition network based on deep learning employs an end-to-end supervised learning approach to automatically extract separable features from sample one-dimensional range profile signals, addressing the shortcomings of traditional methods in sign extraction.

[0005] Synthetic aperture radar (SAR) images typically have poor image quality and significant speckle noise, and after filtering, ship details are significantly lost. One-dimensional range profiles contain more detailed ship information, but the issue of azimuth sensitivity remains unresolved. Consequently, ship classification accuracy using a single SAR image and one-dimensional range profile data is limited, resulting in limited stability and limited stability. Summary of the Invention

[0006] In order to overcome the shortcomings of the above-mentioned existing technologies, the purpose of the present invention is to provide a ship classification method based on the feature-level fusion of HRRP and SAR data, which improves the accuracy and robustness of ship classification and solves the problem that the existing technology only uses a single data source for ship classification, resulting in a bottleneck in accuracy and poor stability.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] A ship classification method based on HRRP and SAR data feature-level fusion includes the following steps:

[0009] S101: Acquire HRRP and SAR detection data of the same ship target at the same time, pre-process them respectively, and divide them into training set and test set;

[0010] S102: constructing a SAR image feature separation module to reduce the correlation between features of each detection data sample and increase the sample feature distance;

[0011] S103: Constructing a SAR image feature aggregation module to aggregate similar features in the separated sample features output by S102, thereby reducing the intra-class distance and increasing the inter-class distance, thereby enhancing the classification performance;

[0012] S104: Construct a one-dimensional range image feature extraction module based on the attention mechanism to extract ship detail features in HRRP;

[0013] S105: Constructing a HRRP and SAR data feature fusion classification module to fuse the features of HRRP and SAR data for target classification;

[0014] S106: Perform supervised training on the built multi-source feature fusion classification model to obtain parameters suitable for the model;

[0015] S107: Send the ship target data to be classified into the trained multi-source feature fusion classification model for classification to obtain a classification result.

[0016] S105 is the last module of the multi-source feature fusion model, which performs feature fusion classification. Classification is completed in this module. S106 is the training and optimization process of the model, and S107 uses the model to classify ships.

[0017] In step S101, the SAR image is subjected to pre-processing operations such as refined Lee filtering and data enhancement; and the HRRP data is normalized and scattering center information is extracted.

[0018] SAR images must first undergo refined Lee filtering to remove speckle noise. Refined Lee filtering is simulated using a neural network model based on an autoencoder architecture and adds a channel attention mechanism. Data augmentation is then performed to increase the amount of training data. The preprocessing steps can be expressed as follows:

[0019]

[0020] Lee(X)=X+f conv (Cat(f conv (X), f CBAM (f ReLU (f conv (X)))))

[0021] in are the input and preprocessed SAR images, H and W are the image sizes; Lee(·) is the refined Lee filter simulated by the network; f conv (·),f ReLU (·) are convolution and activation operations respectively; Cat(·) represents the concatenation of feature channel dimensions; f CBAM (·) represents the channel attention mechanism;

[0022] HRRP must first perform normalization and extract information related to the scattering center, extracting prior information such as the target radial length, the number of scattering centers, the profile skewness, and the variance. The steps can be expressed as follows:

[0023] h w =F enh (F norm (h))

[0024] h info =F info (h)

[0025]

[0026] where h=[h1,h2,...,h M ] represents the original HRRP data, M represents the total number of distance units contained in the HRRP data, h w is the HRRP data output by preprocessing, h info is the extracted HRRP scattering center information.

[0027] The step S102 is specifically as follows:

[0028] First, a pair of SAR images were enhanced using the ResNet50 network. Perform feature extraction to obtain features Processing in batches B is the batch size. A feature separation module is constructed to separate the features of different samples, that is, to project the features of different samples into a feature space with high separation. The feature separation module consists of a convolutional layer, an activation layer, and a batch normalization layer. This structure can effectively project the feature space. The steps can be expressed as follows:

[0029]

[0030]

[0031] Separate(·)=f ReLU (f BN (f conv (f ReLU (f BN (f conv (·))))))

[0032] in It is the SAR data after different data enhancement changes; is the feature extracted by ResNet50, where c, h, and w are the number of channels, length, and width of the feature, respectively. Separate(·) is a feature separation module that projects different sample features into a feature space with high separation.

[0033] The structure of the SAR image feature aggregation module in step S103 is as follows:

[0034] Constructing an integration module will reduce the dimension of the separated sample features and aggregate similar features at the same time. The feature integration module consists of two-step convolution. The first 1×1 convolution layer is used to reduce the number of features from the channel dimension, and the second convolution layer is used to fuse information from the spatial dimension with a 3×3 kernel. Through the integration module, the aggregated features are Indicates that a class code P is added to guide feature aggregation. Each class consists of multiple class codes P. The steps can be expressed as follows:

[0035]

[0036] Integration(·)=f ReLU (f BN (f conv (f ReLU (f BN (f conv (·))))))

[0037]

[0038] in is the separated sample feature; Integration(·) is the feature integration module; Represents the class encoding vector, C, N, and K represent the number of channels, the number of codes per class, and the number of classes, respectively.

[0039] The structure of the one-dimensional range profile feature extraction module in step S104 is as follows:

[0040] A VGG11 network with an attention mechanism is constructed to extract features from HRRP data. The attention mechanism uses a 1D convolutional channel attention module (CAM), which can effectively extract HRRP data features and can be expressed as follows:

[0041] h f =VGG11 CAM (h)

[0042] Where h represents the original HRRP data; VGG11 CBAM (·) represents the feature extraction network with channel attention mechanism; h f is the obtained one-dimensional range image feature.

[0043] In step S105, a HRRP and SAR data feature fusion classification module is constructed to fuse the features of HRRP and SAR data to classify the target;

[0044] The aggregated SAR features, the extracted HRRP features, and the HRRP prior information are fused to obtain the joint features, which are then used for classification. The structure is shown in the following formula:

[0045] z f =f flatten (Z af )

[0046]

[0047] f classifer (·)=f Linear (f ReLU (f Linear (f Dropout (·))))

[0048] Among them, Z af is the SAR image features aggregated by category; is the feature after flattening; is the classification result; f flatten (·),f classifer (·) are flattening operation and classification model respectively; f Dropout (·) is a random dropout layer with a dropout ratio of 0.2.

[0049] The S106 performs supervised training on the built multi-source feature fusion classification model to obtain parameters suitable for the model;

[0050] (1) Input the labeled training samples into the network model to be trained and output the label prediction of the training samples;

[0051] (2) Use the following intersection loss function to calculate the feature separation loss and aggregation loss function, and the loss function between the predicted label and the true label:

[0052] L=L Agg +L Sep +L Cls

[0053]

[0054]

[0055]

[0056] in is the characteristic between different samples; Class encoding for sample features and corresponding classes; are the predicted label and the true label respectively; Sep(a,b)=a·b / (||a||2·||b||2), Agg(a,b)=-a·b / (||a||2·||b||2) are the feature separation loss and aggregation loss functions respectively; CE(a,b) is the cross entropy loss function;

[0057] (3) Use the stochastic gradient descent method to train the network parameters until the network converges, and save the optimal network parameters to complete the classification of ships.

[0058] Beneficial effects of the present invention:

[0059] 1. The present invention first preprocesses HRRP and SAR data to eliminate irrelevant information in the data, recover useful real information, enhance the detectability of relevant information, and improve the reliability and accuracy of recognition. It also extracts prior information from HRRP data to accelerate the convergence of the model.

[0060] 2. The present invention adopts a strategy of first separating samples by features and then aggregating them by class in the SAR feature extraction module. This can alleviate the problem of small inter-class differences and large intra-class differences in ship detection. This improves the effectiveness of feature extraction and enhances the accuracy and robustness of ship classification.

[0061] 3. The present invention adopts the method of fusion of HRRP and SAR data feature level for classification, which fully combines the geometric information of the target in the SAR image and the detailed information of the target in the HRRP data, thereby improving the accuracy and robustness of ship classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of a ship classification method provided by an embodiment of the present invention.

[0063] Figure 2 This is a schematic diagram of the SAR preprocessing process provided by an embodiment of the present invention.

[0064] Figure 3 It is a schematic diagram of the structure of the SAR feature separation and aggregation module provided by an embodiment of the present invention.

[0065] Figure 4 It is a structural diagram of the HRRP feature extraction module provided by an embodiment of the present invention.

[0066] Figure 5 It is a schematic diagram of the overall structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The present invention will be further described in detail below with reference to the accompanying drawings.

[0068] The details of the implementation scheme of the present invention are as follows: a ship classification method based on the feature-level fusion of HRRP and SAR data. The details of the present invention are further explained below with reference to the accompanying drawings.

[0069] like Figure 1 As shown, the ship classification method based on HRRP and SAR data feature-level fusion provided by the present invention includes the following steps:

[0070] S101: Input HRRP and SAR detection data of the same target at the same time, pre-process them respectively, and divide them into training set and test set;

[0071] S102: Constructing a SAR image feature separation module to reduce the correlation between features of each sample and increase the distance between sample features;

[0072] S103: Constructing a SAR image feature aggregation module to aggregate similar features in the separated sample features output by S102, thereby reducing the intra-class distance and increasing the inter-class distance, thereby enhancing the classification performance;

[0073] S104: Construct a one-dimensional range image feature extraction module based on the attention mechanism to extract ship detail features in HRRP;

[0074] S105: Constructing a HRRP and SAR data feature fusion classification module to fuse the features of HRRP and SAR data to classify the target;

[0075] S106: Perform supervised training on the built multi-source feature fusion classification model to obtain parameters suitable for the model;

[0076] S107: Send the ship target data to be classified into the trained multi-source feature fusion classification model for classification, and obtain the classification result

[0077] like Figure 1 As shown, the ship classification method based on HRRP and SAR data feature-level fusion provided by the present invention has the following implementation process:

[0078] (1) Input the HRRP and SAR detection data of the same target at the same time, and perform preprocessing operations such as refined Lee filtering and data enhancement on the SAR image; normalize the HRRP data and extract the scattering center information.

[0079] (1a) To eliminate irrelevant information in the data and enhance the detectability of relevant information, HRRP and SAR data must be preprocessed separately. SAR images must first be subjected to refined Lee filtering to remove speckle noise. Refined Lee filtering is simulated using a neural network model based on an autoencoder architecture and with a channel attention mechanism. Data augmentation is then performed to increase the amount of training data. The preprocessing steps can be expressed as follows:

[0080]

[0081] Lee(X)=X+f conv (Cat(f conv (X), f CBAM (f ReLU (f conv (X)))))

[0082] in are the input and preprocessed SAR images, H and W are the image sizes; Lee(·) is the refined Lee filter simulated by the network; f conv (·),f ReLU (·) are convolution and activation operations respectively; Cat(·) represents the concatenation of feature channel dimensions; f CBAM (·) represents the channel attention mechanism;

[0083] (1b) HRRP first needs to perform normalization and extract information related to scattering centers, such as target radial length, number of scattering centers, profile skewness, and variance. The steps can be expressed as follows:

[0084] h w =F enh (F norm (h))

[0085] h info =F info (h)

[0086]

[0087] where h=[h1,h2,...,h M ] represents the original HRRP data, M represents the total number of distance units contained in the HRRP data, h w is the HRRP data output by preprocessing, h info is the extracted HRRP scattering center information.

[0088] (2) Construct a SAR image feature separation module to reduce the correlation between features of each sample and increase the sample feature distance.

[0089] First, a pair of SAR images were enhanced using the ResNet50 network. Perform feature extraction to obtain features Processing in batches B is the batch size. A feature separation module is constructed to separate features from different samples, projecting the features of different samples into a highly separable feature space. The feature separation module consists of a convolutional layer, an activation layer, and a batch normalization layer. This structure effectively projects the feature space, and its steps can be expressed as follows:

[0090]

[0091]

[0092]

[0093] in It is the SAR data after different data enhancement changes; is the feature extracted by ResNet50, where c, h, and w are the number of channels, length, and width of the feature, respectively. Separate(·) is a feature separation module that projects different sample features into a feature space with high separation.

[0094] (3) Construct a SAR image feature aggregation module to aggregate similar features in the separated sample features, reduce the intra-class distance, increase the inter-class distance, and enhance the classification performance.

[0095] (3a) Construct an integration module to reduce the dimension of the separated sample features and aggregate similar features at the same time. The feature integration module consists of two-step convolution. The first 1×1 convolution layer is used to reduce the number of features from the channel dimension. The second convolution layer is used to fuse information from the spatial dimension with a 3×3 kernel. Through the integration module, the network can better handle the potential relationship of the previously separated features, thereby accelerating the class feature aggregation process. The aggregated features are express.

[0096] (3b) In order to aggregate the features of each class, a class code P is added to guide feature aggregation. Each class has multiple class codes P. The steps can be expressed as follows:

[0097]

[0098] Integration(·)=f ReLU (f BN (f conv (f ReLU (f BN (f conv (·))))))

[0099]

[0100] in is the separated sample feature; Integration(·) is the feature integration module; Represents the class encoding vector, C, N, and K represent the number of channels, the number of codes per class, and the number of classes, respectively.

[0101] (4) Construct a one-dimensional range image feature extraction module based on the attention mechanism to extract the detailed features of ships in HRRP.

[0102] A VGG11 network with an attention mechanism is constructed to extract features from HRRP data. The attention mechanism uses a 1D convolutional channel attention module (CAM), which can effectively extract HRRP data features. It can be expressed as follows:

[0103] h f =VGG11 CAM (h)

[0104] Where h represents the original HRRP data; VGG11 CBAM (·) represents the feature extraction network with channel attention mechanism; h f is the obtained one-dimensional range image feature.

[0105] (5) Construct a HRRP and SAR data feature fusion classification module to fuse the features of HRRP and SAR data to classify the target.

[0106] The aggregated SAR features, the extracted HRRP features, and the HRRP prior information are fused to obtain the joint features, which are then used for classification. The structure is shown in the following formula:

[0107] z f =f flatten (Z af )

[0108]

[0109] f classifer (·)=f Linear (f ReLU (f Linear (f Dropout (·))))

[0110] Among them, Z af is the SAR image features aggregated by category; is the feature after flattening; is the classification result; f flatten (·),f classifer (·) are flattening operation and classification model respectively; f Dropout (·) is a random dropout layer with a dropout ratio of 0.2.

[0111] (6) Perform supervised training on the constructed multi-source feature fusion classification model to obtain parameters suitable for the model.

[0112] (6a) Input the labeled training samples into the network model to be trained and output the label prediction of the training samples;

[0113] (6b) Use the following intersection loss function to calculate the feature separation loss and aggregation loss function, and the loss function between the predicted label and the true label:

[0114] L=L Agg +L Sep +L Cls

[0115]

[0116]

[0117]

[0118] in is the characteristic between different samples; Class encoding for sample features and corresponding classes; are the predicted label and the true label respectively; Sep(a,b)=a·b / (||a||2·||b||2), Agg(a,b)=-a·b / (||a||2·||b||2) are the feature separation loss and aggregation loss functions respectively; CE(a,b) is the cross entropy loss function.

[0119] (6c) Use the stochastic gradient descent method to train the network parameters until the network converges, and save the optimal network parameters to complete the classification of ships.

[0120] In summary, the present invention implements a classification model based on the feature-level fusion of HRRP and SAR data for ship classification.

[0121] Those skilled in the art can make various corresponding changes and modifications based on the above technical solutions and concepts, and all of these changes and modifications should be included in the scope of protection of the claims of the present invention.

Claims

1. A ship classification method based on feature-level fusion of HRRP and SAR data, characterized in that: The following steps are included: S101: Acquire HRRP and SAR detection data of the same ship target at the same time, pre-process them respectively, and divide them into training set and test set; S102: constructing a SAR image feature separation module to reduce the correlation between features of each detection data sample and increase the sample feature distance; S103: Constructing a SAR image feature aggregation module to aggregate similar features in the separated sample features output by S102, thereby reducing the intra-class distance and increasing the inter-class distance, thereby enhancing the classification performance; S104: Construct a one-dimensional range image feature extraction module based on the attention mechanism to extract ship detail features in HRRP; S105: Constructing a HRRP and SAR data feature fusion classification module to fuse the features of HRRP and SAR data for target classification; S106: Perform supervised training on the built multi-source feature fusion classification model to obtain parameters suitable for the model; S107: sending the ship target data to be classified into the trained multi-source feature fusion classification model for classification to obtain a classification result; The step S102 is specifically as follows: First, a pair of SAR images were enhanced using the ResNet50 network. Perform feature extraction to obtain features Processing in batches B is the batch size. A feature separation module is constructed to separate the features of different samples, that is, to project the features of different samples into a feature space with high separation. The feature separation module consists of a convolutional layer, an activation layer, and a batch normalization layer. This structure can effectively project the feature space. The steps can be expressed as follows: Separate(·)=f ReLU (f BN (f conv (f ReLU (f BN (f conv (·)))))) in It is the SAR data after different data enhancement changes; is the feature extracted by ResNet50, where c, h, and w are the number of channels, length, and width of the feature, respectively. Separate(·) is a feature separation module that projects different sample features into a feature space with high separation. The structure of the SAR image feature aggregation module in step S103 is as follows: Constructing an integration module will reduce the dimension of the separated sample features and aggregate similar features at the same time. The feature integration module consists of two-step convolution. The first 1×1 convolution layer is used to reduce the number of features from the channel dimension, and the second convolution layer is used to fuse information from the spatial dimension with a 3×3 kernel. Through the integration module, the aggregated features are Indicates that a class code P is added to guide feature aggregation. Each class consists of multiple class codes P. The steps can be expressed as follows: Integration(·)=f ReLU (f BN (f conv (f ReLU (f BN (f conv (·)))))) in is the separated sample feature; Integration(·) is the feature integration module; Represents the class encoding vector, C, N, and K represent the number of channels, the number of codes per class, and the number of classes, respectively.

2. The ship classification method based on HRRP and SAR data feature-level fusion according to claim 1 is characterized in that: In step S101, the SAR image is subjected to pre-processing operations such as refined Lee filtering and data enhancement; the HRRP data is normalized and scattering center information is extracted; SAR images must first undergo refined Lee filtering to remove speckle noise. Refined Lee filtering is simulated using a neural network model based on an autoencoder architecture and adds a channel attention mechanism. Data augmentation is then performed to increase the amount of training data. The preprocessing steps can be expressed as follows: Lee(X)=X+f conv (Cat(f conv (X),f CBAM (f ReLU (f conv (X))))) in are the input and preprocessed SAR images, H and W are the image sizes; Lee(·) is the refined Lee filter simulated by the network; f conv (·),f ReLU (·) are convolution and activation operations respectively; Cat(·) represents the concatenation of feature channel dimensions; f CBAM (·) represents the channel attention mechanism; HRRP must first perform normalization and extract information related to the scattering center, extracting prior information such as the target radial length, the number of scattering centers, the profile skewness, and the variance. The steps can be expressed as follows: h w =F enh (F norm (h)) h info =F info (h) where h=[h1,h2,...,h M ] represents the original HRRP data, M represents the total number of distance units contained in the HRRP data, h w is the HRRP data output by preprocessing, h info is the extracted HRRP scattering center information.

3. The ship classification method based on HRRP and SAR data feature-level fusion according to claim 1 is characterized in that: The structure of the one-dimensional range profile feature extraction module in step S104 is as follows: A VGG11 network with an attention mechanism is constructed to extract features from HRRP data. The attention mechanism uses a 1D convolutional channel attention module (CAM), which can effectively extract HRRP data features and can be expressed as follows: h f =VGG11 CAM (h) Where h represents the original HRRP data; VGG11 CBAM (·) represents the feature extraction network with channel attention mechanism; h f is the obtained one-dimensional range image feature.

4. The ship classification method based on feature-level fusion of HRRP and SAR data according to claim 1, characterized in that: In step S105, a HRRP and SAR data feature fusion classification module is constructed to fuse the features of HRRP and SAR data to classify the target; The aggregated SAR features, the extracted HRRP features, and the HRRP prior information are fused to obtain the joint features, which are then used for classification. The structure is shown in the following formula: With f =f flatten (WITH af ) f classifer (·)=f Linear (f ReLU (f Linear (f Dropout (·)))) Among them, Z af is the SAR image features aggregated by category; is the feature after flattening; is the classification result; f flatten (·),f classifer (·) are flattening operation and classification model respectively; f Dropout (·) is a random dropout layer with a dropout ratio of 0.

2.

5. The ship classification method based on feature-level fusion of HRRP and SAR data according to claim 1 is characterized in that: The S106 performs supervised training on the built multi-source feature fusion classification model to obtain parameters suitable for the model; (1) Input the labeled training samples into the network model to be trained and output the label prediction of the training samples; (2) Use the following intersection loss function to calculate the feature separation loss and aggregation loss function, and the loss function between the predicted label and the true label: L=L Agg +L Sep +L Cls in is the characteristic between different samples; p k Class encoding for sample features and corresponding classes; are predicted labels and true labels respectively; Sep(a,b)=a·b / (||a||2·||b||2), Agg(a,b)=-a·b / (||a||2·||b||2) are feature separation loss and aggregation loss functions respectively; CE(a,b) is the cross entropy loss function; (3) Use the stochastic gradient descent method to train the network parameters until the network converges, and save the optimal network parameters to complete the classification of ships.

Images