A ship classification method based on feature-level fusion of multi-source information

Abstract

The present invention discloses a ship classification method based on multi-source information feature-level fusion. First, the HRRP and SAR raw radar data of the ship target are obtained, and the HRRP raw radar data are preprocessed to generate HRRP samples in two formats. Secondly, feature extraction is performed on the HRRP samples in the first format to form traditional features. Then, a temporal network model branch is constructed, and the HRRP samples in the second format are used as input, and traditional features are embedded to output mixed features; a two-dimensional convolution model branch is constructed, and the SAR raw radar data is passed through a lightweight CNN network to extract the ship target structure features. Finally, a feature fusion module of the attention weighted mechanism is designed to weightedly fuse the mixed features with the ship target structure features, and output the ship classification result. The present invention solves the problem of limited recognition accuracy and performance of automatic target radar recognition based on single modal data, and improves the classification accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a ship classification method based on multi-source information feature-level fusion. Background Art

[0002] High Range Resolution Profile (HRRP) is the vector sum of the target scattering point sub-echoes projected in the direction of the radar ray obtained by broadband radar signals, which contains rich target structural features, such as target size, scatterer distribution, etc. Due to its easy acquisition, storage and processing characteristics, HRRP target recognition has received great attention from the radar target recognition community. HRRP target recognition methods have gradually evolved from traditional methods to data-driven deep learning methods. Although deep network models have achieved excellent results in some HRRP recognition studies, they are too dependent on sample data. When the sample data quality is poor, their recognition performance is even worse than that of traditional methods. Therefore, it is particularly important to fully combine the advantages of deep algorithms and traditional algorithms, integrate target mechanism characteristics on the basis of existing deep network models, and jointly drive data and knowledge to solve the limitations of current methods. Synthetic Aperture Radar (SAR) target recognition is the development of radar automatic target recognition technology. Since SAR has all-weather and long-range perception capabilities, effective feature extraction of target SAR images plays an important role in target detection and recognition. Faced with massive amounts of SAR data, there is an urgent need to identify SAR targets. The SAR target recognition problem has also received widespread attention at home and abroad. Current radar automatic target recognition usually only uses a single type of high-resolution radar signal. Although research in the field of single-mode recognition has made corresponding progress, the recognition accuracy and performance are limited. There are still major problems in the recognition of non-cooperative targets with a small number of incomplete samples. Therefore, it is of great significance to fuse data from different sources and fully explore their independent information for radar target recognition. Summary of the invention

[0003] The purpose of the present invention is to propose a ship classification method based on multi-source information feature-level fusion in view of the defects of the prior art. The method is based on deep learning, and different branch models are designed for two different modal data. The upper branch based on the HRRP sequence: the manually extracted mechanism model features are embedded into the deep time series network to extract the fused mixed features; the lower branch based on the SAR image: a lightweight convolutional neural network model with fewer parameters is constructed to extract the target structure features. A feature fusion module with an attention weighting mechanism is designed to weightedly fuse the feature vectors output by the two branches, and finally the fused features are used as the input of the classifier for ship target classification.

[0004] In order to achieve the above purpose, the present invention adopts the following technical solutions:

[0005] A ship classification method based on multi-source information feature-level fusion includes the following steps:

[0006] S101: Acquire the HRRP and SAR original radar data corresponding to the ship target and construct a ship target classification data set, wherein the original HRRP is preprocessed to finally generate HRRP samples in b×c1 format and b×c2 format.

[0007] S102: Feature extraction is performed on HRRP samples with a data format of b×c1, including features with physical meanings: strong scattering center features (number of scattering centers and peak width), high-order central moment features and radial energy features; features with mathematical meanings: mean, variance, average fluctuation characteristics of range images and differential fluctuation characteristics; and traditional process features.

[0008] S103: Construct a temporal network model branch based on HRRP, with a bidirectional long short-term memory network (BiLSTM) as the main framework, and HRRP with a data format of b×c2 as the model input. Design a feature fusion method to embed the traditional features extracted in S102 into the temporal network model branch, and finally output the fused mixed features that represent the target HRRP information;

[0009] S104: Construct a SAR-based two-dimensional convolutional model branch, design a lightweight CNN network, and pass the SAR raw radar data through the lightweight CNN network to extract the ship target structure features.

[0010] S105: Design a feature fusion module with an attention weighting mechanism to weightedly fuse the mixed features extracted by the HRRP-based temporal network model with the ship target structure features extracted by the SAR-based two-dimensional convolutional model, complete the construction of the final dual-channel fusion network model, realize the fusion recognition of ship targets based on HRRP and SAR data, and output the ship classification results.

[0011] The specific steps of preprocessing the acquired data in S101 are as follows:

[0012] (1) The L2 norm normalization method is used to normalize the intensity of the HRRP raw radar data. The specific implementation is as follows:

[0013]

[0014] Among them, X src =[x1,x2,...,x N ] represents the HRRP raw radar data, N is the total number of range cells in a single HRRP sample, xn For X src The value of the nth distance unit in the sample, X norm ∈R 1×N is the normalized HRRP radar data.

[0015] (2) The normalized HRRP radar data is aligned using the center of gravity alignment method in the absolute alignment method. The center of gravity of the HRRP is translated to the vicinity of the HRRP center point. The center of gravity alignment method requires first obtaining X norm The center of gravity g norm , the calculation formula is as follows:

[0016]

[0017] where x′ n For X norm The value of the nth distance unit in the sample, N is the total number of distance units in a single HRRP sample, and then g is calculated according to the following formulas norm Start position P after alignment begin and end position P end :

[0018]

[0019] in, It means rounding down. The HRRP data after processing by the center of gravity alignment method is:

[0020] X pre =X norm [P begin ,...,N]||X norm [0,1,...,P end ]

[0021] Among them, ‖ represents the concatenation of two vectors, X norm [P begin ,...,N] represents the norm Extracted from P begin Data in the N index range to the end of the data, X norm [0,1,...,P end ] indicates that from X norm Extract data from the starting position 0 to P end Index range data, X pre ∈R 1×N HRRP radar data after intensity normalization and center of gravity alignment.

[0022] (3) The aligned HRRP is time-series segmented through different sliding windows c1 and c2 to obtain samples in b×c1 and b×c2 formats respectively. The specific formula for time-series segmentation is as follows:

[0023]

[0024] Among them, N max Indicates the maximum number of subsequences, L src Indicates the original sequence length, L win Indicates the window size, and S indicates the sliding step size. When performing time series segmentation, it also includes expanding the data using cubic spline interpolation and then performing time series segmentation.

[0025] The feature extraction of the pre-processed HRRP sequence in S102 is specifically as follows:

[0026] Referring to the traditional feature extraction method, for the HRRP samples with a data dimension of 84×256 after preprocessing, feature extraction is performed on each subsequence with a dimension of 256. The selected features are: mean (f m ), variance(f v ), the number of scattering centers (f p ), peak width characteristics (f w ), average fluctuation characteristics (f f ), differential fluctuation characteristics (f d ), radial energy characteristics (f e ), each order central moment (f ck , k represents the order);

[0027] N is the total number of distance units in a single original HRRP sample. The sliding window is set as described in S101(3). The total number of distance units of the obtained subsequence is also N. Let {x″(n)|n=1,2,...N} be the single HRRP subsequence after preprocessing. The calculation method of the above features is as follows:

[0028] (1) Mean feature, calculated as follows:

[0029]

[0030] (2) Variance characteristics, the calculation formula is as follows:

[0031]

[0032] (3) Number of scattering centers. Since the deck and island building of a ship are its main components, the number of strong scattering centers will basically not change regardless of how the attitude angle changes. Therefore, the number of scattering centers is attitude-insensitive. In addition, the structure of the ship's superstructure is relatively complex, and its HRRP presents multiple peaks. In particular, the number of peaks, i.e., the number of scattering centers, can reflect the structural information of the ship target. The calculation formula is as follows:

[0033] f p=length{n|x″(n)≥λ1X max ,x″(n)>x″(n-1),x″(n)>x″(n+1)},n=1,2,...,N

[0034] Among them, λ1 is the threshold coefficient, X max It represents the maximum value of the HRRP subsequence, and length represents the number of n that satisfy the above conditions.

[0035] (4) Peak width characteristics, calculated as follows:

[0036] f w =length{n|x″(n)≥λ2X max}

[0037] Among them, λ2 is the threshold coefficient, X max It represents the maximum value of the HRRP subsequence, and length represents the number of n that satisfy the above conditions.

[0038] (5) Average fluctuation characteristics. The fluctuation characteristics of the one-dimensional range image of the ship target reflect the complexity of the superstructure structure of the ship at sea. A smaller fluctuation characteristic indicates that the ship structure is simple and there is no large fluctuation in the structure. A larger fluctuation characteristic indicates that the ship structure is complex and the superstructure fluctuates greatly. The calculation formula is as follows:

[0039]

[0040] (6) Differential fluctuation characteristics, calculated as follows:

[0041]

[0042] (7) The radial energy characteristic reflects the scattering capability of the ship target. The calculation formula is as follows:

[0043]

[0044] (8) The central moment feature is translation invariant and scale invariant. The second-order and higher-order central moments compensate the translation component of HRRP through the first-order moment and are independent of the translation of the range image and are insensitive. The calculation formula of the k-order central moment feature is as follows:

[0045]

[0046] The specific steps of constructing the temporal network model branch based on HRRP single-modal data in S103 are as follows:

[0047] The HRRP sample data in the format of b×c2 is used as the input sequence of the temporal network model branch; the input sequence is automatically extracted into deep features through two layers of BiLSTM, the automatically extracted deep features are concatenated with the traditional features, and then the concatenated vector is input into the two layers of BiLSTM again, the two features are fused, the forward and reverse hidden states of the last time step of the second layer of BiLSTM are extracted, and then the fused mixed features are output through a fully connected layer.

[0048] The construction of the two-dimensional convolution model branch based on SAR single-mode data described in S104 is specifically implemented as follows:

[0049] The two-dimensional convolution model branch is composed of several convolution units in cascade, and finally the three-dimensional tensor is flattened into a one-dimensional vector using the Flatten operation to obtain the ship target structure features;

[0050] Each convolution unit consists of a convolution layer and a pooling layer, and after each convolution layer, there is a rectified linear unit ReLU as an activation function and a batch normalization layer.

[0051] The specific implementation steps of the feature fusion module in S105 are as follows:

[0052] (1) The HRRP sequence mixed feature x extracted in S103 and the ship target structure feature y of the SAR image extracted in S104 are sent to the fully connected layer respectively, and the activation function ReLU is used to generate the feature and

[0053]

[0054] in and have the same dimension d, W X and W Y is the weight parameter matrix of the fully connected layer; the fully connected layer not only further maps the two features into the global space, but also makes the dimension and order contain a consistent correspondence, that is, The i-th feature in There is a one-to-one correspondence between the i-th features in;

[0055] (2) The mixed feature x and the ship target structure feature y are concatenated into a long feature vector, and then the attention vector α∈R is learned through a fully connected layer. d×1 :

[0056] α=sigmoid(W α [x,y])

[0057] Among them, the dimension of α is also d, Wα represents the parameters of the fully connected layer; sigmoid(·) represents the sigmoid activation operation, which makes the value of the attention vector α in the range of [0,1]; by using the fully connected layer to perceive all information, determine the focus area, and then pay more attention to the focus features while suppressing other useless information;

[0058] (3) Using the attention vector α as the weight and Sum up; define α as the feature The weights of , and the features are obtained through 1-α operation The weight of the attention vector α and the feature Perform element-by-element multiplication to multiply the vector 1-α by the feature Perform element-by-element multiplication, and then use the element-by-element summation operation to perform weighted summation on the two features to obtain the fused feature F:

[0059]

[0060] in, Represents element-by-element multiplication operation, where 1 is a vector whose elements are all 1.

[0061] Finally, the fusion feature F is passed through the Softmax classifier to output the ship classification result.

[0062] The present invention has the following beneficial effects:

[0063] 1. The present invention uses L2 normalization and center of gravity alignment method to preprocess the HRRP data, which effectively reduces the amplitude sensitivity and translation sensitivity of the HRRP data and improves the generalization ability and classification accuracy of the deep learning model.

[0064] 2. The present invention proposes a dual-driven model of deep learning and traditional mechanisms, which fully integrates the deep features automatically extracted by the deep network with the traditional manual features, solves the problems of slow model convergence and poor recognition accuracy under poor sample data quality and small sample problems, and improves the model robustness and interpretability.

[0065] 3. The present invention uses a lightweight convolutional neural network to model SAR images. The model has fewer parameters but can still achieve performance similar to other large models, reducing computational complexity and improving model recognition efficiency.

[0066] 4. The present invention proposes a feature fusion module with an attention weighting mechanism, which effectively integrates the independent information contained in different radar data of the target, solves the problem of limited recognition accuracy and performance based on single modal data in the field of automatic target radar recognition, and improves the classification accuracy and system stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a flow chart of a ship classification method based on multi-source information feature-level fusion provided by an embodiment;

[0068] Figure 2 It is a branch structure diagram of a temporal network model based on HRRP data provided by an embodiment;

[0069] Figure 3 It is a structural diagram of the feature fusion module provided in the embodiment. DETAILED DESCRIPTION

[0070] The following further describes the embodiments and effects of the present invention in conjunction with the accompanying drawings. A ship classification method based on multi-source information feature-level fusion, such as Figure 1 As shown, the following steps are included:

[0071] Step 1: Data preprocessing.

[0072] The HRRP and SAR original radar data corresponding to the ship target at the same time are obtained. Since the quality of the HRRP original sample data is poor, the HRRP data is mainly preprocessed.

[0073] (1) The L2 norm normalization method is used to normalize the HRRP data intensity, which is specifically implemented as follows:

[0074]

[0075] Among them, X src =[x1,x2,...,x N ] represents the original HRRP signal, N is the total number of distance units in a single HRRP sample, x n For X src The value of the nth distance unit in the sample, X norm ∈R 1×N is the normalized HRRP signal.

[0076] (2) The center of gravity alignment method in the absolute alignment method is used to align the HRRP data and translate the center of gravity of the HRRP to the vicinity of the HRRP center point. The center of gravity alignment method requires first obtaining X norm The center of gravity g norm , the calculation formula is as follows:

[0077]

[0078] where x′ n For X norm The value of the nth distance unit in the sample, N is the total number of distance units in a single HRRP sample, and then g is calculated according to the following formulas norm Start position P after alignmentbegin and end position P end :

[0079]

[0080]

[0081] in, It means rounding down. The HRRP data after processing by the center of gravity alignment method is:

[0082] X pre =X norm [P begin ,...,N]||X norm [0,1,...,P end ]

[0083] Among them, ‖ represents the concatenation of two vectors, X norm [P begin ,...,N] represents the norm Extracted from P begin Data in the N index range to the end of the data, X norm [0,1,...,P end ] indicates that from X norm Extract data from the starting position 0 to P end Index range data, X pre ∈R 1×N HRRP radar data after intensity normalization and center of gravity alignment.

[0084] (3) The processed HRRP is segmented into time series through a sliding window to better meet the model input requirements and improve the model performance. The reference formula is as follows:

[0085]

[0086] Among them, N max Indicates the maximum number of subsequences, L src Indicates the original sequence length, L winIndicates the window size, and S indicates the sliding step size. According to the formula, the window size is set to 7 and the sliding step size is set to 3. Then, the original HRRP one-dimensional sequence with a total number of distance units of 256 can be transformed into 84 7-dimensional subsequences, that is, the single sample data format is 84×7 to meet the input format of the subsequent time series deep network; at the same time, the original data with a length of 256 is expanded to 2746 distance units using cubic spline interpolation. According to the formula, the window size is set to 256 and the sliding step size is set to 30, and 84 subsequences with a length of 256 can be obtained, that is, the single sample data format is 84×256. Each subsequence with a dimension of 256 will replace the original HRRP sequence for subsequent feature extraction, resulting in the features manually extracted from a single sample being two-dimensional vectors in the format of 84×H, where H represents the dimension of the manually extracted features.

[0087] Step 2: Extract features from the preprocessed HRRP subsequences.

[0088] Referring to the traditional feature extraction method, for the HRRP samples with a data dimension of 84×256 after preprocessing, feature extraction is performed on each subsequence; the selected features are: mean (f m ), variance(f v ), the number of scattering centers (f p ), peak width characteristics (f w ), average fluctuation characteristics (f f ), differential fluctuation characteristics (f d ), radial energy characteristics (f e ), each order central moment (f ck , k represents the order);

[0089] N is the total number of distance units in a single original HRRP sample. According to the above sliding window parameter settings, the total number of distance units in each subsequence is also N. Let {x″(n)|n=1,2,...N} be a single HRRP subsequence after preprocessing. The calculation method of the above features is as follows:

[0090] (1) Mean feature, calculated as follows:

[0091]

[0092] (2) Variance characteristics, the calculation formula is as follows:

[0093]

[0094] (3) Number of scattering centers. Since the deck and island building of a ship are its main components, the number of strong scattering centers will basically not change regardless of how the attitude angle changes. Therefore, the number of scattering centers is attitude-insensitive. In addition, the structure of the ship's superstructure is relatively complex, and its HRRP presents multiple peaks. In particular, the number of peaks, i.e., the number of scattering centers, can reflect the structural information of the ship target. The calculation formula is as follows:

[0095] f p =length{n|x″(n)≥λ1X max ,x″(n)>x″(n-1),x″(n)>x″(n+1)},n=1,2,...,N

[0096] Among them, λ1 is the threshold coefficient and takes 0.5, X max It represents the maximum value of the HRRP subsequence, and length represents the number of n that satisfy the above conditions.

[0097] (4) Peak width characteristics, calculated as follows:

[0098] f w =length{n|x″(n)≥λ2X max}

[0099] Among them, λ2 is the threshold coefficient and takes 0.5, X max It represents the maximum value of the HRRP subsequence, and length represents the number of n that satisfy the above conditions.

[0100] (5) Average fluctuation characteristics. The fluctuation characteristics of the one-dimensional range image of the ship target reflect the complexity of the superstructure structure of the ship at sea. A smaller fluctuation characteristic indicates that the ship structure is simple and there is no large fluctuation in the structure. A larger fluctuation characteristic indicates that the ship structure is complex and the superstructure fluctuates greatly. The calculation formula is as follows:

[0101]

[0102] (6) Differential fluctuation characteristics, calculated as follows:

[0103]

[0104] (7) The radial energy characteristic reflects the scattering capability of the ship target. The calculation formula is as follows:

[0105]

[0106] (8) The central moment feature is translation invariant and scale invariant. The second-order and higher-order central moments compensate the translation component of HRRP through the first-order moment and are independent of the translation of the range image and are insensitive. The calculation formula of the k-order central moment feature is as follows:

[0107]

[0108] Step 3: Build and train the temporal network model branch based on HRRP data to determine the model-related parameters.

[0109] A classification model based on manual features and BiLSTM is constructed to perform supervised training on the HRRP dataset. The training set and test set are divided into two sets in a ratio of 6:4. Then, a certain number of samples are divided from the training set as the validation set for training to determine the optimal parameters of the HRRP single-modal recognition framework model.

[0110] The overall framework of the model branch is as follows Figure 2 As shown in the figure, the HRRP data with the data format of (batch_size, 84, 7) after sliding window dimension transformation is used as the input of BiLSTM network; the core of ordinary unidirectional LSTM is the internal three gate structures, namely forget gate, input gate and output gate; t represents the hidden state inside the LSTM unit at the current time t, and the input x at time t t and the previous hidden state h t-1 Together determine the forgetting unit F at the current moment t , Input Unit I t , Candidate state C t ' and output unit O t :

[0111] F t =σ(W fh h t-1 +W fx x t +b f )

[0112] I t =σ(W ih h t-1 +W ix x t +b i )

[0113] C t '=tanh(W ch h t-1 +W cx x t +b c )

[0114] O t =σ(W oh h t-1 +W ox x t +b o )

[0115] Where W fh , W ih , W ch , W oh They are the forget gate, input gate, candidate state, output gate and the hidden state h at the previous moment. t-1 The multiplied weight matrix; W fx , W ix , W cx , W ox They are the forget gate, input gate, candidate state, and output gate and the input x at the current time t. t The multiplied weight matrix; b f 、b i 、b c 、b o are the bias terms of the forget gate, input gate, candidate state, and output gate respectively; σ(·) represents the sigmoid function; the cell state C in the memory unit at the current moment t and the hidden state h t The calculation formula is as follows:

[0116]

[0117] in, Indicates the multiplication of the corresponding elements of the matrix; for BiLSTM, each time step has two hidden states, one from the forward LSTM and the other from the backward LSTM. The final output of the bidirectional LSTM is the forward hidden state h ft and the backward hidden state h lt The bidirectional LSTM considers the sequence information from both the front-to-backward and the back-to-front directions at the same time, so that the output at each time step can combine the entire context information and more effectively capture the symmetry of the front and back information in the HRRP sequence.

[0118] The input data of the format (batch_size, 84, 7) passes through two layers of BiLSTM, and the output format of all time steps is (batch_size, 84, hidden_size × num_directions), where the size of hidden_size is determined by the number of hidden neurons in the gate, and is also a hyperparameter to be debugged. num_directions represents the number of directions. Here, it is a bidirectional LSTM, so it is 2. The manually extracted features of the format (batch_size, 84, N) are input to the fully connected layer for feature dimension transformation, where N represents the dimension of the manual features, and the output format is (batch_size, 84, fc_output_size), where fc_output_size is a hyperparameter to be debugged.

[0119] The output of all the above time steps is concatenated with the manual features after dimension transformation, and then the concatenated vector is input into the two-layer BiLSTM. The two features are fused with the help of the BiLSTM network model. Finally, the forward and reverse hidden states of the last time step of the second layer are extracted and merged to obtain features in the format of (batch_size, hidden_size×2). Finally, it is mapped into the desired dimension through a fully connected layer and the fused mixed feature vector is output. The training method is implemented as follows:

[0120] (3a) A portion of the training set samples is divided into a validation set with a training-validation ratio of 3:1, and cross-validation is started.

[0121] (3b) Initialize the BiLSTM network parameters.

[0122] (3c) Input the HRRP data of the training sample set and the manually extracted features, forward propagate through each BiLSTM layer, and obtain the output value.

[0123] (3d) The cross entropy loss function is used to calculate the error between the network output value and the target value.

[0124] (3e) Adam optimizer is used to update network parameters.

[0125] (3f) Repeat (3b) to (3e) until the error between the network output value and the target value is basically stable.

[0126] (3g) Input the validation set data, obtain the classification results, and record the classification accuracy and model parameters of this training.

[0127] (3h) Repeat (3a) to (3g) and retain the model parameters with the best classification accuracy.

[0128] Step 4: Build and train a two-dimensional convolutional model branch based on SAR images to determine model-related parameters.

[0129] A CNN-based classification model is constructed to perform supervised training on the SAR image dataset. The training set and the test set are divided into a ratio of 6:4. A certain number of samples are then divided from the training set as a validation set for training to determine the optimal parameters of the SAR single-modal recognition framework model. The model is mainly composed of convolutional layers and pooling layers. After each convolutional layer, there is a rectifier linear unit (ReLU) as an activation function, and a batch normalization layer to enable the network to use a higher learning rate. The specific structure and parameters are shown in Table 1 below:

[0130] Table 1 Two-dimensional convolutional network model architecture of the present invention

[0131]

[0132] To achieve lightweight, the entire branch model has only 5 convolutional layers. The convolution kernel size of the first convolutional layer is 11×11. A relatively large convolution kernel is used to obtain a larger receptive field and improve the feature representation capability. In the next three convolutional layers, the convolution kernel size is 5×5, and there is a downsampling operation after the first three convolutional layers to extract significant features, reduce network parameters, and make the model lighter. Considering that the fully connected layer parameters used to convert feature maps into feature vectors at the final position in the network are relatively large, a convolutional layer with a convolution kernel size of 3×3 is finally used to replace the fully connected layer, in which the number of output channels is set to 100 and the stride is 1, so that it generates features with a dimension of 1×1×100 from the feature map, and finally the Flatten operation is used to flatten the three-dimensional tensor into a one-dimensional vector. The training method is implemented as follows:

[0133] (4a) A portion of the training set samples is divided into a validation set with a training-validation ratio of 3:1, and cross-validation is started.

[0134] (4b) Initialize the CNN network parameters.

[0135] (4c) The SAR data of the input training sample set is forward propagated through each convolutional layer to obtain the output value.

[0136] (4d) The cross entropy loss function is used to calculate the error between the network output value and the target value.

[0137] (4e) Adam optimizer is used to update network parameters.

[0138] (4f) Repeat (4b) to (4e) until the error between the network output value and the target value is basically stable.

[0139] (4g) Input the validation set data, obtain the classification results, and record the classification accuracy and model parameters of this training.

[0140] (4h) Repeat (4a) to (4g) and retain the model parameters with the best classification accuracy.

[0141] Step 5: Construct a feature fusion module to perform attention-weighted fusion of two features from different model branches. The specific structure is as follows: Figure 3 shown.

[0142] (5.1) The target mixed feature x extracted from the HRRP branch and the target structural feature y extracted from the SAR branch are sent to the fully connected layer respectively, and the activation function ReLU is used to generate features and

[0143]

[0144] in and have the same dimension d, W X and W Y is the weight parameter matrix of the fully connected layer; the fully connected layer not only further maps the two features into the global space, but also makes the dimensions and order contain consistent correspondence, that is, the i-th feature in X~ and There is a one-to-one correspondence between the i-th features in;

[0145] (5.2) Concatenate the mixed feature x and the target structure feature y into a long feature vector, and then learn the attention vector α∈R through a fully connected layer d×1 :

[0146] α=sigmoid(W α [x,y])

[0147] Among them, the dimension of α is also d, W α represents the parameters of the fully connected layer; sigmoid(·) represents the sigmoid activation operation, which makes the value of the attention vector α in the range of [0,1]; by using the fully connected layer to perceive all information, determine the focus area, and then pay more attention to the focus features while suppressing other useless information;

[0148] (5.3) Using the attention vector α as the weight and Sum up; define α as the feature The weights of , and the features are obtained through 1-α operation The weight of the attention vector α and the feature Perform element-by-element multiplication to multiply the vector 1-α by the feature Perform element-by-element multiplication, and then use the element-by-element summation operation to perform weighted summation on the two features to obtain the fused feature F:

[0149]

[0150] in, represents the element-by-element multiplication operation, and 1 is a vector whose elements are all 1; finally, F is input into the Softmax classifier for classification, thereby realizing the fusion recognition of ship targets based on HRRP and SAR data; define f fusion Represents the entire fusion module, with parameter ψ fusion , then the fusion module is summarized as follows:

[0151] F=f fusion (x,y;ψfusion )

[0152] The effect of the present invention can be further illustrated by the following experiments.

[0153] 1. Experimental conditions:

[0154] The present invention conducts experiments on a simulation data set, which includes: HRRP sequences and SAR images corresponding to three categories of ship targets, with a total of 500 groups of data. In each group of data, HRRP and SAR correspond one to one, and the number of samples for each type of target is balanced. The HRRP data is in csv format with a dimension of 256; the SAR data is in png format with an image size of 128×128 pixels.

[0155] The experimental hardware platform is Core i7-8700k CPU 3.10GHz, NVIDIA GTX 3080 (11G) GPU and 16-GB RAM;

[0156] The experimental software platform is Python 3.6 and Pytorch.

[0157] 2. Experimental content and results analysis:

[0158] Based on the overall framework model and training method of the present invention, the recognition accuracy of each branch model and the overall two-stream network model is obtained by training, and an ablation control experiment is performed at the same time. When one or more specific components of the network are removed or replaced, the other settings remain the same to observe the impact of each component on the performance. In the quantitative analysis, the overall accuracy and average accuracy are used as the criteria for evaluating the target recognition performance. The experimental results are shown in Table 2 below:

[0159] Table 2 Ablation comparison experimental results

[0160]

[0161]

[0162] In Table 2, Addition represents the element-by-element addition of the fusion operation, and Concat represents the splicing fusion operation. These two fusion modules are usually used in the multi-stream network architecture. It can be seen from the first three rows in the table that the overall accuracy of the recognition accuracy using only HRRP data through the BiLSTM model is 86.75%, and the average accuracy is 83.67%. The two indicators of the BiLSTM that integrates manual features in the present invention are higher than those of the single BiLSTM model. In addition, the overall accuracy of the lightweight CNN of the present invention using only SAR image recognition is 89.64%, and the average accuracy is 90.27%. In the present invention, the two branch models for different data can extract different features from different domains, and both have achieved good recognition performance. However, by comparing rows 1, 2, and 3 with rows 4, 5, and 6, it can be seen that the fusion of the potential features of the HRRP data obtained by BiLSTM and the structural features of the SAR image obtained by CNN is conducive to more comprehensive and sufficient reflection of the target information, thereby obtaining better recognition performance. In addition, based on the fusion of BiLSTM and CNN, the performance improvement brought by different fusion modules is different. The overall accuracy of the Addition module is 91.73%, and the average accuracy is 92.23%; the overall accuracy of the Concat module is 91.07%, and the average accuracy is 92.67%; the overall accuracy of the fusion module of the present invention is 93.57%, and the average accuracy is 94.40%. The recognition accuracy is significantly better than that of other modules. The superiority of the present invention is effectively verified through comparison.

[0163] In summary, the ship classification method based on the feature-level fusion of HRRP and SAR data proposed in the present invention can reflect the target information more comprehensively and fully, thereby having better recognition performance, effectively improving the accuracy of fusion classification based on HRRP and SAR data, and has practical application value.

Claims

1. A ship classification method based on multi-source information feature-level fusion, characterized in that: The following steps are involved: S101: Acquire HRRP and SAR raw radar data of the ship target, construct a data set, and preprocess the HRRP raw radar data to generate HRRP samples in b×c1 format and b×c2 format; S102: performing feature extraction on the HRRP sample in the format of b×c1, and obtaining features with physical meanings and mathematical meanings respectively, which constitute traditional features; S103: Construct a temporal network model branch based on HRRP, using a bidirectional long short-term memory network BiLSTM as a framework, using HRRP samples in a data format of b×c2 as input, and embedding traditional features into the temporal network model branch, outputting a hybrid feature that represents the target HRRP information after fusion; the specific implementation process is as follows: The HRRP sample data in the format of b×c2 is used as the input sequence of the temporal network model branch; the input sequence is automatically extracted into deep features through two layers of BiLSTM, the automatically extracted deep features are concatenated with the traditional features, and then the concatenated vector is input into the two layers of BiLSTM again, the two features are fused, the forward and reverse hidden states of the last time step of the second layer of BiLSTM are extracted, and then the fused mixed features are output through a fully connected layer; S104: Construct a SAR-based two-dimensional convolution model branch, pass the SAR raw radar data through a lightweight CNN network, and extract the ship target structure features; the specific implementation process is as follows: The two-dimensional convolution model branch is composed of several convolution units in cascade, and finally the three-dimensional tensor is flattened into a one-dimensional vector using the Flatten operation to obtain the ship target structure features; Each convolution unit consists of a convolution layer and a pooling layer, and after each convolution layer, there is a rectified linear unit ReLU as an activation function and a batch normalization layer; S105: Design a feature fusion module with an attention weighting mechanism to weightedly fuse the mixed features with the ship target structure features and output the ship classification results.

2. The ship classification method based on multi-source information feature-level fusion according to claim 1 is characterized in that: The specific process of the pretreatment is as follows: S1011: Use L2 norm normalization to normalize the intensity of the HRRP raw radar data, which is implemented as follows: Among them, X src =[x1,x2,...,x N ] represents the HRRP raw radar data, N is the total number of range cells in a single HRRP sample, x n For X src The value of the nth distance unit in the sample, X norm ∈R 1×N is the normalized HRRP radar data; S1012: Use the center of gravity alignment method in the absolute alignment method to align the normalized HRRP radar data, translate the center of gravity of the HRRP to the center point of the HRRP, and use the center of gravity alignment method to first obtain X norm The center of gravity g norm , the formula is as follows: where x′ n For X norm The value of the nth distance unit in the sample is calculated according to the following formulas: norm Start position P after alignment begin and end position P end : in, It means rounding down. The HRRP data after processing by the center of gravity alignment method is: X pre =X norm [P begin ,...,N]||X norm [0,1,...,P end ]; Among them, ‖ represents the concatenation of two vectors, X norm [P begin ,...,N] represents the norm Extracted from P begin Data in the N index range to the end of the data, X norm [0,1,...,P end ] indicates that from X norm Extract data from the starting position 0 to P end Index range data, X pre ∈R 1×N is the HRRP radar data after intensity normalization and center of gravity alignment; S1013: performing time-series segmentation on the aligned HRRP using different sliding windows c1 and c2, respectively, to obtain samples in b×c1 format and b×c2 format; The specific formula for time series segmentation is as follows: Among them, N max Indicates the maximum number of subsequences, L src Indicates the original sequence length, L win represents the window size, and S represents the sliding step size.

3. The ship classification method based on multi-source information feature-level fusion according to claim 2 is characterized in that: When performing time series segmentation in S1013, it also includes expanding the data by cubic spline interpolation and then performing time series segmentation.

4. The ship classification method based on multi-source information feature-level fusion according to claim 3 is characterized in that: The characteristics with physical meanings include: strong scattering center characteristics, high-order central moment characteristics and radial energy characteristics; The strong scattering center characteristics include the number of scattering centers and the peak width; The features with mathematical meanings include: mean, variance, average fluctuation feature of range image and differential fluctuation feature.

5. The ship classification method based on multi-source information feature-level fusion according to claim 4 is characterized in that: Let the mean value f m , variance f v , the number of scattering centers f p , peak width characteristic f w , average fluctuation characteristics f f , differential fluctuation characteristics f d , radial energy characteristics f e , each order central moment f ck , k represents the order; N is the total number of distance units in a single original HRRP sample. The sliding window is set as described in S1013, and the total number of distance units of the obtained subsequence is also N. Let {x″(n)|n=1,2,...N} be a single HRRP subsequence after preprocessing. The calculation of the above features is specifically as follows: The mean feature is calculated as follows: Variance feature, the calculation formula is as follows: The number of scattering centers is calculated as follows: f p =length{n|x″(n)≥λ1X max ,x″(n)>x″(n-1),x″(n)>x″(n+1)},n=1,2,...,N; Among them, λ1 is the threshold coefficient, X max represents the maximum value of the HRRP subsequence, and length represents the number of n that satisfy the above conditions; Peak width characteristics, calculated as follows: f w =length{n|x″(n)≥λ2X max }; Among them, λ2 is the threshold coefficient; The average fluctuation characteristics are calculated as follows: The differential fluctuation characteristics are calculated as follows: The radial energy characteristic reflects the scattering capability of the ship target. The calculation formula is as follows: The central moment feature is translation invariant and scale invariant. The second-order and higher-order central moments compensate the translation component of HRRP through the first-order moment. The calculation formula of the k-order central moment feature is as follows:

6. The ship classification method based on multi-source information feature-level fusion according to claim 5 is characterized in that: The specific implementation process of the feature fusion module is as follows: S1051: Send the mixed feature x and the ship target structure feature y to the fully connected layer respectively, and use the activation function ReLU to generate features and in and have the same dimension d, W X and W Y is the weight parameter matrix of the fully connected layer; S1052: Concatenate the mixed feature x and the ship target structure feature y into a feature vector, and then learn the attention vector α∈R through a fully connected layer d×1 : α=sigmoid(W α ·[x,y]); Among them, the dimension of α is also d, W α represents the parameters of the fully connected layer; sigmoid(·) represents the sigmoid activation operation, which makes the value of the attention vector α in the range of [0,1]; S1053: Define α as a feature The weights of , and the features are obtained through 1-α operation The weight of the attention vector α and the feature Perform element-by-element multiplication to multiply the vector 1-α by the feature Perform element-by-element multiplication and then use element-by-element summation to obtain the fused feature F: in, Represents element-by-element multiplication operation, 1 is a vector whose elements are all 1; Finally, the fusion feature F is passed through the Softmax classifier to output the ship classification result.

Images