A SAR image classification method based on deep multi-layer dictionary learning

Abstract

The present invention relates to the field of SAR remote sensing image recognition technology, and specifically provides a SAR image classification method based on deep multi-layer dictionary learning. The implementation of the method includes the following steps: Step 1, acquiring SAR image data; Step 2, partitioning and preprocessing the acquired SAR image dataset; Step 3, inputting the preprocessed image data into a basic network module to obtain underlying universal features; Step 4, inputting the obtained universal features into a class-specific network module of the corresponding category to obtain unique features for each class of data; Step 5, inputting the obtained features of each class of data into the corresponding dictionary learning network module to obtain reconstructed features after encoding and decoding; Step 6, jointly optimizing the proposed network through a reconstruction loss function, a discriminant loss function, and a regularization loss function.

Description

Technical Field

[0001] The present invention relates to the technical field of SAR remote sensing image recognition, and specifically provides a SAR image classification method based on deep multi-layer dictionary learning. Background Art

[0002] Image classification is one of the basic application directions of SAR image processing. At present, people have made many attempts to use deep convolutional neural networks for SAR image classification, and a large number of SAR image classification methods based on deep learning have emerged.

[0003] Sparse representation, or dictionary learning, was originally proposed in the field of signal processing. Its goal is to represent a set of signals using as few atoms as possible in an overcomplete dictionary. Given the strong representation capabilities of dictionary learning, many researchers have begun to introduce it into the field of image classification, with the DPL method being the most practical. Although the DPL method has achieved some success in SAR image classification, it still faces some challenges:

[0004] The DPL algorithm itself ignores the importance of learning image features. Directly using existing deep convolutional neural networks (DCNNs) to extract features is inconsistent with dictionary learning, resulting in limited improvement. Furthermore, traditional DPL algorithms use the alternating direction method of multipliers (ADMM) to optimize the DPL model, which is a relatively complex process. Summary of the Invention

[0005] To address these shortcomings, the present invention provides a SAR image classification method based on deep multi-layer dictionary pair learning. This method integrates deep feature learning with dictionary learning, proposing an end-to-end network architecture—a deep multi-layer dictionary pair learning network. The network is designed to simultaneously learn deep features and a discriminative dictionary. More specifically, the network incorporates multiple dictionary learning layers on top of the DCNN model to learn multiple synthetic and analytical dictionaries. Backpropagation is used to optimize the network, enabling joint learning of features and dictionaries, resulting in more discriminative dictionary pairs and effectively improving SAR image classification accuracy.

[0006] The technical solution adopted by the present invention to solve the technical problem is that the implementation of the method includes the following steps:

[0007] Step 1, obtaining SAR image data;

[0008] Step 2: dividing and preprocessing the SAR image dataset obtained in step 1;

[0009] Step 3: Input the preprocessed image data into the basic network module to obtain the underlying general features;

[0010] Step 4: Input the common features obtained in step 3 into the class-specific network module of the corresponding category to obtain the unique features of each type of data;

[0011] Step 5: Input the features of each type of data obtained in step 4 into the corresponding dictionary learning network module to obtain the reconstructed features after encoding and decoding;

[0012] Step 6, jointly optimize the proposed network through reconstruction loss function, discriminant loss function and regularization loss function;

[0013] Step 7: The test samples are fed into the trained network for each class and classified based on the minimum reconstruction error distance.

[0014] Step 8: Obtain the image recognition result.

[0015] As an optimization, step 2 includes the following steps:

[0016] Partition and preprocess the SAR image dataset: X = [X1, ..., X K ], where K is the number of categories of the image dataset, each image size can be processed as 224×224, and the image dataset includes: training set, validation set, and test set.

[0017] As an optimization, step 3 includes the following steps:

[0018] a. Set the data set X = [X1, ..., X K ] is input into the basic network module, and ResNet50 based on ImageNet pre-training is selected as the backbone network. The network is truncated at Conv4_x and four Non-Local modules and an AdaptiveAvgPool layer are added to the top layer. The final basic network is recorded as: ResNet50-NonLocal;

[0019] b. Fine-tune the basic network module ResNet50-NonLocal on the training set to generate a pre-trained model based on the data set.

[0020] As an optimization, step 4 includes the following steps:

[0021] c. Reorder the data set and sort the non-X k Reorder the image samples of this class to get X k The complementary matrix and

[0022] d. Input the samples in the training set into the pre-trained model obtained in step b, truncate them after the AdaptiveAvgPool layer, and then obtain the feature F of each class of samples.k , k = 1, 2, ..., K. Finally, construct F k The complementary matrix

[0023] e. The general feature F obtained in step d k and Input to the class-specific network module of the corresponding category, the class-specific network is a single FC layer or multiple FC layers, obtain the unique features of each type of data, and record them as Z k and described is the complementary matrix corresponding to the kth class.

[0024] As an optimization, the step 5 includes the following steps:

[0025] f. Construct a dictionary learning network module for each type of data: The dictionary learning network module contains a multi-layer synthetic dictionary and an analysis dictionary. The analysis dictionary is used to generate a sparse coding matrix of sample features and provide feature discriminability, and the synthetic dictionary is used to reconstruct the features of the sample after encoding. Use a fully connected linear layer without bias and nonlinear activation to represent the analysis dictionary and the synthetic dictionary respectively;

[0026] g. Input each type of features obtained in step e into the corresponding dictionary learning network module to obtain the reconstructed features after encoding and decoding: and

[0027] As an optimization, step 6 includes the following steps:

[0028] h. The reconstruction loss function L1, the discrimination loss function L2, and the regularization loss function L3 are constructed as follows:

[0029]

[0030]

[0031]

[0032] i. Z generated in step e k and Substitute into the following overall loss function and jointly optimize the proposed network:

[0033]

[0034] j. Use the back propagation algorithm to jointly learn the dictionary learning network module, class-specific network module and basic network module, guide the parameter update of the deep multi-layer dictionary learning network, and generate the deep multi-layer dictionary learning network model M for each class. k ,in

[0035] As an optimization, step 7 includes the following steps:

[0036] k. Input the verification data into the pre-trained model of each class generated in step j , we get the result after encoding and decoding:

[0037] l. In the test phase, input the test sample y. If y comes from the kth class, then after P k The generated sparse coding will be discriminative. By comparing the output results of each type of model, the model corresponding to the minimum value is taken as the classification result of the verification data.

[0038] The beneficial effects of the present invention are as follows: the present invention provides a SAR image classification method based on deep multi-layer dictionary pair learning, which obtains SAR image data; divides and preprocesses the SAR image data set; inputs the preprocessed image data into a basic network module to obtain underlying common features; inputs the obtained common features into a class-specific network module of the corresponding category to obtain features unique to each category of data; inputs the obtained features of each category of data into the corresponding dictionary learning network module to obtain reconstructed features after encoding and decoding; jointly optimizes the proposed network through a reconstruction loss function, a discriminant loss function, and a regularization loss function; introduces test samples into the trained network of each category and classifies them based on the minimum reconstruction error distance; and obtains recognition results. By adopting the technical solution of the present invention, joint learning of features and dictionaries is achieved, and more discriminative dictionary pairs are learned, thereby effectively improving the accuracy of SAR image classification and simplifying the optimization process. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart for implementing the method of the present invention;

[0040] Figure 2 This is the overall framework diagram of the network of the present invention;

[0041] Figure 3 This is a classification effect diagram of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0043] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0044] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer" and the like indicate positions or locations based on the positions shown in the accompanying drawings, or the positions or locations in which the inventive product is typically placed when in use. These terms are intended solely to facilitate the description of the present invention and to simplify the description, and are not intended to indicate or imply that the devices or components referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third," etc., are used solely to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0045] like Figure 1 A SAR image classification method based on deep multi-layer dictionary learning is shown, and the implementation of the method includes the following steps:

[0046] Step 1, obtaining SAR image data;

[0047] Step 2: dividing and preprocessing the SAR image dataset obtained in step 1;

[0048] Step 3: Input the preprocessed image data into the basic network module to obtain the underlying general features;

[0049] Step 4: Input the common features obtained in step 3 into the class-specific network module of the corresponding category to obtain the unique features of each type of data;

[0050] Step 5: Input the features of each type of data obtained in step 4 into the corresponding dictionary learning network module to obtain the reconstructed features after encoding and decoding;

[0051] Step 6, jointly optimize the proposed network through reconstruction loss function, discriminant loss function and regularization loss function;

[0052] Step 7: The test samples are fed into the trained network for each class and classified based on the minimum reconstruction error distance.

[0053] Step 8: Obtain the image recognition result.

[0054] In this embodiment, step 2 includes: dividing and preprocessing the SAR image data set: X=[X1, ..., X K ], where K is the number of categories in the image dataset, and each image can be processed to a size of 224×224, which is specifically divided into: training set, validation set, and test set.

[0055] In this embodiment, step 3 includes the following steps: a. transform the data set X=[X1, ..., X K ] is input to the base network module. For this base network module, we choose ResNet50, pre-trained on ImageNet, as the backbone network. This network is truncated at Conv4_x and four Non-Local modules and an AdaptiveAvgPool layer are added to the top layer. The final base network is denoted as ResNet50-NonLocal.

[0056] b. Fine-tune the above basic network module (ResNet50-NonLocal) on the training set to generate a pre-trained model based on the dataset.

[0057] In this embodiment, step 4 includes the following steps:

[0058] c. Reorder the data set and sort the non-X k The image samples of this class are reordered to obtain X k The complementary matrix and

[0059] d. Input the samples in the training set into the pre-trained model (i.e., the basic network module) obtained in step b, and truncate them after the AdaptiveAvgPool layer to obtain the feature F of each class of samples. k , k = 1, 2, ..., K. Finally, construct F k The complementary matrix

[0060] e. The general feature F obtained in step d k and Input to the class-specific network module of the corresponding category, where the class-specific network can be a single FC layer or multiple FC layers, to obtain the unique features of each type of data, and record them as Z k and Here represents the complementary matrix corresponding to the kth class.

[0061] In this embodiment, step 5 includes the following steps: f. Constructing a dictionary learning network module for each type of data: This module includes multiple layers of synthetic dictionaries and analysis dictionaries, where the analysis dictionary is used to generate a sparse coding matrix of sample features and provide feature discriminability, and the synthetic dictionary is used to reconstruct the features of the sample after encoding. Specifically, a fully connected linear layer without bias terms and nonlinear activation can be used to represent the analysis dictionary (encoder) and synthetic dictionary (decoder), respectively. The synthetic dictionary and the analysis dictionary are jointly learned to achieve the same idea as traditional DPL, and the dictionary pair used (a synthetic dictionary plus an analysis dictionary) can include multiple.

[0062] g. Input each type of features obtained in step e into the corresponding dictionary learning network module to obtain the reconstructed features after encoding and decoding: and

[0063] In this embodiment, step 6 includes the following steps:

[0064] h. The reconstruction loss function L1, the discrimination loss function L2, and the regularization loss function L3 are constructed as follows:

[0065]

[0066]

[0067]

[0068] i. Z generated in step e k and Substitute into the following overall loss function and jointly optimize the proposed network:

[0069]

[0070] j. Use the back propagation algorithm to jointly learn the dictionary learning network module, class-specific network module and basic network module, guide the parameter update of the deep multi-layer dictionary learning network, and generate the deep multi-layer dictionary learning network model M for each class. k ,in

[0071] In this embodiment, step 7 includes the following steps: k. Input the verification data into the pre-trained model of each class generated in step j. , we get the result after encoding and decoding:

[0072] l. After the k-th sample feature is encoded and decoded through the k-th dictionary, the reconstruction error is obtained will be small; while the reconstruction error obtained by encoding and decoding the features of other types of samples through the k-th dictionary Therefore, in the test phase, for the input test sample y, if y comes from the kth class, then after P k The resulting sparse coding will be discriminative, so the reconstruction error after the synthetic dictionary will be small; if y does not come from the kth class, then the reconstruction error will be large. By comparing the output results of each class model, the model corresponding to the minimum value is taken as the classification result of the verification data.

[0073]

[0074] The effects of the present invention are further described below in conjunction with simulation:

[0075] Simulation experiment conditions:

[0076] The simulation experiment conditions of the present invention are as follows: server GPU: NVIDIA Tesla T4 GPU, video memory 15G.

[0077] The software platform for the simulation experiment of the present invention is: Ubuntu 18.04 system, Python 3.6, and PyTorch-GPU 1.19.0.

[0078] Simulation content and experimental results analysis:

[0079] This simulation uses a multi-layer dictionary to perform SAR remote sensing image recognition on the existing ResNet50 and ResNet50-NonLocal methods. The simulation uses the following datasets: Landcover datasets, FUSAR-Ship, and TenGeoP-SARwv.

[0080] The Land Cover dataset consists of SAR image pairs from the Sentinel-1 satellite, provided by the Technical University of Munich. We searched this raw dataset for images taken in autumn and selected images belonging to four categories: wasteland, grassland, agricultural land, and urban areas. All images are 256×256 pixels in size.

[0081] The FUSAR-Ship dataset consists of the high-resolution GF-3SAR dataset, which contains more than 5,000 ship chips with AIS information, including 15 major ship categories divided into 98 subcategories, covering various ocean, land, coastal, river and island scenes.

[0082] The TenGeoP-SARwv dataset is derived from the 2016 Sentinel-1A WV. The dataset labels a subset of over 37,000 synthetic aperture radar images corresponding to 10 geophysical phenomena, covering the entire high ocean, including both oceanographic and meteorological features. This dataset is the first to address diverse oceanographic or atmospheric phenomena over the high ocean.

[0083] Simulation experiment 1 is the average classification results of ResNet50, DCNN (ResNet50-NonLocal) and the method of the present invention (DMDPL) on three data sets.

[0084] Simulation Experiment 2 compares ResNet50+DPL and DCNN(ResNet50-NonLocal)+DPL. ResNet50+DPL first uses ResNet50 to extract deep features, then classifies them using the traditional DPL algorithm; DCNN+DPL uses DCNN for feature learning. The final average classification results are shown in Table 2.

[0085] Table 1. Comparison of simulation results of the present invention on three data sets

[0086]

[0087] Table 2. Comparison of simulation results of the present invention on the Land cover dataset

[0088]

[0089]

[0090] From Table 1, we can see that compared with the baseline method, the method proposed in this invention achieves the best classification results on all data sets, indicating that dictionary learning can indeed improve the SAR image classification effect.

[0091] From Table 2, we can see that compared with the traditional classification method (i.e., learning features and dictionaries separately), the deep multi-layer dictionary joint learning method proposed in this invention can effectively improve the matching degree between features and classifiers, thereby improving classification performance.

[0092] The k-th class sample features generate a sparse coding matrix by analyzing the dictionary P, which is only for X kThe sample is sparsely coded, so the sample X is reconstructed by synthesizing the dictionary D k Feature F k Obtained will be small; on the other hand, the sample features F i Results obtained by analyzing and synthesizing dictionaries Therefore, in the test phase, for the input test sample y, if y comes from the kth class, then after P k The generated sparse code will be discriminative, so the reconstruction error after the synthetic dictionary will be small; if y comes from other classes, the reconstruction error will be large. By comparing the output results of each class model, the model corresponding to the minimum value is taken as the classification result of the verification data. Figure 3 It can be seen that in the recognition process, we can approximate a block diagonal matrix and take the block diagonal matrix part as the recognition result.

[0093] The above-mentioned specific embodiments are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the product form and style of the above-mentioned specific embodiments. Any appropriate changes or modifications made to them by ordinary technicians in the relevant technical field in accordance with the claims of the present invention shall fall within the patent protection scope of the present invention.

Claims

1. A SAR image classification method based on deep multi-layer dictionary learning, characterized in that: The implementation of the method includes the following steps: Step 1: Acquire SAR image data; Step 2: dividing and preprocessing the SAR image dataset obtained in step 1; Step 3: Input the preprocessed image data into the basic network module to obtain the underlying general features; Step 4: Input the common features obtained in step 3 into the class-specific network module of the corresponding category to obtain the unique features of each type of data; Step 5: Input the features of each type of data obtained in step 4 into the corresponding dictionary learning network module to obtain the reconstructed features after encoding and decoding; Step 6, jointly optimize the proposed network through reconstruction loss function, discriminant loss function and regularization loss function; Step 7: The test samples are fed into the trained network for each class and classified based on the minimum reconstruction error distance. Step 8, obtaining the image recognition result; The step 2 comprises the following steps: Divide and preprocess the SAR image data set: X=[X1,…,X K ], where K is the number of categories in the image dataset, each image is processed to a size of 224×224, and the image dataset is specifically divided into: training set, validation set, and test set; The step 3 comprises the following steps: a、Set the data set X=[X1,…,X K ] is input into the basic network module, and ResNet50 based on ImageNet pre-training is selected as the backbone network. The network is truncated at Conv4_x and four Non-Local modules and an AdaptiveAvgPool layer are added to the top layer. The final basic network is recorded as: ResNet50-NonLocal; b. Fine-tune the basic network module ResNet50-NonLocal on the training set to generate a pre-trained model based on the dataset; The step 4 comprises the following steps: c. Reorder the data set and sort the non-X k Reorder the image samples of this class to get X k The complementary matrix and d. Input the samples in the training set into the pre-trained model obtained in step 32, truncate them after the AdaptiveAvgPool layer, and then obtain the feature F of each class of samples. k ,k=1,2,…,K, finally, construct F k The complementary matrix e. The general feature F obtained in step d k and Input to the class-specific network module of the corresponding category, the class-specific network is a single FC layer or multiple FC layers, obtain the unique features of each type of data, and record them as Z k and described is the complementary matrix corresponding to the kth class.

2. The SAR image classification method based on deep multi-layer dictionary learning according to claim 1, characterized in that: The step 5 comprises the following steps: f. Construct a dictionary learning network module for each type of data: The dictionary learning network module contains a multi-layer synthetic dictionary and an analysis dictionary. The analysis dictionary is used to generate a sparse coding matrix of sample features and provide feature discriminability, and the synthetic dictionary is used to reconstruct the features of the sample after encoding. Use a fully connected linear layer without bias and nonlinear activation to represent the analysis dictionary and the synthetic dictionary respectively; g. Input each type of features obtained in step e into the corresponding dictionary learning network module to obtain the reconstructed features after encoding and decoding: and 3. The SAR image classification method based on deep multi-layer dictionary learning according to claim 2, characterized in that: The step 6 comprises the following steps: h. The reconstruction loss function L1, the discrimination loss function L2, and the regularization loss function L3 are constructed as follows: L1: L2: L3: i. Z generated in step e k and Substitute into the following overall loss function and jointly optimize the proposed network: j. Use the back propagation algorithm to jointly learn the dictionary learning network module, class-specific network module and basic network module, guide the parameter update of the deep multi-layer dictionary learning network, and generate the deep multi-layer dictionary learning network model M for each class. k ,in 4. The SAR image classification method based on deep multi-layer dictionary learning according to claim 3, characterized in that: The step 7 comprises the following steps: k. Input the verification data into the pre-trained model of each class generated in step j , we get the result after encoding and decoding: l. In the test phase, input the test sample y. If y comes from the kth class, then after P k The resulting sparse coding will be discriminative. By comparing the output results of each type of model, the model corresponding to the minimum value is taken as the classification result of the verification data.

Images