Image classification method based on multi-manifold joint metric learning

Abstract

This invention discloses an image classification method based on multi-manifold joint metric learning, comprising the following steps: 1) Data preprocessing, including constructing Grassmann features and SPD features; 2) Constructing dual-manifold kernel matrices, including constructing Grassmann manifold kernel matrices and SPD manifold kernel matrices, and calculating the test kernel matrices of Grassmann and SPD respectively; 3) Implementing multi-manifold joint metric learning, extracting kernel feature vectors from the kernel matrices, calculating the intra-class scatter matrix and inter-class scatter matrix; solving the projection matrix, sorting the feature vectors according to the size of the eigenvalues, and selecting the feature vectors corresponding to the largest eigenvalues ​​to form the projection matrix; 4) Feature fusion, completing the extraction, and implementing sample mapping; 5) Constructing a sample-level fusion convolutional neural network to complete the classification of different land features in the original image. This invention belongs to the field of image processing and machine learning technology and can significantly improve classification accuracy.

Description

Technical Field

[0001] This invention belongs to the field of image processing and machine learning technology, and relates to an image classification method based on multi-manifold joint metric learning. Background Technology

[0002] Image classification is one of the fundamental tasks in computer vision and machine learning. In image classification, traditional Euclidean space feature representation methods often fail to adequately capture the intrinsic geometric structure of image data. Riemannian manifolds, topological spaces with nonlinear geometric structures, can better characterize the intrinsic relationships and geometric properties of high-dimensional image data by mapping them to manifold spaces. In particular, symmetric positive definite SPD matrix manifolds and Grassmann manifolds, as two important Riemannian manifolds, have been proven to effectively represent the covariance structure and subspace features of images. SPD manifolds model image sets through covariance matrices, capturing the second-order statistical properties and global correlations of image data. Grassmann manifolds model image sets through linear subspaces, offering advantages such as low computational complexity and strong discriminative power. However, these methods based on a single manifold primarily focus on subspace features, with limited utilization of covariance information. To overcome the limitations of single-manifold representation, some researchers have proposed manifold dimensionality reduction methods, attempting to learn more discriminative low-dimensional representations while preserving the geometric properties of the manifold. However, these methods perform linear transformations on nonlinear manifolds, inevitably leading to suboptimal results, and the optimization process is time-consuming during the training phase. In recent years, deep learning has made breakthroughs in image recognition. Its multi-layer nonlinear learning mechanism can automatically learn more expressive feature representations. To extend the advantages of deep learning to manifold spaces, researchers have proposed manifold deep networks. Although these networks improve classification accuracy through multi-stage nonlinear transformations, they still suffer from high computational overhead and a tendency to overfit on finite datasets.

[0003] To integrate complementary features from multiple manifolds, researchers have attempted to model images using different Riemannian manifolds. Lu et al. proposed the Localized Multi-Kernel Metric Learning (LMKML) method, which uses multi-order statistical features to model image sets. However, because it uses the same kernel function to map different manifolds to the Hilbert space, it ignores the differences between manifolds. Huang et al. proposed the Hybrid Euclidean-and-Riemannian Metric Learning (HERML) method, which combines multiple heterogeneous statistical features such as mean, covariance, and Gaussian distribution, and designs a metric learning algorithm to fuse them into a common subspace. Wang et al. proposed the Multiple Manifolds Metric Learning (MMML) method, which uses both SPD and Grassmann manifolds to model image sets, and maps the original manifolds to a high-dimensional Hilbert space using a Riemannian kernel function for metric learning. However, these multi-manifold fusion methods still have the following shortcomings: (1) Kernel function mapping changes the distribution of the original data in high-dimensional space, inevitably distorting the geometric structure; (2) The construction of the kernel matrix and the calculation of the scatter matrix bring high computational complexity and low efficiency on large-scale datasets; (3) A large amount of redundant information is introduced in the heterogeneous feature fusion process, all feature mappings are treated equally, and there is a lack of selective attention to important features.

[0004] In summary, existing manifold methods suffer from the following shortcomings: 1) Single-manifold-based methods have limited representational capabilities; the SPD manifold method underutilizes subspace features, and the Grassmann manifold method has limited ability to fuse covariance information; 2) Traditional kernel methods and manifold dimensionality reduction methods struggle to preserve the geometric structure of the original data, and linear mapping on nonlinear manifolds leads to suboptimal classification performance; 3) Existing multi-manifold fusion methods suffer from high computational complexity, inappropriate heterogeneous feature fusion strategies, and the introduction of redundant information reduces the model's generalization ability; 4) Existing methods often employ traditional classifiers, making it difficult to fully utilize the representation learning advantages of deep learning. To address these issues, there is an urgent need to develop an image classification method that can simultaneously utilize multiple manifold geometric structures and combine the advantages of deep learning. Summary of the Invention

[0005] The purpose of this invention is to provide an image classification method based on multi-manifold joint metric learning, which solves the problems of insufficient single-manifold representation, poor feature fusion effect, and low classification accuracy in the prior art; high computational complexity of multi-manifold fusion methods and inappropriate heterogeneous feature fusion strategies.

[0006] The technical solution adopted in this invention is an image classification method based on multi-manifold joint metric learning, implemented according to the following steps:

[0007] Step 1, data preprocessing, including constructing Grassmann features and SPD features;

[0008] Step 2: Construct the dual manifold kernel matrix, including constructing the Grassmann manifold kernel matrix and the SPD manifold kernel matrix, and calculating the test kernel matrices of Grassmann and SPD respectively;

[0009] Step 3, implement multi-manifold joint metric learning, specifically:

[0010] Extracting kernel eigenvectors from the kernel matrix Calculate the intra-class scatter matrix and the inter-class scatter matrix; calculate the projection matrix. V Solve by adjusting the values ​​of the eigenvalues. U Sort the eigenvectors in the matrix, and select the eigenvectors corresponding to the largest eigenvalues ​​to form the projection matrix. V ;

[0011] Step 4: Feature fusion, completion of extraction, and implementation of sample mapping;

[0012] Step 5: Construct a sample-level fusion convolutional neural network to classify different land features in the original image.

[0013] The beneficial effects of this invention are as follows: it employs unified multi-manifold modeling, utilizing the geometric structures of both the SPD manifold and the Grassmann manifold to fully mine the covariance information and subspace features of the image; it learns the optimal manifold metric projection by optimizing the intra-class and inter-class scatter matrices, thereby improving feature discrimination ability; it combines manifold learning with convolutional neural networks to fully utilize the representation learning capabilities of CNNs; furthermore, the method of this invention uses kernel matrix methods and feature fusion strategies to effectively reduce computational complexity and improve classification accuracy. Experiments show that compared with single-manifold methods, the method of this invention can significantly improve classification accuracy. Attached Figure Description

[0014] Figure 1 This is a flowchart of the method of the present invention;

[0015] Figure 2 This is a false color image of the fully polarimetric SAR of the Xian region in Embodiment 1 of the present invention;

[0016] Figure 3 This is the result obtained using the existing HPD+CNN method;

[0017] Figure 4 This is a screenshot of the result obtained using the existing HybridCVnet method;

[0018] Figure 5 This is a diagram showing the effect obtained by the method of the present invention;

[0019] Figure 6 This is a full polarimetric SAR pseudocolor image of the Flevoland region from Embodiment 2 of the present invention;

[0020] Figure 7 yes Figure 6 Corresponding class label reference diagram;

[0021] Figure 8 This is the result obtained using the existing HPD+CNN method;

[0022] Figure 9 This is a diagram showing the effect obtained by the method of the present invention;

[0023] Figure 10 It is the confusion matrix of the existing HybridCVnet method for classifying fully polarimetric SAR images of the Xian area;

[0024] Figure 11 This is the confusion matrix for classifying fully polarimetric SAR images of the Xian area using the method of this invention. Detailed Implementation

[0025] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0026] Reference Figure 1 The image classification method of the present invention is implemented according to the following steps:

[0027] Step 1, data preprocessing, the specific process is as follows:

[0028] 1.1) Constructing Grassmann features: Acquire polarimetric SAR images to obtain a 512×512×57-dimensional polarimetric SAR feature matrix. Randomly select 500 pixels by category as the training set and select pixels for the test set at a ratio of 0.5% of the total data. For each pixel in the image, extract a 5×5 spatial neighborhood window centered on that pixel. Since each pixel contains 57-dimensional polarimetric feature information, this spatial neighborhood window constitutes a 5×5×57=1425-dimensional feature, which is a high-dimensional feature here.

[0029] Then, the constructed 1425-dimensional features are subjected to dimensionality reduction: using principal component analysis (PCA), the high-dimensional features are compressed into a 400-dimensional low-dimensional feature representation through the PCA transformation matrix obtained from training, which effectively reduces the computational burden while maintaining the main information.

[0030] Next, 25 samples are collected in the 5×5 neighborhood to form a 400×25 matrix X, which is then subjected to singular value decomposition. We take the 10 largest eigenvectors to form a 400×10 orthogonal basis matrix;

[0031] 1.2) Constructing SPD Features: Obtain a 512×512 image size HPD feature matrix with a matrix dimension of 3×3. Transform the HPD feature matrix into a real SPD matrix using a "complex-to-real" embedding method. For a complex HPD matrix, the expression is: C = A + Bi, where A is the real part matrix and B is the imaginary part matrix. The transformation expression is:

[0032]

[0033] The transformation yields a 6×6 real symmetric matrix. This is the SPD matrix;

[0034] Then, select the same pixels as those in the aforementioned Grassmann features according to the category.

[0035] Thus, step 1 yielded 3000 Grassmann features of 400×10 and SPD feature samples of 6×6.

[0036] Step 2, construct the dual-manifold kernel matrix, the specific process is as follows:

[0037] 2.1) Construct the Grassmann manifold kernel matrix, as shown in the following expression:

[0038]

[0039] in, Represents the Grassmann manifold kernel matrix. This represents the trace operation. F It is the Frobenius norm (F norm). X i For the first i Grassmann subspace matrix of training samples X j For the first j The Grassmann subspace matrix of each training sample (as mentioned in step 1, 500 pixels are randomly selected by category as the training set and pixels of the test set are selected at a ratio of 0.5% of the total data). T Indicates transpose;

[0040] 2.2) Construct the SPD manifold kernel matrix, as shown in the following expression:

[0041]

[0042] in, SPD manifold kernel matrix, S i For the first i The SPD matrix of each training sample. S j For the first j SPD matrix of training samples;

[0043] 2.3) Calculate the test kernel matrices for Grassmann and SPD respectively:

[0044] The cross-set kernel matrix is ​​constructed separately for the test samples and the training samples, as explained below:

[0045] Construct the Grassmann test kernel matrix, expressed as:

[0046]

[0047] in, X i For the first i Grassmann subspace matrix of training samples X t For the first t Grassmann subspace matrix of each test sample;

[0048] Construct the SPD test kernel matrix, the expression is:

[0049]

[0050] in, S i For the first i The SPD matrix of each training sample. S t For the first t SPD matrix of each test sample;

[0051] Step 2 yielded the SPD training and testing kernel matrices, as well as the Grassmann training and testing kernel matrices.

[0052] Step 3, implement multi-manifold joint metric learning, the specific process is as follows:

[0053] 3.1) Extract kernel eigenvectors from the kernel matrix. The i List ,extract The i List

[0054] 3.1) Calculate the intra-class scatter matrix S w Inter-class scatter matrixS b The expression is as follows:

[0055] Intraclass scatter matrix S w The expression is as follows:

[0056]

[0057] Inter-class scatter matrix S b The expression is as follows:

[0058]

[0059] in, To measure the dispersion of similar samples in the kernel feature space. To measure the dispersion of different classes of samples in the kernel feature space. These are Grassmann manifold weights. For SPD manifold weights, W Represents the set of indexes for similar sample pairs; T Indicates transpose. B For all heterogeneous sample pairs, use an index set; i For the first i Grassmann kernel feature vectors of training samples j For the first j Grassmann kernel feature vectors of training samples; s i For the first i SPD kernel feature vectors of training samples, s j For the first j SPD kernel feature vectors of training samples;

[0060] 3.2) Projection matrix V The solution is obtained through generalized eigenvalue decomposition, expressed as follows:

[0061]

[0062] in, U The eigenvector matrix, D For the eigenvalue matrix, S W Represents the within-class scatter matrix. S b Represents the inter-class scatter matrix. eig (.) represents the eigenvalue decomposition function;

[0063] Based on the magnitude of the eigenvalueU Sort the feature vectors in the data and select the top ones. d The eigenvectors corresponding to the largest eigenvalues ​​constitute the projection matrix. V The expression is:

[0064]

[0065] in, N train This represents the number of training samples.

[0066] Thus, step 3 finally yields the projection matrix. V .

[0067] Step 4, feature fusion, completion of extraction, and sample mapping, the specific process is as follows:

[0068] 4.1) Perform feature fusion on the training set obtained in step 1. For the first... i The expression for the fused features of each training sample is as follows:

[0069]

[0070] in, For the first i The fusion features of each training sample For the first i Grassmann kernel feature vectors of each sample; For the first i SPD kernel feature vector of each sample, V The projection matrix obtained in step 3;

[0071] 4.2) Perform feature fusion on the test set obtained in step 1. For the first... j The expression for the fusion feature of each test sample is as follows:

[0072]

[0073] in, For the first j The fusion features of each test sample Indicates the first j Grassmann kernel feature vectors of each test sample Indicates the first j SPD kernel feature vectors of each test sample;

[0074] 4.3) Implement the mapping of sample-level fusion features and construct a label-based fusion feature dictionary, expressed as:

[0075]

[0076] in,c For category indexing, M The total number of categories, D c For the first c Class fusion feature dictionary For the first i The fusion features of each training sample y i For the first i The category label of each sample.

[0077] From this point on, step 4 yields the fused features of the training and test sets.

[0078] Step 5, construct the sample-level fusion convolutional neural network, the specific process is as follows:

[0079] 5.1) Configure the network architecture.

[0080] A dual-path fusion architecture is adopted, including a feature extraction branch for the original image and a feature fusion branch, expressed as:

[0081]

[0082] in, This is the concatenated composite feature vector. Image features extracted by CNN To integrate manifold features;

[0083] 5.2) Input the concatenated composite feature vector obtained above into a multilayer fully connected neural network (multilayer fully connected neural networks are existing technology) for classification, and output the classification results. The expression is:

[0084]

[0085] in, y This is the classification output vector, where the dimension corresponding to the maximum value is the predicted class. Softmax (.) is the normalized activation function, used to output the probability distribution; ReLU (.) is the activation function; Dropout (.)for Dropout A random deactivation function is used to prevent overfitting. f concat This is the concatenated composite feature vector; W 2, W 3 represents the weight matrix of a two-layer fully connected neural network. b 2, b 3 represents the bias term for the corresponding layer.

[0086] In step 5, the sample-level fusion CNN processes the original image and multi-manifold fusion features simultaneously through a dual-path structure: the convolutional branch extracts 256-dimensional spatial-spectral features from the 9×9 image patch, and the manifold branch uses step 5 to generate 8-dimensional fusion features (containing discriminative information from the SPD covariance and Grassmann subspace); the two features are concatenated into a 264-dimensional vector in a fully connected layer, and then processed... Dropout Regularization and Softmax The layer outputs category probabilities, enabling accurate classification of different land features such as water, grassland, and buildings in the original image. It organically combines the local details of the original image with the global geometric structure of the manifold space, significantly improving classification accuracy.

[0087] Example 1

[0088] I. Classification criteria are as follows:

[0089] 1) In the classification experiment, 512×512×57-dimensional polarimetric SAR image data of a certain area of ​​Xian were selected. After preprocessing, suitable Grassmann manifold features were formed; and a 512×512 SPD feature matrix was obtained.

[0090] 2) In the classification experiment, the polarimetric SAR image data was sampled in layers, with 500 training samples selected for each class, and the test set was selected at a ratio of 0.5% to ensure class balance;

[0091] 3) In the classification experiment, the dimension of the target subspace for multi-manifold metric learning was set to 8, and the weight parameters were... =0.7, =0.3;

[0092] II. Following the steps outlined above in this invention, the following procedures shall be performed:

[0093] Step 1, data preprocessing, including constructing Grassmann features and SPD features;

[0094] Step 2: Construct the dual manifold kernel matrix, including constructing the Grassmann manifold kernel matrix and the SPD manifold kernel matrix, and calculating the test kernel matrices for Grassmann and SPD respectively;

[0095] Step 3, implement multi-manifold joint metric learning, specifically:

[0096] Extracting kernel eigenvectors from the kernel matrix

[0097] Calculate the intra-class scatter matrix S w Inter-class scatter matrix S b ;

[0098] For projection matrix V Solve by adjusting the values ​​of the eigenvalues. U Sort the feature vectors in the data and select the top ones. d The eigenvectors corresponding to the largest eigenvalues ​​constitute the projection matrix. V ;

[0099] Step 4: Feature fusion, completion of extraction, and implementation of sample mapping;

[0100] Step 5: Construct a sample-level fusion convolutional neural network to classify different land features in the original image.

[0101] III. The classification content and results analysis are as follows:

[0102] Table 1 compares the results of the method of the present invention with other methods in the prior art.

[0103]

[0104] Figure 2 A pseudo-color map of the fully polarimetric SAR in the Xian region; Figure 3 The classification results of the existing HPD+CNN method on the fully polarimetric SAR image of the Xian area are shown, with a classification accuracy of 94.99%. Figure 4 The classification result obtained by the existing HybridCVnet method has a classification accuracy of 94.56%. Figure 5 This is the classification result of the fully polarimetric SAR image of the Xian area using the method of this invention, with a classification accuracy of 98.87%. It can be seen that... Figure 4 It will produce a lot of noise and cannot effectively suppress noise, while according to Figure 5 The method of this invention, as shown, yields more consistent classification results and achieves better performance. In the classification results, different land cover categories such as ocean, forest, and buildings are accurately identified, with clear boundaries between categories. The classification accuracy reaches 98.87%; the average accuracy is 99.06%; and the Kappa coefficient is 98.14%.

[0105] Example 2

[0106] I. Classification Criteria

[0107] 1) In the classification experiment, 300×270×57-dimensional polarimetric SAR image data of NASA / JPL AIRSAR satellite L-band in Flevoland, Netherlands were input, and after preprocessing, suitable Grassmann manifold features were formed; 300×270 SPD feature matrix was obtained.

[0108] 2) In the classification experiment, the polarimetric SAR image data was sampled in layers, with 500 training samples selected for each class, and the test set was selected at a ratio of 0.5% to ensure class balance;

[0109] 3) In the classification experiment, the target subspace dimension of the multi-manifold metric learning was set to 8, and the weight parameters were set to... =0.7, =0.3.

[0110] II. The steps and processes described above in this invention are as follows, with specific steps and processes referred to in Example 1.

[0111] III. The classification content and results are as follows:

[0112] Figure 6 This is a pseudocolor SAR image of the Flevoland region with full polarization, represented using the Pauli base as the RGB three-channel color representation. Figure 7 for Figure 6 The corresponding class label reference diagram shows that the white area has no reference class label. Therefore, this invention does not consider the classification results of the white area. In the reference diagram, the Flevoland region is divided into 6 categories. Figure 8 The image shows the classification results obtained by the existing HPD+CNN method, with a classification accuracy of 95.9%. Figure 9 This is a classification result image obtained after processing the Flevoland region's fully polarimetric SAR image using the method of this invention. The classification accuracy is 96.8%. It can be seen that... Figure 8 This method will result in many noise points and cannot effectively suppress noise, while the method of this invention can obtain more consistent classification results.

[0113] Example 3

[0114] I. Classification criteria are as follows:

[0115] 1) In the classification experiment, 512×512×57-dimensional polarimetric SAR image data of a certain area of ​​Xian were selected. After preprocessing, suitable Grassmann manifold features were formed; and a 512×512 SPD feature matrix was obtained.

[0116] 2) In the classification experiment, the polarimetric SAR image data was sampled in layers, with 500 training samples selected for each class, and the test set was selected at a ratio of 0.5% to ensure class balance;

[0117] 3) In the classification experiment, the dimension of the target subspace for multi-manifold metric learning was set to 8, and the weight parameters were... =0.7, =0.3.

[0118] II. The steps and procedures described above in this invention are as described in Example 1.

[0119] III. The classification content and results are as follows:

[0120] Figure 10 The confusion matrix diagram is generated using the existing HybridCVnet method. Figure 11 This is the confusion matrix diagram of the fully polarimetric SAR image of the Xian area obtained by the method of this invention. It can be seen that... Figure 11 The fewer misclassifications will result, indicating that the method of the present invention can obtain more consistent classification results.

[0121] Example 4

[0122] I. Classification criteria are as follows:

[0123] 1) In the classification experiment, 512×512×57-dimensional polarimetric SAR image data of a certain area of ​​Xian were selected. After preprocessing, suitable Grassmann manifold features were formed; and a 512×512 SPD feature matrix was obtained.

[0124] 2) In the classification experiment, the polarimetric SAR image data was sampled in layers, with 1000 training samples selected for each class, and the test set was selected at a ratio of 0.5% to ensure class balance;

[0125] 3) In the classification experiment, the dimension of the target subspace for multi-manifold metric learning was set to 8, and the weight parameters were... =0.60, =0.40.

[0126] II. Following the steps and procedures outlined above in this invention, data preprocessing yielded 3000 Grassmann features of 400×10 and SPD feature samples of 6×6. Other steps and procedures are described in Example 1.

[0127] III. The classification content and results are as follows:

[0128] The method of this invention enables accurate classification of different land features such as water bodies, grasslands, and buildings, and can obtain better classification results.

[0129] Example 5

[0130] I. Classification criteria are as follows:

[0131] 1) In the classification experiment, 512×512×57-dimensional polarimetric SAR image data of a certain area of ​​Xian were selected. After preprocessing, suitable Grassmann manifold features were formed; and a 512×512 SPD feature matrix was obtained.

[0132] 2) In the classification experiment, the polarimetric SAR image data was sampled in layers, with 800 training samples selected for each class, and the test set was selected at a ratio of 0.5% to ensure class balance;

[0133] 3) In the classification experiment, the dimension of the target subspace for multi-manifold metric learning was set to 8, and the weight parameters were... =0.75, =0.25.

[0134] II. Following the steps described above in this invention, 3000 Grassmann features of 400×10 and SPD feature samples of 6×6 were obtained. The steps are described in Example 1.

[0135] III. The classification content and results are as follows:

[0136] It achieves accurate classification of different land features such as water bodies, grasslands, and buildings. It exhibits fewer misclassifications and provides excellent classification results.

[0137] Example 6

[0138] I. Classification criteria are as follows:

[0139] 1) In the classification experiment, 512×512×57-dimensional polarimetric SAR image data of a certain area of ​​Xian were selected. After preprocessing, suitable Grassmann manifold features were formed; and a 512×512 SPD feature matrix was obtained.

[0140] 2) In the classification experiment, the polarimetric SAR image data was sampled in layers, with 600 training samples selected for each class, and the test set was selected at a ratio of 0.5% to ensure class balance;

[0141] 3) In the classification experiment, the dimension of the target subspace for multi-manifold metric learning was set to 8, and the weight parameters were... =0.80, =0.20.

[0142] II. Following the steps described above in this invention, 3000 Grassmann features of 400×10 and SPD feature samples of 6×6 were obtained. The steps are described in Example 1.

[0143] III. The classification content and results are as follows:

[0144] It achieves accurate classification of different land features such as water bodies, grasslands, and buildings. The classification results are superior to existing classification methods.

Claims

1. An image classification method based on multi-manifold joint metric learning, characterized in that, Follow these steps: Step 1, data preprocessing, including constructing Grassmann features and SPD features; Step 2: Construct the dual manifold kernel matrix, including constructing the Grassmann manifold kernel matrix and the SPD manifold kernel matrix, and calculating the test kernel matrices of Grassmann and SPD respectively; Step 3, implement multi-manifold joint metric learning, specifically: Extract kernel eigenvectors from the kernel matrix and calculate the intra-class scatter matrix. Inter-class scatter matrix For the projection matrix V Solve by considering the magnitude of the eigenvalues. U Sort the feature vectors in the data and select the top ones. d The eigenvectors corresponding to the largest eigenvalues ​​constitute the projection matrix. V , The specific process is as follows: 3.1) Extract kernel eigenvectors from the kernel matrix. The i List ,extract The i List Calculate the intra-class scatter matrix Inter-class scatter matrix The expressions are as follows: Intraclass scatter matrix The expression is as follows: Inter-class scatter matrix The expression is as follows: in, This represents the degree of dispersion of similar samples in the kernel feature space. This represents the degree of dispersion of different classes of samples in the kernel feature space. Represents Grassmann manifold weights. Represents the SPD manifold weights. T Indicates transpose; W Represents the set of indexes for similar sample pairs; Z For all heterogeneous sample pairs, use an index set; For the first i Grassmann kernel feature vectors of training samples For the first j Grassmann kernel feature vectors of training samples; For the first i SPD kernel feature vectors of training samples, For the first j SPD kernel feature vectors of training samples; 3.2) Projection matrix V The solution is obtained through generalized eigenvalue decomposition, expressed as follows: in, U Let D be the eigenvector matrix and D be the eigenvalue matrix. S W Represents the within-class scatter matrix. S b Represents the inter-class scatter matrix. eig (.) represents the eigenvalue decomposition function; Step 4, feature fusion, completion of extraction, and sample mapping, the specific process is as follows: 4.1) Perform feature fusion on the training set obtained in step 1. For the first... i The expression for the fused features of each training sample is as follows: in, For the first i The fusion features of each training sample V The projection matrix; 4.2) Perform feature fusion on the test set obtained in step 1. For the first... j The expression for the fusion feature of each test sample is as follows: in, For the first j The fusion features of each test sample Indicates the first j Grassmann kernel feature vectors of each test sample Indicates the first j SPD kernel feature vectors of each test sample; 4.3) Implement sample-level fusion feature mapping and construct a label-based fusion feature dictionary; Step 5: Construct a sample-level fusion convolutional neural network to classify different land features in the original image. This includes sample-level fusion CNNs that process the original image and multi-manifold fusion features simultaneously through a dual-path structure, resulting in a concatenated composite feature vector, which is then input into a multi-layer fully connected neural network for classification.

2. The image classification method based on multi-manifold joint metric learning according to claim 1, characterized in that, In step 1, the specific process is as follows: 1.1) Constructing Grassmann features: Acquire polarimetric SAR images to obtain a 512×512×57-dimensional polarimetric SAR feature matrix. Randomly select 500 pixels by category as the training set and select pixels for the test set at a ratio of 0.5% of the total data. For each pixel in the image, extract a 5×5 spatial neighborhood window centered on that pixel. Since each pixel contains 57-dimensional polarimetric feature information, this spatial neighborhood window constitutes a 5×5×57=1425-dimensional feature, which is called a high-dimensional feature. Then, the constructed 1425-dimensional features are subjected to dimensionality reduction: using principal component analysis (PCA), the high-dimensional features are compressed into a 400-dimensional low-dimensional feature representation through the PCA transformation matrix obtained from training. Next, 25 samples are collected in the 5×5 neighborhood to form a 400×25 matrix X. Then, singular value decomposition is performed on it, and the 10 largest eigenvectors are taken to form a 400×10 orthogonal basis matrix. 1.2) Constructing SPD Features: Obtain a 512×512 image size HPD feature matrix with a matrix dimension of 3×3. Transform the HPD feature matrix into a real SPD matrix using a "complex-to-real" embedding method. For a complex HPD matrix, the expression is: C = A + Bi, where A is the real part matrix and B is the imaginary part matrix. The transformation expression is: The transformation yields a 6×6 real symmetric matrix. This is the SPD matrix; Then, select the same pixels as those in the aforementioned Grassmann features according to the category.

3. The image classification method based on multi-manifold joint metric learning according to claim 1, characterized in that, Step 2, the specific process is as follows: 2.1) Construct the Grassmann manifold kernel matrix, as shown in the following expression: in, Represents the Grassmann manifold kernel matrix. This represents the trace operation. F It is the Frobenius norm. For the first i Grassmann subspace matrix of training samples For the first j Grassmann subspace matrix of training samples; 2.2) Construct the SPD manifold kernel matrix, as shown in the following expression: in, Represents the SPD manifold kernel matrix. For the first i The SPD matrix of each training sample. For the first j SPD matrix of training samples; 2.3) Calculate the test kernel matrices for Grassmann and SPD respectively: The cross-set kernel matrix is ​​constructed separately for the test samples and the training samples, as explained below: Construct the Grassmann test kernel matrix, expressed as: in, For the first t Grassmann subspace matrix of each test sample; Construct the SPD test kernel matrix, the expression is: in, For the first t SPD matrix of each test sample.

4. The image classification method based on multi-manifold joint metric learning according to claim 1, characterized in that, In step 3.2), the specific calculation process is as follows: Based on the magnitude of the eigenvalue U Sort the feature vectors in the data and select the top ones. d The eigenvectors corresponding to the largest eigenvalues ​​constitute the projection matrix. V The expression is: in, This represents the number of training samples.

5. The image classification method based on multi-manifold joint metric learning according to claim 1, characterized in that, In step 4.3), the specific calculation process is as follows: Implement sample-level fusion feature mapping and construct a label-based fusion feature dictionary, expressed as: in, c For category indexing, M The total number of categories, c= 1,2,…, M ; For the first c Class fusion feature dictionary For the first i The fusion features of each training sample For the first i The category label of each sample.

6. The image classification method based on multi-manifold joint metric learning according to claim 1, characterized in that, Step 5, the specific process is as follows: 5.1) Configure the network architecture. A dual-path fusion architecture is adopted, including a feature extraction branch for the original image and a feature fusion branch, expressed as: in, This is the concatenated composite feature vector. Image features extracted by CNN To integrate manifold features; 5.2) Input the concatenated composite feature vector obtained above into a multi-layer fully connected neural network for classification, and output the classification vector. The expression is: in, For the classification output vector, the dimension corresponding to its maximum value is the predicted class. Softmax (.) is the normalized activation function, used to output the probability distribution; ReLU (.) is the activation function; Dropout (.)for Dropout A random deactivation function is used to prevent overfitting. This is the concatenated composite feature vector; , This is the weight matrix of a two-layer fully connected neural network; , This is the bias term for the corresponding layer.

7. The image classification method based on multi-manifold joint metric learning according to claim 1, characterized in that, In step 5, the dual-path structure simultaneously processes the original image and multi-manifold fusion features as follows: the convolutional branch extracts 256-dimensional spatial-spectral features from the 9×9 image patch, and the manifold branch uses step 5 to generate 8-dimensional fusion features, including SPD covariance and Grassmann subspace discriminative information; the two features are concatenated into a 264-dimensional vector in the fully connected layer, and then... Dropout Regularization and Softmax The layer outputs category probabilities to classify different land features in the original image.

Images