Hyperspectral data multi-view clustering method based on spectrum and space fusion

Abstract

The invention relates to the technical field of image processing and unsupervised machine learning, and discloses a hyperspectral data multi-view clustering method based on spectrum and space fusion. Comprising the following steps: step 1, inputting hyperspectral image data to form a hyperspectral data matrix X; 2, spectral features are extracted, and hyperspectral view data X (1) are generated; step 3, extracting spatial features of X, and generating spatial view data X (2); 4, combining X (1) and X (2), and constructing a multi-view data set X (v); 5, establishing a multi-view weighting objective function; 6, iteratively updating P (v), G and alpha (v); and 7, outputting a clustering result. According to the method, multi-view clustering of hyperspectral data is carried out by utilizing a spectrum and space fusion method, and large-scale data clustering on a multi-core processor can be realized; meanwhile, combination and characteristic decomposition do not need to be carried out, and a large amount of calculation burden is effectively avoided; the problems that an existing traditional processing method is large in data processing amount and cannot be suitable for large-scale data processing are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and unsupervised machine learning, and particularly relates to a multi-view clustering method for hyperspectral data based on spectral and spatial fusion. Background Technique

[0002] The breakthrough development of hyperspectral imaging technology has provided a revolutionary analysis means for industrial inspection and remote sensing monitoring. By collecting information on dozens to hundreds of continuous spectral bands, hyperspectral data not only contains rich material composition characteristics but also can accurately record the spatial distribution characteristics of the target. This spectral-spatial two-dimensional information fusion enables it to show unique advantages in fields such as target recognition and environmental monitoring. However, the problem of high-dimensional data processing brought about by a large number of spectral features poses a severe challenge to traditional algorithms. Especially under the condition of lacking pixel-level label constraints, how to mine the internal structure of data through unsupervised methods has become a key issue, which has also promoted the continuous innovation of hyperspectral clustering algorithms.

[0003] The rise of multi-view clustering algorithms has provided a new idea for solving the dilemma of hyperspectral analysis. Compared with the conventional methods that only rely on single spectral features, multi-view technology constructs a more complete feature representation system by jointly interpreting spectral features and spatial texture information. Typical studies such as the fusion framework of spectral segmentation morphological features (SSMLC) and principal component analysis (PCA), and the spatial-spectral joint learning method based on multi-view low-rank subspace clustering (MLRSSC) have all confirmed that multi-feature collaboration can significantly improve the clustering accuracy. By introducing redundant discrete wavelet transform (RDWT) to construct multi-band views and a multi-feature fusion framework without regularization parameters, the application dimension of multi-view technology has been further expanded. However, there is still room for improvement in its methods in terms of feature weight optimization, view generation strategy, etc. For example: Although the current multi-view hyperspectral clustering research has made phased progress, there is still a significant gap between the theoretical system and engineering applications. Existing algorithms mostly focus on the verification of feature combinations in specific scenarios, lacking general view construction criteria and adaptive fusion mechanisms. Especially when dealing with real-world scenarios such as complex surface cover and noise interference, the robustness and generalization ability of the algorithms need to be strengthened urgently and cannot be applied to large-scale data processing.

[0004] Based on this, the present invention proposes a multi-view clustering method for hyperspectral data based on spectral and spatial fusion to solve the problems existing in the above-mentioned prior art. Summary of the Invention

[0005] (1) Technical Problems to be Solved

[0006] Aiming at the deficiencies of the existing technology, the present invention provides a multi-view clustering method for hyperspectral data based on spectral and spatial fusion. This method can achieve large-scale data clustering on a multi-core processor. At the same time, this method does not require graph construction and feature decomposition, avoiding a large amount of computational burden; it solves the problems of large data processing volume and inapplicability to large-scale data processing existing in the existing graph-based processing methods.

[0007] (II) Technical solution

[0008] To achieve the above object, the present invention provides the following technical solution:

[0009] A multi-view clustering method for hyperspectral data based on spectral and spatial fusion, including:

[0010] Step 1: Input hyperspectral image data to form a hyperspectral data matrix X;

[0011] Step 2: Extract the spectral features of the hyperspectral image data through principal component analysis to generate hyperspectral view data X( 1 );

[0012] Step 3: Extract the spatial features of the hyperspectral data matrix X to generate spatial view data X( 2 );

[0013] Step 4: Merge the spectral view data X (1) obtained in Step 2 with the spatial view data X (2) obtained in Step 3 to construct a multi-view data set X (v) ;

[0014] where V represents the total number of views;

[0015] Step 5: Based on the multi-view data set X (v) , establish a multi-view weighted objective function;

[0016] Step 6: Iteratively optimize and update the clustering center matrix P (v) , the clustering indicator matrix G and the view weight α (v) until the objective function converges;

[0017] Step 7: Output the clustering result to complete the multi-view clustering of hyperspectral data.

[0018] In a preferred embodiment, the hyperspectral data matrix described in Step 1

[0019]

[0020] where d is the number of bands and n is the total number of pixels.

[0021] In a preferred embodiment, the process of extracting the spectral features of the hyperspectral image data in step 2 includes:

[0022] Step 2.1: Data standardization. For the hyperspectral data matrix X, standardize each band:

[0023]

[0024] where μ i is the mean of the i-th band, and σ i is the standard deviation;

[0025] Step 2.2: Calculate the covariance matrix. The covariance matrix C of the standardized data is:

[0026]

[0027] where the matrix dimension of the covariance matrix C is d×d;

[0028] Step 2.3: Eigenvalue decomposition. Perform eigenvalue decomposition on the covariance matrix C:

[0029] C = VΛV T ;

[0030] where V is an orthogonal matrix, and its column vectors are eigenvectors; Λ is a diagonal matrix, and its diagonal elements are eigenvalues;

[0031] Step 2.4: Select the principal components. Select the first p principal components according to the cumulative variance contribution rate;

[0032]

[0033] Step 2.5: Project onto a low-dimensional space. Use the first p eigenvectors to construct a projection matrix V p ∈R d×p , and map the original data to the low-dimensional space:

[0034]

[0035] to obtain the hyperspectral view data X (1) .

[0036] In a preferred embodiment, step 3 uses a 1D convolutional neural network to extract the spatial features of the hyperspectral data matrix X and generate the spatial view data X (2) ;

[0037] where d is the number of bands, and n is the total number of pixels.

[0038] In a preferred embodiment, the multi-view weighted objective function in step 5 is

[0039]

[0040] Among them, α (v) represents the weight value of each view, P (v) represents the cluster center matrix, G represents the cluster indicator matrix, and γ is a regularization parameter that can avoid trivial solutions.

[0041] In a preferred embodiment, the process of iteratively optimizing and updating the cluster center matrix P (v) , the cluster indicator matrix G, and the view weight α (v) includes:

[0042] Step 6.1: Fix G and α (v) , and update D (v) ;

[0043] Step 6.2: Fix P (v) , D (v) , α (v) , and update the cluster indicator matrix G by pixel-by-pixel search;

[0044] Step 6.3: Fix P (v) , G, and D (v) , and update the view weight α by the method of Lagrange multipliers (v) .

[0045] In a preferred embodiment, in Step 6.1, by fixing G and α (v) , and updating D (v) , the obtained cluster center matrix is:

[0046]

[0047] where D (v) represents the diagonal weight matrix; G represents the cluster indicator matrix.

[0048] In a preferred embodiment, in Step 6.2, fix P (v) , D (v) , α (v) , and update the cluster indicator matrix G to obtain:

[0049]

[0050] where g * represents the optimal solution.

[0051] In a preferred embodiment, in Step 6.3, fix P (v) , G, and D (v) , and update α (v) , to obtain:

[0052]

[0053] In a preferred embodiment, the clustering result described in step 7 includes a common clustering indication matrix G and view weights α for each view. (v) 。

[0054] (III) Advantageous Effects

[0055] Compared with the prior art, the present invention provides a multi-view clustering method for hyperspectral data based on spectral and spatial fusion, having the following advantageous effects:

[0056] By fusing spectral and spatial features to construct a multi-view data set and using a dynamic weighted objective function to optimize the clustering process, the parallel computing power of a multi-core processor can be fully utilized to achieve efficient clustering of hyperspectral data; by using principal component analysis (PCA) and three-dimensional convolutional neural network (3DCNN) to extract spectral and spatial features respectively, complex feature decomposition and combination operations in traditional graph methods are avoided, effectively reducing the computational complexity; by iteratively updating the clustering centers, indication matrix, and view weights, an adaptive allocation of the contributions of different views is achieved, and finally the following core effects are achieved:

[0057] 1. Improved computational efficiency: By avoiding the graph construction and feature decomposition steps, the time complexity is reduced from O(n 3 ) in traditional methods to O(knpdv), where k is the number of clusters, n is the total number of data, p is the number of iterations, d is the data dimension, and v is the number of views. It supports parallel processing of large-scale data on a multi-core processor;

[0058] 2. Optimized clustering accuracy: By dynamically balancing the contributions of the spectral and spatial views, accurate classification of complex surface coverages is achieved, and the overall accuracy (OA) is increased by 12% - 30% compared with K-means;

[0059] 3. Enhanced scalability: The limitation that graph-based methods cannot handle high-dimensional massive data is solved. Through PCA dimensionality reduction and 3DCNN feature extraction, clustering is successfully completed, verifying the applicability to large-scale remote sensing data;

[0060] 4. Enhanced robustness: By using the regularization parameter γ to avoid trivial solutions, in subsequent experimental result demonstrations, it is significantly superior to the comparative algorithms, proving its robustness to noisy and heterogeneous data. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a system framework diagram of the multi-view clustering method for hyperspectral data based on spectral and spatial fusion of the present invention;

[0062] Figure 2For the false color images and true label maps of the three data sets in Embodiment 2 of the present invention;

[0063] Figure 3 For the accuracy result graphs of extracting different dimensions using PCA and 3DCNN for different data sets in Embodiment 2 of the present invention;

[0064] Figure 4 For the clustering graph on the Indian Pines data set in Embodiment 2 of the present invention;

[0065] Figure 5 For the clustering graph on the Salinas data set in Embodiment 2 of the present invention;

[0066] Figure 6 For the clustering graph on the Pavia University data set in Embodiment 2 of the present invention;

[0067] Figure 7 For the convergence analysis graph of the hyperspectral data multi-view clustering method based on spectral and spatial fusion on the Indian Pines and Salinas data sets in Embodiment 2 of the present invention.

[0068] Wherein:

[0069] In Figure 2 : Figure (a) is the false color image of the Indian Pines data set; Figure (b) is the true label map of the Indian Pines data set; Figure (c) is the false color image of the Pavia University data set; Figure (d) is the true label map of the Pavia University data set; Figure (e) is the false color image of the Salinas data set; Figure (f) is the true label map of the Salinas data set;

[0070] In Figure 3 : Figure (a) is the graph of the relationship between the PCA feature dimension and accuracy of the Indian Pines data set; Figure (b) is the graph of the relationship between the 3DCNN feature dimension and accuracy of the Indian Pines data set; Figure (c) is the graph of the relationship between the PCA feature dimension and accuracy of the Salinas data set; Figure (d) is the graph of the relationship between the 3DCNN feature dimension and accuracy of the Salinas data set;

[0071] In Figure 4 : Figure (a) is the K-means clustering result graph; Figure (b) is the FCM clustering result graph; Figure (c) is the AWP clustering result graph; Figure (d) is the OPMC clustering result graph; Figure (e) is the FCM-S clustering result graph; Figure (f) is the SSF-MV clustering result graph; Figure (g) is the true label comparison graph;

[0072] In Figure 5In; Figure (a) is the K-means clustering result graph; Figure (b) is the FCM clustering result graph; Figure (c) is the AWP clustering result graph; Figure (d) is the OPMC clustering result graph; Figure (e) is the FCM-S clustering result graph; Figure (f) is the SSF-MV clustering result graph;

[0073] In Figure 6 In: Figure (a) is the K-means clustering result graph; Figure (b) is the FCM clustering result graph; Figure (c) is the AWP clustering result graph; Figure (d) is the OPMC clustering result graph; Figure (e) is the FCM-S clustering result graph; Figure (f) is the SSF-MV clustering result graph;

[0074] In Figure 7 In: Figure (a) is the objective function convergence curve graph of the Indian Pines dataset; Figure (b) is the objective function convergence curve graph of the Salinas dataset. Specific implementation mode

[0075] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0076] Embodiment 1:

[0077] Please refer to Figure 1 , this embodiment provides a technical solution: a multi-view clustering method for hyperspectral data based on spectral and spatial fusion, including:

[0078] Step 1: Input hyperspectral image data to obtain a hyperspectral data matrix as:

[0079]

[0080] where d is the number of bands and n is the total number of pixels.

[0081] Step 2: Data preprocessing, extract the spectral features of the hyperspectral image data through principal component analysis (PCA) to generate hyperspectral view data Specifically include:

[0082] Step 2.1: Data standardization. For the hyperspectral data matrix X ∈ R d×n , perform standardization processing on each band:

[0083]

[0084] where, μ iis the mean value of the i-th band, and σ i is the standard deviation;

[0085] Step 2.2: Calculate the covariance matrix. The covariance matrix C of the standardized data is:

[0086]

[0087] where X std represents the data after standardization. Its core purpose is to eliminate the dimensional difference between different bands and ensure the fairness of covariance calculation. The matrix dimension of the covariance matrix C is d×d, which is used to reflect the correlation between different bands;

[0088] Step 2.3: Eigenvalue decomposition. Perform eigen decomposition on the covariance matrix C:

[0089] C = VΛV T ;

[0090] where V is an orthogonal matrix, and its column vectors are eigenvectors (principal component directions); Λ is a diagonal matrix, and its diagonal elements are eigenvalues (arranged in descending order as λ1≥λ2≥L≥λ d );

[0091] Step 2.4: Select the principal components. Select the first p principal components (p << d) according to the cumulative variance contribution rate;

[0092]

[0093] where the cumulative variance contribution rate is a key indicator in principal component analysis (PCA), which is used to measure the proportion of the original data information retained by the first p principal components. The higher the cumulative variance contribution rate, the more original information is retained in the data after dimensionality reduction. λ i represents the eigenvalue of the i-th principal component, indicating the variance of the original data explained by this principal component. Usually, the minimum p value with a cumulative contribution rate exceeding 85% is selected; in this embodiment, p is selected at intervals of 10% to 100% of the original number of bands (for example, the first 30 principal components are selected for the Indian Pines dataset);

[0094] Step 2.5: Project onto the low-dimensional space. Use the first p eigenvectors to construct the projection matrix V p ∈R d×p , and map the original data into the low-dimensional space:

[0095]

[0096] to obtain the hyperspectral view data

[0097] Step 3: Secondary data processing, using a three-dimensional convolutional neural network (3DCNN) to extract the hyperspectral data matrix spatial features to generate spatial view data Specifically, it includes:

[0098] Step 3.1: Input the hyperspectral view data as a cube, (width × height × number of bands × number of input channels). Hyperspectral data is usually single-channel, i.e., C in = 1;

[0099] Step 3.2: Three-dimensional convolutional operation. The three-dimensional convolutional kernel slides in the spatial-spectral dimension to extract local features:

[0100] Step 3.2.1: Let the convolutional kernel be:

[0101]

[0102] where k w ,k h ,k d represent the sizes of the convolutional kernel in the width, height, and band directions respectively; C out represents the number of output channels;

[0103] Step 3.2.2: The calculation formula for each element of the output feature map is:

[0104]

[0105] where s represents the convolutional stride (stride), defaulting to 1; b c represents the bias term, which is used to enhance the flexibility of the model; W', H', and D' all represent the output sizes, and the calculation formulas are:

[0106]

[0107] H' and D' are the same;

[0108] padding represents the operation of adding extra pixels (or values) around the edges of the input data. These padding values are usually 0 (referred to as "zero padding"), aiming to control the size of the output feature map after the convolutional operation and retain the information at the input edges, with no padding (padding = 0).

[0109] Step 3.3: Three-dimensional max pooling operation. The pooling layer reduces the computational amount through downsampling and enhances the robustness of the features. The formula for three-dimensional max pooling is:

[0110]

[0111] where pw , p h , p d represent the dimensions of the pooling window in the width, height, and band directions respectively, and s represents the pooling stride, which is usually the same as the window size.

[0112] Step 3.4: Global average pooling generates spatial view data, compresses the three-dimensional feature map into a two-dimensional matrix as the spatial view input. Let the output of the last convolutional layer be

[0113]

[0114] Output: After global average pooling: Take the mean value for each channel to obtain d2 = C out .

[0115] Step 4: Combine the spectral view data obtained in Step 2 with the spatial view data obtained in Step 3 to construct a multi-view dataset X (v) (v = 1, 2, K, V);

[0116] Among them, V represents the total number of views. In this embodiment, V is equal to 2.

[0117] Step 5: Based on the multi-view dataset X (v) (v = 1, 2, K, V), establish a multi-view weighted objective function, which specifically includes:

[0118] k-means, as a widely used unsupervised clustering algorithm, is used to divide a given dataset into k clusters according to the Euclidean distance between the data, so that the samples within the clusters are as closely connected as possible, while the distance between the clusters is as large as possible, and it is optimized by minimizing the squared loss objective function.

[0119] Assume a given data matrix where n represents the number of data and d represents the dimension of the data. This problem can be described as follows:

[0120]

[0121] Among them, represents the cluster center matrix, represents the cluster assignment matrix. If the sample x i is assigned to the k-th class, then G ik = 1. Otherwise, the sample x i does not belong to the k-th class, and G ik= 0. However, K-means is only applicable to single-view datasets. With the development of technology, single-view data can no longer fully reflect all the characteristics of the data. Therefore, it is very important to perform a reasonable analysis of multi-view data. Because data from different perspectives can reveal more information in the data, and the connections between them can be used to explore deeper structural information. The most direct method of clustering using multi-view data is to concatenate the views of all the data for clustering. Unfortunately, this method can only treat all data perspectives equally. In practical problems, the contributions of different perspectives to clustering are usually different, which results in the model not being able to obtain the optimal result. To overcome this problem, this step must solve the following two problems: how to reasonably and naturally fuse the clustering results of different views. How to learn different feature views to discover the contribution rates of different views to clustering? Therefore, we must solve the above two problems simultaneously.

[0122] To solve the problem of clustering large-scale multi-view image data, this embodiment proposes a new multi-view weighted clustering method:

[0123] Suppose represents the features of the v-th view, represents the data center matrix of the v-th view. Naturally, take as the clustering indicator matrix of the v-th view, and there are V views. Then, establish the multi-view weighted objective function as follows:

[0124]

[0125] where α (v) represents the weight value of each view, and γ is a regularization parameter that can avoid trivial solutions. By applying different weights to different feature views, important feature views can provide more contributions in the clustering process, that is, important feature views can obtain larger weights.

[0126] Step 6: Iteratively optimize and update the clustering center matrix P (v) , the clustering indicator matrix G and the view weight α (v) , until the objective function converges, specifically including:

[0127] Step 6.1: When using the multi-view clustering algorithm for clustering, the clustering results under different feature views should be the same and unique, that is, the clustering indicator matrix G (v) obtained from different views should have the same view. Therefore, this embodiment enforces that all clustering assignment matrices have the same view, that is, use a recognized clustering indicator matrix and make it satisfy the one-out-of-K coding strategy.

[0128] Step 6.1.1: The objective function (2) is not easy to solve (hereinafter referred to as problem (2)) mainly because each row of the clustering indicator matrix is a discrete integer and each row must satisfy the 1-out-of-K coding strategy. To solve problem (2), an effective iterative optimization algorithm is designed in this embodiment. To clearly show the solution process, problem (2) is abbreviated as follows:

[0129]

[0130] Step 6.1.2: Next, optimize each variable in turn, that is, fix G, α (v) , update D (v) :

[0131] Take the partial derivative of P in the objective function (v) to get:

[0132]

[0133] where is a diagonal matrix with each diagonal element being (α (v) ), which corresponds to the weight of each view. Let problem (5) be equal to 0 to get γ Then get: Then obtain:

[0134]

[0135] Step 6.2: Fix P (v) , D (v) , α (v) , and update the clustering indicator matrix G by pixel-by-pixel search, which specifically includes:

[0136] Fix P (v) , D (v) , α (v) , and update G to get:

[0137]

[0138]

[0139] where d ii is the i-th diagonal element of the diagonal matrix D (v) .

[0140] Obviously, the above problem (7) cannot be directly solved. Therefore, the above problem can be solved by decoupling the data and independently assigning clustering indicators one by one. Specifically, for the vector the following problem can be solved by fixing i:

[0141]

[0142] Among them, d (v) = d ii . According to the optimization conditions of g, there are K candidate solutions. For convenience, in this embodiment, a search algorithm is used to find the optimal solution of problem (8). Assume that g * is the optimal solution, then k can be solved in the following way:

[0143]

[0144] Step 6.3: Fix P (v) , G and D (v) , and update the view weight α by the Lagrange multiplier method (v)

[0145] Fix P (v) , G, D (v) , and update α (v) , to obtain:

[0146]

[0147] Among them, the Lagrangian function of equation (10) is defined as follows:

[0148]

[0149] Then let problem (11) be equal to 0, and we can get:

[0150]

[0151] According to the constraint conditions and substitute it into problem (12), we have:

[0152]

[0153] Among them, the value range of the regularization parameter γ is 1 < γ < 5, which is used to balance the sparsity of the view weight.

[0154] By iteratively repeating the variables G, P, and α (v) , until the objective function converges, the optimal solution of problem (2) can be obtained.

[0155] Step 7: Output the clustering results, including the common clustering indicator matrix G and each view weight α (v) , and complete the multi-view clustering of hyperspectral data.

[0156] Specifically, the process of iteratively optimizing and updating the clustering center matrix P (v) , the clustering indicator matrix G, and the view weight α (v) , until the objective function converges can also be represented by the following algorithm process:

[0157] Input the data matrix X from V views (1) , X (2) , …, X (V) And The number of clusters K, the regularization parameter γ;

[0158] Initialize the common clustering indicator matrix And G satisfies the 1-encoding strategy in K, and the weight of each view

[0159] When not converged:

[0160] (1) Update the clustering center matrix P of each view through Equation (6) (v) ;

[0161] (2) Update the clustering indicator vector g of each data through Equation (9);

[0162] (3) Update the weight α of each view through Equation (13) (v) ;

[0163] End the loop

[0164] Output the common clustering indicator matrix G, the clustering centroid matrix P of each view (v) , the weight α of each view (v) .

[0165] Specifically, in the process of the above multi-view clustering method for hyperspectral data based on spectral and spatial fusion, capital letters, bold lowercase letters, and lowercase letters are used to represent matrices, vectors, and letters respectively; Tr(.) represents the trace of a matrix, A T represents the transpose of a matrix, A -1 represents the inverse of a matrix,

[0166] Example 2:

[0167] Different from the above Example 1, this example conducts a theoretical analysis of the convergence process of Step 6 in Example 1 from three dimensions: parameters, computational complexity, and convergence, to ensure that the model has theoretical guarantees.

[0168] Parameter dimension: There is a regularization parameter γ in Problem (2), which assigns a weight to each view. It can be seen from Problem (13) that when γ approaches infinity, the weights of all views are equal. On the contrary, when γ approaches a certain value, the view corresponding to the minimum H value is given a weight of 1, and the weights of other views are 0. This method of using the regularization parameter not only avoids the problem of the model falling into a trivial solution, that is, the problem when γ approaches 1, but also allows the parameter γ to control the weights of different perspectives.

[0169] Computational complexity dimension: Problem (2) is a center-based multi-view clustering method, which is obtained by extending and transforming the K-means algorithm. Therefore, its computational complexity is very similar to that of the K-means algorithm. Therefore, assuming the computational complexity of the K-means algorithm is O(knpd), where n is the number of data, d is the dimension of the data, and p is the number of iterations. Correspondingly, the computational complexity of Problem (2) is O(knpdv), where v is the number of views. Generally speaking, p << n, v << n, and k << n hold. Therefore, in practical applications, if the dataset to be calculated is too large to be fully stored in memory, we can regard the algorithm in Embodiment 1 of the present invention as an external memory-based algorithm, which can process one data block at a time and iteratively optimize all data blocks simultaneously on multiple processors. After all data blocks are processed, the clustering center matrix is updated. It can be seen that the method proposed in Embodiment 1 of the present invention can be applied to the clustering problem of large-scale data.

[0170] Convergence dimension: Briefly analyze the convergence of the method described in Embodiment 1, that is, the method described in Embodiment 1 will gradually decrease the value of the objective function until the model obtains a local optimal solution. Specifically, through the solution process of Problem (2), it can be seen that it actually has four optimization variables, namely G, α, P, and D. It can be regarded as four sub-problems, each of which is a convex optimization problem with respect to the corresponding variable and satisfies the KKT conditions. Therefore, iterative solution through the method described in Embodiment 1 can ensure that each sub-problem can obtain an optimal solution. In addition, each sub-problem is always greater than 0, so each sub-problem has an uncertain lower bound of 0, which will ultimately make Problem (2) converge to a local optimal solution.

[0171] It should be noted that the above Problem (2) and Problem (13) correspond to Equation (2) and Equation (13) in Embodiment 1 respectively.

[0172] Embodiment 3:

[0173] As Figures 2 - 7 shown, different from the above embodiments, to verify the credibility of the method described in the above Embodiment 1, this embodiment is used to verify the algorithm process of the above multi-view clustering method for hyperspectral data based on spectral and spatial fusion.

[0174] This embodiment will evaluate the performance of the hyperspectral data multi-view clustering method based on spectral and spatial fusion (SSF-MV) proposed in Embodiment 1 on hyperspectral datasets (i.e., Indian Pines, Pavia University, and Salinas), and compare it with several other unsupervised clustering methods. The evaluation criteria used in the experiment are k-means, fuzzy c-means (FCM), FCM-S, AWP, and OPMC.

[0175] Example 1: Input the Indian Pines dataset (145×145×220145×145×220). Extract the first 30 principal components as the spectral view through PCA, and extract spatial features as the second view through 3DCNN. Set the number of clusters K = 16K = 16, the regularization parameter γ = 2, and it converges after 50 iterations. The results show that the OA is 41.99%, which is 12% higher than that of K-means.

[0176] Example 2: In the Pavia University dataset (610×340×103610×340×103), fuse spectral and texture features, and set γ = 3γ = 3. The final OA reaches 70.15%, and the accuracy in the "painted metal sheet" category is increased by 53%.

[0177] Example 3: For the Salinas dataset (512×217×204512×217×204), optimize the weight update formula, and it converges after 20 iterations. The experiment shows that the accuracy of SSF-MV in the "6-week lettuce" category reaches 100%, which is significantly better than the comparison algorithms.

[0178] The following are the specific measures and analyses for the implementation experiment:

[0179] First, the detailed information of the dataset is mainly introduced. Table 1 and Figure 2 show the number of categories, the number of dimensions, and the visualization results of the dataset.

[0180] Indian Pines: The Indian Pines dataset is obtained through aerial or satellite remote sensing technology, and collects hyperspectral images of the Indian Lake area in southwestern India. This dataset contains 16 different types of features, such as farmland, forest, road, water body, etc., and has a total of 220 bands. This dataset is a three-dimensional dataset with dimensions of 145×145×220.

[0181] Pavia University dataset: The Pavia University dataset is taken from the city of Pavia and its surrounding areas in northern Italy. The dimensions of this dataset are 610×340×103, that is, the number of samples is 610×340. Each sample contains the features of 103 bands, covering 9 different types of features such as urban, farmland, and natural features.

[0182] Salinas Dataset: The Salinas dataset was acquired in the Salinas Valley region of California, USA, through aerial or satellite remote sensing technology. This region is known for its diverse terrain types and rich agricultural landscapes. The dimensions of this dataset are 512×217×204. Each sample contains the characteristics of 204 bands, including 16 different types of ground objects such as corn, cucumber, and land.

[0183] Table 1: Dataset Description

[0184] Dataset Number of data Dimension Category Indian Pines Dataset 145×145 220 16 Pavia University Dataset 610×340 103 9 Salinas Dataset 512×217 204 16

[0185] To obtain more multi-view information from hyperspectral data, in this embodiment, the dataset is preprocessed. Specifically, we divide the hyperspectral data into two views and obtain their spectral information and spatial information respectively. For spectral features, feature extraction is usually used to reduce the dimension of the spectrum. Therefore, in Embodiment 1 of the present invention, the principal component analysis (PCA) method is used to extract the principal components of different dimensions and use them as the spectral perspective. For spatial information, the convolutional neural network (CNN) framework is usually used to automatically extract high-level depth information. Therefore, in Embodiment 1 of the present invention, the three-dimensional convolutional neural network (3DCNN) framework is used to extract the spatial information features from the data to obtain the spatial perspective of the data. Subsequently, the two perspective data obtained above are combined to obtain a multi-view hyperspectral dataset.

[0186] To obtain the multi-view hyperspectral dataset more efficiently, in Embodiment 1 of the present invention, the above two methods (i.e., PCA and 3DCNN) are used to cluster and evaluate the features in the dataset. First, the features of the dataset with different dimensions are extracted, that is, the feature dimension is set from 10% to 100% with an interval of 10%, and then the above two methods are used to cluster each group of selected features. The experimental results are as Figure 3 shown. Due to space limitations, only the clustering results of the Indian Pines and Salinas datasets are shown in this embodiment. It can be seen that in the same dataset, using different methods to cluster data with different dimensions will show the best performance in different dimensions. Based on this, in this embodiment, the above two methods are used to select the best-performing data spectral features and spatial features from all datasets, and then a multi-view dataset is formed for subsequent multi-view experiments.

[0187] This embodiment evaluated the effectiveness of the proposed SSF-MV algorithm on the Indian Pines, Pavia University, and Salinas datasets. For the SSF-MV algorithm, this embodiment set the parameter γ to 2. For the comparative algorithms with parameters, this embodiment set their parameters to the recommended parameter ranges in the original text. In the experiment, this embodiment used a 64G Windows machine equipped with an i7 Intel processor and conducted the experiment in the MATLAB environment.

[0188] As Figure 4 shown in Table 2, the clustering quantitative evaluation results and clustering diagrams on the Indian Pines dataset are given respectively. In the table, the best results are shown in bold. From the experimental results, it can be seen that the overall experimental results of FCM-S are the worst, including the average accuracy (AA), overall accuracy (OA), and Kappa coefficient indicators. The AAF-MV proposed in Embodiment 1 of the present invention shows the best experimental results in the OA and Kappa indicators. Although K-means performs best in the AA indicator, it performs poorly in other indicators and subclass accuracies. Especially in the Corn and Soybean-notill subclasses, the method proposed in Embodiment 1 of the present invention far exceeds other algorithms.

[0189] Table 2: Performance evaluation on the Indian Pines dataset.

[0190]

[0191] As Figure 5 shown in Table 3, the clustering quantitative evaluation results and clustering diagrams on the Salinas dataset are given respectively. In the table, the best results are shown in bold, where "OM" represents "out of memory".

[0192] From the experimental results, it can be seen that the SSF-MV proposed in Embodiment 1 of the present invention achieved the best clustering results in many subclasses, including Brocoli_weeds_2, Stubble, Celery, Corn, Lettuce_6wk, Lettuce_7wk, Vinyard_untrained, and Vinyard_trellis. Relatively speaking, the clustering results of FCM are poor in most subclasses, while FCM-S performs better, especially at the overall level, with the best clustering performance in the OA and Kappa indicators. However, overall, the SSF-MV proposed in Embodiment 1 of the present invention can achieve the best performance in most cases.

[0193] Table 3: Performance Evaluation of Salinas Dataset

[0194]

[0195]

[0196] As Figure 6 shown in Table 4, the quantitative clustering evaluation results and clustering diagrams on the University dataset are given respectively. In the table, the best results are shown in bold, where "OM" represents "overflow memory". It can be seen from the experimental results that SSF-MV proposed in Embodiment 1 of the present invention has achieved the best clustering results in many subclasses, including Gravel, Trees, and Painted metal sheets. Especially in the Painted metal sheet subclass, the clustering accuracy of SSF-MV far exceeds that of the sub-optimal algorithm and reaches the best accuracy in terms of AA, OA, and Kappa metrics. Except for the Asphalt subclass, FCM-S has achieved the best accuracy in other subclasses, and the K-means algorithm also shows sub-optimal results. As the same multi-view clustering algorithm, OPMC does not show good results on all datasets, which further demonstrates the superiority of the SSF-MV algorithm.

[0197] Table 4: Performance Evaluation of Pavia University Dataset.

[0198]

[0199]

[0200] The convergence of problem (2) is analyzed through experiments. The entire algorithm is strictly executed according to the method described in Embodiment 1 of the present invention, and the number of iterations of the algorithm is set to 50 in this embodiment. Due to space limitations, only the convergence results of the Indian Pines and Salinas datasets are shown in this embodiment. The experimental results are as Figure 7 shown. It can be seen that Embodiment 1 of the present invention gradually reduces the value of the objective function until the algorithm converges, and usually converges within 20 steps.

[0201] It can be seen from the above results that the new unsupervised multi-view weighted clustering algorithm (SSF-MV) for hyperspectral data processing proposed in Embodiment 1 of the invention is easy to run in parallel, that is, large-scale data clustering can be achieved on a multi-core processor. Secondly, since the algorithm does not require graph construction and eigen-decomposition, a large amount of computational burden is avoided to a great extent.

[0202] It should be noted that the above problem (2) corresponds to formula (2) in Embodiment 1.

[0203] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-view clustering method for hyperspectral data based on spectral and spatial fusion, characterized in that: Step 1: Input hyperspectral image data to form a hyperspectral data matrix X; Step 2: Extract the spectral features of the hyperspectral image data through principal component analysis to generate the hyperspectral view data X (1) ; Step 3: Extract the spatial features of the hyperspectral data matrix X to generate the spatial view data X (2) ; Step 4: Combine the spectral view data X obtained in Step 2 (1) with the spatial view data X obtained in Step 3 (2) to construct a multi-view dataset X (v) ; Where V represents the total number of views; Step 5: Based on the multi-view dataset X (v) , establish a multi-view weighted objective function; Step 6: Iteratively optimize and update the cluster center matrix P (v) , the cluster indicator matrix G and the view weight α (v) , until the objective function converges; Step 7: Output the clustering result to complete the multi-view clustering of hyperspectral data.

2. The multispectral data multi-view clustering method based on spectrum and space fusion according to claim 1, wherein: The hyperspectral data matrix described in Step 1 Where d is the number of bands and n is the total number of pixels.

3. A hyperspectral data multi-view clustering method based on spectral and spatial fusion as described in claim 1, characterized in that: The process of extracting the spectral features of hyperspectral image data described in Step 2 includes: Step 2.1: Data standardization. For the hyperspectral data matrix X, perform standardization processing on each band: Among them, μ i is the mean of the i-th band, and σ i is the standard deviation; Step 2.2: Calculate the covariance matrix. The covariance matrix C of the standardized data is: Where the matrix dimension of the covariance matrix C is d×d; Step 2.3: Eigenvalue decomposition. Perform eigen decomposition on the covariance matrix C: C = VΛV T ; Where V is an orthogonal matrix and the column vectors are eigenvectors; Λ is a diagonal matrix and the diagonal elements are eigenvalues; Step 2.4: Select the principal components. Select the first p principal components according to the cumulative variance contribution rate; Step 2.5: Project onto a low-dimensional space and construct a projection matrix V using the first p eigenvectors p ∈R d×p , and map the original data into the low-dimensional space: Obtain hyperspectral view data X (1) .

4. A hyperspectral data multi-view clustering method based on spectral and spatial fusion according to claim 1, characterized in that: Step 3 uses a 3D convolutional neural network to extract the spatial features of the hyperspectral data matrix X and generate spatial view data X (2) ; Among them, d is the number of bands, and n is the total number of pixels.

5. A hyperspectral data multi-view clustering method based on spectral and spatial fusion according to claim 1, characterized in that: The multi-view weighted objective function described in Step 5 is Among them, α (v) represents the weight value of each view, P (v) represents the cluster center matrix, G represents the cluster indication matrix, and γ is a regularization parameter that can avoid trivial solutions.

6. The multispectral data multi-view clustering method based on spectral and spatial fusion according to claim 4, wherein: The process of iteratively optimizing and updating the cluster center matrix P (v) , the cluster indicator matrix G, and the view weight α (v) includes: Step 6.1: Fix G, α (v) , update D (v) ; Step 6.2: Fix P (v) , D (v) , α (v) , update the clustering indicator matrix G by pixel-by-pixel search; Step 6.3: Fix P (v) , G and D (v) , and update the view weight α by the Lagrange multiplier method (v) .

7. A hyperspectral data multi-view clustering method based on spectral and spatial fusion according to claim 6, characterized in that: Step 6.1 Update D by fixing G and α (v) to obtain (v) the clustering center matrix as follows: P (v) = X (v) D (v) G(G T d (v) G) -1 (6) Among them, D (v) represents the diagonal weight matrix; G represents the clustering indicator matrix.

8. A hyperspectral data multi-view clustering method based on spectral and spatial fusion according to claim 6, characterized in that: Step 6.2 Fix P (v) , D (v) , α (v) , update the clustering indicator matrix G to obtain: Among them, g * represents the optimal solution.

9. A hyperspectral data multi-view clustering method based on spectral and spatial fusion according to claim 6, characterized in that: Step 6.3 Fix P (v) , G and D (v) , update α (v) , obtain:

10. A hyperspectral data multi-view clustering method based on spectral and spatial fusion as claimed in claim 6, characterized in that: The clustering result described in step 7 includes a common clustering indication matrix G and view weights α (v) .

Images