Incomplete hyperspectral image classification method based on sample-level weighting

Abstract

The invention discloses an incomplete hyperspectral image classification method based on sample-level weighting. The method comprises the following steps: acquiring incomplete hyperspectral samples of different wavebands; a Laplacian matrix is introduced for existing incomplete hyperspectral samples of each wave band for completion; learning a particular representation from the complemented hyperspectral sample; introducing a generalized non-convex tensor; performing sample-level weighting on the specific representation of each wave band; iteratively updating the strategy, and updating a certain variable under the condition that other variables are fixed; and performing spectral clustering on the learned consensus representation to obtain an image classification result. According to the method, the incomplete hyperspectral data is complemented through the geometric structure of the original sample so as to maintain the manifold structure of the sample in the original space, the specific representation of the wave band is subjected to sample-level weighting, and the influence of different samples on the contribution of the wave band during hyperspectral information fusion is considered, so that the image classification accuracy is improved.

Description

Technical Field

[0001] The invention relates to the field of remote sensing image processing, and in particular to an incomplete hyperspectral image classification method based on sample-level weighting. Background Art

[0002] Hyperspectral imaging is an important remote sensing technology that collects electromagnetic spectra from visible light to near-infrared bands. In a hyperspectral image, each pixel can be regarded as a high-dimensional vector, and the value of the pixel corresponds to the spectral reflectance of a specific wavelength.

[0003] Hyperspectral image classification is the task of classifying the class label of each pixel in the image. At present, hyperspectral remote sensing image classification is widely used in agriculture, military, marine management and geological exploration, and hyperspectral image classification technology has become an important part of modern science and technology.

[0004] It is difficult to obtain a large number of manually labeled classification samples of hyperspectral images, and there may be occlusions or data loss in the process of taking hyperspectral images. Therefore, how to restore missing data based on existing data and improve the accuracy of unsupervised classification is the main problem that we urgently need to solve. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides an incomplete hyperspectral image classification method based on sample-level weighting. The present invention completes the incomplete hyperspectral data through the geometric structure of the original sample to maintain the manifold structure of the sample in the original space, and performs sample-level weighting on the specific representation of the band, taking into account the influence of different samples on the contribution of the band when the hyperspectral information is fused, thereby improving the image classification accuracy.

[0006] The technical means adopted by the present invention are as follows:

[0007] A method for classifying incomplete hyperspectral images based on sample-level weighting includes the following steps:

[0008] S1, obtain incomplete hyperspectral samples in different bands;

[0009] S2. constructing an incomplete hyperspectral matrix based on incomplete hyperspectral samples of different bands, introducing a Laplace matrix for existing incomplete hyperspectral samples of each band, and completing the missing data of the incomplete hyperspectral matrix under the constraint of the Laplace matrix, thereby generating a completed hyperspectral sample matrix;

[0010] S3, learning specific representations from the completed hyperspectral samples to construct specific representations for each band;

[0011] S4. Introduce a generalized tensor based on non-convex functions to perform sample-level weighted fusion on the specific representation of each band. In the process of weighted fusion, the influence of different samples on the band contribution is established through the Euclidean norm, thereby constructing a consensus representation;

[0012] S5, build an image classification model based on the completed hyperspectral sample matrix, the specific representation of each band and the consensus representation;

[0013] S6, introducing auxiliary variables into the image classification model, and using an iterative update strategy to update the variables in the model in sequence while keeping other variables fixed, until the image classification model converges;

[0014] S7. Obtain a consensus representation when the model converges, perform spectral clustering on the consensus representation, and obtain an image classification result.

[0015] Furthermore, a Laplace matrix is ​​introduced for the existing incomplete hyperspectral samples in each band, and the missing data of the incomplete hyperspectral samples are supplemented under the constraints of the Laplace matrix. The specific calculation formula is as follows:

[0016]

[0017] Among them, X (v) represents the completed hyperspectral sample matrix, represents the existing feature matrix of the vth band, represents the missing feature matrix of the vth band restored under the constraint of the Laplacian matrix, and Represents a predefined permutation matrix based on the locations of existing and missing samples in the band, combining the existing feature matrix and the missing feature matrix into a complete matrix d v and n v They represent the dimension of existing samples and the number of existing samples in the vth band respectively, and n represents the total number of samples.

[0018] Furthermore, specific representations are learned from the completed hyperspectral samples through the following model:

[0019]

[0020]

[0021] Among them, tr represents the trace of the matrix, L (v) Represents the Laplacian matrix of the existing hyperspectral samples in the vth band, Z (v) ∈R n×n represents the specific affinity matrix of the vth band, represents the L2 norm, λ1 and λ2 represent regularization parameters.

[0022] Furthermore, a generalized non-convex tensor is introduced through the following model to perform sample-level weighting on the specific representation of each band:

[0023]

[0024] in represents a tensor,

[0025] represents the non-convex function performed on the singular values ​​of its horizontal slices to enhance the low rank property, Z (v) ∈R n×n represents the affinity matrix of the vth band, W (v) ∈R n×n Represents the weight matrix of the vth band, A∈R n×n It represents the fused affinity matrix obtained by fusing the band-specific affinity matrices.

[0026] Furthermore, the image classification model is:

[0027]

[0028] The model after introducing auxiliary variables is:

[0029]

[0030] in, It is an auxiliary variable introduced to assist the tensor The solution, is the Lagrange multiplier, μ is the penalty factor, used to balance

[0031] The weights in the model.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] The present invention discloses an incomplete hyperspectral image classification method that obtains a unified affinity matrix through sample-level weighting. This is a novel incomplete classification method. First, the data of missing samples are completed under the constraint of the Laplace matrix of existing samples to obtain complete hyperspectral data. Then, the completed hyperspectral data is fitted to obtain a specific affinity matrix for each band. Then, a generalized non-convex approximation based on the low-rank property of tensors is introduced to explore the high-dimensional spatial structure of the data. Then, the band-specific affinity matrix is ​​weighted at the sample level to obtain a unified affinity matrix. Finally, the learned affinity matrix is ​​spectrally clustered to obtain the final result. The present invention completes the incomplete hyperspectral data through the geometric structure of the original sample to maintain the manifold structure of the sample in the original space, and performs sample-level weighting on the specific representation of the band, taking into account the influence of different samples on the contribution of the band when the hyperspectral information is fused. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0035] Figure 1 The present invention is a flow chart of an incomplete hyperspectral image classification method based on sample-level weighting.

[0036] Figure 2 The present invention provides a specific execution flow of an incomplete hyperspectral image classification method based on sample-level weighting in an embodiment of the present invention.

[0037] Figure 3 This is a network architecture diagram for implementing an incomplete hyperspectral image classification method based on sample-level weighting in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0039] like Figure 1 As shown, the present invention discloses an incomplete hyperspectral image classification method based on sample-level weighting, comprising the following steps:

[0040] S1. Obtain incomplete hyperspectral samples in different bands.

[0041] S2. An incomplete hyperspectral matrix is ​​constructed based on incomplete hyperspectral samples of different bands. A Laplace matrix is ​​introduced for the existing incomplete hyperspectral samples of each band. The missing data of the incomplete hyperspectral matrix is ​​completed under the constraint of the Laplace matrix, thereby generating a completed hyperspectral sample matrix.

[0042] S3. Learn specific representations from the completed hyperspectral samples to construct an affinity matrix for each band.

[0043] S4. A generalized tensor based on non-convex functions is introduced to perform sample-level weighted fusion on the affinity matrix of each band. In the process of weighted fusion, the influence of different samples on the contribution of the band is established through the Euclidean norm, thereby constructing the fused affinity matrix.

[0044] S5. Build an image classification model based on the completed hyperspectral sample matrix, the affinity matrix of each band, and the fused affinity matrix.

[0045] S6, introducing auxiliary variables into the image classification model, and using an iterative update strategy to update each variable in the model in turn while keeping other variables fixed, until the image classification model converges;

[0046] S7. Obtain a fused affinity matrix when the model converges, and perform spectral clustering on the fused affinity matrix to obtain an image classification result.

[0047] The solution of the present invention is further described below through specific application examples.

[0048] like Figure 2-3 As shown, this embodiment provides an incomplete hyperspectral image classification method based on sample-level weighting, comprising the following steps:

[0049] Step 1: Given incomplete hyperspectral data X = {X (1) ,X (2) ,...,X (v)}, where X (v) Represents the data of the vth band, where missing data is filled with 0.

[0050] Step 2: Introduce the Laplacian matrix of existing samples in each band and complete the missing data under its constraints to maintain the geometric structure of the original data;

[0051] The missing data mentioned are completed as follows:

[0052]

[0053] in represents the existing feature matrix of the vth band, represents the missing feature matrix of the vth band, and Represents two permutation matrices, combining the existing feature matrix and the missing feature matrix into a complete matrix L (v) ∈R n×n It represents the Laplacian matrix of the complete data constructed by the Laplacian matrix of the original sample through the indicator matrix of the vth band. This formula fully explores the role of the similarity graph of the original sample in completing the missing data.

[0054] Step 3: Learn a specific representation Z from the completed band data (v) , i.e., affinity graph data, to preserve the specific structure within the band;

[0055] The learning of the intra-band structure can be obtained by the following formula:

[0056]

[0057] Among them, Z (v) ∈R n×n represents the specific representation of the vth band, and λ2 represents the regularization parameter.

[0058] Step 4: Introduce a generalized non-convex tensor to alleviate the biased estimation of the rank function while maintaining its low rank and inter-band structure;

[0059] The learning of inter-band structure is as follows:

[0060]

[0061] in, represents a tensor, and ψ γ (·) is about the adjustment parameter γ∈R + A non-convex function of

[0062] Step 5: Perform sample-level weighting on the specific representation of each band, and obtain the consensus representation A based on the influence of different samples on the band contribution when multi-band information is fused, that is, the fused affinity graph data;

[0063] The learning of consensus representation can be obtained by the following formula:

[0064]

[0065] stdiag(A)=0,A ij ≥0

[0066] Among them, W (v) ∈R n×n Represents the weight matrix of the vth band, A∈R n×n is the learned consensus representation.

[0067] Step 6: Combine the above steps into a unified framework and introduce corresponding auxiliary variables for optimization;

[0068] Step 6.1 The overall formula is as follows:

[0069]

[0070] Step 6.2 The optimization formula after adding auxiliary variables is as follows:

[0071]

[0072] in, is an auxiliary variable introduced to help tensors The solution, is the Lagrange multiplier, μ is the parameter.

[0073] Step 7: Using an iterative update strategy, update a variable while keeping other variables fixed. The variable includes the completed hyperspectral sample matrix, auxiliary variables, specific representation, consensus representation, weight matrix, Lagrange multiplier, and penalty factor. Specifically:

[0074] Step 7.1 Update X (v) , fixing other variables, we can get:

[0075]

[0076] The above formula is about X (v) Taking the derivative and setting it to 0, we get:

[0077]

[0078] Among them, M (v) =λ1L (v) +λ2(Z (v) -I)((Z (v) ) T -I),

[0079] Step 7.2 Update Fixing other variables yields:

[0080]

[0081] Using the generalized weighted singular value threshold operator to solve, we can get:

[0082]

[0083] Among them, UΣV T Yes D (v) The singular value decomposition of δ i D (v) The i-th singular value of .

[0084] Step 7.3 Update Z (v) , fixing other variables, we can get:

[0085]

[0086] For the above formula about Z (v) Taking the derivative and setting it to 0, we get:

[0087] (2λ2(X (v) ) T X (v) +μI)Z (v) +Z (v) (2W (v) )=2W (v) +2λ2(X (v) ) T X (v) +μQ (v) -C (v)

[0088] This is a typical Sylvester equation and can be solved using the Bartels-Stewart algorithm.

[0089] Step 7.4 Update A and fix other variables to obtain:

[0090]

[0091] Taking the derivative of the above formula with respect to A and setting it to 0, we get:

[0092]

[0093] Step 7.5 Update W (v) , fixing other variables, we can get:

[0094]

[0095] Among them, W (v) is a diagonal matrix, Represents the values ​​of the elements on its diagonal.

[0096] Step 7.6 Update C (v) and μ, and fixing other variables, we can get:

[0097] C (v) =C (v) +μ(Z (v) -Q (v) )

[0098] μ=min(ρμ,μ0)

[0099] Step 8: Perform spectral clustering on the learned consensus representation A to obtain the corresponding results.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for classifying incomplete hyperspectral images based on sample-level weighting, characterized in that: The following steps are involved: S1, obtain incomplete hyperspectral samples in different bands; S2. constructing an incomplete hyperspectral matrix based on incomplete hyperspectral samples of different bands, introducing a Laplace matrix for existing incomplete hyperspectral samples of each band, and completing the missing data of the incomplete hyperspectral matrix under the constraint of the Laplace matrix, thereby generating a completed hyperspectral sample matrix; S3, learning specific representations from the completed hyperspectral samples to construct specific representations for each band; S4. Introduce a generalized tensor based on non-convex functions to perform sample-level weighted fusion on the specific representation of each band. In the process of weighted fusion, the influence of different samples on the band contribution is established through the Euclidean norm, thereby constructing a consensus representation; S5, build an image classification model based on the completed hyperspectral sample matrix, the specific representation of each band and the consensus representation; S6, introducing auxiliary variables into the image classification model, and using an iterative update strategy to update the variables in the model in sequence while keeping other variables fixed, until the image classification model converges; S7. Obtain a consensus representation when the model converges, perform spectral clustering on the consensus representation, and obtain an image classification result.

2. The incomplete hyperspectral image classification method based on sample-level weighting according to claim 1 is characterized in that: The Laplace matrix is ​​introduced for the existing incomplete hyperspectral samples in each band, and the missing data of the incomplete hyperspectral samples are supplemented under the constraints of the Laplace matrix. The specific calculation formula is as follows: Among them, X (v) represents the completed hyperspectral sample matrix, represents the existing feature matrix of the vth band, represents the missing feature matrix of the vth band restored under the constraint of the Laplacian matrix, and Represents a predefined permutation matrix based on the locations of existing and missing samples in the band, combining the existing feature matrix and the missing feature matrix into a complete matrix d v and n v They represent the dimension of existing samples and the number of existing samples in the vth band respectively, and n represents the total number of samples.

3. The incomplete hyperspectral image classification method based on sample-level weighting according to claim 2 is characterized in that: Specific representations are learned from the completed hyperspectral samples through the following model: Among them, tr represents the trace of the matrix, L (v) Represents the Laplacian matrix of the existing hyperspectral samples in the vth band, Z (v) ∈R n ×n represents the specific affinity matrix of the vth band, represents the L2 norm, λ1 and λ2 represent regularization parameters.

4. The incomplete hyperspectral image classification method based on sample-level weighting according to claim 3 is characterized in that: A generalized non-convex tensor is introduced through the following model to perform sample-level weighting on the specific representation of each band: in represents a tensor, represents the non-convex function performed on the singular values ​​of its horizontal slices to enhance the low rank property, Z (v) ∈R n×n represents the affinity matrix of the vth band, W (v) ∈R n×n Represents the weight matrix of the vth band, A∈R n×n It represents the fused affinity matrix obtained by fusing the band-specific affinity matrices.

5. The incomplete hyperspectral image classification method based on sample-level weighting according to claim 4 is characterized in that: The image classification model is: The model after introducing auxiliary variables is: in, It is an auxiliary variable introduced to assist the tensor The solution, is the Lagrange multiplier, μ is the penalty factor, used to balance The weights in the model.

Images