Incomplete multispectral image classification method based on manifold double consistency guidance

Abstract

The invention discloses an incomplete multispectral image classification method based on manifold double-consistency guidance. The method comprises the following steps: acquiring incomplete spectral data in different wavebands; respectively introducing an error matrix into the incomplete spectral data of each wave band to carry out dimension completion, and recovering the dimension of the incomplete spectral data of each wave band so as to obtain recovered spectral data of each wave band; constructing an incomplete multispectral image classification model; solving the incomplete multispectral image classification model by adopting an iterative updating strategy, including fixing one variable in the model, and alternately updating other variables until convergence; and obtaining affinity graph data solved by the model, and clustering the affinity graph through K-means to obtain a classification result. According to the method, the multispectral image data is reversely recovered through the popular embedding learning represented by consensus, and the information brought by different wavebands is balanced through the adaptive weight, so that the influence of unbalanced information on the classification result is avoided.

Description

Technical Field

[0001] The invention relates to the field of remote sensing image processing, and in particular to an incomplete multispectral image classification method based on manifold double consistency guidance. Background Art

[0002] Multispectral images are usually special images taken by aircraft such as drones with multispectral imaging devices. They contain more bands and higher resolution than ordinary images, and can be continuously imaged within a certain band. They contain a large amount of spatial and spectral information of ground objects. They are widely used in the field of earth observation and play an important role in economic, agricultural, and environmental monitoring.

[0003] Multispectral image classification refers to the identification of each pixel in the image based on the obtained sample features and the classification of its category. The multispectral image classification method in the field of image processing mainly relies on the unique spectral information characteristics of different objects to classify images. With the advancement of machine learning technology, there are already methods such as support vector machines, sparse representation-based methods, and convolutional neural network-based methods.

[0004] Since the labeling process of classification labels in the acquisition of multispectral image training samples is very costly and there will be data loss during the data stringing process, how to perform unsupervised multispectral classification and restore the original multispectral data to improve the classification accuracy, that is, restore multispectral data and unsupervised classification, is an extremely challenging and practical problem. Summary of the invention

[0005] According to the technical problems of high cost of acquiring training samples and data loss in the supervised image classification method proposed above, an incomplete multispectral image classification method based on manifold double consistency guidance is provided. The present invention realizes double consistency guidance by restoring incomplete data between different multispectral data and reverse regularization. At the same time, in the representation learning of restoring missing instances, the term of reconstruction is added to dynamically explore potential consistency. In addition, an adaptive weight module is introduced for each band to avoid reducing the classification performance, thereby achieving a balance in the importance of data in different bands.

[0006] The technical means adopted by the present invention are as follows:

[0007] An incomplete multispectral image classification method based on manifold double consistency guidance is characterized by comprising the following steps:

[0008] S1, obtaining incomplete spectral data in different bands;

[0009] S2, introducing an error matrix into the incomplete spectral data of each band to complete the dimension, and restoring the dimension of the incomplete spectral data of each band, so as to obtain the restored spectral data of each band;

[0010] S3, mapping the restored spectral data of each band into a potential feature space, thereby realizing potential consensus representation learning;

[0011] S4. Perform popular embedding learning on the obtained latent consensus representation to obtain affinity graphs of each band, on the one hand, to obtain the consistency information of spectral data of different bands, and on the other hand, to further restore the spectral data of each band;

[0012] S5, constructing an objective function of an incomplete multispectral image classification model, wherein the objective function balances information brought by data of different bands through adaptive weights, and the adaptive weights are obtained by calculating the Euclidean distances of different affinity graphs;

[0013] S6, solving the objective function of the incomplete multispectral image classification model, using an iterative update strategy, fixing a certain variable in turn and updating other variables alternately until convergence; the variables include potential consensus representation, affinity graph and adaptive weight;

[0014] S7. Obtain affinity graph data for solving the objective function, cluster the affinity graph using K-means, and thus obtain classification results.

[0015] Furthermore, the restored spectral data is obtained according to the following calculation:

[0016] Recover spectral data = X (v) +E (v) W (v)

[0017] in, represents the incomplete spectral data of the υth band, d v represents the data dimension, n represents the number of samples, represents the error matrix, Represents an index matrix.

[0018] Furthermore, the latent consensus representation is learned according to the following model:

[0019]

[0020] Where P∈R c×n represents the potential consensus representation of the data, c represents the dimension of the view, n represents the number of samples, λ1 represents the preset parameter, Tr represents the trace of the matrix, and L (v) About E (v) The Laplacian matrix, U (v)∈R n×c Represents the basis matrix, which is initialized to zero and then obtained through update iteration. Represents the index matrix, which is specifically expressed as follows:

[0021]

[0022] Furthermore, popular embedding learning is performed according to the following model:

[0023]

[0024] Among them, S∈R n×n It represents the affinity graph of the υth band, and β and λ2 represent the preset parameters.

[0025] Furthermore, the objective function of the incomplete multispectral image classification model is:

[0026]

[0027] Among them, α (v) Represents the weight of the υth band data.

[0028] Furthermore, the affinity graph is clustered by K-means to obtain the classification results, including:

[0029] Use spectral clustering model to low-rank constrain affinity graph:

[0030]

[0031] F T F=I

[0032] where F∈R n×c is the low-dimensional representation matrix, F T F = I is a constraint to avoid trivial solutions, λ is the regularization parameter, c is the number of clusters, L S represents the Laplacian matrix of the affinity graph S.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] The present invention discloses an incomplete multispectral image classification method based on manifold dual consistency guidance, firstly, the original incomplete multispectral data is dimensionally restored by introducing an error matrix, then the potential consensus representation is learned from the aligned multispectral data; then the learned consensus representation is subjected to popular embedding learning to explore the consistency information of the multispectral data, thereby aligning the error matrix; the information brought by different bands is balanced by adaptive weights; an optimization objective function is constructed and solved; an iterative update strategy is used to alternately update other variables while fixing one variable until convergence; finally, the consensus representation is subjected to K-means clustering to obtain the final classification result. The present invention reversely restores multispectral data through popular embedding learning of consensus representation, and balances the information brought by different bands through adaptive weights, thereby avoiding the influence of unbalanced information on the classification result. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 The present invention provides a specific implementation process of an incomplete multispectral image classification method based on manifold dual consistency guidance in an embodiment of the present invention.

[0037] Figure 2 It is a model architecture for implementing the incomplete multispectral image classification method based on manifold double consistency guidance in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0039] like Figure 1-2 As shown, the present invention provides an incomplete multispectral image classification method based on manifold double consistency guidance, comprising the following steps:

[0040] Step 1: Obtain incomplete spectral data in different bands. In this embodiment, given incomplete multi-spectral data X = {X 1 ,X2 ,...,X υ}, where X υ Represents the data of the υth view.

[0041] Step 2: The error matrix is ​​introduced into the incomplete spectral data of each band to complete the dimension, so as to restore the dimension of the incomplete spectral data of each band, thereby obtaining the restored spectral data of each band.

[0042] In this embodiment, the error matrix is ​​introduced to obtain the complete restored spectrum data as X (v) +E (v) W (v) ;in Represents the υth band, where the data dimension is d v , there are n samples. The error matrix is ​​initialized to zero and needs to be updated and iterated later to restore the true dimension of the multispectral data. Represents the index matrix, whose purpose is to save the location of missing data. Specifically, it can be expressed as follows:

[0043]

[0044] Step 3: Use the restored spectral data obtained above to perform potential consensus representation learning.

[0045] Specifically, the potential consensus representation learning model in this application is:

[0046]

[0047] Where P∈R c×n represents the potential consensus representation of the data, λ1 represents the first preset parameter, which is set to 0.001. Its function is to balance the impact of different views, Tr represents the trace of the matrix, and L (v) About E (v) The Laplacian matrix, U (v) ∈R n×c Represents the basis matrix, the purpose is to map the original data space to the potential feature space of the data, initialized to a zero matrix and then obtained through update iterations. represents the index matrix and c represents the dimension of the view.

[0048] Step 4: In order to fully explore the consistency information of different views and align the error matrix to ensure that the multispectral data can be restored and obtain high-performance classification results, the obtained potential consensus representation is subjected to popular embedding learning to obtain affinity graphs for each band, including:

[0049] The obtained latent consensus representation is subjected to popular embedding learning:

[0050]

[0051] Where S represents the affinity graph, and constraints are imposed on it To ensure the block diagonal structure of the affinity graph, β and λ2 represent preset parameters. Both parameters are set to 0.001 to balance the impact of different views.

[0052] Step 5: Balance the information of different bands through adaptive weighting to avoid reducing the classification performance. The overall model constructed is as follows:

[0053]

[0054] Among them, α (v) represents the weight of the band, To ensure the smoothness of weight distribution, U (v)T U (v) =I is to avoid infinite solutions.

[0055] Step 6: Solve the objective function of the incomplete multispectral image classification model, and use an iterative update strategy to fix one variable and update other variables alternately until convergence; the variables include potential consensus representation, affinity graph, and adaptive weight. This step specifically includes:

[0056] Step 6.1 Update P. Based on the matrix trace operation rules and iterative solution method, P is finally written as follows:

[0057]

[0058] Step 6.2 Update S. Under the condition that P is constant, we can get:

[0059]

[0060] in

[0061] Step 6.3, Update α (v) , under certain conditions of S and P, we can get:

[0062]

[0063] Step 7: Finally, perform K-means clustering on the affinity graph to obtain the classification result. This includes: using the spectral clustering model to constrain the affinity graph to a low rank:

[0064]

[0065] F T F=I

[0066] where F∈R n×c is the low-dimensional representation matrix, F T F = I is a constraint to avoid trivial solutions, λ is the regularization parameter, c is the number of clusters, L S represents the Laplacian matrix of the affinity graph S.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An incomplete multispectral image classification method based on manifold bi-consistency guidance, characterized in that: The following steps are involved: S1, obtaining incomplete spectral data in different bands; S2, introducing an error matrix into the incomplete spectral data of each band to complete the dimension, and restoring the dimension of the incomplete spectral data of each band, so as to obtain the restored spectral data of each band; S3, mapping the restored spectral data of each band into a potential feature space, thereby realizing potential consensus representation learning; S4. Perform popular embedding learning on the obtained latent consensus representation to obtain affinity graphs of each band, on the one hand, to obtain the consistency information of spectral data of different bands, and on the other hand, to further restore the spectral data of each band; S5, constructing an objective function of an incomplete multispectral image classification model, wherein the objective function balances information brought by data of different bands through adaptive weights, and the adaptive weights are obtained by calculating the Euclidean distances of different affinity graphs; S6, solving the objective function of the incomplete multispectral image classification model, using an iterative update strategy, fixing a certain variable in turn and updating other variables alternately until convergence; the variables include potential consensus representation, affinity graph and adaptive weight; S7. Obtain affinity graph data for solving the objective function, cluster the affinity graph using K-means, and thus obtain classification results.

2. The incomplete multispectral image classification method based on manifold bi-consistency guidance according to claim 1 is characterized in that: The restored spectral data is obtained according to the following calculation: Recover spectral data = X (v) +E (v) W (v) in, represents the incomplete spectral data of the υth band, d v represents the data dimension, n represents the number of samples, represents the error matrix, Represents an index matrix.

3. The incomplete multispectral image classification method based on manifold bi-consistency guidance according to claim 2 is characterized in that: The latent consensus representation is learned according to the following model: Where P∈R c×n represents the potential consensus representation of the data, c represents the dimension of the view, n represents the number of samples, λ1 represents the preset parameter, Tr represents the trace of the matrix, and L (v) About E (v) The Laplacian matrix, U (v) ∈R n×c Represents the basis matrix, which is initialized to zero and then obtained through update iteration. Represents the index matrix, which is specifically expressed as follows:

4. The incomplete multispectral image classification method based on manifold bi-consistency guidance according to claim 3 is characterized in that: The resulting latent consensus representations are subjected to popular embedding learning according to the following model: Among them, S∈R n×n It represents the affinity graph of the υth band, and β and λ2 represent the preset parameters.

5. The incomplete multispectral image classification method based on manifold bi-consistency guidance according to claim 1 is characterized in that: The objective function of the incomplete multispectral image classification model is: Among them, α (v) Represents the weight of the υth band data.

6. The incomplete multispectral image classification method based on manifold bi-consistency guidance according to claim 5 is characterized in that: Finally, the affinity graph is clustered by K-means to obtain the classification results, including: Use spectral clustering model to low-rank constrain affinity graph: s.t.F T F=I where F∈R n×c is the low-dimensional representation matrix, F T F = I is a constraint to avoid trivial solutions, λ is the regularization parameter, c is the number of clusters, L S represents the Laplacian matrix of the affinity graph S.

Images