Classification Method Based on Kernel Tensor Slice Sparse Coding

Abstract

Accurate crop classification plays an important role in the agricultural field. An ideal data source for accurate crop classification is high spatial resolution hyperspectral imagery (H2 imagery) obtained by drones. However, for images with many different types of crops, crop classification of drone H2 imagery is a huge challenge. The significant spectral diversity, spatial heterogeneity, and non-linear data structure of drone H2 imagery result in poor spectral discriminability. To improve discriminability, the present invention proposes a classification method based on kernel tensor sliced sparse coding for accurate crop classification of drone H2 imagery. The kernel tensor representation mechanism in KTSSCC can reduce linear inseparability while well preserving the spectral features and spatial constraints of land covers, thus greatly improving discriminability. In addition, the present invention proposes a kernel tensor sliced sparse orthogonal matching pursuit method for optimizing the kernel tensor sliced sparse coding in the spectral space, greatly reducing the computational cost.

Description

Technical Field

[0001] The present invention relates to the technical field of precise crop classification, and particularly to a classification method based on kernel tensor slice sparse coding. Background Art

[0002] At present, precise crop classification is important for agricultural monitoring and food security assessment. Benefiting from the fine spectral responses of crop attributes, hyperspectral remote sensing has been a popular crop classification tool in recent years. In southern China, there are many small agricultural areas. Therefore, the average plot size is much smaller than the coverage area of a single pixel in satellite-borne or airborne hyperspectral images, making it difficult to correctly classify crops. Unmanned aerial vehicle (UAV) technology has developed rapidly in the past decade. Therefore, we can obtain rich hyperspectral imagery with high spatial resolutions (H2 imagery) in real time at low cost through an airborne UAV system. The airborne H2 imagery not only provides detailed spectral attribute information of ground objects but also provides their precise spatial geometries. Therefore, it is an ideal data source for precise crop classification.

[0003] However, crop classification of airborne H2 imagery faces many challenges. Some of them are common problems in hyperspectral imagery classification (HIC), such as the curse of dimensionality and complex non-linear data structures. Other problems are more specific to airborne H2 imagery; for example, the extremely high spatial resolution results in significant spectral diversity and spatial heterogeneity, leading to poor spectral discriminability. Therefore, classifiers that only use spectral information cannot obtain satisfactory classification results and introduce serious salt-and-pepper (SP) noise into the classification map. Therefore, the spectral features of airborne H2 imagery must be well represented to improve spectral discriminability.

[0004] In recent years, deep learning has been a research hotspot in many fields. To improve the generalization ability of the HIC method, various deep learning models have been improved and used to mine the high-level semantic features of hyperspectral images, such as convolutional neural network (CNN), deep residual network, and graph convolutional network (GCN). These models have led to a huge leap in HIC. However, to obtain robust classification performance, some requirements need to be met. First, many model parameters need to be adjusted, which is a very arduous and boring task. Second, they usually suffer from high computational costs. Finally, these models require a large number of labeled samples, which are very limited in the crop classification of UAV H2 images.

[0005] Sparse representation is a simple and effective method for spectral feature representation. Although hyperspectral images are high-dimensional data, hyperspectral pixels from the same class are roughly located in the same low-dimensional spectral subspace. Therefore, a small number of atoms in the structured dictionary can sparsely represent unknown pixels. In addition, the relevant sparse representation vectors can implicitly encode class information. By combining spatial homogeneity with this sparse representation mechanism, Chen et al. proposed JSCC for HIC. JSCC is a parameter-free method that achieved the highest HIC accuracy at that time. Since its proposal, it has attracted extensive attention in the field of remote sensing, and various improvements to JSCC have been proposed in recent years. The JSCC model and its various improvements can well extract and utilize the sparsity and spatial homogeneity of hyperspectral data to obtain accurate HIC results.

[0006] However, as we all know, hyperspectral images are data generated by the interaction of three factors: spectrum, spatial width, and spatial height. The JSCC model and its improvements represent hyperspectral images as a matrix, which only considers two spatial factors and loses its three-factor structure. Therefore, a more suitable data representation method is needed to effectively retain the inherent spatial-spectral structure of hyperspectral data. A natural choice is to describe it as a three-dimensional tensor. Yang et al. proposed SSCTC-CDR and HPSCC-SNT, which achieved sparse coding classification by using the sparsity of spatial neighbor tensors along the spectral modality. The tensor sparse coding of hyperspectral data can better combine its sparsity with spatial information, effectively retain the spatial-spectral structure information, and thus significantly improve the classification accuracy. In addition, the spatial information contained in UAV H2 images is more abundant, so the impact of tensor representation on the classification of UAV H2 images will be more significant.

[0007] However, it should be noted that the HPSCC-SNT and SSCTC-CDR methods cannot effectively handle complex non-linear data structures. A proven solution to this problem is to project the data into a high-dimensional feature space, in which the data becomes linearly separable. As is well known, the kernel trick can achieve such a non-linear transformation. In addition, it avoids the explicit evaluation of the data in the feature space. Many studies have shown that the use of kernel-based methods can significantly improve the performance of hyperspectral or UAV H2 image classification. Summary of the Invention

[0008] In view of this, the present application proposes a classification method based on kernel tensor slice sparse coding, including the following steps:

[0009] Step 1, the UAV H2 image is a high spatial resolution hyperspectral image (high spatial resolutions hyperspectral imagery, H2 image). For each unknown UAV H2 pixel, select the pixels in the small window centered on it as its spatial neighbors;

[0010] Step 2, generate its kernel spatial neighbor tensor (KSNT) by arranging the representations of these spatially adjacent pixels in the feature space according to their spatial positions;

[0011] Step 3, due to the sparsity and spatial homogeneity of the UAV H2 image, the spectral vectors in each generated KSNT are expected to be sparsely represented by the spectral vectors of several common training pixels;

[0012] Step 4, the sparse coding coefficients related to the KSNT form a slice sparse tensor;

[0013] Step 5, denote this tensor as the slice sparse coding tensor (SSCT).

[0014] Preferably, in Step 5, instead of directly calculating the SSCT in the high-dimensional feature space, the kernel trick is used in the spectral space, and the kernel tensor slice sparse orthogonal matching pursuit (KTSSOMP) method is used to adaptively learn the SSCT. The SSCT is discriminative and is used to achieve crop classification of UAV H2 images.

[0015] Preferably, in Step 3, represent the spectral vector corresponding to the spectral pixel as The classification of UAV H2 images aims to infer its label y∈{1,2,…,C} from x and a set of labeled hyperspectral pixels S={(x1,y1),(x2,y2),…,(x L ,y L )}; in this classification task, the labeled pixels are selected as training samples, and the other unlabeled pixels are used for testing.

[0016] Preferably, in order to better describe the new classifier we proposed, we briefly introduce some definitions related to tensors; a tensor is a multi-dimensional array of numbers

[27] ; a tensor can be expressed as where the dimension M is the order of the tensor; in particular, scalars, vectors, and matrices are also tensors; X Any element in is denoted as k ≤I k , 1≤k≤M; then, we briefly introduce several related definitions of tensors;

[0017] (1) d-mode vector

[0018] This concept is similar to the rows and columns of a matrix; a vector in the d-mode vector can be obtained by changing the index i d while fixing other indices; for example, the 1-mode can be denoted as

[0019] (2) Mode-d matrixization

[0020] Matrixization is the process of expanding an M-order tensor into a matrix; the mode-d matrixization of the tensor X is denoted as where This indicates that the index i d should be fixed and other indices should be changed; Figure 3 Visual descriptions of the d-mode vector and mode-d matrixization of a 3-order tensor are given in

[0021] (3) Mode-d product

[0022] The mode-d product of a tensor and a matrix is also a tensor, which can be expressed as Its element form is as follows

[0023]

[0024] (4) Frobenius norm

[0025] The Frobenius norm of the tensor is given by

[0026]

[0027] Preferably, for non-linearly separable hyperspectral pixels, the mapping can project their spectral features into a high-dimensional feature space where the pixels are linearly separable; Let be an unknown test sample, and assume that it can be linearly represented by several training samples in the feature space; and is the representation of x in the feature space, then φ(x) can be sparsely represented as follows

[0028]

[0029] where the structured dictionary the l-th column (l = 1, …, L) of φ(x l ) represents the representation of the training sample x l in the feature space, and α is an unknown sparse vector;

[0030] In addition to the sparsity assumption, it is well known that spatially adjacent hyperspectral pixels are likely to belong to the same class. This means that under the dictionary D φ , the spectral vectors of x and its spatial neighbors share a common sparse pattern; To utilize the common sparsity to improve the classification performance, this paper selects the pixels in the T×T spatial window centered on x as its spatial neighbors, and then transforms the representations of these pixels according to their spatial positions in the feature space, thereby generating its KSNT, denoted as 2 the tensor tensor can be Tucker decomposed into

[0031]

[0032] where is the core tensor, and the factor matrices Ψ1 = Ψ2 = I T , Ψ3 = D φ . Benefiting from the particularity of the matrices Ψ1 and Ψ2, formula (4) can be simplified to

[0033]

[0034] From the common sparse pattern of spatially adjacent pixels under the dictionary D φ , it can be inferred that the core tensor A is slice-sparse with respect to the 3-modal dictionary D φ . The SSCTA can be calculated by solving the following optimization problem

[0035]

[0036] where · Fis the Frobenius norm of the tensor, A0 is the number of positive slices of the tensor of A with non - zero values, and S is the preset Sparsity;

[0037] Instead of directly solving SSCT in the feature space, this application proposes a KTSSOMP algorithm that uses the kernel trick to solve the optimization problem formula (6); then the pixels are classified into the category closest to KSNT.

[0038] Preferably, in step five, the kernel function κ: (κ(x,y) = φ(x),φ(y)) is used to simplify the calculation. consists of training samples sorted by category. is a tensor generated by rearranging the spectral vectors of T 2 pixels in the spatial neighborhood of x. It can be found that D φ = φ(D), X φ = φ(X); Denote each 3 - mode vector of X φ as (X φ )(i,j = 1,2,…,T); The correlation between it and the atoms of the dictionary D i,j,: can be calculated by the following formula φ

[0039] (X φ ) i,j,: ,φ(x l ) = κ(X i,j,: ,x l )(i,j = 1,2,…,T; l = 1,2,…,L). Similarly for formula (7), the correlation between two atoms of the dictionary D φ can be calculated using

[0040] φ(x l ),φ(x m ) = κ(x l ,x m )(l,m = 1,2,…,L) formula (8)

[0041] Calculate; Let be the kernel tensor with the (i,j,l) - th element being κ(X i,j,: ,x l ), be the kernel matrix with the (l,m) - th element being κ(x l ,x m ); Assume a set of atoms {φ(x l )} l∈Λ is selected, and (X φ ) i,j,: (i,j = 1,2,…,T) can be approximately represented as

[0042]

[0043] Here, λ is a very small scalar, and the regularization term is added to obtain a stable inverse; then ( X φ ) i,j,: (i, j = 1, 2, …, T) and the residual vector between its approximation is

[0044] ( R φ ) i,j,: = ( X φ ) i,j,: - ( K X ,D ) :,:,Λ ×3(D φ ) :,Λ ((K D ) Λ,Λ + λI) -1 Equation (10)

[0045] ( R φ ) i,j,: and φ(x l ) is

[0046] <( R φ ) i,j,: , φ(x l )> = κ( X i,j,: , x l ) - ( K X ,D ) :,:,Λ ×3(K D ) Λ,l ((K D ) Λ,Λ + λI) -1 Equation (11)

[0047] Then the (i, j, l)-th element of the correlation tensor <( R φ ) i,j,: , φ(x l )> can be obtained from

[0048] C = K X ,D - ( K X ,D ) :,:,Λ ×3(K D ) :,Λ ((KD ) Λ,Λ +λI) -1 Equation (12)

[0049] Calculate; in KTSSOMP, at each iteration, the atoms that simultaneously approximate all T 2 pixels should be selected; specifically, in the initialization step, the selected atoms should be related to ( X φ ) i,j,: (i, j = 1, 2,..., T) with the maximum correlation; in other words, we select the new atom with the highest || K X ,D || F from the positive slice; after initialization, the newly selected atoms should be related to ( R φ ) i,j,: (i, j = 1, 2,..., T) with the maximum correlation; in other words, the newly selected atoms we choose should have the highest || C || F .

[0050] Preferably, the kernel tensor slice sparse orthogonal matching pursuit (KTSSOMP) method is as follows:

[0051] Input: L labeled hyperspectral vectors and unlabeled hyperspectral vectors kernel function κ, sparsity level S;

[0052] Output: SSCT A ;

[0053] Step 1: Construct the dictionary matrix and the spatial neighbor tensor

[0054] Step 2: Calculate the kernel matrix K with the (l, m) - th element being κ(x l , x m ) and the kernel tensor with the (i, j, l) - th element being κ( D i,j,: X i,j,: , x l )

[0055] Step 3: Set s = 0 and initialize

[0056] Step 4: When s ≤ S, execute the loop

[0057] 1) Calculate the correlation tensor

[0058] 2) Search for new indices

[0059] 3) Merge the selected indices Λ s = [Λ s-1 , λ s ;

[0060] 4) s = s + 1;

[0061] 5) Return the index set Λ = Λ s-1 , SSCT A , its non - zero slices ( K X u,D ) :,:,Λ ×3((K D )) Λ,Λ + λI) -1 Indexed by Λ

[0062] End the loop.

[0063] Preferably, once SSCT A is obtained, the residual between the KSNT of the unknown pixel x and the approximation of the training samples of the c - th class can be calculated by the following formula:

[0064]

[0065] where is the sub - dictionary composed of the spectral features of the training samples of the c - th class, is the representation of D c in the feature space, consisting of the positive slices of L corresponding to D c in A, c is the kernel tensor whose (i, j, l c ) - th element is κ() X i,j,: , x lc ), is the kernel matrix whose (l c , m c ) - th element is κ(x lc , x mc )(x lc , x mc is an atom of D c );

[0066] Then the label of x is predicted by

[0067]

[0068] Predict.

[0069] Classification method based on kernel tensor slice sparse coding (KTSSCC). For each unknown UAV H2 pixel, pixels in a small window centered on it are selected as its spatial neighbors. Then, a kernel spatial neighbor tensor (KSNT) is generated by arranging the representations of these spatially adjacent pixels in the feature space according to their spatial positions. Due to the sparsity and spatial homogeneity of UAV H2 images, the spectral vectors in each generated KSNT are expected to be sparsely represented by the spectral vectors of several common training pixels. As a result, the sparse coding coefficients associated with the KSNT form a slice sparse tensor. This application denotes this tensor as the slice sparse coding tensor (SSCT). Instead of directly calculating the SSCT in the high-dimensional feature space, this application uses the kernel trick in the spectral space. This application proposes a kernel tensor slice sparse orthogonal matching pursuit (KTSSOMP) algorithm to adaptively learn the SSCT. The SSCT is discriminative and can be used to achieve crop classification of UAV H2 images.

[0070] The advantages of this invention are as follows: 1) Using the kernel tensor representation scheme in the feature space, for UAV H2 image pixels, KTSSCC can fully preserve spectral features and spatial constraints while improving linear separability. Thus, the crop classification performance of KTSSCC is greatly improved.

[0071] 2) The KTSSOMP algorithm can adaptively learn sparse coding coefficients in the spectral space. Therefore, the computational cost can be reduced while ensuring linear separability.

[0072] 3) Different from deep learning methods, there are few parameters in the KTSSCC model. Therefore, its performance is relatively robust. Moreover, it can avoid the boring and arduous work of adjusting a large number of parameters. Description of the Drawings

[0073] Figure 1 Schematic diagram of the pattern of the first experimental WHU-Hi-HongHu data;

[0074] Figure 2 Schematic diagram of the pattern of the WHU-Hi-HanChuan data;

[0075] Figure 3 Schematic diagram of the OA curve of the test samples obtained by different methods with different numbers of training samples;

[0076] Figure 4 Schematic diagram of the OA curve of KTSSCC on the validation samples of two experimental datasets under different sparsity levels and iteration times;

[0077] Figure 5 Schematic diagram of the OA change curve of KTSSCC evaluated on the validation samples of two experimental datasets under the influence of the regularization parameter γ;

[0078] Figure 6 Schematic diagram of the OA change curve of KTSSCC evaluated on the validation samples of two experimental datasets under the influence of the regularization parameter λ. Specific implementation manner

[0079] In order to make the technical solution of the present invention easier to understand, the technical solution of the present invention will be clearly and completely described by way of specific embodiments in combination with the accompanying drawings.

[0080] A classification method based on kernel tensor sliced sparse coding includes the following steps:

[0081] Step 1, the UAV H2 image is a high spatial resolution hyperspectral image (high spatial resolutions hyperspectral imagery, H2 image). For each unknown UAV H2 pixel, the pixels in the small window centered on it are selected as its spatial neighbors;

[0082] Step 2, generate its kernel spatial neighbor tensor (KSNT) by arranging the representations of these spatially adjacent pixels in the feature space according to their spatial positions;

[0083] Step 3, due to the sparsity and spatial homogeneity of the UAV H2 image, the spectral vectors in each generated KSNT are expected to be sparsely represented by the spectral vectors of several common training pixels;

[0084] Step 4, the sparse coding coefficients related to the KSNT form a sliced sparse tensor;

[0085] Step 5: Denote this tensor as the slice sparse coding tensor (SSCT). Instead of directly calculating the SSCT in the high-dimensional feature space, the kernel trick is used in the spectral space, and the kernel tensor slice sparse orthogonal matching pursuit (KTSSOMP) method is adopted to adaptively learn the SSCT. The SSCT is discriminative and is used to achieve crop classification for UAV H2 images.

[0086] In Step 3, the spectral vector corresponding to the spectral pixel is expressed as The classification of UAV H2 images aims to infer its label y ∈ {1, 2, …, C} from x and a set of labeled hyperspectral pixels S = {(x1, y1), (x2, y2), …, (x L , y L )}; in this classification task, the labeled pixels are selected as training samples, and the other unlabeled pixels are used for testing.

[0087] To better describe the new kernel tensor slice sparse coding-based classification method of this application, we briefly introduce some tensor-related definitions; a tensor is a multi-dimensional array of numerical values, which can also be used to describe the linear relationship between vectors and other tensors; a tensor can be expressed as where the dimension M is the order of the tensor, also known as the mode; in particular, scalars, vectors, and matrices are also tensors; any element in X is denoted as where 1 ≤ i k ≤ I k , 1 ≤ k ≤ M; then, we briefly introduce several tensor-related definitions;

[0088] (1) d-mode vector

[0089] This concept is similar to the rows and columns of a matrix; a vector in the d-mode vector can be obtained by changing the index i d while fixing other indices; for example, the 1-mode can be denoted as

[0090] (2) Mode-d matricization

[0091] Matricization is the process of expanding an M-order tensor into a matrix; the mode-d matricization of the tensor X is denoted as where This indicates that the index i d should be fixed and other indices should be changed; please refer to Figure 2 , Figure 2A visual description of the d-mode vector and mode-d matricization of a 3rd-order tensor is given in

[0092] (3) Mode-d product

[0093] A tensor and a matrix The mode-d product of is also a tensor, which can be expressed as Its element form is as follows

[0094]

[0095] (4) Frobenius norm

[0096] The tensor The Frobenius norm of is given by

[0097]

[0098] For non-linearly separable hyperspectral pixels, the mapping φ: (d > B) can project their spectral features into a high-dimensional feature space where the pixels are linearly separable; Let be an unknown test sample, assuming that it can be linearly represented by several training samples in the feature space; and is the representation of x in the feature space, then φ(x) can be sparsely represented as follows

[0099] φ(x) = [φ(x1), φ(x2), …, φ(x L )]α = D φ α(3)

[0100] where the structured dictionary The l-th column (l = 1, …, L) of φ(x l ) represents the representation of the training sample x l in the feature space, and α is an unknown sparse vector;

[0101] In addition to the sparsity assumption, it is well known that spatially adjacent hyperspectral pixels are likely to belong to the same class. This means that, under the dictionary D φ , the spectral vectors of x and its spatial neighbors share a common sparse pattern; To utilize the common sparsity to improve the classification performance, the present invention selects the pixels in the T×T spatial window centered on x as its spatial neighbors, and then transforms the representations of these pixels according to their spatial positions in the feature space, thereby generating its KSNT, denoted as 2 The tensor X can be Tucker decomposed into φ

[0102] ​X φ = A × 1Ψ1 × 2Ψ2 × 3Ψ3(4)

[0103] where is the core tensor, and the factor matrices Ψ1 = Ψ2 = I T , Ψ3 = D φ . Benefiting from the particularity of matrices Ψ1 and Ψ2, equation (4) can be simplified to

[0104] X φ = A × 3D φ (5)

[0105] From the common sparse patterns of spatially adjacent pixels under the dictionary D φ , it can be inferred that the core tensor A is slice-sparse with respect to the 3-modal dictionary D φ . The SSCTA can be calculated by solving the following optimization problem

[0106]

[0107] where ||·|| F is the Frobenius norm of the tensor, || A ||0 is A the number of positive slices of the tensor with non-zero values, and S is the preset Sparsity;

[0108] This application does not directly solve the SSCT in the feature space, but proposes a KTSSOMP algorithm that uses the kernel trick to solve the optimization problem (6); then the pixels are classified into the category closest to KSNT.

[0109] In step five, the kernel function κ: (κ(x, y) = <φ(x), φ(y)>) is used to simplify the calculation, which is composed of training samples sorted by category, is a tensor generated by rearranging the spectral vectors of T 2 pixels in the spatial neighborhood of x, and it can be found that D φ = φ(D), X φ = φ( X ); Denote each 3-modal vector of X φ as ( X φ ) i,j,: (i, j = 1, 2,..., T); The correlation between it and the atoms of the dictionary D φ can be calculated by the following formula

[0110] <( X φ )i,j,: , φ(x l ) > = κ( X i,j,: , x l )(i, j = 1, 2, …, T; l = 1, 2, …, L). (7)

[0111] Similarly, the correlation between two atoms of the dictionary D φ can be calculated using

[0112]

[0113] ; Let be the kernel tensor with the (i, j, l)-th element being κ( X i,j,: , x l ), be the kernel matrix with the (l, m)-th element being κ(x l , x m ); Assume a set of atoms {φ(x l )} l∈Λ , ([[]] X φ ) i,j,: (i, j = 1, 2, …, T) can be approximately represented as

[0114] (([[]] K X ,D ) i,j,Λ ×3((K D ) Λ,Λ + λI) -1 )×3(D φ ) :,Λ = ([[]] K X ,D ) i,j,Λ ×3(D φ ) :,Λ ((K D ) Λ,Λ + λI) -1 (9)

[0115] Here λ is a very small scalar, and adding the regularization term is to obtain a stable inverse; then the residual vector between ( X φ ) i,j,: (i, j = 1, 2, …, T) and its approximation is

[0116] ( R φ ) i,j,: = ([[]] X φ ) i,j,: - ([[]] K X ,D) :,:,Λ ×3(D φ ) :,Λ ((K D ) Λ,Λ +λI) -1 (10)

[0117] ( R φ ) i,j,: and the correlation between φ(x l ) is

[0118] <( R φ ) i,j,: , φ(x l )> = κ( X i,j,: , x l ) - ( K X ,D ) :,:,Λ ×3(K D ) Λ,l ((K D ) Λ,Λ +λI) -1 (11)

[0119] Then the (i, j, l)-th element of the correlation tensor <( R φ ) i,j,: , φ(x l )> can be calculated by

[0120] C = K X ,D - ( K X ,D ) :,:,Λ ×3(K D ) :,Λ ((K D ) Λ,Λ +λI) -1 (12)

[0121] Calculate; in KTSSOMP, at each iteration, the atoms that simultaneously approximate all T 2 pixels most closely are selected; specifically, in the initialization step, the selected atoms should have the maximum correlation with ( X φ ) i,j,: (i, j = 1, 2,..., T); in other words, we select the new atom with the highest || K X ,D || F from the positive slice; after initialization, the newly selected atoms should simultaneously be with (R φ ) i,j,: (i,j = 1,2,…,T) has the maximum correlation; the newly selected atoms of the present invention should have the highest || C || F .

[0122] The Kernel Tensor Slice Sparse Orthogonal Matching Pursuit (KTSSOMP) method is as follows:

[0123] Input: L labeled hyperspectral vectors and unlabeled hyperspectral vectors kernel function κ, sparsity level S;

[0124] Output: SSCT A ;

[0125] Step 1: Construct the dictionary matrix and the spatial neighbor tensor

[0126] Step 2: Calculate the kernel matrix K whose (l,m)-th element is κ(x l , x m ) and the kernel tensor whose (i,j,l)-th element is κ( D X i,j,: , x, x l )

[0127] Step 3: Set s = 0 and initialize

[0128] Step 4: When s ≤ S, execute the loop

[0129] 1) Calculate the correlation tensor

[0130] 2) Find the new index

[0131] 3) Combine the selected indices Λ s = [Λ s-1 , λ s ;

[0132] 4) s = s + 1;

[0133] 5) Return the index set Λ = Λ s-1 , SSCTA, and its non-zero slices are indexed by Λ

[0134] End the loop.

[0135] Once SSCT is obtained A, the residual between the KSNT of the unknown pixel x and the approximation of the training samples of the c-th class can be calculated by the following formula:

[0136]

[0137] where is the sub-dictionary composed of the spectral features of the training samples of the c-th class, is the representation of D c in the feature space, is composed of the L c corresponding to D in A c frontal slices, is the kernel tensor whose (i, j, l c )-th element is , is the kernel matrix whose (l c , m c )-th element is ( is the atom of D c ); then the label of x is predicted by

[0138]

[0139] Formula (14).

[0140] The performance of the KTSSCC proposed in this invention is compared and evaluated with several state-of-the-art methods: support vector machine (SVM), CNN with a conditional random field classifier (CNNCRF), spectral patching network (SPNet), local correntropy matrix with SVM (LCEM), CNN-enhanced GCN (CEGCN)

[12] , and HPSCC-SNT. On two UAV hyperspectral image datasets collected and provided by the RSIDEA research group of Wuhan University, this application evaluates the performance of the proposed method and these comparison methods.

[0141] The experimental verification data is as follows:

[0142] This application evaluated the performance of the proposed KTSSCC on two UAV H2 image datasets. These two datasets were obtained from agricultural areas in Hubei Province, China, with several different crop categories. The two experimental datasets contained 270 and 274 bands from 400 to 1000 nm respectively. Their spatial resolutions were 0.043 m and 0.109 m respectively.

[0143] Table 1 WHU-Hi-HongHu Data Description

[0144]

[0145] The first dataset of this application is the WHU-Hi-HongHu data collected in Table 1, which was collected in Honghu City on November 20, 2017. The test area is a complex agricultural environment with a wide variety of crops and many varieties of the same crop.

[0146] Please refer to Figure 1 , Figure 1 (a) and Table 1 give an overview of the WHU-Hi-HongHu data.

[0147] The second experimental dataset of this application, WHU-Hi-HanChuan data, was obtained in Hanchuan on June 17, 2016, as Figure 2 shown. This area is an urban-rural fringe, containing a mixture of buildings, water, and farmland. It should be noted that the image contains many shadow areas. The detailed information of this dataset is provided in Figure 2 (a) and Table 2.

[0148] Table 2 WHU-Hi-HanChuan Data Description

[0149]

[0150] Algorithm Performance Evaluation:

[0151] Research on Algorithm Performance on WHU-Hi-HongHu

[0152] Through comparison with methods such as SVM, CNNCRF, SPNet, LCEM, CEGCN, and HPSCC-SNT, this application conducted qualitative and quantitative performance evaluations of the proposed KTSSCC on WHU-Hi-HongHu data. This experiment followed the experimental settings of Zhong et al., selecting 100 pixels per class as training samples. Please refer to Figure 1 (b), and other pixels were used as test samples. For the support vector machine, a one-versus-one strategy was adopted to solve the multi-class problem, and its parameters were obtained through cross-validation. In LCEM, the reduced dimension B was set to 20, the window factor T λ and the scaling factor Kp They are set to 30 and 0.55 respectively. In CEGCN, the learning rate and value ratio are 0.0005 and 0.01 respectively, and the number of iterations is set to 200. The maximum number of iterations L in HPSCC-SNT is set to 5. SparsityS in KTSSCC is set to 60. The spatial window width T in both HPSCC-SNT and KTSSCC is set to 11. The regularization parameters in HPSCC-SNT and KTSSCC are adjusted by cross-validation in the range. In addition, the radial basis kernel is used as the kernel function of the proposed KTSSCC. And the kernel parameter γ is fine-tuned by five-fold cross-validation in the range. The experimental results of CNNCRF and SPNet are from the papers respectively.

[0153] From Figure 1 it can be seen that the SVM classification results based only on spectral information show obvious SP noise and misclassification. By simultaneously considering the spatial information and spectral information with the CNNCRF and LCEM methods, the SP noise and misclassification are significantly reduced. However, there are still quite a few misclassifications. For example, cotton is mislabeled as the cottonfirewood class by CNNCRF. LCEM misjudges cotton as cotton firewood, Chinese cabbage, pakchoi, and brassica chinensis, etc. Benefiting from graph deep learning and tensor data representation schemes, CEGCN and HPSCC-SNT can well distinguish ground objects, and their classification maps look smoother. However, there are some small holes and isolated misclassification areas, and SPNet and KTSSCC can well improve these problems. The spectral block mechanism and deep encoder-decoder network in SPNet enable it to well extract global spatial information and multi-scale high-level semantic features. The classification map of SPNet is very smooth, but some blocks are misclassified. For example, some cotton at the top is misclassified as the cottonfirewood class. And some small brassica chinensis at the bottom are misclassified as the Romaine lettuce class. Using the kernel trick and tensor sparse coding, the classification results of KTSSCC are also satisfactory. However, pixel-level classification and fixed spatial window scales bring a small number of mislabeled pixels, especially at the boundaries of different classes.

[0154] In addition to qualitative evaluation, these seven methods were also quantitatively evaluated using four evaluation metrics: the precision of each class, OA, AA, and the kappa coefficient (Kappa). Table 3 Classification accuracies (%) of different methods on the WHU-Hi-HongHu data lists the quantitative evaluation metrics for the test samples of different methods, and the best results for each row are shown in bold. As can be seen from Table 3, among the 22 classes, KTSSCC achieved 100% classification accuracy for six classes, namely cottonfirewood, celtuce, romaine lettuce, garlic sprout, broad bean, and tree. In addition, among all the methods, it obtained the highest precision in six other classes and the second highest precision in seven other classes. Moreover, both the OA and AA of KTSSCC were higher than 98%, and its Kappa was 0.9756, far higher than the respective values of the comparison methods.

[0155] Table 3 Classification accuracies (%) of different methods on the WHU-Hi-HongHu data

[0156]

[0157] Research on the algorithm performance on WHU-Hi-HanChuan

[0158] This application respectively visually and quantitatively evaluated the performance of KTSSCC on the WHU-Hi-HanChuan data. According to the experimental settings of Zhong et al., 100 pixels were selected for training in each class in this experiment. Please refer to Figure 2 (b), and the other pixels were used for testing. The parameter settings were the same as those in the experiment on the WHU-Hi-HongHu data. Figure 2 (c)-(i) show the visual classification maps of these seven methods. As Figure 2 shown, in the classification map of SVM, there are obvious SP noises and misclassifications. By mining the high-level semantic features of hyperspectral images, the classification maps of CEGCN and CNNCRF significantly improved this problem. However, there are still many misclassifications in the shadow areas. For example, Road was mislabeled as redroof and bare soil classes by CEGCN. CNNCRF mislabeled some grayroof as water. By utilizing the spatial information, the classification maps of LCEM and HPSCC-SNT are spatially homogeneous, but there are some holes and isolated areas in the classification results. By using the kernel trick, our KTSSCC method eliminated the misclassifications in LCEM and HPSCC-SNT. Most of the holes and isolated areas in the LCEM and HPSCC-SNT classifications Figure 3 do not exist in the classification maps generated by SPNet and our proposed KTSSCC. From Figure 3It can be found that the classification results of KTSSCC are very close to the ground truth.

[0159] Similar conclusions can also be obtained from the quantitative evaluation indicators in Table 4. Table 4 shows that among the 16 classes, KTSSCC achieved 100% classification accuracy for 6 classes, such as soybean, waterspinach, watermelon, redroof, plastic, and brightobject. In addition, in the other 10 classes, it reached the highest and second-highest accuracies among all methods for 3 classes respectively. Moreover, the OA, AA, and Kappa of KTSSCC were 98.45%, 98.48%, and 0.9818 respectively, which were much higher than the respective values of the SVM, CNN-CRF, LCEM, CEGCN, and HPSCC-SNT methods. Although the classification accuracy of SPNet was slightly worse than that of KTSSCC, its requirements for the computing environment were much higher than those of KTSSCC. And there were more parameters in SPNet.

[0160] Table 4 Classification accuracies (%) of different methods on WHU-Hi-HongHu data

[0161]

[0162]

[0163] Algorithm computational efficiency

[0164] In addition, this application studied the computational efficiency of the proposed KTSSCC. We performed simulation experiments of CNNCRF, LCEM, HPSCC-SNT, and KTSSCC in MATLAB R2020a under CUDA 10.1 in an environment with 4 TESLA-V100 16G GPUs and 20 128GB memories of Intel(R) Xeon(R) CPU E5-2640v4 @ 2.40. The experimental conditions were the same as those used in the previous two studies. Table 5 lists the training and test times consumed by different methods on the WHU-Hi-HongHu and WHU-Hi-HanChu datasets. As can be seen from Table 5, on the WHU-Hi-HanChuan dataset, KTSSCC takes more time than CNNCRF, LCEM, and HPSCC-SNT because KTSSOMP uses greedy optimization to solve the SSCT for each test pixel. On the WHU-Hi-HongHu dataset with more object classes, the time consumption of KTSSCC increases very little, which benefits from the fact that solving the SSCT for each test pixel does not increase much computational effort. However, the test speed of CNNCRF is relatively slow because the CRF model has to infer the connection between each pixel and its adjacent pixels. In addition, the scale of solving its inverse matrix in HPSCC-SNT becomes larger, so the test time of HPSCC-SNT increases significantly. In summary, the total time consumption of KTSSCC is much less than that of CNNCRF and HPSCC-SNT. In addition, the very complex network model in SPNet not only brings the second-best performance on the WHU-Hi-HongHu and WHU-HiHanChan datasets but also consumes more time than KTSSCC. KTSSCC has an advantage in terms of computational efficiency in classifying UAV H2 images with more categories and samples.

[0165] Table 5 Classification times (seconds) of different methods on WHU-Hi data

[0166]

[0167] Effect of the number of training samples on the performance of KTSSCC

[0168] In this experiment, the relationship between the training sample size and the classification accuracy was studied by comparing KTSSCC with five other methods. According to the setting, for each experimental dataset, 50 to 300 samples were used for training for each class, and the other samples were used for testing. Figure 3 The OA of the test samples obtained by different methods is shown. Figure 3It shows that for any number of training samples, KTSSCC achieves the highest OA on all datasets. In addition, the OA of various methods increases with the increase in the number of training samples. However, when the number of training samples per class on the WHU-Hi-HongHu dataset exceeds 150, the growth rate of the OA obtained by the KTSSCC method is much slower than that of CNNCRF and SPNet. The reason may be that the KTSSCC method has converged to a relatively high OA for 150 training samples per class. On the WHU-Hi-HanChuan dataset, the KTSSCC method achieved a relatively high OA when there were 100 training samples per class.

[0169] Influence of parameters T and S on the performance of KTSSCC:

[0170] This application evaluates how parameters T and S affect the performance of our proposed KTSSCC. For each experimental data, 1000 test samples are randomly selected per class as validation samples. The spatial window width T ranges from 3 to 11, and the Sparsity ranges from {5, 10, 15, 20, 30, 40, 60, 80, 100}. To eliminate the influence brought by the randomness of the validation samples, twenty independent experiments are carried out under the same experimental conditions, and the average results are reported. Please refer to Figure 4 , Figure 4 (a) and (b) respectively show the OA of our proposed KTSSCC on the validation samples of two experimental datasets. Figure 4 (a) and (b) show that the OA of WHU-Hi-HongHu and WHU-Hi-HanChuan data increases with the increase in T. In addition, for all values of T, as S increases, the OA first increases rapidly and then stabilizes when S reaches 60. This indicates that the spatial homogeneous neighborhood of WHU-Hi-HongHu and WHU-Hi-Hanchuan data is an important factor for classification. Therefore, for the first three experiments, we set the spatial window width T to 11 and the Sparsity S to 60.

[0171] Influence of parameter γ on the performance of KTSSCC:

[0172] This experiment evaluates the influence of the RBF kernel parameter γ on the performance of our proposed KTSSCC. Here, γ varies within , and the validation samples are randomly selected in the same way as in the previous experiment. As γ varies within , by using the samples in Figure 1 (b) and Figure 2 (b) for training, please also refer to Figure 5 , and the average OA on the validation samples is evaluated and described in Figure 5 (a) and (b) respectively. We start fromFigure 5 It is found in (a) that as γ increases, OA rises rapidly until 2 -9 , and then OA rises slowly until 2 -2 , when γ is greater than 2 -2 , OA drops rapidly. When γ is within the range of , the change in OA of WHU-Hi-HongHu data is no greater than 2.61. In other words, KTSSCC is relatively robust to the parameter γ within the range of on WHU-Hi-HongHu data. Similar conclusions can be drawn from Figure 5 (b). The relatively robust range of γ for WHU-Hi-HanCuan data is

[0173] Effect of parameter λ on the performance of KTSSCC:

[0174] To study the effect of the regularization parameter λ, we evaluated the change in OA of KTSSCC on the validation samples of two experimental datasets respectively. Other experimental settings follow the descriptions of the above experimental conclusions. Please refer to Figure 6 , it can be seen from Figure 6 (a) that when λ is less than 10 -4 , the OA of KTSSCC changes slightly. However, when λ is greater than 10 -4 , the OA of KTSSCC drops rapidly. Therefore, KTSSCC is robust to the parameter λ within the range of on WHU-Hi-HongHu data. In addition, it can be seen from Figure 6 (b) that on WHU-Hi-HanChuan data, the robust range is

[0175] It should be noted that the embodiments described herein are only partial embodiments of the present invention, rather than all implementation manners of the present invention. The embodiments are only exemplary, and their functions are only to provide a more intuitive and clear way to understand the content of the present invention, rather than a limitation on the technical solutions described in the present invention. Without departing from the concept of the present invention, all other implementation manners that can be thought of by those of ordinary skill in the art without creative efforts, as well as other simple substitutions and various changes to the technical solutions of the present invention, all fall within the protection scope of the present invention.

Claims

1. A classification method based on kernel tensor slice sparse coding, characterized by: The classification method based on kernel tensor slice sparse coding includes the following steps: Step 1: Select the pixels in a small window centered on the unknown pixel in the UAV H2 image as its spatial neighboring pixels; Step 2: Generate the kernel space neighbor tensor by arranging the representations of these spatially adjacent pixels in the feature space according to their spatial positions; Step 3: Due to the sparsity and spatial homogeneity of UAV H2 images, the spectral vector in each generated kernel space neighbor tensor is expected to be sparsely represented by the spectral vectors of several common training pixels; Step 4: The sparse coding coefficients associated with the kernel space neighbor tensor form a slice sparse tensor; Step 5: record the tensor as slice sparse coding tensor SSCT; In step 5, instead of directly computing the SSCT in the high-dimensional feature space, the kernel technique is used in the spectral space. The kernel tensor slicing sparse orthogonal matching pursuit (KTSSOMP) method is used to adaptively learn the SSCT. The SSCT is discriminative and can be used to achieve crop classification in UAV H2 imagery. The kernel tensor slicing sparse orthogonal matching pursuit KTSSOMP method is: Input: L labeled hyperspectral vectors and unlabeled hyperspectral vectors Kernel function κ, sparsity level S; Output: SSCT A ; Step 1: Construct the dictionary matrix and the spatial neighbor tensor Step 2: Calculate the l,mth element as κ(x l ,x m )’s kernel matrix K D and the i, j, lth elements are κ( X i,j,: ,x l )'s core tensor Step 3: Set s=0, initialize Step 4: When s≤S, execute the loop 1) Calculate relevant tensors 2) Find new index 3) Merge the selected index Λ s =[Λ s-1 ,λ s ]; 4) s = s + 1; 5) Return index set Λ=Λ s-1 ,SSCT A , its non-zero slice Indexed with Λ End the loop; Obtaining SSCT A , the residual between the KSNT of the unknown pixel x and the approximate value of the training sample of class c can be calculated as follows: in is a sub-dictionary consisting of the spectral features of the c-th type of training samples, It's D c Representation in feature space, Depend on A Middle and D c The corresponding L c Frontal slice composition, is the i,j,l c The elements are The core tensor of It is the first c ,m c The elements are It's D c The nuclear matrix of the atom of predict.

2. The classification method based on kernel tensor slice sparse coding according to claim 1, wherein: In step 3, the spectral vector corresponding to the spectral pixel is expressed as UAV H2 image classification aims to classify the image from x and a set of labeled hyperspectral pixels S = {(x1,y1),(x2,y2),…,(x L ,y L )} infer its label y∈{1,2,…,C}; in this classification task, the labeled pixels are selected as training samples, and other unlabeled pixels are used for testing.

3. The classification method based on kernel tensor slice sparse coding according to claim 2, wherein: A tensor is a multidimensional array of values; a tensor can be represented as Among them, the dimension M is the rank of the tensor; scalars, vectors and matrices are also tensors; X Any element in Where 1≤i k ≤I k ,1≤k≤M; Set the tensor: (1) d-modal vectors are the rows and columns of the matrix; a vector in the d-modal vector can be transformed by changing the index i d While fixing other indicators, the 1-mode can be recorded as (2) Modal-d matrixing: Matrixing is the process of expanding an M-order tensor into a matrix; X The modal-d matrix of in This indicates that to maintain the index i d Fix and vary other indicators; (3) Modal-d product, a tensor With a matrix The modal-d product of is also a tensor and can be expressed as Its elementized form is as follows (4) Frobenius norm, tensor The Frobenius norm of 4. The classification method based on kernel tensor slice sparse coding according to claim 3, wherein: For nonlinearly separable hyperspectral pixels, the mapping φ: Their spectral features can be projected into a high-dimensional feature space where pixels are linearly separable; set up is an unknown test sample, which is assumed to be linearly represented in the feature space by several training samples; and is the representation of x in the feature space, then φ(x) can be sparsely represented as follows: φ(x) = [φ(x1), φ(x2), …, φ(x L )]α = D φ α formula (3) The structured dictionary The lth column (l=1,…,L)φ(x l ) represents the training sample x l In the feature space, α is an unknown sparse vector; In the dictionary D φ In this case, the spectral vector of x and its spatial neighbors share a common sparse pattern; the pixels in the T×T spatial window centered on x are taken as its spatial neighbors, and then according to this T 2 The spatial position of pixels in the feature space transforms the representation of these pixels to generate their kernel space nearest neighbor tensor KSNT, which is recorded as Tensor X φ is decomposed into X φ = A ×1Ψ1×2Ψ2×3Ψ3 Formula (4) in is the core tensor, the factor matrix Ψ1=Ψ2=I T ,Ψ3=D φ , formula (4) is simplified to X φ = A ×3D φ Formula (5) From dictionary D φ Derivation of common sparsity patterns of adjacent pixels in the lower space, core tensor A It is about 3-modal dictionary D φ Slice sparsity; SSCT is calculated by solving the following optimization problem A where ||·|| F is the Frobenius norm of the tensor, || A ||0 is A The number of positive slices of the tensor with non-zero values, S is the Sparsity set beforehand.

5. The classification method based on kernel tensor slice sparse coding according to claim 4, characterized in that: In step 5, the kernel function κ is used: κ(x,y)=<φ(x),φ(y)> to simplify the calculation, It consists of training samples sorted by category, is obtained by rearranging the spatial neighborhood of x in T 2 The tensor generated by the spectral vector of pixels can be found to be D φ =φ(D), X φ =φ( X );Will X φ Each 3-modal vector of is recorded as ( X φ ) i,j,: i,j=1,2,…,T; it is consistent with the dictionary D φ The correlation between atoms can be calculated by <( X φ ) i,j,: ,φ(x l )>=κ( X i,j,: ,x l )i,j=1,2,…,T;l=1,2,…,L Formula (7) Dictionary D φ The correlation between two atoms is expressed as <φ(x l ),φ(x m )>=κ(x l ,x m )l,m=1,2,…,L Formula (8) is used for calculation; let The i, j, lth element is κ(X _i,j,: ,x l ), The lth,mth element is κ(x l ,x m )’s nuclear matrix; suppose we choose a set of atoms {φ(x l )} l∈Λ , ( X φ ) i,j,: i,j=1,2,…,T can be approximately expressed as (( K X,D ) i,j,Λ ×3((K D ) Λ,Λ +λI) -1 )×3(D φ ) :,Λ =( K X ,D ) i,j,Λ ×3(D φ ) :,Λ ((K D ) Λ,Λ +λI) -1 Formula (9) Here λ is a scalar, and the regularization term is added to obtain a stable inverse; then ( X φ ) i,j,: The residual vector between i,j=1,2,…,T and its approximation is ( R φ ) i,j,: =( X φ ) i,j,: -( K X,D ) :,:,Λ ×3(D φ ) :,Λ ((K D ) Λ,Λ +λI) -1 Formula (10) where ( R φ ) i,j,: and φ(x l ) is the correlation between <( R φ ) i,j,: ,φ(x l )>Zκ( X i,j,: ,x l )-( K X ,D ) :,:,Λ ×3(K D ) Λ,l ((K D ) Λ,Λ +λI) -1 Enclosure(11) Then the relevant tensor The (i,j,l)th element of <( R φ ) i,j,: ,φ(x l )>by C = K X ,D -( K X ,D ) :,:,Λ ×3(K D ) :,Λ ((K D ) Λ,Λ +λI) -1 Formula (12) Calculation; In KTSSOMP, each iteration selects the closest approximation to all T 2 Atoms per pixel; In the initialization step, the selected atoms are compared with ( X φ ) i,j,: i,j=1,2,…,T has the greatest correlation; Select from the frontal slices the one with the highest || K X,D || F ; after initialization, the newly selected atom is simultaneously with ( R φ ) i,j,: i,j=1,2,…,T has the greatest correlation; The new atom and the frontal slice have the highest || C || F .

Images