A hyperspectral image dimension reduction method based on tensor graph embedding

Abstract

The application discloses a hyperspectral image dimension reduction method based on tensor graph embedding, and comprises the following steps: a global spectral attention mechanism is weighted, hyperspectral data is divided into a plurality of tensor blocks, a spectral domain covariance descriptor is used to define the characteristics of the tensor blocks, the discriminant of a labeled training set is divided, an intra-class and inter-class graph of high-discriminant samples, an intra-class and inter-class neighborhood graph of common labeled samples and a neighborhood structure graph of unlabeled training samples are constructed, and a graph embedding method is used to calculate an optimal projection matrix for dimension reduction. The application can effectively reduce the band number of a high-dimensional image, fully retains useful information of original data, improves image processing efficiency, and can be applied to 3-order and higher-order data.

Description

Technical Field

[0001] This invention belongs to the field of hyperspectral image dimensionality reduction technology, specifically relating to a hyperspectral image dimensionality reduction method based on tensor graph embedding. Background Technology

[0002] Medical hyperspectral technology has attracted widespread attention in recent years, bringing new opportunities for pathological diagnosis. Microscopic hyperspectral imaging can acquire a three-dimensional data cube of a target, where the spatial dimension provides the target's spatial structure, and the spectral dimension provides rich spectral information to express the chemical characteristics of biological tissues. However, the large number of bands introduces a significant amount of redundant information, which not only adversely affects the recognition and classification of classifiers but also increases computational complexity and storage resource consumption. Therefore, dimensionality reduction preprocessing is necessary before processing medical hyperspectral images.

[0003] In practical applications, the scarcity of labeled medical samples makes supervised algorithms costly in terms of time and manpower, while unsupervised algorithms often fail to achieve the expected performance. Therefore, this invention focuses on semi-supervised methods, using a limited number of labeled samples in the tensor space to improve dimensionality reduction. In medical microscopic hyperspectral images, cancerous and normal regions are usually closely connected. To improve recognition performance, enhancing the discriminative power of the dimensionality-reduced image is crucial. However, existing semi-supervised dimensionality reduction methods do not fully utilize recognition information because they treat all labeled samples equally, without considering the varying discriminative power of the samples. Furthermore, hyperspectral images exhibit global spectral correlation; increasing the weight of important frequency bands and reducing the weight of redundant and noisy frequency bands can improve dimensionality reduction efficiency. Summary of the Invention

[0004] Purpose of the invention: In order to address the problem that existing hyperspectral image dimensionality reduction methods do not consider the global correlation and discrimination strength of hyperspectral images, and to make up for the deficiencies of current microscopic hyperspectral dimensionality reduction technology, this invention provides a hyperspectral image dimensionality reduction method based on tensor graph embedding, which improves the performance of dimensionality reduction and the effect of subsequent classification.

[0005] Technical Solution: This dimensionality reduction method first utilizes the global spectral correlation of hyperspectral images to construct a global spectral attention mechanism to weight the original hyperspectral images. Next, it divides the hyperspectral data into several tensor blocks, introduces a spectral domain covariance descriptor to define the features of the tensor blocks, and then divides the labeled training set into categories based on their discriminative power. It constructs intra-class and inter-class graphs for high-discriminative-power samples, intra-class and inter-class neighborhood graphs for other samples, and neighborhood graphs for unlabeled training samples. Finally, it uses a graph embedding method to find the optimal projection matrix for dimensionality reduction.

[0006] The specific steps are as follows:

[0007] S1: The global spectral attention mechanism weights each spectral band of the original hyperspectral image;

[0008] S2: The weighted hyperspectral data is divided into several tensor blocks, and a spectral domain covariance descriptor is introduced to extract representative features of the tensor blocks;

[0009] S3: Assign pseudo-labels to some unlabeled samples based on the nearest neighbor relationship between labeled and unlabeled samples, expand the number of labeled samples, and divide the existing labeled samples, including those with pseudo-labels, into different discriminative categories.

[0010] S4: Construct intra-class and inter-class graphs for high-discrimination samples, intra-class and inter-class neighborhood graphs for other ordinary samples, and local neighborhood graphs for unlabeled training samples based on Log-Euclidean distance metric and hot kernel similarity.

[0011] S5: Graph embedding, which calculates the projection matrix by preserving the discriminative information and local neighborhood information of the original data in a low-dimensional subspace;

[0012] S6: Project the training and test set samples into a low-dimensional subspace using the obtained optimal projection matrix.

[0013] Furthermore, the specific process of the global spectral attention mechanism in step S1 is as follows:

[0014] A1: Obtain the original hyperspectral image X∈R H×W×C The probability of gray values ​​in each band Standard deviation σ Band，i and mean μ Band，i In this step, i represents the band number, and the information entropy E of that band is calculated. Band，i Then, the information content I of this band is calculated. Band，i：

[0015]

[0016] A2: Calculate the correlation coefficient ρ between each band of the original hyperspectral image. i，j :

[0017]

[0018] In the formula, Cov(X) Band，i X Band，j Let be the covariance between the i-th band and the j-th band, and Var(X) be the variance between the i-th band and the j-th band. Band，i ) represents the variance of the i-th band, Var(X) Band,j ) represents the variance of the j-th band.

[0019] A3: Based on the principle that bands with high information content and low average correlation coefficients play a greater role in dimensionality reduction, the ratio of band information content to average correlation coefficient is used as the attention value. A Softmax function is used for smoothing, and finally, the weight w for each band is calculated. Band，i for:

[0020]

[0021] A4: Calculate the hyperspectral image X′ after global spectral attention weighting. Band，i :

[0022] X′ Band，i =X Band，i ×w Band，i i = 1, 2, ... C

[0023] Furthermore, the specific process of dividing the sample discriminative power described in step S3 is as follows:

[0024] B1: Calculate all labeled tensor samples X i Spectral domain covariance characteristics C i The spectral domain covariance features lie on the Riemannian manifold, and the two features C are measured using the Log-Euclidean distance metric. i and C j Distance between:

[0025] D LE (C i C j )=||log(C i )-log(C j) || F

[0026] B2: Construct feature C based on the distances between features obtained in B1. i homogeneous nearest neighbors heterogeneous neighbors Calculate feature C i Improved average nearest neighbor boundary δ i :

[0027]

[0028] The neighborhood size k in the method e and k o All are set to 5.

[0029] B3: Select the labeled samples corresponding to the top k largest improved average nearest neighbor boundary values ​​as high-discrimination samples, and the remaining labeled samples as ordinary samples.

[0030] Furthermore, the specific process of constructing the intra-class and inter-class graphs of high-discrimination samples, the intra-class and inter-class neighborhood graphs of other samples, and the local neighborhood graphs of unlabeled training samples based on Log-Euclidean distance metric and hot kernel similarity in step S4 is as follows:

[0031] C1: Construct intra-class neighborhood graphs for ordinary samples Inter-class neighborhood graph Connect ordinary sample points X in the intra-class neighborhood graph i and X j weight of the edge Connecting ordinary sample points X in the inter-class neighborhood graph i and X j weight of the edge for:

[0032]

[0033] C2: Constructing an in-class graph for high-discrimination samples Inter-class graph Connecting two high-discrimination sample points X in the intra-class graph and the inter-class graph i and X j weight of the edge and Defined as:

[0034]

[0035] C3: Construct the neighborhood graph G of unlabeled samples u Used to preserve local neighborhood structure, used to connect unlabeled sample points X i and X j weight of the edge Defined as:

[0036]

[0037] Furthermore, the calculation of the projection matrix in step S5, which preserves the discriminative information and local neighborhood information of the original data in the low-dimensional subspace, specifically involves minimizing the inter-class distance after dimensionality reduction, maximizing the intra-class distance after dimensionality reduction, and preserving the local geometric structure of the original data. This process solves the following generalized eigenvalue problem, and the optimal projection matrix is ​​the first d... k The eigenvector corresponding to the smallest eigenvalue, d k The target dimension for the dimensionality reduction method is:

[0038]

[0039] In the formula, α1, α2 and α3 represent the weights of the discrimination information of ordinary samples, the discrimination information of high-discrimination samples and the local geometric structure of unlabeled samples, respectively. The sum of the three is 1, and their magnitude is inversely proportional to the corresponding number of samples.

[0040] Y (k) It is Y k The k-model expansion.

[0041] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0042] (1) This invention considers the global spectral correlation of high-dimensional data and designs a global spectral attention mechanism, which can focus on information that is more critical to dimensionality reduction among many spectral bands, making subsequent dimensionality reduction and classification more efficient.

[0043] (2) By defining the distinction between high-discrimination samples and ordinary samples, this invention selects samples with high discrimination ability from the labeled samples and focuses on them in the subsequent dimensionality reduction, giving full play to their powerful discrimination function. This breaks through the limitation of treating all samples equally in the existing methods, making the data after dimensionality reduction more discriminative and more suitable for processing medical microscopic hyperspectral data.

[0044] (3) The present invention determines the weighting parameters of three different graph structures in the embedding stage based on the number of samples, thus avoiding the complex problem of parameter selection in other multi-graph embedding methods. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the main process of training the image dimensionality reduction method in an embodiment of the present invention;

[0046] Figure 2-a A schematic diagram of the 29th band image of the cholangiocarcinoma dataset 031368c-20x-roi2;

[0047] Figure 2-b True-labeled image of the cholangiocarcinoma dataset 031368c-20x-roi2;

[0048] Figure 3-a A schematic diagram of the 29th band image in the 032236-20x-roi4 dataset for cholangiocarcinoma.

[0049] Figure 3-b The ground truth map for the cholangiocarcinoma dataset 032236-20x-roi4;

[0050] Figure 4-a A schematic diagram of the 29th band image in the 033850c-20x-roi3 dataset for cholangiocarcinoma.

[0051] Figure 4-b True-labeled image of the cholangiocarcinoma dataset 033850c-20x-roi3;

[0052] Figure 5 The images show the classification results of the 031368c-20x-roi2 image after applying different dimensionality reduction methods and then using support vector machines. Among them, (a) is the result of the real ground cover, (b) is the classification result of the original hyperspectral image, (c) PCA, (d) LPP, (e) SSLPP, (f) SLPP, (g) TLPP, (h) MTLPP, and (i) EDLCPP of the present invention.

[0053] Figure 6 The images show the classification results of the 032236-20x-roi4 image after applying different dimensionality reduction methods and then using support vector machines. Among them, (a) is the result of the real ground cover, (b) is the classification result of the original hyperspectral image, (c) PCA, (d) LPP, (e) SSLPP, (f) SLPP, (g) TLPP, (h) MTLPP, and (i) EDLCPP of the present invention.

[0054] Figure 7 The images show the classification results of the 033850c-20x-roi3 image after applying different dimensionality reduction methods and then using support vector machines. Among them, (a) is the result of the real ground cover, (b) is the classification result of the original hyperspectral image, (c) PCA, (d) LPP, (e) SSLPP, (f) SLPP, (g) TLPP, (h) MTLPP, and (i) EDLCPP of the present invention. Detailed Implementation

[0055] The embodiments of the present invention will now be described in detail with reference to specific accompanying drawings and examples. The following embodiments are for illustrative purposes only and should not be construed as limiting the invention.

[0056] like Figure 1 As shown, this invention discloses a hyperspectral image dimensionality reduction method based on tensor graph embedding, comprising the following steps:

[0057] S1: The global spectral attention mechanism weights each spectral band of the original hyperspectral image;

[0058] S2: The weighted hyperspectral data is divided into several tensor blocks. The spectral domain covariance descriptor is introduced to extract the representative features of the tensor blocks. The subsequent distance calculations are all based on the spectral domain covariance features of the tensor.

[0059] The calculation method for describing spectral domain covariance can be expressed as follows:

[0060]

[0061] In the formula, S is the number of spectral vectors contained in each tensor block, x t It is the t-th spectral vector in the tensor block.

[0062] S3: Assign pseudo-labels to some unlabeled samples based on the nearest neighbor relationship between labeled and unlabeled samples, expand the number of labeled samples, and divide the existing labeled samples, including those with pseudo-labels, into different discriminative categories.

[0063] Specifically, the method for assigning pseudo-labels involves calculating the neighborhoods of all labeled samples in the unlabeled sample using the K-nearest neighbor algorithm. The distance is measured by Log-Euclidean distance and hot kernel similarity. If an unlabeled sample is located in the intersection of neighborhoods of two or more classes, it is not assigned a pseudo-label. If the sample is located in the neighborhood of only one class, it is assigned the same label as the central labeled sample. In this method, k is chosen to be 3.

[0064] S4: Construct intra-class and inter-class graphs for high-discrimination samples, intra-class and inter-class neighborhood graphs for other samples, and local neighborhood graphs for unlabeled training samples based on Log-Euclidean distance metric and hot kernel similarity.

[0065] S5: Graph embedding, which calculates the projection matrix by preserving the discriminative information and local neighborhood information of the original data in a low-dimensional subspace;

[0066] S6: Project the training and test set samples into a low-dimensional subspace using the obtained optimal projection matrix.

[0067] Furthermore, the specific process of the global spectral attention mechanism in step S1 is as follows:

[0068] A1: Obtain the original hyperspectral image X∈R H×W×C The probability of gray values ​​in each band Standard deviation σ Band，i and mean μ Band，i In this step, i represents the band number, and the information entropy E of that band is calculated. Band，i Then, the information content I of this band is calculated. Band,i :

[0069]

[0070]

[0071] A2: Calculate the correlation coefficient ρ between each band of the original hyperspectral image. i，j :

[0072]

[0073] In the formula, Cov(X) Band,i X Band，j Let be the covariance between the i-th band and the j-th band, and Var(X) be the variance between the i-th band and the j-th band. Band，i ) represents the variance of the i-th band, Var(X) Band，j ) represents the variance of the j-th band.

[0074] A3: Based on the principle that bands with high information content and low average correlation coefficients play a greater role in dimensionality reduction, the ratio of band information content to average correlation coefficient is used as the attention value. A Softmax function is used for smoothing, and finally, the weight w for each band is calculated. Band，i for:

[0075]

[0076] A4: Calculate the hyperspectral image X′ after global spectral attention weighting. Band，i :

[0077] X′ Band,i =X Band，i ×w Band,i i = 1, 2, ... C

[0078] Furthermore, the specific process of dividing the sample discriminative power described in step S3 is as follows:

[0079] B1: Calculate all labeled tensor samples X i Spectral domain covariance characteristics C i The spectral domain covariance features lie on the Riemannian manifold, and the two features C are measured using the Log-Euclidean distance metric. i and C j Distance between:

[0080] D LE (C i C j )=||log(C i )-log(C j )|| F

[0081] B2: Construct feature C based on the distances between features obtained in B1. i homogeneous nearest neighbors heterogeneous neighbors Calculate feature C i Improved average nearest neighbor boundary δ i :

[0082]

[0083] The neighborhood size k in the method e and k oAll are set to 5.

[0084] B3: Select the labeled samples corresponding to the top k largest improved average nearest neighbor boundary values ​​as high-discrimination samples, and the remaining labeled samples as ordinary samples.

[0085] Furthermore, the specific process of constructing the intra-class and inter-class graphs of high-discrimination samples, the intra-class and inter-class neighborhood graphs of other samples, and the local neighborhood graphs of unlabeled training samples based on Log-Euclidean distance metric and hot kernel similarity in step S4 is as follows:

[0086] C1: Construct intra-class neighborhood graphs for ordinary samples Inter-class neighborhood graph Connect ordinary sample points X in the intra-class neighborhood graph i and X j weight of the edge Connecting ordinary sample points X in the inter-class neighborhood graph i and X j weight of the edge for:

[0087]

[0088] C2: Constructing an in-class graph for high-discrimination samples Inter-class graph Connecting two high-discrimination sample points X in the intra-class graph and the inter-class graph i and X j weight of the edge and Defined as:

[0089]

[0090] C3: Construct the neighborhood graph G of unlabeled samples u Used to preserve local neighborhood structure, used to connect unlabeled sample points X i and X j weight of the edge Defined as:

[0091]

[0092] Furthermore, the calculation of the projection matrix in step S5, which preserves the discriminative information and local neighborhood information of the original data in the low-dimensional subspace, specifically involves minimizing the inter-class distance after dimensionality reduction, maximizing the intra-class distance after dimensionality reduction, and preserving the local geometric structure of the original data. This process solves the following generalized eigenvalue problem, and the optimal projection matrix is ​​the first d... k The eigenvector corresponding to the smallest eigenvalue, d k The target dimension for the dimensionality reduction method is:

[0093]

[0094] In the formula, α1, α2 and α3 represent the weights of the discrimination information of ordinary samples, the discrimination information of high-discrimination samples and the local geometric structure of unlabeled samples, respectively. The sum of the three is 1, and their magnitude is inversely proportional to the corresponding number of samples.

[0095] Y (k) It is Y k The k-model expansion.

[0096] Analysis of Simulation Experiment Results

[0097] 1. Experimental images

[0098] This invention will test the proposed hyperspectral image dimensionality reduction method based on tensor graph embedding on three datasets of cholangiocarcinoma microscopic hyperspectral images to verify the effectiveness and reliability of the method. The three datasets are named 031368c-20x-roi2, 032236-20x-roi4 and 033850c-20x-roi3, and the single-band images are shown in Figures 2 to 4.

[0099] Three hyperspectral microscopic images of cholangiocarcinoma were acquired using a Molecular Hyperspectral Imaging (MHI) system under a 20x objective lens. These images comprise 60 bands within a wavelength range of 550 to 1000 nm, with a data size of 1024 × 1280 × 60 pixels. All images are accompanied by well-documented cholangiocarcinoma segmentation boundaries annotated by a qualified pathologist. Due to the large size of the entire dataset and limitations of the equipment, a quarter of the spatial size of each dataset was used for dimensionality reduction experiments. Based on the distribution of cancerous and non-cancer regions, the top-left quarter of the images from datasets 031368c-20x-roi2 and 032236-20x-roi4 was selected, while the bottom-left quarter of the image from dataset 033850c-20x-roi3 was selected. Therefore, the original hyperspectral images in this example are 512 × 640 × 60 pixels.

[0100] 2. Experimental methods and related parameter settings

[0101] The experimental environment was configured as follows: CPU: Intel(R) Xeon(R) CPU E5-2680 v4@2.40GHz, programming language: Python, environment: Visual Studio Code.

[0102] For evaluation metrics, three indicators were selected: overall accuracy (OA), average accuracy (AA), and Kappa coefficient (KA). All data were the average of 10 experimental data sets.

[0103] In the partitioning of the training and test sets, only 1% of the samples in each of the three datasets were selected as the training set, of which 10% were labeled training samples and the remainder were unlabeled training samples. The remaining 99% of the samples were used as the test set. In the experiments on the three datasets, the tensor block window size was 11×11, the neighborhood size was set to 5 when constructing the nearest neighbor graph, the number of high-discriminative-power samples selected for discriminative partitioning was 30, the spectral dimension after dimensionality reduction was 14, and the spatial dimension was 1×1. In comparison, the vector-based dimensionality reduction method retained a spectral dimension of 20.

[0104] 3. Experimental Results

[0105] Table 1-031368c-20x-roi2 dataset test results

[0106]

[0107]

[0108] Table 2-032236-20x-roi4 dataset test results

[0109]

[0110] Table 3-033850c-20x-roi3 dataset test results

[0111]

[0112] Refer to Tables 1, 2, and 3, and Figure 5-7 It can be seen that the hyperspectral image dimensionality reduction method based on tensor graph embedding proposed in this invention can better characterize the intrinsic structure of high-dimensional data and use stronger discriminative feature constraints for dimensionality reduction, which is superior to other dimensionality reduction methods.

[0113] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand it. It should be noted that any improvements and modifications made by those skilled in the art without departing from the technical principles of the present invention should fall within the protection scope of the present invention.

Claims

1. A hyperspectral image dimensionality reduction method based on tensor graph embedding, characterized in that: The method specifically includes the following steps: S1: The global spectral attention mechanism weights each spectral band of the original hyperspectral image. The process of the global spectral attention mechanism is as follows: S11: Acquire the raw hyperspectral image The probability of gray values ​​in each band Standard deviation σ Band，i and mean μ Band，i In this step, i represents the band number, and the information entropy E of that band is calculated. Band，i Then, the information content I of this band is calculated. Band，i : S12: Calculate the correlation coefficient ρ between each band of the original hyperspectral image. i，j : In the formula, Cov(X) Band，i X Band，j Let be the covariance between the i-th band and the j-th band, and Var(X) be the variance between the i-th band and the j-th band. Band，i ) represents the variance of the i-th band, Var(X) Band，j Let ) be the variance of the j-th band; S13: Based on the principle that bands with high information content and low average correlation coefficients play a greater role in dimensionality reduction, the ratio of band information content to average correlation coefficient is used as the attention value. A Softmax function is employed for smoothing, and the final weight w for each band is calculated. Band，i for: S14: Calculate the hyperspectral image X′ after global spectral attention weighting. Band，i ; X′ Band，i =X Band，i ×w Band，i ,i=1,2,...C S2: The weighted hyperspectral data is divided into several tensor blocks, and a spectral domain covariance descriptor is introduced to extract representative features of the tensor blocks; S3: Based on the nearest neighbor relationship between labeled and unlabeled samples, pseudo-labels are assigned to some unlabeled samples to increase the number of labeled samples. The existing labeled samples, including those with pseudo-labels, are then divided by their discriminative power. The specific process for dividing the sample discriminative power is as follows: S31: Calculate X for all labeled tensor samples. i Spectral domain covariance characteristics C i The spectral domain covariance features lie on the Riemannian manifold, and the two features C are measured using the Log-Euclidean distance metric. i and C j Distance between: D LE (C i ,C j )=||log(C i )-log(C j )|| F S32: Construct feature C based on the distances between features obtained from B1. i homogeneous nearest neighbors heterogeneous neighbors Calculate feature C i Improved average nearest neighbor boundary δ i : The neighborhood size k in the method e and k o All are set to 5; S33: Select the labeled samples corresponding to the top k largest improved average nearest neighbor boundary values ​​as high-discrimination samples, and the remaining labeled samples as ordinary samples; S4: Construct intra-class and inter-class graphs for high-discrimination samples, intra-class and inter-class neighborhood graphs for ordinary samples, and local neighborhood graphs for unlabeled training samples based on Log-Euclidean distance metric and hot kernel similarity. S5: Graph embedding, which calculates the projection matrix by preserving the discriminative information and local neighborhood information of the original data in a low-dimensional subspace; S6: Project the training and test set samples into a low-dimensional subspace using the obtained optimal projection matrix.

2. The hyperspectral image dimensionality reduction method based on tensor graph embedding according to claim 1, characterized in that... The construction of intra-class and inter-class graphs for high-discrimination samples, intra-class and inter-class neighborhood graphs for other samples, and local neighborhood graphs for unlabeled training samples based on Log-Euclidean distance metric and hot kernel similarity in step S4 specifically includes: A1: Construct an intra-class neighborhood graph for ordinary samples Inter-class neighborhood graph Connecting ordinary sample points X in the intra-class neighborhood graph i and X j weight of the edge Connecting ordinary sample points X in the inter-class neighborhood graph i and X j weight of the edge for: A2: Constructing intra-class graphs for high-discrimination samples Inter-class graph Connecting two high-discrimination sample points X in the intra-class graph and the inter-class graph i and X j weight of the edge and Defined as: A3: Construct the neighborhood graph G of unlabeled samples u Used to preserve local neighborhood structure, used to connect unlabeled sample points X i and X j weight of the edge Defined as follows: 。 3. The hyperspectral image dimensionality reduction method based on tensor graph embedding according to claim 1, characterized in that... Step S5, which involves retaining the discriminative information and local neighborhood information of the original data in the low-dimensional subspace to calculate the projection matrix, specifically involves solving the following generalized eigenvalue problem, where the optimal projection matrix is ​​the first d... k The eigenvector corresponding to the smallest eigenvalue, d k Let k = 1, 2, 3 represent the target dimension of the tensor, where k is the target dimension of the dimensionality reduction method. In the formula, α1, α2 and α3 represent the weights of ordinary sample discrimination information, high discrimination information of high discrimination sample discrimination information and local geometric structure of unlabeled sample, respectively. The sum of the three is 1, and their magnitude is inversely proportional to the corresponding number of samples. Y (k) It is the k-mode expansion of Y.

Images