Hyperspectral remote sensing image classification method based on unsupervised multi-view image comparative learning

Through the unsupervised multi-view diagram comparison learning and adaptive data enhancement module, the problem of imperfect training samples in hyperspectral remote sensing image classification is solved, and high-precision and efficient image classification is achieved, which can be used in the case of few samples. Extract local and remote information of the image below.

CN118918459AActive Publication Date: 2024-11-08WUHAN UNIV

Patent Information

Application Number
CN202410903675.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-08
Publication Date
2024-11-08
Estimated Expiration
2044-07-08

AI Technical Summary

Technical Problem

The existing hyperspectral remote sensing image classification method requires a large number of training samples and the initial construction of the initial map is incomplete, resulting in only local information of the image being extracted and the mining of remote information is ignored.

Method used

Using the method based on unsupervised multi-view diagram comparison learning, through adaptively enhanced contrast learning model and multi-view construction module, a complete adjacency matrix is ​​established from the two dimensions of space and spectrality, and combined with the adaptive data enhancement module, deep features are extracted for Classification.

Benefits of technology

High-precision and efficient hyperspectral image classification are achieved, reducing dependence on labeled samples, and the local and remote information of the images can be extracted in the case of few samples, improving classification efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918459A_ABST
    Figure CN118918459A_ABST
Patent Text Reader

Abstract

The invention discloses a hyperspectral remote sensing image classification method based on unsupervised multi-view image comparative learning, and the method comprises the steps: firstly carrying out the data preprocessing of an obtained hyperspectral remote sensing image X, and obtaining superpixels and corresponding labels and segmentation matrixes; according to the segmentation matrix, establishing a spatial adjacency matrix and a spectral adjacency matrix from multiple view angles to obtain multiple views; and finally, inputting the multiple views into an adaptive data enhancement module to obtain deep features for classification. According to the method, image classification is carried out by fully utilizing unsupervised deep contrast learning, a multi-view form of graph construction is combined, various representations of samples are fully considered, each level of features is comprehensively considered, and the classification precision is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image processing, and relates to a hyperspectral remote sensing image classification method, and in particular to a hyperspectral remote sensing image classification method, device and software product based on unsupervised multi-view image contrast learning. Background Art

[0002] Hyperspectral remote sensing image processing plays an important role in material information detection and is an important topic in the field of remote sensing. Hyperspectral remote sensing images can provide continuous radiation spectral bands, carry rich ground object information, and can be used to process different application fields, such as house change detection, crop assessment, geological mineral resource investigation, etc. (References 1, 2, 3). In the field of hyperspectral remote sensing image processing, classification is one of the main tasks. The so-called hyperspectral image classification is to determine the category of each hyperspectral image pixel. Existing classification methods are mainly based on convolutional neural network method research, that is, using convolutional neural network models for end-to-end image classification.

[0003] As a deep learning model (References 4, 5), graph convolutional neural network has been successfully applied to hyperspectral remote sensing image classification. The method based on graph convolutional neural network can process high-dimensional data and quickly aggregate surrounding node information, and can quickly extract deep abstract information. In the existing hyperspectral image classification methods, a large number of prior training samples are required to achieve relatively good results. However, in the application of hyperspectral image classification, this will consume a lot of human resources. At the same time, the existing methods of constructing graph structures Most of the existing graph classification methods have the problem of imperfect initial graph construction, so that only local information of the image can be extracted during the training process, while the mining of remote information is ignored.

[0004] References:

[0005] [1]He L,Li J,Liu C,Li S.Recent Advances on Spectral-SpatialHyperspectral Image Classification:An Overview and New Guidelines.IEEE TransGeosci Remote Sens.2018Mar;56(3):1579-97.

[0006] [2]Ahmad M,Shabbir S,Roy SK,Hong D,Wu X,Yao J,et al.HyperspectralImage Classification-Traditional to Deep Models:A Survey for FutureProspects.IEEE J Sel Top Appl Earth Observations Remote Sens.2022Oct 15;15:968-99.

[0007] [3]Ye J,He J,Peng X,Wu W,Qiao Y.Attention-Driven Dynamic GraphConvolutional Network for Multi-label Image Recognition.In:Vedaldi A,BischofH,Brox T,Frahm J-M,editors.Computer Vision-ECCV 2020[Internet].Cham:SpringerInternational Publishing;2020[cited 2021Apr 26].p.649-65.(Lecture Notes inComputer Science;vol.12366).Available from:https: / / link.springer.com / 10.1007 / 978-3-030-58589-1_39

[0008] [4]Liang H,Li Q.Hyperspectral Imagery Classification Using SparseRepresentations of Convolutional Neural Network Features.RemoteSens.2016Jan27;8(2):99.

[0009] [5]Zhang H, Zou J, and Zhang L, EMS-GCN: An End-to-end MixhopSuperpixel-based Graph Convolutional Network for Hyperspectral Image Classification. IEEETrans.Geosci.Remote Sens. 2022Mar 30; 60:1-16. Summary of the invention

[0010] In view of the problem that the prior art requires a large number of training samples and constructs an incomplete initial image, the method of the present invention provides a high-precision and high-efficiency hyperspectral image classification method based on unsupervised multi-view image comparison learning.

[0011] In a first aspect, the present invention provides a hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning, comprising the following steps:

[0012] Step 1: Perform data preprocessing on the acquired hyperspectral remote sensing image X to obtain superpixels and corresponding labels, and segment the matrix;

[0013] Step 2: Based on the segmentation matrix, establish the spatial adjacency matrix and the spectral adjacency matrix from multiple perspectives to obtain multiple views;

[0014] Step 3: Input multiple views into the adaptive enhanced contrastive learning model to obtain deep features for classification;

[0015] Step 3: Input multiple views into the adaptive data enhancement module to obtain deep features for classification;

[0016] The adaptive data enhancement module includes a spectral graph adaptive data enhancement branch and a spatial graph adaptive data enhancement branch which are arranged in parallel; the spectral graph adaptive data enhancement branch and the spatial graph adaptive data enhancement branch are both composed of a serially connected edge screening layer and a feature mask layer; the edge screening layer first uses a reference node centrality measurement parameter to measure the importance of the current node, and according to the parameter, the importance of the edge is represented by the average of the parameters of the nodes at both ends; then the edges are sorted by importance, the first K nodes are selected, their edge relationships are retained, and the remaining edges are deleted; the feature mask layer first sorts the importance of the nodes, and then sets a discard probability, and masks the features of unimportant nodes to 0 to achieve data enhancement.

[0017] Preferably, in step 1, principal component analysis (PCA) is used to perform dimensionality reduction and denoising on the hyperspectral remote sensing image.

[0018] Preferably, in step 2, the spectral adjacency matrix A is obtained using the Euclidean distance with K-order nearest neighbors. spe :

[0019] N(v i ) = argsort(||v i -v j || 2 );

[0020]

[0021] Among them, N(v i ) is the Euclidean distance between node i and all other nodes j; N K (v i ) represents the screening process of all nodes, N K (v i ) represents the set of the first K nodes with the closest Euclidean distance to node i; v i and v j It is the characteristic of the nodes in the graph; the argsort() function sorts the elements in the matrix from small to large and returns the array subscript of the corresponding sequence element;

[0022] Construct the spatial adjacency matrix A using spatial second-order nearest neighbors based on self-connection spa :

[0023] A spa =A×A+I;

[0024] a=max(w),b=min(w),if a≠b,A ab =1;

[0025] Among them, a 3×3 template w is used to take the maximum value a and the minimum value b of the segmentation matrix to obtain the adjacency matrix A based on the spatial first-order neighbor; I is the unit matrix.

[0026] As a preference, in step 3, the obtained multi-view A is first subjected to adaptive edge discarding to obtain an edge matrix

[0027] E = vstack (coo_matrix (A)),

[0028] Among them, J represents the number of edges, coo_matrix and vstack are two matrix transformation functions in Python;

[0029] Then calculate the importance metric parameter of the edge between nodes a and b

[0030]

[0031] where θa (E) Calculate the degree matrix for node a, θ b (E) Calculate the degree matrix for node b, where e is the abbreviation of edge;

[0032] Calculate the normalized edge order subscript idx e ;

[0033]

[0034] Where W e max and W e mean It is W e The maximum and average values ​​of is a metric parameter specifically referring to nodes a and b, w e It refers to the metric parameter of a certain node; the argsort() function sorts the elements in the matrix from small to large and returns the array subscript of the corresponding sequence element; idx e Represents the output of all edge weights W e Array subscript of ;

[0035] Sort the edges by importance, keep the edges with large weights, and remove the unimportant edges to reduce computing memory and improve efficiency;

[0036]

[0037] Among them, T = J × p is used to calculate the number of discarded edges, p is a hyperparameter representing the discard probability, and J represents the number of edges;

[0038] Then multiple views are passed through the adaptive node feature mask in a cascaded manner:

[0039] W f =x T θ(E),

[0040] Among them, x T is the eigenvector of the node, θ(E) is the degree matrix of the node, W f is the importance measurement parameter of the node;

[0041] Sort the node features in the graph by importance and get the normalized edge sort subscript idx f ;

[0042]

[0043] Among them, w f max and w f mean Yes f The maximum and average values ​​of

[0044] Remove unimportant node features according to the discard probability p:

[0045]

[0046] Among them, T2 = N × p2, p2∈[0,1] represents the probability of discarding node features; N represents the number of bands after PCA dimensionality reduction; F i Represents the characteristics of node i.

[0047] Preferably, the adaptive enhanced contrastive learning model is a trained network model; during the training process, the NT-Xent loss function is used for back propagation:

[0048]

[0049] Among them, sim(·,·) represents the cosine similarity function, z spa,i , z spe,i represents the spatial and spectral features after MLP; τ represents the temperature parameter, n represents the number of nodes; ζ(z spa,i ,z spe,i ) represents the similarity between the spatial branch and the spectral branch corresponding to the i-node.

[0050] In a second aspect, the present invention provides a hyperspectral remote sensing image classification system based on unsupervised multi-view image contrast learning, comprising the following modules:

[0051] The hyperspectral remote sensing image preprocessing module is used to perform data preprocessing on the acquired hyperspectral remote sensing image X to obtain superpixels and corresponding labels and segmentation matrices;

[0052] A multi-view construction module is used to establish a spatial adjacency matrix and a spectral adjacency matrix from multiple perspectives according to the segmentation matrix to obtain multiple views;

[0053] Hyperspectral remote sensing image classification module, used to input multiple views into the adaptive enhanced contrast learning model to obtain deep features for classification;

[0054] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning is implemented.

[0055] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning.

[0056] In a fifth aspect, the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning.

[0057] The present invention has the following advantages and positive effects:

[0058] 1. This invention innovatively proposes a deep learning network based on unsupervised graph contrast learning, which realizes the unsupervised hyperspectral image classification for the first time and can alleviate the model's dependence on labeled samples.

[0059] 2. This paper designs a new multi-view adjacency matrix construction method to improve the initial graph structure. This method can build adjacency matrices from two aspects, spatial and spectral, respectively, and obtain remote information while paying attention to local information of nodes, making it more perfect when extracting features.

[0060] 3. The present invention designs an adaptive data enhancement module to discard unimportant data from the perspectives of edge and node features. This can reduce the loss of important information caused by random enhancement and improve network performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The technical solution of the present invention is further described below using embodiments and specific implementation methods. In addition, in the process of describing the technical solution, some drawings are also used. For those skilled in the art, other drawings and the intention of the present invention can also be obtained based on these drawings without paying creative work.

[0062] Figure 1 It is a specific schematic diagram of the process of the present invention;

[0063] Figure 2 It is a schematic diagram of the structure of the adaptive data enhancement module of the method of the present invention. DETAILED DESCRIPTION

[0064] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0065] Utilize multi-perspective graph contrast learning to maximize the consistency of the same category and the difference of different categories. Especially when the number of samples is small and the number of positive and negative samples is unbalanced, it can make up for the defects of convolutional neural networks. Data enhancement is a key component of contrastive learning. In order to adaptively enhance data when using graph structures, reduce the damage of random enhancement to important information on the way. The present invention introduces an effective method based on unsupervised multi-perspective graph contrast learning. In view of the problems of existing methods, the local and remote information of hyperspectral images is extracted from the two dimensions of space and spectrum in a multi-perspective graph construction method, while selectively retaining important structures for data enhancement, so as to better complete the hyperspectral image classification task.

[0066] Please see Figure 1 The present embodiment provides a hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning, comprising the following steps:

[0067] Step 1: Perform data preprocessing on the acquired hyperspectral remote sensing image X to obtain superpixels and corresponding labels, and segment the matrix;

[0068] For hyperspectral images Where H, W, and B refer to the length, width, and height of the hyperspectral image, respectively, and are passed into the model. The hyperspectral image is a data cube and can be represented by a tensor. The vector of each point in the tensor is the pixel radiation value corresponding to each band. Since it is composed of hundreds of very narrow continuous spectra, it has rich information while also containing a lot of redundant information. In addition, due to the atmosphere, sensors, environmental interference, and human annotation of the image, the spectral information may not be accurate. Therefore, the embodiment uses principal component analysis PCA to reduce the dimension and denoise the image.

[0069] Step 2: Based on the segmentation matrix, establish the spatial adjacency matrix and the spectral adjacency matrix from multiple perspectives to obtain multiple views;

[0070] In one implementation, the graph convolutional neural network calculates the adjacency matrix based on all nodes of the entire graph. If the pixels of the image are directly converted into nodes, a large amount of memory resources will be consumed and it cannot be applied to large images such as remote sensing images. Therefore, a clustering function is needed to group multiple similar pixels into a superpixel to reduce memory usage. First, an encoder and decoder are needed to construct pixel-level features and superpixel features.

[0071]

[0072] Among them, Q ij Represents the value of the association matrix Q at position i, j, S is the superpixel, and Flatten(X) represents the expansion of the node feature X. Construct the association matrix, and then construct the encoder and decoder of superpixels and pixels based on the connection matrix, that is,

[0073]

[0074] Where Decoder(·; Q) is the superpixel feature decoder, Rshape(·) performs dimension transformation, and V is the superpixel feature. Convert node data into raster data.

[0075] The hyperspectral spatial-spectral multi-view adjacency matrix is ​​constructed using the association matrix, and then the initial graph structure is constructed and improved. First, when constructing the spectral adjacency matrix, the pixel features are converted to superpixel features through the relationship matrix Q:

[0076] F S =Q T F P ,

[0077] where F S is the feature of superpixel, Q T is the transpose of the incidence matrix, F P is the feature of the pixel. Then, the Euclidean distance of the spectrum between each two nodes is determined, so that nodes with far distance in space can be connected according to the spectral similarity, thereby realizing the extraction of remote information. Then, the Euclidean distance of each node needs to be sorted, and the spectral adjacency matrix A is constructed by judging whether the nodes are connected through the K-order nearest neighbors. spe :

[0078] N(v i ) = argsort(||v i -v j || 2 );

[0079]

[0080] Among them, N(v i ) is the Euclidean distance between node i and all other nodes j; N K (v i ) represents the screening process of all nodes, N K (v i ) represents the set of the first K nodes with the closest Euclidean distance to node i; v i and v j is the characteristic of the nodes in the graph; the argsort() function sorts the elements in the matrix from small to large and returns the array subscript of the corresponding sequence element; A spe is the obtained spectral adjacency matrix.

[0081] Next, we construct a spatial adjacency matrix. The previous methods all used a template to take the maximum and minimum values ​​on the segmentation matrix M to determine whether the nodes are first-order neighbors:

[0082] a=max(w),b=min(w),if a≠b,A ab =1.

[0083] Among them, a 3×3 template w is used to take the maximum and minimum values ​​of the segmentation matrix to obtain the adjacency matrix A based on the spatial first-order nearest neighbor. This implementation uses the second-order nearest neighbor based on spatial self-connection to extract spatial information, so that the information of the surrounding nodes can be aggregated while paying attention to the internal pixels of the node.

[0084] A spa =A×A+I.

[0085] Where I is the identity matrix. spa is the obtained spatial adjacency matrix.

[0086] Step 3: Input multiple views into the adaptive data enhancement module to obtain deep features for classification;

[0087] Please see Figure 2 The adaptive data enhancement module of this embodiment includes a spectral graph adaptive data enhancement branch and a spatial graph adaptive data enhancement branch arranged in parallel; the spectral graph adaptive data enhancement branch and the spatial graph adaptive data enhancement branch are both composed of a serially connected edge screening layer and a feature mask layer; the edge screening layer first uses a reference node centrality measurement parameter to measure the importance of the current node, and according to the parameter, the importance of the edge is expressed by averaging the parameters of the nodes at both ends; then the edges are sorted by importance, the top K nodes are selected, their edge relationships are retained, and the remaining edges are deleted; the feature mask layer first sorts the importance of the nodes, and then sets a discard probability, and masks the features of unimportant nodes to 0 to achieve data enhancement.

[0088] In order to enhance the ability of the contrastive learning framework to extract key features, in this embodiment, an adaptive method is used to selectively enhance the edge and node features of the graph, and unimportant nodes and edges are discarded to enhance the feature extraction ability of the model. The perfect initial graph obtained by the multi-view construction module is first subjected to adaptive edge discarding to obtain the edge matrix

[0089] E = vstack (coo_matrix (A)),

[0090] Among them, J represents the number of edges, coo_matrix and vstack are two matrix transformation functions in Python;

[0091] Then calculate the importance metric parameter of the edge between nodes a and b

[0092]

[0093] where θ a (E) Calculate the degree matrix for node a, θ b (E) Calculate the degree matrix for node b, where e is the abbreviation of edge;

[0094] Then calculate the normalized edge sort subscript idx e ;

[0095]

[0096] Where W e max and W e mean It is W e The maximum and average values ​​of is a metric parameter specifically referring to nodes a and b, w e It refers to the metric parameter of a certain node; the argsort() function sorts the elements in the matrix from small to large and returns the array subscript of the corresponding sequence element; idx e Represents the output of all edge weights W e Array subscript of ;

[0097] Sort the edges by importance, keep the edges with large weights, and remove the unimportant edges to reduce computing memory and improve efficiency;

[0098]

[0099] Among them, T = J × p is used to calculate the number of discarded edges, p is a hyperparameter representing the discard probability, and J represents the number of edges;

[0100] Then multiple views are passed through the adaptive node feature mask in a cascaded manner:

[0101] W f =x T θ(E),

[0102] Among them, x T is the eigenvector of the node, θ(E) is the degree matrix of the node, W f is the importance measurement parameter of the node;

[0103] Sort the node features in the graph by importance and get the normalized edge sort subscript idx f ;

[0104]

[0105] Among them, w f max and w f meanYes f The maximum and average values ​​of

[0106] Remove unimportant node features according to the discard probability p:

[0107]

[0108] Among them, T2 = N × p2, p2∈[0,1] represents the probability of discarding node features; N represents the number of bands after PCA dimensionality reduction; F i Represents the characteristics of node i.

[0109] In this implementation, an adaptive contrastive learning framework is used for unsupervised feature extraction. During the training process, the MoCo structure is used to perform back propagation using the NT-Xent loss function:

[0110]

[0111] Among them, sim(·,·) represents the cosine similarity function, z spa,i , z spe,i represents the spatial and spectral features after MLP; τ represents the temperature parameter, n represents the number of nodes; ζ(z spa,i ,z spe,i ) represents the similarity between the spatial branch and the spectral branch corresponding to the i-node.

[0112] This embodiment also provides a hyperspectral remote sensing image classification system based on unsupervised multi-view image contrast learning, including the following modules:

[0113] The hyperspectral remote sensing image preprocessing module is used to perform data preprocessing on the acquired hyperspectral remote sensing image X to obtain superpixels and corresponding labels and segmentation matrices;

[0114] A multi-view construction module is used to establish a spatial adjacency matrix and a spectral adjacency matrix from multiple perspectives according to the segmentation matrix to obtain multiple views;

[0115] Hyperspectral remote sensing image classification module, used to input multiple views into the adaptive enhanced contrast learning model to obtain deep features for classification;

[0116] This embodiment also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning is implemented.

[0117] This embodiment also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning is implemented.

[0118] This embodiment also provides a computer program product, including a computer program, which, when executed by a processor, implements the hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning.

[0119] The key invention of the present invention is to propose a hyperspectral image classification method based on unsupervised multi-view graph contrast learning, which combines multi-view adjacency matrix construction and adaptive pair learning to deal with the problem of no training samples. This method is based on the graph convolutional neural network framework. By constructing multiple views, it can not only extract the local spatial information of the image, but also obtain long-range information through remote node connections, which helps to fully obtain the deep abstract information of the image. At the same time, the use of adaptive data enhancement can enhance the information extraction ability of the model. In essence, this method uses the connection feature information between nodes in multi-view calculation to improve the initial graph structure, and uses an adaptive contrast learning framework to improve the classification efficiency.

[0120] The present invention is further described below through experiments.

[0121] This experiment is written in PYTHON language and implemented using the classic deep learning framework PYTORCH. The PYTHON remote sensing image reading and writing function is the implementation basis. Call the remote sensing image processing library SPECTRAL, input the remote sensing image file name to be read, and the remote sensing image is read into a tensor of size . Each element in the tensor is the pixel radiation value corresponding to each band, where H is the length of the remote sensing image, is the width of the remote sensing image, and B is the number of remote sensing image bands. Call the remote sensing image reading and writing function to read the hyperspectral remote sensing image to be analyzed into the tensor X. The PYTHON remote sensing image reading and writing function is a well-known technology in the technical field and will not be described here.

[0122] The data used in this experiment are the Salinas and Longkou datasets of Wuhan UAV hyperspectral images, which are used as experimental datasets to verify the effectiveness of the model. The Salinas dataset was taken by the AVIRIS sensor in Salinas Valley, California. The dataset size is 512×217, the spatial resolution is 3.7 meters, and it contains 224 continuous bands. After removing 20 water absorption bands (108-112, 154-167, and 224 bands), the actual number of bands used for training is 204. There are 16 types of land features in the study area, including vegetables, bare soils, and vineyard fields. The Wuhan UAV hyperspectral image dataset (WHU-Hi) is collected and shared by the RSIDEA research group of Wuhan University, and can be used as a benchmark dataset for accurate crop classification and hyperspectral image classification research. The WHU-Hi dataset contains three independent UAV hyperspectral datasets: WHU-Hi-LongKou, WHU-Hi-HanChuan, and WHU-Hi-HongHu. All datasets were acquired in agricultural areas with different crop types in Hubei Province, China, using Headwall Nano-Hyperspec sensors mounted on UAV platforms. Compared with satellite-borne and airborne hyperspectral platforms, unmanned aerial vehicle (UAV)-borne hyperspectral systems can obtain hyperspectral images with high spatial resolution. The Longkou dataset was collected in Longkou Town, Hubei Province, China, from 13:49 to 14:37 on July 17, 2018, using an 8mm focal length Headwall Nano-Hyperspec imaging sensor mounted on a DJI Matrice 600Pro (DJI M600 Pro) UAV platform. During the data collection period, the weather was clear and cloudless, the temperature was around 36℃, and the relative humidity of the air was around 65%. The study area is a simple agricultural scene containing six crops: corn, cotton, sesame, broad-leaf soybean, narrow-leaf soybean, and rice. The UAV flight altitude was 500m, the image size was 550×400, there were 270 bands within 40-1000nm, and the spatial resolution of the UAV-borne hyperspectral image was about 0.463m.

[0123] Classification evaluation index: Using quantitative evaluation method, a total of 1% of pixels in data set 1 were selected as training samples, and the rest were used as test samples. A total of 0.5% of pixels in data set 2 were selected as training samples, and the rest were used as test samples. The evaluation index uses the following two indicators:

[0124] 1) Kappa coefficient:

[0125] The kappa coefficient is an authoritative evaluation indicator used to evaluate classification problems. The larger the kappa coefficient, the higher the accuracy. The kappa coefficient calculation method is:

[0126] The confusion matrix is ​​obtained according to the samples, see Table 1.

[0127] Table 1 Confusion matrix

[0128]

[0129] In Table 1, TTO indicates the number of cases whose labels are category 1 and are predicted to be category 1; TF indicates the number of cases whose actual category is category 2 but are predicted to be category 1. FT indicates the number of cases whose actual category is category 1 but are predicted to be category 2. TTT indicates the number of cases whose labels are category 2 and are predicted to be category 2; NCO is the sum of TTO and FT, NCT is the sum of TF and TTT, NRO is the sum of TTO and TF, NRT is the sum of FT and TTT, and N is the total number of samples.

[0130] The calculation formula of the kappa coefficient is:

[0131]

[0132] 2) Overall accuracy:

[0133] Overall Accuracy (OA) is an evaluation index used to evaluate classification problems. The higher the overall accuracy, the higher the detection accuracy. The calculation method of OA is also based on the confusion matrix shown in Table 1. The calculation formula of overall accuracy OA is:

[0134]

[0135] The kappa coefficient and overall accuracy were used to evaluate the change detection capabilities of methods 1 to 5 and the method of the present invention. The evaluation indicators are shown in Table 2.

[0136] Table 2 Comparative test results 1

[0137]

[0138] As can be seen from Table 2, the method of the present invention can obtain higher overall accuracy and kappa value, indicating that the method of the present invention has stronger feature extraction and classification capabilities. The method of the present invention and the comparative method achieved the best results in terms of overall classification accuracy and kappa value, with a significant improvement.

[0139] It can be concluded that the method of the present invention has higher classification accuracy than traditional and recent deep learning unsupervised classification methods. The present invention makes full use of unsupervised deep contrast learning for image classification, combines the multi-view form of graph construction, fully considers the multiple representations of samples, comprehensively considers all levels of features, and improves classification accuracy.

[0140] It should be understood that the embodiments described above are part of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be combined with each other arbitrarily to form a feasible technical solution. Such combination is not restricted by the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0141] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. A hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning, It is characterized in that The following steps are involved: Step 1: Perform data preprocessing on the acquired hyperspectral remote sensing image X to obtain superpixels and corresponding labels, and segment the matrix; Step 2: Based on the segmentation matrix, establish the spatial adjacency matrix and the spectral adjacency matrix from multiple perspectives to obtain multiple views; Step 3: Input multiple views into the adaptive data enhancement module to obtain deep features for classification; The adaptive data enhancement module includes a spectral graph adaptive data enhancement branch and a spatial graph adaptive data enhancement branch arranged in parallel; The spectral graph adaptive data enhancement branch and the spatial graph adaptive data enhancement branch are both composed of a serially connected edge screening layer and a feature mask layer; the edge screening layer first uses a reference node centrality measurement parameter to measure the importance of the current node, and according to the parameter, the importance of the edge is expressed by taking the average of the parameters of the nodes at both ends; then the edges are sorted by importance, the first K nodes are selected, their edge relationships are retained, and the remaining edges are deleted; The feature mask layer first sorts the importance of the nodes, and then sets a discard probability to mask the features of unimportant nodes to 0 to achieve data enhancement.

2. The hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning according to claim 1, Features: In step 1, principal component analysis (PCA) is used to perform dimensionality reduction and denoising on the hyperspectral remote sensing image.

3. The hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning according to claim 1, Features: In step 2, the spectral adjacency matrix A is obtained using the Euclidean distance with K-order nearest neighbors spe : N(v i )=argsort(||v i -v j || 2 ); Among them, N(v i ) is the Euclidean distance between node i and all other nodes j; N K (v i ) represents the screening process of all nodes, N K (v i ) represents the set of the first K nodes with the closest Euclidean distance to node i; v i and v j It is the characteristic of the nodes in the graph; the argsort() function sorts the elements in the matrix from small to large and returns the array subscript of the corresponding sequence element; Construct the spatial adjacency matrix A using spatial second-order nearest neighbors based on self-connection spa : a=max(w),b=min(w),if a≠b,A ab =1; A spa =A×A+I; Among them, a 3×3 template w is used to take the maximum value a and the minimum value b of the segmentation matrix to obtain the adjacency matrix A based on the spatial first-order neighbor; I is the unit matrix.

4. The hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning according to claim 1, Features: In step 3, the obtained multi-view A is first subjected to adaptive edge discarding to obtain the edge matrix E = vstack (coo_matrix (A)), Among them, J represents the number of edges, coo_matrix and vstack are two matrix transformation functions in Python; Then calculate the importance metric parameter of the edge between nodes a and b where θ a (E) Calculate the degree matrix for node a, θ b (E) Calculate the degree matrix for node b, where e is the abbreviation of edge; Calculate the normalized edge order subscript idx e ; Where W e max and W e mean It is W e The maximum and average values ​​of is a metric parameter specifically referring to nodes a and b, w e It refers to the metric parameter of a certain node; the argsort() function sorts the elements in the matrix from small to large and returns the array subscript of the corresponding sequence element; idx e Represents the output of all edge weights W e Array subscript of ; Sort the edges by importance, keep the edges with large weights, and remove the unimportant edges to reduce computing memory and improve efficiency; Among them, T = J × p is used to calculate the number of discarded edges, p is a hyperparameter representing the discard probability, and J represents the number of edges; Then multiple views are passed through the adaptive node feature mask in a cascaded manner: W f =x T θ(E), Among them, x T is the eigenvector of the node, θ(E) is the degree matrix of the node, W f is the importance measurement parameter of the node; Sort the node features in the graph by importance and get the normalized edge sort subscript idx f ; Among them, w f max and w f mean Yes f The maximum and average values ​​of Remove unimportant node features according to the discard probability p: Among them, T 2 =N×p 2 , p 2 ∈[0,1] represents the probability of discarding node features; N represents the number of bands after PCA dimensionality reduction; F i Represents the characteristics of node i.

5. The hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning according to any one of claims 1 to 4, Features: The adaptive enhanced contrastive learning model is used for feature extraction; during training, the NT-Xent loss function is used for back propagation: Among them, sim(·,·) represents the cosine similarity function, z spa,i , z spe,i represents the spatial and spectral features after MLP; τ represents the temperature parameter, n represents the number of nodes; ζ(z spa,i ,z spe,i ) represents the similarity between the spatial branch and the spectral branch corresponding to the i-node.

6. A hyperspectral remote sensing image classification system based on unsupervised multi-view image contrast learning, It is characterized in that Includes the following modules: The hyperspectral remote sensing image preprocessing module is used to perform data preprocessing on the acquired hyperspectral remote sensing image X to obtain superpixels and corresponding labels and segmentation matrices; A multi-view construction module is used to establish a spatial adjacency matrix and a spectral adjacency matrix from multiple perspectives according to the segmentation matrix to obtain multiple views; Hyperspectral remote sensing image classification module, used to input multiple views into the adaptive enhanced contrast learning model to obtain deep features for classification; An adaptive data enhancement module, including a spectral map adaptive data enhancement branch and a spatial map adaptive data enhancement branch arranged in parallel; The spectral graph adaptive data enhancement branch and the spatial graph adaptive data enhancement branch are both composed of a serially connected edge screening layer and a feature mask layer; the edge screening layer first uses a reference node centrality measurement parameter to measure the importance of the current node, and according to the parameter, the importance of the edge is expressed by taking the average of the parameters of the nodes at both ends; then the edges are sorted by importance, the first K nodes are selected, their edge relationships are retained, and the remaining edges are deleted; The feature mask layer first sorts the importance of the nodes, and then sets a discard probability to mask the features of unimportant nodes to 0 to achieve data enhancement.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning as described in any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning as described in any one of claims 1 to 5 is implemented.

9. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the hyperspectral remote sensing image classification method based on unsupervised multi-view image contrast learning as described in any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • A Fast Hyperspectral Image Classification Method Based on Dynamic Graph Convolutional Networks

    CN114937173A

  • Hyperspectral remote sensing image ground object clustering method of self-supervised double-branch Transform structure

    CN118072059A

  • Brain atlas individualization method and system based on magnetic resonance and twin graph neural network

    US20230301542A1

Cited By

  • Hyperspectral image classification method for feature extraction of unsupervised double-mask graph auto-encoder

    CN119810657A

  • Hyperspectral remote sensing image classification method and device based on graph neural network

    CN119888367A