Hyperspectral Target Detection Method Based on Graph Convolutional Neural Network

Abstract

The present invention proposes a hyperspectral image target detection method based on a graph convolutional neural network, which mainly solves the technical problem of low detection accuracy in the prior art, and includes the following steps: obtaining a training sample set and a test sample set; constructing a graph convolutional neural network H; performing iterative training on the graph convolutional neural network H; and obtaining hyperspectral target detection results. The present invention can reduce redundant information in hyperspectral images, make full use of the spatial information of hyperspectral images, distinguish targets from complex backgrounds in hyperspectral images, improve the detection accuracy of target points, and can be used in fields such as social security and food safety.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and further relates to a hyperspectral target detection method based on a graph convolutional neural network in the field of image target detection technology. It can be used in fields such as social security and food safety. Background Art

[0002] Hyperspectral images are different from natural images that contain three bands of red, green, and blue. They contain more energy information in more bands and have high resolution, so they contain rich spatial information and spectral information. And because different spectral information reflects different material types, combining spectral features can greatly improve the detection accuracy of targets and backgrounds. For example, detecting drugs and gunpowder attached to the surface of clothes, detecting chemical drug residues attached to food, etc. Therefore, using hyperspectral images for target detection is of great significance in aspects such as social security and food safety.

[0003] Since hyperspectral images contain far more information than natural images, using machine learning methods for hyperspectral target detection is also a hot topic of current concern. Based on the powerful feature extraction ability of machine learning, rich spectral information in hyperspectral images can be better extracted, and then the similarity with known spectral vectors is used to distinguish target vectors from background vectors, thus completing target detection. However, the rich spatial information and spectral information also bring interference from many redundant information, resulting in a higher complexity of feature extraction, which becomes an important factor affecting the detection accuracy of hyperspectral image targets.

[0004] On May 13, 2022, a patent application with application publication number CN115115933A and titled "Hyperspectral Image Target Detection Method Based on Self-supervised Contrastive Learning" disclosed a hyperspectral image target detection method based on self-supervised contrastive learning. The method first samples the odd and even bands of the hyperspectral image to be detected, and then uses the obtained odd and even band hyperspectral images to train the corresponding adversarial convolutional autoencoder, and uses the feature extraction part of the trained encoder as a data enhancement function for data enhancement. Positive and negative pairs are constructed through data enhancement, and the backbone is used to extract the representation vector of the enhanced sample. Then, the representation vectors are mapped to the spectral and cluster contrast spaces respectively using their corresponding contrast heads. In the contrast space, the similarity and dissimilarity of spectra and clusters are learned by maximizing the similarity of positive pairs while minimizing the similarity of negative pairs to increase the difference between the representation vectors of the target and the background. Finally, combined with spatial information, an edge-preserving filter is used to process the detection results obtained by cosine similarity using spectral information to obtain the final detection result. This method can improve the efficiency of target detection by using self-supervised contrastive learning of the similarities and differences between spectral vectors. However, its shortcomings are that the hyperspectral image is divided into odd bands and even bands for feature extraction respectively, and the relationship between the spectral vectors of odd bands and even bands is not fully utilized. The spatial information is insufficiently utilized, resulting in low target detection accuracy. Moreover, due to the use of data enhancement, the sample size is increased while more redundant information interference is also increased, resulting in low target detection accuracy. Summary of the invention

[0005] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and propose a hyperspectral target detection method based on graph convolutional neural network to solve the technical problem of low detection accuracy in the prior art.

[0006] To achieve the above object, the technical solution adopted by the present invention includes the following steps:

[0007] (1) Obtain training sample set and test sample set:

[0008] (1a) A hyperspectral image I with a dimension of M×N×L containing M×N spectral vectors, each of which contains L bands, a true label corresponding to each spectral vector, and a true spectral vector d similar to the spectral vector of the target to be detected contained in the hyperspectral image I are obtained, and the hyperspectral image I is preprocessed using the true spectral vector d to obtain a two-dimensional matrix X containing M×N spectral vectors and a pseudo label matrix y with a dimension of M×N l , where M ≥ 100, N ≥ 100, L ≥ 200;

[0009] (1b) The C randomly selected spectral vectors and y lThe pseudo - labels corresponding to each spectral vector in [[]] form a training sample set, and the M×N spectral vectors and their corresponding true labels form a test sample set, where 100 ≤ C < M×N;

[0010] (2) Construct a graph convolutional neural network H:

[0011] Construct a graph convolutional neural network H including a graph convolutional layer, a feature extraction sub - network W, and a discriminator connected in sequence; among them, the feature extraction sub - network W includes multiple composite layers connected in sequence and an activation layer, and the composite layer includes a feature mapping layer and a normalization layer connected in sequence;

[0012] (3) Iteratively train the graph convolutional neural network H:

[0013] (3a) Initialize the iteration number as t, the maximum iteration number as T, where T ≥ 200, and the weight of the current graph convolutional neural network H t is ω t , and let t = 1;

[0014] (3b) Use the training sample set as the input of the graph convolutional neural network H. The graph convolutional layer performs graph embedding representation on each training sample to obtain a feature matrix G of dimension C×L containing the features of each sample; the feature extraction sub - network W performs feature mapping on each sample feature in the feature matrix G to obtain a detection probability matrix Y containing the detection probabilities of each training sample, and the discriminator makes a decision on each detection probability in the matrix Y to obtain C detection results;

[0015] (3c) Adopt the cross - entropy loss function to calculate the loss value Loss of W l through the detection probability corresponding to each training sample in the detection probability matrix Y and the pseudo - label corresponding to each training sample in the pseudo - label matrix y t , and update the weight ω t of the feature extraction sub - network W t through the loss value Loss t to obtain the graph convolutional neural network H t after this iteration; t ;

[0016] (3c) Judge whether t = T holds. If so, obtain the trained graph convolutional neural network H * , otherwise, let t = t + 1, H t = H, and execute step (3b);

[0017] (4) Obtain the hyperspectral target detection results:

[0018] (4a) Use the test sample set as the input of the trained graph convolutional neural network H * to perform forward propagation to obtain M×N detection results;

[0019] (4b) For each detection result, use the row where its corresponding spectral vector is located in the hyperspectral image I to be detected as the abscissa and the column where it is located as the ordinate to obtain a prediction map Y for target detection with a dimension of M×N out 。

[0020] Compared with the existing technology, the present invention has the following advantages:

[0021] 1. The graph convolutional neural network constructed by the present invention includes a feature extraction sub-network. During the training of the graph convolutional neural network and the acquisition of detection results, the feature extraction sub-network first increases the dimension of the spectral features of each spectral vector to amplify the main components and redundant information, and then extracts the main components that can more effectively represent the spectral vector dimensional features through dimensionality reduction, avoiding the influence of more redundant information interference on the detection accuracy in the existing technology.

[0022] 2. The graph convolutional neural network constructed by the present invention includes a graph convolutional layer. During the training of the graph convolutional neural network and the acquisition of detection results, the graph convolutional layer selects the spectral vector with the closest Euclidean distance to all spectral vectors in the entire hyperspectral image, and uses the Laplacian matrix to perform graph embedding representation on all spectral vectors, which can make full use of the spatial information of the hyperspectral image, avoid the defect of insufficient utilization of spatial information in the existing technology, and further improve the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is the implementation flowchart of the present invention;

[0024] Figure 2 is the distribution diagram of the hyperspectral image to be detected and its attached real target points used in the simulation experiment;

[0025] Figure 3 is the simulation comparison diagram of the detection accuracy between the present invention and the existing technology. DETAILED DESCRIPTION OF THE INVENTION

[0026] The present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0027] Refer to Figure 1 , the present invention includes the following steps:

[0028] Step 1) Obtain a training sample set and a test sample set:

[0029] (1a) Obtain a hyperspectral image I of dimension M×N×L that contains M×N spectral vectors, where each spectral vector contains L bands, the true label corresponding to each spectral vector, and a true spectral vector d that is similar to the spectral vectors of the target to be detected in the hyperspectral image I. Here, M≥100, N≥100, L≥200. In this example, the hyperspectral image I to be detected is a real hyperspectral image collected by a Hydice hyperspectral sensor, with a dimension of 100×100×205, and the true spectral vector d is the spectral vector of a certain mineral;

[0030] (1b) Obtain the pseudo-labels corresponding to each spectral vector in the hyperspectral image to be detected:

[0031] (1b1) Construct a spectral vector matrix X of dimension (M×N)×L with the M×N spectral vectors contained in the hyperspectral image I as rows and the L bands contained in each spectral vector as columns;

[0032] (1b2) Use the hierarchical constrained energy minimization algorithm, linearly filter the spectral vector matrix X through multiple FIR linear filters, and after normalization, obtain an energy map containing the energy values of M×N spectral vectors. Then, change the energy value of each spectral vector greater than a pre-set threshold β to 1, and change the energy values of the remaining spectral vectors to 0, to obtain a pseudo-label matrix y of dimension M×N. l , 0<β<1. In this example, considering both accuracy and efficiency, the hierarchical constrained energy minimization algorithm is used, and the obtained pseudo-labels are used to simulate the true labels to complete the subsequent training of the graph convolutional neural network;

[0033] (1c) Construct a training sample set from C spectral vectors randomly selected from the two-dimensional matrix X and the pseudo-labels corresponding to each spectral vector in y l , and construct a test sample set from the M×N spectral vectors in the two-dimensional matrix X and their corresponding true labels. 100≤C<M×N. In this example, C is 200;

[0034] Step 2) Construct a graph convolutional neural network H:

[0035] Construct a graph convolutional neural network H including a graph convolutional layer, a feature extraction sub-network W, and a discriminator connected in sequence. Among them, the feature extraction sub-network W includes two composite layers connected in sequence and an activation layer. The weight dimensions of the feature mapping layers in the first and second composite layers are L×300 and 300×2 respectively;

[0036] Step 3) Iteratively train the graph convolutional neural network H:

[0037] (3a) Initialize the number of iterations as t, the maximum number of iterations as T, where T ≥ 200, and the weights of the current graph convolutional neural network H t are ω t , and let t = 1;

[0038] (3b1) In the graph convolutional layer, calculate the adjacency relationship S between each training sample and its neighbor samples among the Q training samples with the smallest Euclidean distance to each training sample c,q :

[0039]

[0040] where d c,q represents the Euclidean distance between the c-th training sample and the q-th neighbor sample, e represents the natural constant, and 1 < Q < C;

[0041] (3b2) In the graph convolutional layer, construct a training sample matrix C' of dimension C × L with each spectral vector contained in the C training samples as rows and the L bands of each spectral vector as columns. Use the adjacency relationship S f,g between each training sample and its neighbor samples as the abscissa with the row where the corresponding training sample is located in the training sample matrix C' and the ordinate with the row where the corresponding neighbor sample is located in the training sample matrix C' to construct an adjacency matrix S' of dimension C × C;

[0042] (3b3) Perform symmetric normalization on the adjacency matrix S' to obtain the Laplacian matrix S sym , and use the Laplacian matrix S sym to perform graph embedding representation on the training sample matrix C' to obtain a feature matrix G of dimension C × L containing the features of each sample: G = S sym C';

[0043] (3b4) The feature extraction sub-network W performs feature mapping on the sample features after each graph embedding representation to obtain a detection probability matrix Y of dimension C × 2 containing the detection probabilities of each training sample:

[0044] Y = h(G × ω + b)

[0045] where h represents the activation function ReLU, ω represents the weights of the feature extraction sub-network W, and b represents the bias of the feature extraction sub-network W;

[0046] (3b5) The decision maker takes the maximum value by row for the detection probability matrix Y. If the maximum value is in the first column, it means that the training sample corresponding to this row is a target vector and is marked as 1, otherwise it is a background vector and is marked as 0, obtaining C detection results.

[0047] (3c) The cross - entropy loss function is adopted to calculate the loss value Loss of W through the detection probability corresponding to each training sample in matrix Y and the pseudo - label corresponding to each training sample in pseudo - label matrix y l in the pseudo - label corresponding to each training sample in t the loss value Loss of W t is calculated. Calculate the loss value Loss of W t The formula for the loss value Loss of W t and the formula for updating the weight ω of the feature extraction sub - network W t are respectively as follows: t

[0048]

[0049]

[0050]

[0051] where represents the cross - entropy loss function of the z - th composite layer at the t - th iteration, represents the detection probability output by the z - th composite layer for each training sample at the t - th iteration, y lc represents the pseudo - label corresponding to each training sample, η represents the preset gradient - descent parameter, ω t+1 represents ω t the update result of ω represents the partial - derivative operation.

[0052] (3d) Judge whether t = T holds. If so, obtain the trained graph convolutional neural network H * , otherwise, let t = t + 1, H t = H, and execute step (3b1).

[0053] Step 4) Obtain the hyperspectral target detection result:

[0054] (4a) Use the test sample set as the input of the trained graph convolutional neural network H * to perform forward propagation, and obtain M×N detection results;

[0055] (4b) Take the spectral vector corresponding to each detection result as the abscissa of the row where it is located and the ordinate of the column where it is located in the hyperspectral image I to be detected, and obtain the prediction map Y of target detection with a dimension of M×N out .

[0056] Next, the technical effects of the present invention will be described in combination with simulation experiments.

[0057] 1. Simulation conditions and content:

[0058] ​The hardware environment for the simulation experiment is an Intel(R) Core(TM) i7-10700U CPU with a main frequency of 2.90 GHz and 16 GB of memory. The software environment is PyCharm and MATLAB, and the operating system is Windows10 x64;

[0059] The hyperspectral images to be detected and the distribution maps of the attached real target points used in the simulation experiment are respectively as follows Figure 2 (a) and Figure 2 (b) shown. Among them Figure 2 (a) is a real hyperspectral image collected by a Hydice hyperspectral sensor;

[0060] A comparative simulation of the target detection accuracy of the present invention and the existing hyperspectral image target detection method based on self-supervised contrast learning is carried out, and the results are as follows Figure 3 and Table 1 shown.

[0061] 2. Analysis of simulation results:

[0062] Referring to Figure 3 , among which Figure 3 (a) is the simulation experiment result diagram of the existing hyperspectral image target detection method based on self-supervised contrast learning, and the white bright spots therein represent the target points detected for the hyperspectral image to be detected; Figure 3 (b) is the simulation experiment result diagram of the present invention, and the white bright spots therein represent the target points detected for the hyperspectral image to be detected.

[0063] Comparing Figure 3 (a) and Figure 3 (b) it can be seen that: the present invention has fewer misdetected and missed detected target points, effectively improving the detection accuracy.

[0064] In order to evaluate the detection performance of the two methods, according to the following formula, the detection accuracy AUC value is calculated:

[0065]

[0066]

[0067] Among them, TPR represents the true positive rate of the detection result, represents the false positive rate of the detection result, TP represents the number of target points predicted as target points, FN represents the number of target points predicted as background points, FP represents the number of background points predicted as target points, and TN represents the number of background points predicted as target points. Then, the detection accuracy (AUC) is calculated using the ROC curve with TPR as the vertical axis and FPR as the horizontal axis. The detection accuracy (AUC) values calculated by the two methods are shown in the following table.

[0068] Table 1 Comparison Table of Object Detection Accuracy between the Method of the Present Invention and the Prior Art Method

[0069] Method type Detection accuracy (AUC) Existing technology 88.23％ The present invention 97.67％

[0070] As can be seen from Table 1, compared with the prior art hyperspectral image object detection method based on self-supervised contrast learning, the detection accuracy of the detection results obtained by using the method of the present invention is significantly improved.

[0071] In summary, the present invention extracts spectral vector features in the hyperspectral image to be detected by using graph convolution, reduces the dimensionality of the spectral vector features by using feature mapping, and judges the dimensionality-reduced spectral vector features by using a discriminator to obtain a prediction map for hyperspectral image object detection, making better use of spatial information, reducing redundant information at the same time, and improving the detection accuracy.

Claims

1. A hyperspectral target detection method based on graph convolutional neural network, characterized in that, It includes the following steps: (1) Obtain a training sample set and a test sample set: (1a) Obtain a hyperspectral image I of dimension M×N×L that contains M×N spectral vectors, with each spectral vector containing L bands, the true label corresponding to each spectral vector, and a true spectral vector d that is similar to the spectral vectors of the target to be detected in the hyperspectral image I. Then, preprocess the hyperspectral image I using the true spectral vector d to obtain a spectral vector matrix X of dimension (M×N)×L that contains M×N spectral vectors and a pseudo-label matrix y of dimension M×N l , where M≥100, N≥100, and L≥200; (1b) C spectral vectors randomly selected from the two-dimensional matrix X and the pseudo-labels corresponding to each spectral vector in y l constitute a training sample set, and the M×N spectral vectors in the two-dimensional matrix X and their corresponding true labels constitute a test sample set, where 100 ≤ C < M×N; (2) Construct a graph convolutional neural network H: Construct a graph convolutional neural network H including a graph convolutional layer, a feature extraction sub-network W, and a discriminator connected in sequence; among them, the feature extraction sub-network W includes a plurality of sequentially connected composite layers and an activation layer, and the composite layer includes a feature mapping layer and a normalization layer connected in sequence; (3) Iteratively train the graph convolutional neural network H: (3a) Initialize the number of iterations as t, the maximum number of iterations as T, where T ≥ 200, and the weight of the current graph convolutional neural network H t is ω t , and set t = 1; (3b) Use the training sample set as the input of the graph convolutional neural network H. The graph convolutional layer performs graph embedding representation on each training sample to obtain a feature matrix G of dimension C×L containing the features of each sample; the feature extraction sub-network W performs feature mapping on each sample feature in the feature matrix G to obtain a detection probability matrix Y containing the detection probabilities of each training sample, and the discriminator makes a decision on each detection probability in the matrix Y to obtain C detection results; (3c) The cross-entropy loss function is adopted to calculate the loss value Loss of W by using the detection probability corresponding to each training sample in the detection probability matrix Y and the pseudo-label corresponding to each training sample in the pseudo-label matrix y l ; and the weights ω of the feature extraction sub-network W are updated by the loss value Loss t to obtain the graph convolutional neural network H after this iteration t ; t t t t ;​​​ (3d) Determine whether t = T holds. If so, obtain the trained graph convolutional neural network H * , otherwise, set t = t + 1, H t = H, and execute step (3b); (4) Obtain the hyperspectral target detection result: (4a) Use the test sample set as the input of the trained graph convolutional neural network H * to perform forward propagation and obtain M×N detection results; (4b) For each detection result, using the row where its corresponding spectral vector is located in the hyperspectral image I to be detected as the abscissa and the column where it is located as the ordinate, a prediction map Y for target detection with dimensions M×N is obtained out 。 2. The hyperspectral target detection method based on graph convolutional neural network according to claim 1, characterized in that The preprocessing of the hyperspectral image I using the true spectral vector d in step (1a) is realized as follows: (1a1) Construct a spectral vector matrix X of dimension (M×N)×L with the M×N spectral vectors contained in the hyperspectral image I as rows and the L bands contained in each spectral vector as columns; (1a2) Adopt a hierarchical constrained energy minimization algorithm, linearly filter the spectral vector matrix X through multiple FIR linear filters, and after normalization, obtain an energy map containing M×N spectral vector energy values. Then, change the energy value of each spectral vector greater than the pre-set threshold β to 1, and change the energy values of the remaining spectral vectors to 0, resulting in a pseudo-label matrix y with dimensions M×N l , where 0 < β < 1.

3. The hyperspectral target detection method based on graph convolutional neural network according to claim 1, wherein For the feature extraction sub-network W described in step (2), the number of composite layers it contains is 2, and the weight dimensions of the feature mapping layers in the first and second composite layers are L×300 and 300×2 respectively.

4. The hyperspectral target detection method based on graph convolutional neural network according to claim 1, characterized in that For the detection result matrix R described in step (3b), the obtaining process includes the following steps: (3b1) Among the Q training samples with the smallest Euclidean distance from each training sample calculated by the graph convolutional layer, the adjacency relationship S between each training sample and its neighbor samples c,q : where d c,q represents the Euclidean distance between the c-th training sample and the q-th neighbor sample, e represents the natural constant, and 1 < Q < C; (3b2) The graph convolutional layer constructs a training sample matrix C' of dimension C×L, with each spectral vector contained in C training samples as rows and L bands of each spectral vector as columns, and uses the adjacency relationship S between each training sample and its neighbor samples f,g to construct an adjacency matrix S' of dimension C×C, with the row where the corresponding training sample is located in the training sample matrix C' as the abscissa and the row where the corresponding neighbor sample is located in the training sample matrix C' as the ordinate; (3b3) Symmetrically normalize the adjacency matrix S' to obtain the Laplacian matrix S sym , and use the Laplacian matrix S sym to perform graph embedding representation on the training sample matrix C', obtaining a feature matrix G of dimension C×L containing the features of each sample, where G = S sym C'; (3b4) The feature extraction sub-network W performs feature mapping on each sample feature after graph embedding representation to obtain a detection probability matrix Y of dimension C×2 containing the detection probabilities of each training sample: Y = h(G×ω + b) where h represents the activation function ReLU, ω represents the weight of the feature extraction sub-network W, and b represents the bias of the feature extraction sub-network W; (3b5) The discriminator takes the maximum value of each row of the detection probability matrix Y. If the maximum value is in the first column, it means that the training sample corresponding to this row is a target vector and is marked as 1, otherwise it is a background vector and is marked as 0, to obtain C detection results.

5. The hyperspectral target detection method based on graph convolutional neural network according to claim 1, characterized in that The calculation of W described in step (3c) t The loss value Loss t , and for the feature extraction sub-network W t The weight ω t Is updated, and the calculation and update formulas are respectively:[[]]END]] Among them represents the cross-entropy loss function of the z-th composite layer at the t-th iteration, represents the detection probability output by the z-th composite layer for each training sample at the t-th iteration, y lc represents the pseudo-label corresponding to each training sample, η represents the preset gradient descent parameter, ω t+1 represents ω t the update result of, represents the partial derivative operation.

Images