A hyperspectral change detection method across tasks meta-learning

Abstract

The application provides a hyperspectral change detection method based on meta-learning across tasks, and relates to the technical field of remote sensing image processing, wherein the correlation between multiple binary classification tasks is learned in a meta-learning training stage, so that a binary classification network based on a double-branch graph enhancement attention module has better ability to process new tasks, general meta-knowledge is obtained, then a unified change detection framework is constructed by using a change detection network based on a three-branch graph enhancement attention module, a scoring strategy, a cross-entropy loss function and a cosine loss function, the general meta-knowledge is transferred to a hyperspectral change detection task under the condition of limited training samples, and efficient binary change detection or multi-class change detection is realized through a meta-learning test stage. The application has the beneficial effects that the spatial-spectral features of the hyperspectral image are fully utilized, the meta-knowledge is effectively transferred to realize the hyperspectral change detection under the condition of limited samples, and the change detection precision and generalization are improved.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, and in particular relates to a cross-task meta-learning hyperspectral change detection method. Background Technology

[0002] Change detection is one of the fundamental tasks in understanding remote sensing images, aiming to acquire information on changes in ground features within multi-temporal remote sensing images. Hyperspectral images, with their high spectral resolution, can comprehensively reflect the spectral composition of different objects, providing the possibility of capturing subtle changes in ground features and distinguishing multiple change categories in multi-temporal hyperspectral images. Therefore, fully extracting the spatial-spectral features of hyperspectral images helps to obtain more accurate change detection results. In recent years, deep learning methods have experienced rapid development and widespread application in the field of hyperspectral image processing. Many hyperspectral change detection methods utilize deep learning networks to extract deep features from hyperspectral images, improving the accuracy of change detection.

[0003] Existing deep learning-based hyperspectral change detection methods train a specific model for each change detection dataset by directly learning the mapping between input features and output predictions. This approach is highly inefficient, and due to the differences in the distribution and categories of different data, the model's ability to generalize to new datasets and categories is poor. Furthermore, deep learning-based hyperspectral change detection methods rely on a large number of samples for training; insufficient training samples can lead to overfitting, affecting change detection accuracy, while labeling a large number of samples is extremely time-consuming and difficult. Therefore, we designed an effective and practical framework that fully utilizes the spatial-spectral features of hyperspectral images and conducts in-depth research on the design of a high-generalization and efficient deep learning model under limited sample conditions. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an efficient and highly generalizable cross-task meta-learning hyperspectral change detection method for handling binary and multi-class change detection tasks. This invention utilizes multiple binary classification tasks to optimize the model itself to learn general meta-knowledge, enabling the model to quickly adapt to change detection tasks with only a small number of samples for fine-tuning. The hyperspectral image is a data cube, which can be represented by a tensor, where the vector at each point in the tensor represents the pixel radiance value corresponding to each band. The method mainly includes the following steps:

[0005] S1: Obtain the meta-training hyperspectral remote sensing dataset Heyuan Test Hyperspectral Remote Sensing Dataset The meta-training hyperspectral remote sensing dataset contains T land cover categories, and the meta-test hyperspectral remote sensing dataset contains temporal-hyperspectral images of the same region. and temporal hyperspectral images Where h, w, b represent the length, width, and height of the meta-training hyperspectral data, respectively, and H, W, B represent the length, width, and height of the meta-test hyperspectral data, respectively.

[0006] S2: Based on the meta-trained hyperspectral remote sensing dataset HSI tra The label information is used to construct T binary classification tasks {τ1,τ2,…,τ i ,…τ T Each task contains multiple sample pairs consisting of positive and negative examples. Then, a portion of the sample pairs are randomly selected as the support dataset, and the remaining sample pairs are used as the query dataset.

[0007] S3: For the T binary classification tasks set in step S2, a binary classification network model based on bi-branch graph enhanced attention is constructed using a graph self-attention module, a multi-scale channel attention module, a fully connected layer, and a Softmax layer. This model predicts the classification result of the supporting dataset in each task and calculates the cross-entropy loss of the query dataset in each task. Finally, the average loss of all tasks is minimized to constrain the training process of the binary classification network model, obtain the optimal binary classification network model, and process the binary classification tasks based on the optimal binary classification network model to obtain general meta-knowledge.

[0008] S4: Transfer the general meta-knowledge from step S3 to the change detection task. The change detection network based on three-branch graph enhanced attention is composed of multiple graph self-attention modules with shared parameters, multi-scale channel attention modules, fully connected layers and Softmax layers.

[0009] S5: Using a small number of samples, and combining the overall loss function composed of the cross-entropy loss function and the cosine loss function, fine-tune the change detection network obtained in step S4. After multiple iterations, obtain the final change detection network and test it on the meta-test hyperspectral dataset HSI. tes Change detection is performed; for type II change detection tasks, a change detection network is used to obtain type II change detection results; for multi-type change detection tasks, a scoring strategy is used on the basis of the change detection network to obtain multi-type change detection results.

[0010] A cross-task meta-learning hyperspectral change detection device includes: a processor and a storage device; the processor loads and executes instructions and data in the storage device to implement the cross-task meta-learning hyperspectral change detection method.

[0011] The beneficial effects of the technical solution provided by this invention are as follows: This invention processes multiple binary classification tasks through a meta-learning training phase, learning the correlation between multiple tasks, enabling the binary classification network based on bi-branch graph enhanced attention to better handle new tasks, obtaining general meta-knowledge, and transferring the trained binary classification network to construct a change detection network based on tri-branch graph enhanced attention. Combining a scoring strategy, cross-entropy loss function, and cosine loss function, a unified change detection framework is constructed. Under limited training sample conditions, the general meta-knowledge is transferred to the hyperspectral change detection task, achieving efficient binary or multi-class change detection through the meta-learning testing phase. Ultimately, it can fully utilize the spatial-spectral features of hyperspectral images, possessing high model generalization ability and significantly surpassing current deep learning models in inference speed, thus better completing the hyperspectral image change detection task. Based on meta-learning theory, it learns general meta-knowledge from multiple tasks and effectively transfers meta-knowledge to achieve hyperspectral change detection under limited sample conditions, improving change detection accuracy and generalization. Attached Figure Description

[0012] Figure 1 This is a flowchart of a cross-task meta-learning hyperspectral change detection method according to an embodiment of the present invention.

[0013] Figure 2 This is a schematic diagram of the hardware device working in an embodiment of the present invention. Detailed Implementation

[0014] The following is combined with Figure 1 The specific embodiments of the present invention will be further described in detail below.

[0015] The key inventive point of this invention is the proposal of a cross-task meta-learning hyperspectral change detection framework. This method first constructs multiple binary classification tasks using a meta-trained hyperspectral remote sensing dataset. Through a meta-learning training phase, the parameters of a binary classification network based on bi-branch graph enhanced attention are updated to obtain general meta-knowledge. This meta-knowledge is then transferred to the change detection task, where a change detection network based on tri-branch graph enhanced attention is used for feature extraction. Fine-tuning the change detection network with a small number of samples achieves efficient and highly generalized change detection.

[0016] This invention is specifically implemented using the Python language and the classic deep learning framework PyTORCH. The Python remote sensing image read / write functions and graph structure processing functions form the basis of the implementation. The data processing libraries NUMPY, SCIPY, and SPECTRAL are called, and the filename of the remote sensing image to be read is input. The remote sensing image is then read into a tensor, where each element represents the pixel radiance value corresponding to each band. The PyTORCH GEOMETRIC data processing library is called, and the adjacency matrix and feature matrix of the graph structure are input. The graph structure is then embedded into a tensor, where each element represents the feature representation of all nodes. The Python remote sensing image read / write functions and graph structure processing functions are well-known technologies in this field.

[0017] like Figure 1 As shown, a cross-task meta-learning hyperspectral change detection method includes the following steps:

[0018] S1: Obtain the meta-training hyperspectral remote sensing dataset Heyuan Test Hyperspectral Remote Sensing Dataset The meta-training hyperspectral remote sensing dataset contains T land cover categories, and the meta-test hyperspectral remote sensing dataset contains temporal-hyperspectral images of the same region. and temporal hyperspectral images h, w, b represent the length, width, and height of the meta-training hyperspectral data, respectively; H, W, B represent the length, width, and height of the meta-test hyperspectral data, respectively.

[0019] S2: Based on the meta-trained hyperspectral remote sensing dataset HSI tra The label information is used to construct T binary classification tasks {τ1,τ2,…,τ i ,…τ T Each task contains multiple sample pairs consisting of positive and negative examples. Then, a portion of the sample pairs are randomly selected as the support dataset, and the remaining sample pairs are used as the query dataset.

[0020] S3: For the T binary classification tasks set in step S2, a binary classification network model based on bi-branch graph enhanced attention is constructed using a graph self-attention module, a multi-scale channel attention module, a fully connected layer, and a Softmax layer. This model predicts the classification result of the supporting dataset in each task and calculates the cross-entropy loss of the query dataset in each task. Finally, the average loss of all tasks is minimized to constrain the training process of the binary classification network model, thereby obtaining the optimal binary classification network model. This enables the network to learn to handle binary classification tasks. By processing binary classification tasks according to the optimal binary classification network model, general meta-knowledge is obtained.

[0021] S4: The meta-knowledge common in step S3 is transferred to the change detection task. The change detection network based on three-branch graph enhanced attention is composed of two graph self-attention modules with shared parameters, a multi-scale channel attention module, a fully connected layer and a Softmax layer.

[0022] S5: Using a small number of samples, and combining the overall loss function composed of the cross-entropy loss function and the cosine loss function, fine-tune the change detection network obtained in step S4. After multiple iterations, obtain the final change detection network and test it on the meta-test hyperspectral dataset HSI. tes Change detection is performed; for type II change detection tasks, a change detection network is used to obtain type II change detection results; for multi-type change detection tasks, a scoring strategy is used on the basis of the change detection network to obtain multi-type change detection results.

[0023] Steps S2 and S3 above belong to the meta-learning training phase, while steps S4 and S5 belong to the meta-learning testing phase.

[0024] The specific operations involved in the above steps are as follows:

[0025] (1) Using meta-training of the hyperspectral remote sensing dataset HSI tra Constructing multiple binary classification tasks, querying datasets, and supporting datasets are key steps in acquiring general meta-knowledge, as detailed below:

[0026] Based on the label information of the meta-training hyperspectral remote sensing dataset, samples with label value i (i∈{1,2,…,T}) are designated as positive examples, and samples with label value j (j≠i) are designated as negative examples. A positive example and a negative example together constitute a sample pair. For all sample pairs with positive example label value i, the task of distinguishing between positive and negative examples constitutes the i-th binary classification task τ. i From all sample pairs with positive label value i, a subset of sample pairs are randomly selected to form task τ. i Supported datasets The remaining pairs of positive samples with label value i constitute task τ. i query dataset

[0027] Each task's dataset consists of a supporting dataset and a query dataset, such as task τ. i The dataset is composed of The dataset for all tasks consists of... express.

[0028] (2) Design a binary classification network model based on dual-branch graph-enhanced attention, making full use of the spatial-spectral features of the meta-training data. The specific implementation process is as follows:

[0029] 1) Extracting superpixel-level features of ground objects using a graph self-attention module. The graph self-attention module consists of two graph self-attention blocks, each of which contains multi-head self-attention, residual connections, a LeakyReLU layer, and a Dropout layer. The specific implementation process of the graph self-attention module is as follows:

[0030] ① Meta-training of the hyperspectral remote sensing dataset HSI tra The hyperspectral image X is segmented into superpixels. Based on the segmentation results, the connectivity relationships between superpixel nodes are obtained, forming an adjacency matrix A. The average features of all pixels within a superpixel constitute the node feature matrix H. Principal component analysis (PCA) is then performed on the node features for dimensionality reduction, selecting only the first principal component as the representative band. A superpixel map G = (A, H) is constructed for the hyperspectral image X:

[0031]

[0032] Among them, h i h represents the i-th element of the graph node feature matrix H, i.e., the feature vector of the i-th superpixel. j K represents the feature vector of the j-th node element of the graph node feature matrix H, i.e., the j-th superpixel. i and x i,j Let A represent the number of pixels in the i-th superpixel and the original spectral vector of the j-th pixel in the i-th superpixel, respectively. i,j This represents the adjacency matrix value between the i-th node and the j-th node, where σ represents a set constant, and is set to 0.1 based on experience.

[0033] ② For each edge pointing from node j to node i, calculate the self-attention coefficient using multi-head dot product self-attention:

[0034]

[0035] in, The self-attention coefficient is represented by d, which is the size of the hidden layer when calculating the multi-head dot product self-attention each time. α ij Let represent the weight coefficient of each node j connected to node i with respect to node i, c is the number of times the multi-head dot product self-attention calculation is performed, l represents the l-th graph self-attention block, l∈{1,2,…,L}, and L represents the number of graph self-attention blocks; This represents the query vector when node i performs the c-th multi-head dot product self-attention computation through the l-th graph self-attention block. This represents the transpose of the key vector when node j performs the c-th multi-head dot product self-attention calculation through the l-th graph self-attention block.

[0036] ③ Multiply the self-attention coefficients and the projected features of node i, use a residual connection, and add the outputs of the previous and next layers to obtain the dynamically weighted features.

[0037]

[0038] in, and Let represent the first and second learnable weight coefficients respectively when the l-th graph self-attention block performs the c-th dot product self-attention calculation, where c is the number of times the multi-head dot product self-attention is calculated, and C represents the total number of times the multi-head dot product self-attention is calculated. This represents the i-th node element of the graph node feature matrix H passing through the l-th graph self-attention block. This represents the value vector of the multi-head dot product self-attention. This represents the value vector of node j when it performs the c-th multi-head dot product self-attention calculation through the l-th graph self-attention block.

[0039] ④ Using LeakyReLU and Dropout layers, the features are normalized to obtain superpixel-level feature extraction results. And reduce overfitting issues:

[0040]

[0041] in, H represents the updated superpixel map feature matrix. (L+1) The i-th element, σ(·) is the Leakyrelu layer, and M is the number of superpixels.

[0042] 2) Pixel-level spatial-spectral feature extraction is performed using a multi-scale channel attention module to obtain multi-scale pixel-level features. This multi-scale channel attention module consists of two 1×1 convolutional layers, one 3×3 convolutional layer, one 5×5 convolutional layer, and one channel attention module. The specific implementation process of the multi-scale channel attention module is as follows:

[0043] ① Use a 1×1 convolutional layer to reduce the meta-training time of the hyperspectral remote sensing dataset HSI. tra The channel dimension of the hyperspectral image X is reduced, decreasing the number of parameters and computational cost of the binary classification network model, while unifying the number of bands for different data.

[0044] ② A channel attention mechanism module is used to dynamically fuse the output features of 1×1, 3×3, and 5×5 convolutional layers to obtain multi-scale pixel-level features Pf, fully extracting pixel-level spatial-spectral features:

[0045]

[0046] Where X is a hyperspectral image in the meta-training hyperspectral remote sensing dataset, CAM(·) represents the computation of channel attention update features, and W k These are learnable weight parameters used to dynamically fuse features obtained from k×k convolutional layers. This represents performing a k×k convolution operation, where k is the kernel size.

[0047] 3) By fusing multi-scale pixel-level features and superpixel-level features, pixel-level spatial-spectral features are obtained to solve the problem of pixel information loss within superpixels during superpixel-level feature extraction. The specific implementation process of obtaining pixel-level spatial-spectral features by fully utilizing the pixel-level spatial-spectral features of hyperspectral images is as follows:

[0048] ① First, transform the superpixel-level features to pixel space:

[0049]

[0050] in, It is the pixel-to-superpixel correlation matrix, Q i,j =1 indicates that the i-th pixel belongs to the j-th superpixel, Q i,j =0 indicates that the i-th pixel does not belong to the j-th superpixel; N is the number of pixels, M is the number of superpixels, H is the number of superpixels. (L+1) Represents the superpixel map feature matrix. This is a superpixel-level feature transformed into pixel space.

[0051] ②Then, the multi-scale pixel-level features Pf and the superpixel-level features transformed to pixel space are... The features are then stitched together and fused through two fully connected layers to obtain pixel-level spatial-spectral features P enhanced with graph structure knowledge.

[0052]

[0053] In this context, FC(·) represents two fully connected layers, and Concat(·) represents a concatenation operation.

[0054] 4) Use a classifier consisting of a fully connected layer and a Softmax layer to perform binary classification.

[0055] In summary, for the meta-training hyperspectral remote sensing dataset HSI tra Using the hyperspectral image X, a binary classification network model based on bi-branch graph-enhanced attention is constructed:

[0056]

[0057] in, For the prediction results of the binary classification network model, F φ () represents a binary classification network model with parameter φ, which consists of a feature extractor and a classifier that extracts spatial-spectral features. G is a superpixel image and X is a hyperspectral image.

[0058] (3) The specific implementation process of training a binary classification network using multiple binary classification tasks is as follows:

[0059] First, each task involves multiple in-task network training iterations to predict the classes supporting the dataset and update the randomly initialized parameters φ of the binary classification network model.

[0060]

[0061] in For predicting binary classification tasks τ i Supported datasets The result is the cross-entropy loss. This represents the parameters updated after training in the k-th task. ε is the learning rate for updating parameters through in-task training for each task, and the superscript tra indicates the meta-learning training phase. Indicates parameters Regarding the loss The partial differential of .

[0062] Then, the query dataset is used to verify the binary classification network model's ability to handle binary classification tasks. The cross-entropy loss is calculated, and the final binary classification network model parameters are obtained by minimizing the average loss across all tasks.

[0063]

[0064] Where, φ i Represents a binary classification task τ i Parameters of the binary classification network model;

[0065] Therefore, based on the supporting datasets and query datasets for all tasks, the model parameters are updated through the above process to obtain model parameters that can quickly adapt to new tasks. That is, general meta-knowledge.

[0066] (4) Transfer meta-knowledge to construct a change detection network based on a three-branch graph-enhanced attention:

[0067] 1) For the dual-temporal hyperspectral change detection dataset, the single graph self-attention module is extended into a two-branch module. The two branches share the parameters initialized during the meta-learning training phase. The three-branch graph enhancement attention module is composed of two graph self-attention modules, a multi-scale channel attention module, and two fully connected layers.

[0068]

[0069] Among them, f θ * Represents the freeze parameter θ * The graph self-attention module, where i∈{1,2} represents branch one or branch two. This represents the superpixel-level output features of branch i of the image obtained using the graph self-attention module with frozen parameters. Z1 and Z2 represent the superpixel-level features of the time-varying fine-tuning task, respectively, where Z1 and Z2 are the superpixel-level features of the two temporal hyperspectral images transformed to pixel space. This represents the superpixel-level output features of a branch image obtained using a graph self-attention module with frozen parameters. This represents the superpixel-level output features of the branch-two image obtained using the graph self-attention module with frozen parameters; and These are the pixel-superpixel correlation matrices corresponding to the two temporal hyperspectral images, where N is the number of pixels and N1 = N2, and M is the number of superpixels.

[0070] 2) A change detection network based on three-branch graph-enhanced attention consists of a three-branch graph-enhanced attention module, a fully connected layer, and a Softmax layer.

[0071] (5) The specific process of fine-tuning the change detection network is as follows:

[0072] The fine-tuning strategy consists of three parts: freezing some layer parameters, randomly initializing and retraining some layer parameters, and continuing to train some layer parameters. To improve the inference speed of the binary classification network model, when transferring the general meta-knowledge acquired during the meta-learning training phase to the change detection task, the parameters of the graph-attention module are frozen, and a dual-branch module with shared parameters is used to extract superpixel-level features from the dual-temporal hyperspectral images. Since there may be differences in spectral dimensions between different hyperspectral images, the parameters of the first 1×1 convolutional layer in the pixel-level feature extraction branch of the three-branch graph-enhanced attention module are retrained through random initialization. The remaining network parameters are further trained based on the results of meta-training.

[0073] 1) Meta-test hyperspectral remote sensing dataset Superpixel segmentation is performed, and the adjacency matrix and feature matrix are calculated to obtain superpixel images G1 = (A1, H1) and G2 = (A2, H2). A1 and H1 represent the adjacency matrix and feature matrix obtained after segmenting the hyperspectral image T1 of time phase one, respectively, and A2 and H2 represent the adjacency matrix and feature matrix obtained after segmenting the hyperspectral image T2 of time phase two, respectively. The hyperspectral image Xcd = T1 - T2 is obtained by subtracting the hyperspectral image of time phase one from the hyperspectral image of time phase two. These represent the hyperspectral image of phase one and the hyperspectral image of phase two, respectively.

[0074] 2) Use the change detection network obtained in step S4 to predict the results:

[0075]

[0076] in These are the prediction results from the change detection network. It has meta-training initialization parameters And freeze the parameter θ * Change detection network.

[0077] 3) Calculate the overall loss value consisting of cross-entropy loss and cosine loss using a very small number of samples, perform gradient backpropagation, and iterate a small number of times to obtain the fine-tuned change detection network.

[0078] (6) The specific process of using a scoring strategy for multi-class change detection is as follows:

[0079] If the hyperspectral change detection data contains δ change classes and one unchanged class:

[0080] 1) Using a one-to-many classification strategy, construct δ binary change detection tasks. Set the target class of the i-th binary change detection task to the i-th change class. The purpose of this task is to identify the i-th change class i∈{1,2,…,δ} from other classes (including the unchanged class).

[0081] 2) Based on the fine-tuned change detection network, the prediction results of each binary change detection task are recorded. Then, a scoring strategy is used to determine the category with the most frequent classification for each pixel. The category with the most frequent classification for each pixel is determined as the final predicted change category. If a pixel does not belong to any change category, it is determined as the unchanged category.

[0082]

[0083] in For the final prediction result of the change detection network, This represents the prediction result of the i-th binary change detection task. Record the number of times each pixel is classified and its category, with the result being 0 or 1, where N is the number of pixels.

[0084] (7) Constrain the change detection network using an overall loss function that combines the cross-entropy loss function and the cosine loss function:

[0085] The formula for calculating the cross-entropy loss function is:

[0086]

[0087] in, Y represents the cross-entropy loss function, S represents the number of samples, and Y represents the number of samples. s This represents the true label of the s-th sample. This represents the probability that the change detection network predicts the s-th sample as belonging to the change category.

[0088] The cosine similarity between superpixel-level features of the two-temporal hyperspectral images is calculated to represent the overall difference between these features. A larger value indicates a less significant difference. By reducing the cosine loss value, the feature differences between the two-temporal hyperspectral images are amplified, allowing the final extracted superpixel-level features to contain more variation information. The formula for calculating the cosine loss function is as follows:

[0089]

[0090] Among them, Loss cos (Z1, Z2) represents the cosine loss function, N represents the number of pixels, and Z1 and Z2 represent the superpixel-level features of the two temporal hyperspectral images converted to pixel space, respectively. i and (Z2) i Let i represent the i-th element of the superpixel-level features of the hyperspectral images of phase one and phase two, respectively.

[0091] The formula for calculating the overall loss function is:

[0092]

[0093] Where Loss represents the overall loss function, and λ is the weighting coefficient for measuring the cosine loss. Y represents the final prediction result of the change detection network, and Y represents the true label of the sample.

[0094] The following comparative experiments will verify the beneficial effects of the present invention.

[0095] This embodiment uses the Indian Pine Tree and Santa Barbara datasets as the meta-training hyperspectral remote sensing datasets, and the Chinese Farm, American Farm, and Benton County hyperspectral datasets as the meta-test hyperspectral remote sensing datasets to validate the model's effectiveness. The Indian Pine Tree dataset is a commonly used hyperspectral classification dataset, acquired by an Airborne Visible Infrared Imaging Spectroradiometer (AVIRIS) sensor, with a wavelength range of 400 to 2500 nm. After removing the water vapor absorption band, 200 bands remain. This dataset is 145×145 pixels and is located in Indiana, USA. Due to the diverse types of ground objects in this image (a total of 16 classes), it was used as the meta-training hyperspectral remote sensing dataset. Furthermore, the Santa Barbara dataset is a hyperspectral change detection dataset, where Phase 1 and Phase 2 images were acquired by an Airborne Visible Infrared Imaging Spectroradiometer (AVIRIS) sensor in 2013 and 2014, respectively, and contain 224 bands. The first dataset, measuring 984×740, is located near Santa Barbara, California, USA. Because it contains one change category and one unchanged category, it provides binary change information. Therefore, it was used as a meta-training hyperspectral remote sensing dataset in addition to the Indian pine dataset, allowing the meta-training model to fully learn how to handle different binary classification tasks. The second dataset, representing temporal phases 1 and 2, was acquired by the EO-1 Hyperion hyperspectral sensor on May 3, 2006, and April 23, 2007, respectively, with wavelengths ranging from 400 to 2500 nm. After removing water vapor and noise, 155 bands remained. The third dataset, measuring 420×140, is located in a farmland near Yancheng City, Jiangsu Province, China, and includes three change categories and one unchanged category. The fourth dataset, representing temporal phases 1 and 2, was acquired by the Hyperion hyperspectral sensor on May 1, 2004, and May 8, 2007, respectively, retaining 156 bands. The first dataset, measuring 307×241 pixels, is located in Hermiston, Oregon, USA, and includes changes among six crops and one unchanged category. The second dataset, from Benton County, contains Phase 1 and Phase 2 images acquired by the Hyperion hyperspectral sensor on May 1, 2004, and May 8, 2007, respectively, retaining 159 bands. This third dataset, measuring 180×225 pixels, is also located in Benton County, Oregon, USA, and includes six change categories and one unchanged category. These three datasets are multi-class change detection datasets, but they can also be used as binary change detection datasets to validate the model's effectiveness on different change detection tasks.

[0096] This invention compares the method of the present invention with the classic multi-level encoder-decoder attention network (method 1), multi-scale differential variation feature fusion network (method 2), bi-branch differential enhancement graph convolutional network (method 3), and multi-order graph convolutional network (method 4) to demonstrate the effectiveness of the present invention.

[0097] Change detection evaluation indicators: Overall Accuracy (OA), Kappa coefficient, and running time.

[0098] (1) Overall accuracy:

[0099] Overall Accuracy (OA) is an evaluation metric used to assess change detection problems. Higher overall accuracy indicates higher detection accuracy. The formula for calculating overall accuracy (OA) is:

[0100]

[0101] Wherein, TP represents true positives, which is the number of samples that are actually positive and correctly predicted as positive; FN represents false negatives, which is the number of samples that are actually positive but incorrectly predicted as negative; FP represents false positives, which is the number of samples that are actually negative but incorrectly predicted as positive; and TN represents true negatives, which is the number of samples that are actually negative and correctly predicted as negative.

[0102] (2) Kappa coefficient:

[0103] The Kappa coefficient is an authoritative evaluation metric for assessing change detection problems. A higher Kappa coefficient indicates higher accuracy. The formula for calculating the Kappa coefficient is:

[0104]

[0105] in,

[0106] (3) Running time:

[0107] Model runtime is a key metric for evaluating its computational efficiency and speed. High runtime is especially important in real-time applications. The shorter the runtime, the more efficient the model.

[0108] In this experiment, the change detection capability and computational efficiency of methods 1-4 and the method of the present invention were evaluated using OA, Kappa coefficient values ​​and running time. The OA and Kappa coefficient values ​​are shown in Table 1, and the running time is shown in Table 2. To avoid randomness, all algorithms were run 5 times consecutively, and the median value was taken as the final result.

[0109] Table 1 Comparative Experiment Results

[0110]

[0111] As shown in Table 1, the method of this invention achieves higher OA and Kappa values ​​in both binary change detection and multi-class change detection tasks. Therefore, it can be concluded that the method of this invention has higher change detection accuracy compared to recent deep learning change detection methods. This invention fully considers the relationship between the spatial-spectral features and categories of hyperspectral images, thus improving the performance of change detection.

[0112] Table 2 Comparison of Experimental Results 2 (Time unit: seconds)

[0113]

[0114]

[0115] Table 2 shows the training and testing times of methods 1-4 and the method of the present invention on three datasets, with the fine-tuning time of the meta-learning testing phase provided by the method of the present invention. The method of the present invention achieves the fastest overall runtime, especially in terms of training time. Compared with recent deep learning methods, the method of the present invention has higher runtime efficiency.

[0116] Please see Figure 2 , Figure 2 This is a schematic diagram of the hardware device in operation according to an embodiment of the present invention. The hardware device specifically includes: a cross-task meta-learning hyperspectral change detection device 201, a processor 202, and a storage device 203.

[0117] A cross-task meta-learning hyperspectral change detection device 201: The cross-task meta-learning hyperspectral change detection device 201 implements the cross-task meta-learning hyperspectral change detection method.

[0118] Processor 202: The processor 202 loads and executes the instructions and data in the storage device 203 to implement the cross-task meta-learning hyperspectral change detection method.

[0119] Storage device 203: The storage device 203 stores instructions and data; the storage device 203 is used to implement the cross-task meta-learning hyperspectral change detection method.

[0120] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cross-task meta-learning hyperspectral change detection method, characterized in that: Includes the following steps: S1: Obtain the meta-training hyperspectral remote sensing dataset Heyuan Test Hyperspectral Remote Sensing Dataset The meta-training hyperspectral remote sensing dataset contains Each land cover category, the meta-test hyperspectral remote sensing dataset contains temporal-hyperspectral images of the same region. and temporal hyperspectral images ,in, , , These represent the length, width, and height of the meta-training hyperspectral data, respectively. , , These represent the length, width, and height of the meta-tested hyperspectral data, respectively. S2: Based on meta-trained hyperspectral remote sensing dataset Tag information construction T A binary classification task Each task contains multiple sample pairs consisting of positive and negative examples. Then, a portion of the sample pairs are randomly selected as the support dataset, and the remaining sample pairs are used as the query dataset. S3: Settings for step S2 T For each binary classification task, a binary classification network model based on bi-branch graph enhanced attention is constructed using a graph self-attention module, a multi-scale channel attention module, a fully connected layer, and a Softmax layer. This model predicts the classification result of the supporting dataset in each task and calculates the cross-entropy loss of the query dataset in each task. Ultimately, the average loss of all tasks is minimized to constrain the training process of the binary classification network model, thereby obtaining the optimal binary classification network model. The optimal binary classification network model is then used to process binary classification tasks to obtain general meta-knowledge. In step S3, the specific implementation process of constructing the binary classification network model is as follows: 1) Extract superpixel-level features of ground objects using a graph self-attention module. The graph self-attention module consists of two graph self-attention blocks, each of which contains multi-head self-attention, residual connections, Leakyrelu layers, and Drupout layers. 2) Utilize a multi-scale channel attention module to obtain pixel-level multi-scale pixel-level features. This multi-scale channel attention module consists of two 1 1 convolutional layer, one 3 3 convolutional layers, one 5 It consists of 5 convolutional layers and one channel attention module; 3) By fusing multi-scale pixel-level features and superpixel-level features, pixel-level spatial-spectral features are obtained; 4) A classifier consisting of a fully connected layer and a softmax layer is used for binary classification; this is performed on a meta-trained hyperspectral remote sensing dataset. hyperspectral images in Construct a binary classification network model based on dual-branch graph-enhanced attention: in, The prediction results are from a binary classification network model. Indicates parameters A binary classification network model, It is a superpixel image. This is a hyperspectral image; The specific process of handling binary classification tasks using the optimal binary classification network model is as follows: First, each task involves multiple in-task network training iterations to predict the classes supporting the dataset and update the randomly initialized parameters of the binary classification network model. : in, For predicting binary classification tasks Supported datasets The result is the cross-entropy loss. Representing the Parameters updated after training within this task. Representing the Parameters updated after training within this task. , It is the learning rate used to update parameters for each task through in-task training, with the superscript indicating the learning rate. This indicates the meta-learning training phase. Indicates parameters Regarding the loss The partial derivative; Then, the query dataset is used to verify the binary classification network model's ability to handle binary classification tasks. The cross-entropy loss is calculated, and the final binary classification network model parameters are obtained by minimizing the average loss across all tasks. : in, Represents a binary classification task Parameters of the binary classification network model; Therefore, based on the support datasets and query datasets for all tasks, the binary classification network model parameters are updated through the above process to obtain model parameters that can quickly adapt to new tasks. That is, general meta-knowledge; S4: Transfer the general meta-knowledge from step S3 to the change detection task. The change detection network based on three-branch graph enhanced attention is composed of multiple graph self-attention modules with shared parameters, multi-scale channel attention modules, fully connected layers and Softmax layers. S5: Using a small number of samples, fine-tune the change detection network obtained in step S4 by combining the overall loss function composed of the cross-entropy loss function and the cosine loss function. After multiple iterations, the final change detection network is obtained and tested on the meta-test hyperspectral dataset. Change detection is performed; for type II change detection tasks, a change detection network is used to obtain type II change detection results; for multi-type change detection tasks, a scoring strategy is used on the basis of the change detection network to obtain multi-type change detection results.

2. The cross-task meta-learning hyperspectral change detection method as described in claim 1, characterized in that: In step S2, the hyperspectral remote sensing dataset is used for meta-training. Build The specific implementation process of each binary classification task, supporting dataset, and query dataset is as follows: Based on the label information of the meta-trained hyperspectral remote sensing dataset, the label value is... The sample is taken as a positive example, and the label value is... ,in A positive example is considered a negative example, and a positive example and a negative example together constitute a sample pair; for positive examples, the label value is... All sample pairs, the task of distinguishing positive and negative examples constitutes the first... i A binary classification task From the positive example label value Randomly select a portion of sample pairs from all sample pairs to form the task. Supported datasets The remaining positive examples have label values ​​of The sample pairs constitute the task query dataset ; Each task's dataset consists of a supporting dataset and a query dataset, i.e., the task... The dataset is composed of The dataset for all tasks consists of... express.

3. The cross-task meta-learning hyperspectral change detection method as described in claim 1, characterized in that: The specific implementation process of the graph attention module is as follows: ① Meta-training of hyperspectral remote sensing dataset hyperspectral images in Perform superpixel segmentation, and obtain the connection relationships between superpixel graph nodes based on the superpixel segmentation results to construct an adjacency matrix. A The average features of all pixels within a superpixel constitute the graph node feature matrix. H Then, PCA dimensionality reduction is performed on the features of the graph nodes, selecting only the first principal component as the representative band; for hyperspectral images Constructing superpixel maps : in, Represents the feature matrix of graph nodes The The element, i.e., the th element The feature vector of each superpixel Represents the feature matrix of graph nodes The The nth node element, i.e., the nth node element The feature vector of each superpixel and They represent the first The number of pixels contained in the first superpixel and the number of pixels in the second superpixel The first superpixel in the nth superpixel The original spectral vector of pixels, Indicates the first The node and the first The adjacency matrix values ​​between nodes This represents a set constant; ② For each slave node Pointing to node The self-attention coefficients are calculated using multi-head dot product self-attention on the edges: in, Represents the self-attention coefficient. It is the size of the hidden layer each time the multi-head dot product self-attention is calculated. Represent each node Connected nodes For nodes The weighting coefficients, This indicates the number of times the multi-head dot product self-attention calculation is performed. l Indicates the first l Each graph self-attention block , L Indicates the number of self-attention blocks in the graph; Represents a node Through the first l The first graph self-attention block performs the first... Sub-multi-head dot product self-attention calculation vector, Represents a node Through the first l The first graph self-attention block performs the first... Sub-multi-head dot product self-attention calculation Transpose of a vector; ③ Combine the self-attention coefficients and the projected nodes The features are multiplied together, and a residual connection is used to add the outputs of the previous and next layers to obtain the dynamically weighted features. : in, and They represent the first The first graph self-attention block performs the first... The first and second learnable weight coefficients during the computation of the dot product self-attention. This indicates the total number of times the multi-head dot product self-attention is calculated. Indicates passing through the first Graph node feature matrix of a graph self-attention block The One element, Represents a node Through the first The first graph self-attention block performs the first... Sub-multi-head dot product self-attention calculation vector; ④ Utilizing Leakyrelu layers and The layer normalizes the features to obtain superpixel-level feature extraction results. And reduce overfitting issues: in, Represents the updated superpixel map feature matrix The One element, For Leakyrelu layer, It is the number of superpixels; Multi-scale pixel-level features The specific implementation process is as follows: ①Use a 1 One convolutional layer to reduce the meta-training hyperspectral remote sensing dataset hyperspectral images in The channel dimension reduces the number of parameters and computational cost of binary classification network models, while unifying the number of bands for different data. ② Utilize a channel attention mechanism module to dynamically fuse 1 1 convolutional layer, 3 3 convolutional layers and 5 The output features of 5 convolutional layers yield multi-scale pixel-level features. : in, This represents hyperspectral images in the meta-trained hyperspectral remote sensing dataset. This indicates the computational channel attention update features. Indicates use for dynamic fusion Learnable weight parameters for features acquired by convolutional layers. Indicates to proceed Convolution operation, The kernel size; The specific implementation process for obtaining pixel-level spatial-spectral features is as follows: ① First, transform the superpixel-level features to pixel space: in, It is a pixel-to-superpixel correlation matrix. Indicates the first The pixel belongs to the first One superpixel, Indicates the first The pixel does not belong to the first One superpixel; It's the number of pixels. It's the number of superpixels. Represents the superpixel map feature matrix. For superpixel-level features converted to pixel space; ②Then, multi-scale pixel-level features and superpixel-level features transformed to pixel space The features are then stitched together and fused through two fully connected layers to obtain pixel-level spatial-spectral features enhanced with graph structure knowledge. : in, Represents two fully connected layers. This represents a splicing operation.

4. The cross-task meta-learning hyperspectral change detection method as described in claim 1, characterized in that: In step S4, the process of transferring meta-knowledge to construct a change detection network based on a three-branch graph with enhanced attention is as follows: 1) For the dual-temporal hyperspectral change detection dataset, the single graph self-attention module is extended into a two-branch module. The two branches share the parameters initialized during the meta-learning training phase. The three-branch graph enhancement attention module is composed of two graph self-attention modules, a multi-scale channel attention module, and two fully connected layers. in, Indicates the freeze parameter Graph self-attention module, Indicates branch one or branch two. This represents the branch obtained using the graph self-attention module with frozen parameters. Superpixel-level output features of the image This represents the superpixel-level features for the fine-tuning change detection task. and These represent the superpixel-level features of the two temporal hyperspectral images converted to pixel space, respectively. and These represent the pixel-superpixel correlation matrices corresponding to the two temporal hyperspectral images, respectively. This represents the superpixel-level output features of the branch-image obtained using the graph self-attention module with frozen parameters. This represents the superpixel-level output features of the branch-two image obtained using the graph self-attention module with frozen parameters; 2) A change detection network based on three-branch graph-enhanced attention consists of a three-branch graph-enhanced attention module, a fully connected layer, and a Softmax layer. .

5. The cross-task meta-learning hyperspectral change detection method as described in claim 1, characterized in that: In step S5, the specific process of fine-tuning is as follows: The fine-tuning strategy consists of three parts: freezing some layer parameters, randomly initializing and retraining some layer parameters, and continuing to train some layer parameters. To improve the inference speed of the binary classification network model, when transferring the general meta-knowledge acquired during the meta-learning training phase to the change detection task, the parameters of the graph-attention module are frozen, and a dual-branch module with shared parameters is used to extract superpixel-level features from the dual-temporal hyperspectral images. Since there may be differences in spectral dimensions between different hyperspectral images, the first branch of the pixel-level feature extraction branch in the three-branch graph-enhanced attention module is... The parameters of the first convolutional layer are retrained using random initialization, while the parameters of the remaining networks are trained based on the results of the meta-training. 1) Meta-test hyperspectral remote sensing dataset Superpixel segmentation is performed, and the adjacency matrix and feature matrix are calculated to obtain the superpixel map. and , and These represent temporal-hyperspectral images, respectively. The adjacency matrix and characteristic matrix obtained after segmentation and These represent the two hyperspectral images of different time phases. The adjacency matrix and feature matrix obtained after segmentation are used to subtract the hyperspectral image from the hyperspectral image of time phase one and the hyperspectral image of time phase two to obtain the hyperspectral image. ; 2) Use the change detection network obtained in step S4 to predict the results: in, This represents the prediction results of the change detection network. Indicates the presence of meta-training initialization parameters And freeze parameters Change detection network; 3) Calculate the overall loss value consisting of cross-entropy loss and cosine loss using a small number of samples, perform gradient backpropagation, and iterate a small number of times to obtain the fine-tuned change detection network.

6. The cross-task meta-learning hyperspectral change detection method as described in claim 1, characterized in that: In step S5, the specific process of using a scoring strategy to detect multiple types of changes is as follows: Hyperspectral change detection data includes One changed class and one unchanged class: 1) Using a one-to-many classification strategy, construct... The first binary change detection task will be the first... The target class of the binary change detection task is set to the first... If there is a variable class, then the purpose of this task is to identify the first variable class from the other classes. Changes ; 2) Based on the fine-tuned change detection network, the prediction results of each binary change detection task are recorded. Then, a scoring strategy is used to determine the category with the most frequent classification for each pixel. The category with the most frequent classification for each pixel is determined as the final predicted change category. If a pixel does not belong to any change category, it is determined as the unchanged category. in, For the final prediction result of the change detection network, Indicates the first Prediction results for a binary change detection task. Record the number of times each pixel is classified and its category, with the result being either 0 or 1. This represents the number of pixels.

7. The cross-task meta-learning hyperspectral change detection method as described in claim 1, characterized in that: In step S5, the cross-entropy loss function is: in, Represents the cross-entropy loss function. Indicates the number of samples. Indicates the first The true label of each sample The change detection network represents the first... The probability value of each sample being predicted as the change category; The cosine loss function is: in, Represents the cosine loss function. Indicates the number of pixels. and These represent the superpixel-level features of the two temporal hyperspectral images converted to pixel space, respectively. and The first and second superpixel-level features of the hyperspectral images of time phase one and time phase two are respectively represented by the first superpixel-level features. Each element.

8. The cross-task meta-learning hyperspectral change detection method as described in claim 1, characterized in that: In step S5, the overall loss function is: in, Represents the overall loss function. It is a weighting coefficient for measuring cosine loss. Represents the cross-entropy loss function. Represents the cosine loss function. and These represent the superpixel-level features of the two temporal hyperspectral images converted to pixel space, respectively. This represents the final prediction result of the change detection network. This represents the true label of the sample.

9. A cross-task meta-learning hyperspectral change detection device, characterized in that: include: A processor and a storage device; the processor loads and executes instructions and data in the storage device to implement the cross-task meta-learning hyperspectral change detection method according to any one of claims 1 to 8.

Images