A hyperspectral image change detection method, device and storage medium

Abstract

The present invention discloses a hyperspectral image change detection method, device and storage medium in the field of hyperspectral image technology. The method includes: extracting features from a hyperspectral image to obtain a spatial feature map and a spectral feature map, aggregating the spatial feature map and the spectral feature map to obtain a fused feature map; segmenting a dual-phase hyperspectral image to generate a superpixel label map, projecting the superpixel label map into the fused feature map to form a superpixel feature map, and constructing a topological map based on the superpixel feature map; inputting the node feature vector and the adjacency matrix in the topological map into a KAN-GAT network to obtain a node embedding vector; performing similarity measurement on the node embedding vector to find the changed node, detecting the superpixel area of ​​the superpixel feature map change through the superpixel label map, and generating a pixel-level changed area display. The present invention can solve the technical problem that the traditional difference measurement method leads to an increase in false detection rate and missed detection rate.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image processing in the field of remote sensing, and in particular to a hyperspectral image change detection method, device and storage medium. Background Art

[0002] Hyperspectral images are different from ordinary color images. They often have a higher number of spectral bands, and each band corresponds to different material absorption characteristics. Therefore, hyperspectral images can provide richer information than ordinary color images. When taken at different time points, they can record the changes in the spectral characteristics of the objects in detail. In hyperspectral change detection, changes are defined as the phenomenon that the spectral characteristics of certain pixels in the hyperspectral images at different time points change significantly. Change detection algorithms are designed to automatically identify and locate these change areas. Hyperspectral change detection technology does not need to rely on prior information. It can directly detect and identify changes in images without prior spectral characteristics of the target objects.

[0003] However, algorithms based on feature extraction of dual-phase hyperspectral images and then difference measurement often face the following limitations:

[0004] 1. The spectral characteristics of the objects may be affected by the mixing of surrounding materials, especially when the spatial resolution is low. The spectral mixing phenomenon will cause the extracted features to be impure, increasing the complexity and uncertainty of change detection. 2. When processing complex hyperspectral data, traditional linear feature extraction methods cannot fully capture the nonlinear relationship between spectral features, thus limiting the performance of change detection. 3. The spectral changes of the same object at different time points are inconsistent due to changes in environmental conditions (such as light, humidity, temperature, etc.). Traditional difference measurement methods cannot effectively distinguish between spectral changes caused by changes in the objects themselves and spectral changes caused by changes in environmental conditions, resulting in increased false detection and missed detection rates.

[0005] Therefore, it is urgent to propose a change detection method that can better find the changed pixels in dual-phase hyperspectral images to solve the above technical problems. Summary of the invention

[0006] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide a hyperspectral image change detection method, device and storage medium, which can solve the technical problem that traditional difference measurement methods cannot effectively distinguish between spectral changes caused by changes in the ground objects themselves and spectral changes caused by changes in environmental conditions, resulting in increased false detection rates and missed detection rates.

[0007] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0008] In a first aspect, the present invention provides a hyperspectral image change detection method, comprising:

[0009] Acquire dual-phase hyperspectral images;

[0010] Extracting features at the spatial level and the spectral level of the dual-phase hyperspectral image respectively to obtain a spatial feature map and a spectral feature map, and aggregating the spatial feature map and the spectral feature map through a pre-constructed adaptive fusion block to obtain a fused feature map;

[0011] Segmenting the dual-phase hyperspectral image by a superpixel algorithm to generate a superpixel label map, and projecting the superpixel label map onto the fusion feature map to form a superpixel feature map;

[0012] Constructing a topological map according to the superpixel feature map, wherein the topological map includes node feature vectors and an adjacency matrix;

[0013] The node feature vectors and adjacency matrix in the topological map are input into the pre-trained KAN-GAT network to obtain the node embedding vector; the node embedding vector is measured for similarity to find the changed nodes, the changed nodes are mapped back to the superpixel feature map, the superpixel area of ​​the superpixel feature map where the superpixel feature map is changed is detected through the superpixel label map, and a pixel-level display of the changed area is generated.

[0014] Furthermore, feature extraction is performed on the dual-phase hyperspectral image at the spatial level and the spectral level to obtain a spatial feature map and a spectral feature map, including:

[0015] At the spatial level, the dual-phase hyperspectral image is input into a pre-built RSAB module, the dual-phase hyperspectral image is convolved and normalized by the RSAB module to obtain a first hyperspectral image feature map, the first hyperspectral image feature map is input into a channel attention mechanism in the RSAB module for compression to obtain a first channel attention feature map, the first channel attention feature of the first channel attention feature map is input into the spatial attention mechanism in the RSAB module, the edge and texture information of the first channel attention feature is extracted by the maximum pooling layer and the average pooling layer of the spatial attention mechanism, the edge and texture information is transformed by a two-dimensional convolutional layer, and a spatial feature map is output;

[0016] The description formula of the channel attention mechanism includes:

[0017]

[0018] The description formula of the spatial attention mechanism includes:

[0019]

[0020] in, represents the spatial feature map, represents the activation function, represents the first hyperspectral image feature map, It is a two-dimensional convolution with a convolution kernel of 1x1. is the average pooling layer, is the maximum pooling layer, It is a two-dimensional convolution with a convolution kernel of 7x7. is the first channel attention feature map, is the first channel attention feature;

[0021] At the spectral level, the dual-phase hyperspectral image is input into a pre-built RSCB module, the dual-phase hyperspectral image is convolved and normalized by the RSCB module to obtain a second hyperspectral image feature map, the second hyperspectral image feature map is input into the channel attention mechanism in the RSCB module for compression to obtain a second channel attention feature map, the second channel attention feature of the second channel attention feature map is input into the 1D convolution of the pre-built weighted channel attention mechanism module to obtain the attention weight of the second channel attention feature, the second channel attention feature and the attention weight are input into a two-dimensional convolutional layer for normalization, and a spectral feature map is output.

[0022] Furthermore, segmenting the dual-phase hyperspectral image by a superpixel algorithm to generate a superpixel label map includes:

[0023] The dual-phase hyperspectral image is divided into several blocks of equal size. The central pixel of each block is used as a superpixel center. The spacing between adjacent superpixel centers is:

[0024]

[0025] in, is the total number of pixels in the dual-phase hyperspectral image, is the desired number of superpixel centers, is the spacing between adjacent superpixel centers;

[0026] The color of the dual-phase hyperspectral image is converted from its original color space to the CIELab color space, the color distance between the pixel point and the superpixel center is calculated in the CIELab color space, and the spatial distance between the pixel point and the superpixel center is calculated in the image space of the dual-phase hyperspectral image, and the calculation formula includes:

[0027]

[0028]

[0029] in, is the color distance between the pixel and the superpixel center in the CIELab color space, is the spatial distance between the pixel point and the superpixel center in the image space, , and Respectively represent The CIELab value of the pixel color, , and Indicates The CIELab value of the superpixel center, is the brightness component, which indicates the brightness of the color. is the component from green to red, is the component from blue to yellow; Indicates The horizontal coordinate of the pixel, Indicates The vertical coordinate of the pixel, Indicates The horizontal coordinate of the superpixel center, Indicates The vertical coordinate of the superpixel center, is the pixel ordinal number, and its value range is , is the superpixel center ordinal, ranging from ;

[0030] The distance metric is calculated according to the spacing between the adjacent superpixel centers, the color distance between the pixel point and the superpixel center in the CIELab color space, and the spatial distance between the pixel point and the superpixel center in the image space. The calculation formula includes:

[0031]

[0032] in, is a parameter that balances color distance and spatial distance, represents a distance metric;

[0033] Based on the distance metric, each pixel is assigned to the superpixel center that is closest to it;

[0034] When all pixels are assigned, the center of each superpixel is recalculated based on the color values ​​and position coordinates of all pixels in the superpixel center. The calculation formula includes:

[0035]

[0036]

[0037]

[0038]

[0039]

[0040] in, , and Represent the CIELab values ​​of the new superpixel center, is the brightness component of the new superpixel center, indicating the brightness of the color, is the green to red component of the new superpixel center, is the blue to yellow component of the new superpixel center, Indicates that based on The horizontal coordinate of the new superpixel center obtained by the superpixel center is Indicates that based on The ordinate of the new superpixel center obtained by the superpixel center; It is The set of pixels associated with the superpixel centers, yes The number of pixels in ;

[0041] Repeat the iteration until the changes in the position coordinates and color values ​​of the superpixel center between two iterations are lower than the preset stability threshold, and generate a superpixel label map based on the position coordinates and color values ​​of the superpixel center of the last iteration.

[0042] Furthermore, inputting the node feature vector and the adjacency matrix in the topological graph into the pre-trained KAN-GAT network to obtain the node embedding vector includes:

[0043] The weight coefficient between a node and its neighboring nodes is calculated through the B-spline basis function;

[0044] Generating a weighted adjacency matrix according to the weight coefficients;

[0045] Apply the weighted adjacency matrix to the node feature vector to generate the node feature vector after dimension transformation;

[0046] Performing a linear transformation on the node feature vector after the dimension transformation to generate a linearly transformed node feature vector;

[0047] Based on the linearly transformed node feature vector, the attention score between the node and its neighbor nodes is calculated through a preset self-attention mechanism.

[0048] The attention score is normalized by a Softmax function to obtain an attention coefficient of the node;

[0049] The feature vectors of neighboring nodes are weighted summed according to the attention coefficient of the node to obtain a node embedding vector.

[0050] Furthermore, the training process of the KAN-GAT network includes:

[0051] Generate training sample pairs according to the topological graph;

[0052] The training sample pairs are input into the KAN-GAT network, and the KAN-GAT network is trained in combination with the contrast loss function until the contrast loss function converges. The contrast loss function is defined as:

[0053]

[0054] in, is the contrast loss function, and is the feature vector of the training sample pair, y is the label of the training sample pair, when the samples in the training sample pair belong to the same category , when they do not belong to the same category , is the Euclidean distance between the training sample pairs, is a hyperparameter, which indicates the lower limit of the distance of negative sample pairs.

[0055] Further, performing similarity measurement on the node embedding vector to find the changed node includes:

[0056] The change threshold is determined by the statistical characteristics of the comprehensive metric value, and the change node is identified by comparing the comprehensive metric value with the change threshold. The calculation formula of the comprehensive metric value is:

[0057]

[0058] in, is the comprehensive metric value, node and nodes The eigenvectors of and , Represents the feature vector and The Euclidean distance between Represents the feature vector and The cosine similarity between and is a weight parameter used to adjust the influence of Euclidean distance and cosine similarity in comparing the changed nodes.

[0059] In a second aspect, the present invention provides a hyperspectral image change detection device, comprising:

[0060] An image acquisition module, used for acquiring dual-phase hyperspectral images;

[0061] A feature map acquisition module is used to extract features of the dual-phase hyperspectral image at the spatial level and the spectral level respectively to obtain a spatial feature map and a spectral feature map, and to aggregate the spatial feature map and the spectral feature map through a pre-built adaptive fusion block to obtain a fused feature map;

[0062] A projection module is used to segment the dual-phase hyperspectral image by a superpixel algorithm to generate a superpixel label map, and project the superpixel label map into the fusion feature map to form a superpixel feature map;

[0063] A topology map generation module, used to construct a topology map according to the superpixel feature map, wherein the topology map includes node feature vectors and an adjacency matrix;

[0064] The display module is used to input the node feature vector and adjacency matrix in the topological map into the pre-trained KAN-GAT network to obtain the node embedding vector; perform similarity measurement on the node embedding vector to find the changed node, map the changed node back to the superpixel feature map, detect the superpixel area of ​​the superpixel feature map change through the superpixel label map, and generate a pixel-level change area display.

[0065] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] The present invention can capture more details and global information in the image by extracting and fusing the spatial features and spectral features of the dual-phase hyperspectral image, thereby improving the accuracy of change detection. The pre-built KAN-GAT network is used to implement self-supervised training, which can effectively learn the node feature representation in the feature graph, enhance the robustness of change detection, and reduce the false detection rate and missed detection rate.

[0068] The comprehensive measurement method combining Euclidean distance and cosine similarity can more accurately measure the changes in node features and improve the accuracy of change detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1A method flow chart of a hyperspectral image change detection method provided in Embodiment 1 of the present invention;

[0070] Figure 2 This is a structural block diagram of a hyperspectral image change detection device provided in Embodiment 2 of the present invention;

[0071] Figure 3 An RGB pseudo-color image of a hyperspectral image at a first moment in a hyperspectral image change detection method provided in the first embodiment of the present invention;

[0072] Figure 4 The RGB pseudo-color image of the hyperspectral image at the second moment in the hyperspectral image change detection method provided in the first embodiment of the present invention;

[0073] Figure 5 A fusion feature map obtained by inputting a hyperspectral image at a first moment in a hyperspectral image change detection method provided in the first embodiment of the present invention;

[0074] Figure 6 A fusion feature map obtained by inputting a hyperspectral image at a second moment into a hyperspectral image change detection method provided in the first embodiment of the present invention;

[0075] Figure 7 This is the change detection graph finally outputted in the hyperspectral image change detection method provided in the first embodiment of the present invention. DETAILED DESCRIPTION

[0076] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0077] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0078] Embodiment 1:

[0079] Figure 1 1 is a flow chart of the hyperspectral image change detection method in the first embodiment of the present invention. This flow chart only shows the logical sequence of the method described in this embodiment. In other possible embodiments of the present invention, different Figure 1 The steps shown or described are accomplished in the order shown.

[0080] like Figure 1 As shown, the method disclosed in this embodiment specifically includes the following steps:

[0081] Step 1: Obtain dual-phase hyperspectral images, such as Figure 3 and Figure 4 As shown, it is represented as HSIT1 and HSIT2, i.e., the hyperspectral image at the first moment and the hyperspectral image at the second moment;

[0082] Step 2: Extract features of the dual-phase hyperspectral image at the spatial level and the spectral level, and obtain the spatial feature map and the spectral feature map including:

[0083] At the spatial level, the dual-phase hyperspectral image is input into the pre-built RSAB module. The RSAB module (Recurrent Spatial Attention Block) is an attention mechanism module used in deep learning. It combines the spatial attention mechanism of the recurrent neural network (RNN). This module is designed to improve the model's ability to recognize spatial features in the input data. The RSAB module includes a 3x3 two-dimensional convolution layer, a 1x1 two-dimensional convolution layer, a BN layer (i.e., a batch normalization layer), a spatial attention mechanism, and a channel attention mechanism. Attention), convolve the dual-phase hyperspectral image through the 3x3 two-dimensional convolution layer of the RSAB module, obtain the first hyperspectral image feature map through the BN layer normalization processing after the convolution, input the first hyperspectral image feature map into the channel attention mechanism in the RSAB module for compression to obtain the first channel attention feature map, input the first channel attention feature of the first channel attention feature map into the spatial attention mechanism in the RSAB module, extract the edge and texture information of the first channel attention feature through the maximum pooling layer and the average pooling layer of the spatial attention mechanism, transform the edge and texture information through the 7x7 two-dimensional convolution layer, and output the spatial feature map;

[0084] The description formula of the channel attention mechanism includes:

[0085]

[0086] The description formula of the spatial attention mechanism includes:

[0087]

[0088] in, represents the spatial feature map, () represents the activation function, represents the first hyperspectral image feature map, It is a two-dimensional convolution with a convolution kernel of 1x1. () is the average pooling layer, () is the maximum pooling layer, It is a two-dimensional convolution with a convolution kernel of 7x7. is the first channel attention feature map, is the first channel attention feature;

[0089] Specifically, the average pooling layer and the maximum pooling layer convert each first channel attention feature of the first channel attention feature map into a representative statistical value, and use the other two shared 1x1 two-dimensional convolutional layers as targets to obtain the inspired channel representation, and use the sigmoid activation function () The sum of the extracted first channel attention features is converted into a spatial feature map Mca with a weight range of 0 to 1. The larger the weight of the spatial feature map Mca, the more critical the spectral information of the first hyperspectral image feature map is. By highlighting important spectral information, the sensitivity to key information can be enhanced, and the interference of irrelevant or noise information can be suppressed, thereby improving the overall performance;

[0090] At the spectral level, the dual-phase hyperspectral image is input into the RSCB (Recurrent Spatial ChannelBlock) module. The RSCB module is an attention mechanism module used in deep learning. It combines the channel attention mechanism of the recurrent neural network (RNN). This module aims to improve the model's ability to recognize spectral features in the input data. The RSCB module includes a 1x1 two-dimensional convolution layer, a BN layer, and a channel attention mechanism. The dual-phase hyperspectral image is convolved through the 1x1 two-dimensional convolution layer of the RSCB module. After the convolution, it is normalized through the BN layer to obtain a second hyperspectral image feature map. The second hyperspectral image feature map is input into the channel attention mechanism in the RSCB module for compression to obtain a second channel attention feature map. The second channel attention feature of the second channel attention feature map is input into the pre-built weighted channel attention mechanism (Efficient Channel Attention Mechanism). Attention) module, obtain the attention weight of the second channel attention feature, the number of attention weights is the same as the number of channels of the second channel attention feature map, input the second channel attention feature and attention weight into the two-dimensional convolution layer for normalization, and output the spectral feature map. The convolution kernel size of the two-dimensional convolution layer is related to the complexity of the features to be captured. Small convolution kernels (1x1 or 3x3) can capture local features of the image, and large convolution kernels (7x7) can capture more extensive features. The convolution kernel size can be adjusted according to actual needs and is not limited here.

[0091] Among them, Efficient Channel Attention (ECA) is a lightweight channel attention mechanism that aims to effectively enhance the feature representation ability of deep neural networks with minimal computational cost. ECA achieves this by learning the importance weight of each channel, allowing the network to pay more attention to useful features and suppress unimportant features. It belongs to the existing technology and will not be described here.

[0092] The spatial feature maps and spectral feature maps are obtained by inputting the hyperspectral images HSIT1 and HSIT2 at different times. The spatial feature maps and spectral feature maps at different times are aggregated by pre-built adaptive fusion blocks to obtain fused feature maps at different times, such as Figure 5 and Figure 6 As shown in the figure, the Adaptive Fusion Block is a structure used to fuse different types of features (such as spatial feature maps and spectral feature maps) in deep learning. It is a prior art and will not be described in detail here.

[0093] Step 3: Segmenting the dual-phase hyperspectral image using a superpixel algorithm to generate a superpixel label map includes:

[0094] The dual-phase hyperspectral image is divided into several blocks of equal size. The central pixel of each block is used as a superpixel center. The spacing between adjacent superpixel centers is:

[0095]

[0096] in, is the total number of pixels in the dual-phase hyperspectral image, is the desired number of superpixel centers, is the spacing between adjacent superpixel centers;

[0097] The color of the dual-phase hyperspectral image is converted from its original color space to the CIELab color space, the color distance between the pixel point and the superpixel center is calculated in the CIELab color space, and the spatial distance between the pixel point and the superpixel center is calculated in the image space of the dual-phase hyperspectral image, and the calculation formula includes:

[0098]

[0099]

[0100] in, is the color distance between the pixel and the superpixel center in the CIELab color space, is the spatial distance between the pixel point and the superpixel center in the image space, , and Respectively represent The CIELab value of the pixel color, , and Indicates The CIELab value of the superpixel center, is the brightness component, which indicates the brightness of the color. is the component from green to red, is the component from blue to yellow; Indicates The horizontal coordinate of the pixel, Indicates The vertical coordinate of the pixel, Indicates The horizontal coordinate of the superpixel center, Indicates The vertical coordinate of the superpixel center, is the pixel ordinal number, and its value range is , is the superpixel center ordinal, ranging from ;

[0101] The distance metric is calculated according to the spacing between the adjacent superpixel centers, the color distance between the pixel point and the superpixel center in the CIELab color space, and the spatial distance between the pixel point and the superpixel center in the image space. The calculation formula includes:

[0102]

[0103] in, is a parameter that balances color distance and spatial distance, represents a distance metric;

[0104] Based on the distance metric, each pixel is assigned to the superpixel center that is closest to it;

[0105] When all pixels are assigned, the center of each superpixel is recalculated based on the color values ​​and position coordinates of all pixels in the superpixel center. The calculation formula includes:

[0106]

[0107]

[0108]

[0109]

[0110]

[0111] in, , and Represent the CIELab values ​​of the new superpixel center, is the brightness component of the new superpixel center, indicating the brightness of the color, is the green to red component of the new superpixel center, is the blue to yellow component of the new superpixel center, Indicates that based on The horizontal coordinate of the new superpixel center obtained by the superpixel center is Indicates that based on The ordinate of the new superpixel center obtained by the superpixel center; It is The set of pixels associated with the superpixel centers, yes The number of pixels in ;

[0112] Repeat the iteration until the changes of the position coordinates and color values ​​of the superpixel center between two iterations are lower than the preset stability threshold, and generate a superpixel label map according to the position coordinates and color values ​​of the superpixel center of the last iteration;

[0113] The superpixel label map is projected onto the fused feature map to form a superpixel feature map.

[0114] Step 4: Construct a topological map based on the superpixel feature map. The topological map includes node feature vectors and adjacency matrix:

[0115] Each superpixel region in the superpixel feature map is regarded as a node in the topological map. The node is represented as the mean value of all pixels in each superpixel region. The adjacency matrix is ​​constructed according to the superpixel segmentation result. The adjacency matrix is ​​used to represent the edges in the graph, and finally the topological map G is obtained.

[0116] Among them, the construction of the adjacency matrix can be formally described as the following steps:

[0117] Define the adjacency matrix: Let the superpixel segmentation result be a matrix , whose size is , the set of superpixel labels is , is the number of sets;

[0118] Initialize the adjacency matrix: Define the adjacency matrix , whose size is , initially all elements All are 0;

[0119] Traverse each pixel, for each pixel and its superpixel labels , check whether it and its neighboring pixels in the four directions of up, down, left, and right belong to different superpixels. If so, set the corresponding position in the adjacency matrix to 1;

[0120] Step 5: Input the node feature vector and adjacency matrix in the topology graph into the pre-trained KAN-GAT network to obtain the node embedding vector:

[0121] Calculate the nodes through B-spline basis functions Its neighbor nodes The weighting coefficient between:

[0122]

[0123] in, Representative Node and The characteristic distance between is the corresponding B-spline basis function, Representation Node Its neighbor nodes The weighting coefficient between

[0124] Generate a weighted adjacency matrix based on the weighting coefficients:

[0125]

[0126] in, represents the weighted adjacency matrix, represents the adjacency matrix;

[0127] Apply the weighted adjacency matrix to the node's feature vector to generate the node's feature vector after dimension transformation:

[0128]

[0129] Among them, X represents the feature vector of the node, Represents the node feature vector after the dimension change, is the parameter matrix that can be learned;

[0130] Node feature vector after dimension transformation Perform linear transformation to generate node feature vector after linear transformation :

[0131]

[0132] in, is the node feature vector after linear transformation;

[0133] Based on the linearly transformed node feature vector, the attention score between the node and its neighbor nodes is calculated through the preset self-attention mechanism:

[0134]

[0135] in, () represents the activation function, a represents the weight vector that can be learned, Indicates the node after linear change The characteristic vector of Indicates the node after linear change The characteristic vector of Indicates that the node The eigenvectors and nodes of The feature vectors are concatenated. Representation Node and nodes Attention score;

[0136] The attention score is normalized by the Softmax function to obtain the attention coefficient of the node:

[0137]

[0138] in, Representation Node The neighbor node set of node For Node The neighbor nodes of the node ensure that all attention coefficients are obtained through the Softmax function The sum is 1, Representation Node and nodes The attention coefficient;

[0139] According to the node's attention coefficient, the feature vectors of neighboring nodes are weighted summed to obtain the node embedding vector:

[0140]

[0141] in, is the embedding vector of node i.

[0142] Perform similarity measurement on the node embedding vector to find the changed nodes:

[0143] The change threshold is determined by the statistical characteristics of the comprehensive metric value, and the change node is identified by comparing the comprehensive metric value with the change threshold. The calculation formula of the comprehensive metric value is:

[0144]

[0145] in, is the comprehensive metric value, node and nodes The eigenvectors of are and , Represents the feature vector and The Euclidean distance between Represents the feature vector and The cosine similarity between and is a weight parameter used to adjust the influence of Euclidean distance and cosine similarity in comparing the changed nodes.

[0146] Among them, the comprehensive metric value refers to a comprehensive indicator obtained by calculating or combining multiple indicators (Euclidean distance and cosine similarity) in a certain way, which reflects the overall state of the system, process or object. The change threshold is a critical value used to determine whether the metric value has changed significantly. This threshold can be determined based on the statistical characteristics of the metric value, such as being set to the mean of the metric value plus or minus several times the standard deviation, or using other statistical methods (such as CUSUM, EWMA, etc.) to calculate the threshold. In this embodiment, the change threshold can be the mean of the comprehensive metric value plus or minus one standard deviation.

[0147] Map the changed nodes back to the superpixel feature map, detect the superpixel area where the superpixel feature map changes through the superpixel label map, and generate a pixel-level change area display, such as Figure 7 As shown in the figure, how to map the changed nodes back to the superpixel feature map, detect the superpixel region of the superpixel feature map change through the superpixel label map, and generate pixel-level change region display are existing mature technologies and will not be described here.

[0148] The training process of the KAN-GAT network includes:

[0149] Generate training sample pairs based on the topological map: Based on the generated topological maps G1 and G2, generate positive samples and negative samples. The purpose of this operation is to shorten the distance between positive samples and enlarge the distance between negative samples, which are used to train the subsequent KAN-GAT network.

[0150] The training sample pairs are input into the KAN-GAT network, and the KAN-GAT network is trained in combination with the contrast loss function until the contrast loss function converges. In this embodiment, the number of training is set to 500, and the contrast loss function converges to 0.2 after 500 times, and the training is completed. The definition of the contrast loss function is:

[0151]

[0152] in, is the contrast loss function, and is the feature vector of the training sample pair, y is the label of the training sample pair, when the samples in the training sample pair belong to the same category , when they do not belong to the same category , is the Euclidean distance between the training sample pairs, is a hyperparameter, which indicates the lower limit of the distance of negative sample pairs.

[0153] Embodiment 2:

[0154] like Figure 2 As shown, the present invention provides a hyperspectral image change detection device, comprising:

[0155] An image acquisition module, used for acquiring dual-phase hyperspectral images;

[0156] A feature map acquisition module is used to extract features at the spatial level and the spectral level of the dual-phase hyperspectral image, respectively, to obtain a spatial feature map and a spectral feature map, and to aggregate the spatial feature map and the spectral feature map through a pre-built adaptive fusion block to obtain a fused feature map. The feature map acquisition module includes a spatial feature extraction module and a spectral feature extraction module;

[0157] A projection module is used to segment the dual-phase hyperspectral image by a superpixel algorithm to generate a superpixel label map, and project the superpixel label map into the fusion feature map to form a superpixel feature map;

[0158] A topology map generation module, used to construct a topology map according to the superpixel feature map, wherein the topology map includes node feature vectors and an adjacency matrix;

[0159] The display module is used to input the node feature vector and adjacency matrix in the topological map into the pre-trained KAN-GAT network to obtain the node embedding vector; perform similarity measurement on the node embedding vector to find the changed node, map the changed node back to the superpixel feature map, detect the superpixel area of ​​the superpixel feature map change through the superpixel label map, and generate a pixel-level change area display.

[0160] A hyperspectral image change detection device provided in an embodiment of the present invention can execute a hyperspectral image change detection method provided in an embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0161] about Figure 2 , making the following translation:

[0162] Spatial feature extraction module (SAB): spatial feature extraction module; Spectral feature extraction module (CAE): spectral feature extraction module; Convolutional layer (CL): convolutional layer; KAN-Layer represents an architecture; guidance refers to the information exchange and influence between different network layers or different network modules, and here refers to the information exchange between different inputs of the KAN-GAT network; linear transformation, self-attention mechanism and multi-head attention mechanism are all specific processing processes for obtaining the node embedding vector after inputting the node feature vector and adjacency matrix in the topological graph into the pre-trained KAN-GAT network. The specific processing process has been introduced in Example 1 and will not be repeated here; Regarding the multi-head attention mechanism, an additional supplement is made here: in practice, when given the same query, key and value set, we hope that the model can learn different behaviors based on the same attention mechanism, and then combine the different behaviors as knowledge, such as capturing various ranges of dependencies within the sequence (for example, short-distance dependencies and long-distance dependencies). Therefore, different subspace representations of queries, keys, and values ​​are combined through the attention mechanism. Finally, the outputs of the attention pooling are concatenated together and transformed by another learnable linear projection to produce the final output. This design is called a multi-head attention mechanism. Figure 2 The loss in is the contrast loss mentioned in Example 1; weight sharing refers to the use of the same weight parameters at different positions or levels of the neural network. This concept is common in convolutional neural networks (CNNs), where the parameters of the convolution kernel are shared across the entire input image. This design can greatly reduce the number of parameters that need to be learned, reduce the complexity of the model, reduce the risk of overfitting, and improve the computational efficiency of the model. Node difference metric is a method used in graph theory and network analysis to measure the differences between different nodes in a network. In this embodiment, it is used to judge and analyze the generated pixel-level change area display.

[0163] Embodiment three:

[0164] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method described in the first embodiment are implemented, and the program has functional modules and beneficial effects corresponding to the execution method.

[0165] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0166] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (apparatus), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0167] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0169] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A hyperspectral image change detection method, characterized in that: include: Acquire dual-phase hyperspectral images; Extracting features at the spatial level and the spectral level of the dual-phase hyperspectral image respectively to obtain a spatial feature map and a spectral feature map, and aggregating the spatial feature map and the spectral feature map through a pre-constructed adaptive fusion block to obtain a fused feature map; Segmenting the dual-phase hyperspectral image by a superpixel algorithm to generate a superpixel label map, and projecting the superpixel label map onto the fusion feature map to form a superpixel feature map; Constructing a topological map according to the superpixel feature map, wherein the topological map includes node feature vectors and an adjacency matrix; Input the node feature vector and adjacency matrix in the topological map into the pre-trained KAN-GAT network to obtain the node embedding vector; perform similarity measurement on the node embedding vector to find the changed node, map the changed node back to the superpixel feature map, detect the superpixel area of ​​the superpixel feature map change through the superpixel label map, and generate a pixel-level change area display; Inputting the node feature vector and adjacency matrix in the topological graph into the pre-trained KAN-GAT network to obtain the node embedding vector includes: The weight coefficient between a node and its neighboring nodes is calculated through the B-spline basis function; Generating a weighted adjacency matrix according to the weight coefficients; Apply the weighted adjacency matrix to the node feature vector to generate the node feature vector after dimension transformation; Performing a linear transformation on the node feature vector after the dimension transformation to generate a linearly transformed node feature vector; Based on the linearly transformed node feature vector, the attention score between the node and its neighbor nodes is calculated through a preset self-attention mechanism. The attention score is normalized by a Softmax function to obtain an attention coefficient of the node; The feature vectors of neighboring nodes are weighted summed according to the attention coefficient of the node to obtain a node embedding vector.

2. The hyperspectral image change detection method according to claim 1, characterized in that: The dual-phase hyperspectral image is subjected to feature extraction at the spatial level and the spectral level respectively to obtain a spatial feature map and a spectral feature map, including: At the spatial level, the dual-phase hyperspectral image is input into a pre-built RSAB module, the dual-phase hyperspectral image is convolved and normalized by the RSAB module to obtain a first hyperspectral image feature map, the first hyperspectral image feature map is input into a channel attention mechanism in the RSAB module for compression to obtain a first channel attention feature map, the first channel attention feature of the first channel attention feature map is input into the spatial attention mechanism in the RSAB module, the edge and texture information of the first channel attention feature is extracted by the maximum pooling layer and the average pooling layer of the spatial attention mechanism, the edge and texture information is transformed by a two-dimensional convolutional layer, and a spatial feature map is output; The description formula of the channel attention mechanism includes: ； The description formula of the spatial attention mechanism includes: ； in, represents the spatial feature map, represents the activation function, represents the first hyperspectral image feature map, It is a two-dimensional convolution with a convolution kernel of 1x1. is the average pooling layer, is the maximum pooling layer, It is a two-dimensional convolution with a convolution kernel of 7x7. is the first channel attention feature map, is the first channel attention feature; At the spectral level, the dual-phase hyperspectral image is input into a pre-built RSCB module, the dual-phase hyperspectral image is convolved and normalized by the RSCB module to obtain a second hyperspectral image feature map, the second hyperspectral image feature map is input into the channel attention mechanism in the RSCB module for compression to obtain a second channel attention feature map, the second channel attention feature of the second channel attention feature map is input into the 1D convolution of the pre-built weighted channel attention mechanism module to obtain the attention weight of the second channel attention feature, the second channel attention feature and the attention weight are input into a two-dimensional convolutional layer for normalization, and a spectral feature map is output.

3. The hyperspectral image change detection method according to claim 1, characterized in that: The dual-phase hyperspectral image is segmented by a superpixel algorithm to generate a superpixel label map, including: The dual-phase hyperspectral image is divided into blocks of equal size, and the central pixel of each block is used as a superpixel center. The spacing between adjacent superpixel centers is: ； in, is the total number of pixels in the dual-phase hyperspectral image, is the desired number of superpixel centers, is the spacing between adjacent superpixel centers; The color of the dual-phase hyperspectral image is converted from its original color space to the CIELab color space, the color distance between the pixel point and the superpixel center is calculated in the CIELab color space, and the spatial distance between the pixel point and the superpixel center is calculated in the image space of the dual-phase hyperspectral image, and the calculation formula includes: ； ； in, is the color distance between the pixel and the superpixel center in the CIELab color space, is the spatial distance between the pixel point and the superpixel center in the image space, , and Respectively represent The CIELab value of the pixel color, , and Indicates The CIELab value of the superpixel center, is the brightness component, which indicates the brightness of the color. is the component from green to red, is the component from blue to yellow; Indicates The horizontal coordinate of the pixel, Indicates The vertical coordinate of the pixel, Indicates The horizontal coordinate of the superpixel center, Indicates The vertical coordinate of the superpixel center, is the pixel ordinal number, and its value range is , is the superpixel center ordinal, ranging from ; The distance metric is calculated according to the spacing between the adjacent superpixel centers, the color distance between the pixel point and the superpixel center in the CIELab color space, and the spatial distance between the pixel point and the superpixel center in the image space. The calculation formula includes: ； in, is a parameter that balances color distance and spatial distance, represents a distance metric; Based on the distance metric, each pixel is assigned to the superpixel center that is closest to it; When all pixels are assigned, the center of each superpixel is recalculated based on the color values ​​and position coordinates of all pixels in the superpixel center. The calculation formula includes: ； ； ； ； ； in, , and Represent the CIELab values ​​of the new superpixel center, is the brightness component of the new superpixel center, indicating the brightness of the color, is the green to red component of the new superpixel center, is the blue to yellow component of the new superpixel center, Indicates that based on The horizontal coordinate of the new superpixel center obtained by the superpixel center is Indicates that based on The ordinate of the new superpixel center obtained by the superpixel center; It is The set of pixels associated with the superpixel centers, yes The number of pixels in ; Repeat the iteration until the changes in the position coordinates and color values ​​of the superpixel center between two iterations are lower than the preset stability threshold, and generate a superpixel label map based on the position coordinates and color values ​​of the superpixel center of the last iteration.

4. The hyperspectral image change detection method according to claim 1, characterized in that: The training process of the KAN-GAT network includes: Generate training sample pairs according to the topological graph; The training sample pairs are input into the KAN-GAT network, and the KAN-GAT network is trained in combination with the contrast loss function until the contrast loss function converges. The contrast loss function is defined as: ； in, is the contrast loss function, and is the feature vector of the training sample pair, y is the label of the training sample pair, when the samples in the training sample pair belong to the same category , when they do not belong to the same category , is the Euclidean distance between the training sample pairs, is a hyperparameter, which indicates the lower limit of the distance of negative sample pairs.

5. The hyperspectral image change detection method according to claim 4, characterized in that: Performing similarity measurement on the node embedding vector to find the changed node includes: The change threshold is determined by the statistical characteristics of the comprehensive metric value, and the change node is identified by comparing the comprehensive metric value with the change threshold. The calculation formula of the comprehensive metric value is: ； in, is the comprehensive metric value, node and nodes The eigenvectors of are and , Represents the feature vector and The Euclidean distance between Represents the feature vector and The cosine similarity between and is a weight parameter used to adjust the influence of Euclidean distance and cosine similarity in comparing the changed nodes.

6. A hyperspectral image change detection device, characterized in that: include: An image acquisition module, used for acquiring dual-phase hyperspectral images; A feature map acquisition module is used to extract features of the dual-phase hyperspectral image at the spatial level and the spectral level respectively to obtain a spatial feature map and a spectral feature map, and to aggregate the spatial feature map and the spectral feature map through a pre-built adaptive fusion block to obtain a fused feature map; A projection module is used to segment the dual-phase hyperspectral image by a superpixel algorithm to generate a superpixel label map, and project the superpixel label map into the fusion feature map to form a superpixel feature map; A topology map generation module, used to construct a topology map according to the superpixel feature map, wherein the topology map includes node feature vectors and an adjacency matrix; A display module is used to input the node feature vector and adjacency matrix in the topological map into a pre-trained KAN-GAT network to obtain a node embedding vector; perform similarity measurement on the node embedding vector to find the changed node, map the changed node back to the superpixel feature map, detect the superpixel area of ​​the superpixel feature map change through the superpixel label map, and generate a pixel-level display of the changed area; Inputting the node feature vector and adjacency matrix in the topological graph into the pre-trained KAN-GAT network to obtain the node embedding vector includes: The weight coefficient between a node and its neighboring nodes is calculated through the B-spline basis function; Generating a weighted adjacency matrix according to the weight coefficients; Apply the weighted adjacency matrix to the node feature vector to generate the node feature vector after dimension transformation; Performing a linear transformation on the node feature vector after the dimension transformation to generate a linearly transformed node feature vector; Based on the linearly transformed node feature vector, the attention score between the node and its neighbor nodes is calculated through a preset self-attention mechanism. The attention score is normalized by a Softmax function to obtain an attention coefficient of the node; The feature vectors of neighboring nodes are weighted summed according to the attention coefficient of the node to obtain a node embedding vector.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images