An unsupervised non-training hyperspectral image change detection method

Abstract

The application discloses a kind of unsupervised non-training hyperspectral image change detection method, the unsupervised non-training hyperspectral image change detection method is based on deep learning technique, including multiple expansion feature extraction module, deformable feature extraction module and deep feature comparison module.A kind of unsupervised non-training hyperspectral image change detection method of the application, for the characteristics of relatively low spatial resolution of hyperspectral image, and the problem of lack of labeled sample, non-training deformable network is applied to hyperspectral image change detection technology, fully utilizes the structure information such as ground feature shape and size to realize high-precision completely unsupervised change detection.The designed multiple expansion feature extraction and deformable feature extraction combination module makes network can extract local feature in larger receptive field and any adjacent position, which is beneficial to improve change detection performance, and can be easily embedded into other technologies;A simple and effective feature selection improvement scheme is proposed according to feature importance, which improves network model accuracy and adaptability in different scenarios by emphasizing the contribution of important features.

Description

Technical Field

[0001] This invention relates to an image change detection method, and more particularly to an unsupervised, untrained hyperspectral image change detection method, belonging to the field of image change detection technology. Background Technology

[0002] Deep learning-based change detection is a class of intelligent change detection techniques that can automatically extract discriminative features without requiring manual feature engineering. For example, techniques use Siamese Convolutional Neural Networks (CNNs) to extract spatial-spectral-temporal features of image patches and then classify these patches as changing or invariant. Other techniques employ Fully Convolutional Networks (FCNs) and U-Net-like frameworks to segment each pixel as changing or invariant without dividing the image into smaller patches. Some techniques also apply attention mechanisms, such as spatial attention and channel attention, to extract rich local contextual features to improve change detection performance. Furthermore, some novel techniques do not use CNNs but instead convert images into sequential image patches, developing change detection techniques based on Transformer networks. In general, most deep learning change detection techniques attempt to extract discriminative spatially relevant features from high-resolution images. These techniques typically require a large number of samples to train the network model and may not handle medium- to low-resolution images well, such as hyperspectral images. Hyperspectral images are known to convey more spectral information to distinguish land cover types, which is crucial for accurate change detection. However, due to the scarcity of existing labeled hyperspectral images, techniques for detecting changes in hyperspectral images are very limited.

[0003] In recent years, techniques employing untrained models have attracted increasing attention. Untrained models do not require learning prior knowledge of real images using labeled samples, representing an unsupervised technique based on the assumption that most existing neural network models, even without training, are generally sufficient to capture key primary features. In light of this, Lempitsky et al. achieved significant performance in image denoising and super-resolution tasks by randomly initializing network parameters. Saha et al. utilized untrained networks to accurately detect changes in high-dimensional images. Their approach involved designing a novel high-dimensional image change detection framework. The network model's parameters were initialized using the He initialization method without training. The model was then used to extract depth features from images before and after the change. To distinguish changed pixels from unchanged pixels, depth change vector analysis was used to compare the extracted features, obtaining depth change hypervectors. Finally, clustering and thresholding were used to classify pixels into change types. Due to the relatively low spatial resolution of hyperspectral images (lower spatial complexity compared to high-resolution images) and the lack of labeled images, untrained models are well-suited for hyperspectral image change detection. However, existing untrained deep learning hyperspectral image change detection techniques use convolutional operations with fixed-grid sampling, only acquiring contextual information within a rule window. Numerous studies have shown that pixels outside the regular window convey crucial information about feature representation. Whether this information can further improve the performance of unsupervised hyperspectral image change detection has never been explored.

[0004] In summary, (1) most deep learning change detection techniques are developed for high-resolution images and require a large number of labeled samples to train the model, which may not be able to handle medium- and low-resolution data such as hyperspectral images well. (2) Due to the lack of labeled hyperspectral images, deep learning change detection techniques for hyperspectral images are very limited. (3) Existing untrained deep learning techniques that do not require labeled samples use convolution operations with fixed grid sampling, ignoring information that is beneficial for change detection, such as the shape and size of ground features represented by pixels outside the regular rectangular window. Summary of the Invention

[0005] The purpose of this invention is to provide an unsupervised, untrained hyperspectral image change detection method to solve the problems existing in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: an unsupervised, untrained hyperspectral image change detection method, wherein the unsupervised, untrained hyperspectral image change detection method includes a multi-expansion feature extraction module, a deformable feature extraction module, and a depth feature comparison module.

[0007] The steps of the unsupervised, untrained hyperspectral image change detection method are as follows:

[0008] Step 1: First, use a random initialization method to assign parameters to each layer of the entire network. Then, use a weight-sharing dual-branch network to extract depth features from the hyperspectral images before and after the change.

[0009] Step 2: The dual-branch deep learning network consists of one or more multi-expansion feature extraction and deformable feature extraction modules to obtain multi-scale depth spatial-spectral features at irregular sampling locations;

[0010] Step 3: Finally, the above features are input into the depth feature comparison module to obtain the discriminative difference features between the hyperspectral images before and after the change, and the changes of pixels representing ground features are classified by the threshold segmentation method.

[0011] As a preferred technical solution of the present invention, in step one, before the hyperspectral image is input into the MDFE module, it is first projected to a high-dimensional feature space through convolution operation. At this time, more non-training parameters are randomly initialized for feature extraction by the convolutional layer.

[0012] As a preferred embodiment of the present invention, the multi-dilated feature extraction module utilizes two branches of a dilated convolutional layer to extract local features with different receptive fields. Dilated convolution is a powerful tool for extracting local features within a relatively large receptive field; compared to conventional convolution, it can exponentially increase the receptive field while the parameters only increase linearly. The outputs of the two branches of the dilated convolutional layer are concatenated along the channel dimension and input into the next module to form the multi-dilated feature extraction.

[0013] As a preferred embodiment of the present invention, the deformable feature extraction module is capable of extracting local features from non-fixed grid sampling regions. This design is based on the idea of ​​using additional offsets to increase sample positions. It utilizes conventional convolution to generate positional offsets, and then calculates the convolutional features of the non-fixed grid sampling regions to fully leverage information about changing surface structures that helps in accurately detecting these changes.

[0014] As a preferred embodiment of the present invention, the depth feature comparison module identifies changes by analyzing and comparing the depth features extracted from hyperspectral images before and after the change using a multi-expansion feature extraction module and a deformable feature extraction module. It employs a feature selection scheme based on spatial segmentation and maximum variance. From the difference features between the depth features of the images before and after the change, some discriminative feature channels are selected based on their standard deviation, ranked from highest to lowest, thus avoiding redundancy in high-dimensional features. Then, the differences between the selected feature channels are quantified using Euclidean norms, and a threshold segmentation method is used to analyze pixel changes.

[0015] As a preferred embodiment of the present invention, in step three, considering that the selected discriminative features contribute differently to the final change detection result, this method assumes that feature channels with greater differences can provide more reliable change information than feature channels with smaller differences. A weighted strategy is employed in the Euclidean norm to further quantify the differences in image depth features before and after the change. By emphasizing the contribution of important features, the model accuracy and adaptability to different scenarios are improved.

[0016] Compared with the prior art, the beneficial effects of the present invention are: the present invention provides an unsupervised, untrained hyperspectral image change detection method.

[0017] (1) Given the relatively low spatial resolution of hyperspectral images and the lack of labeled samples, deformable untrained networks are applied to hyperspectral image change detection technology to make full use of structural information such as the shape and size of ground objects to achieve high-precision fully unsupervised change detection.

[0018] (2) The combined module of MDFE and DFE enables the network to extract local features in a larger receptive field and at any adjacent location, which is beneficial for improving change detection performance and can be easily embedded into other technologies.

[0019] (3) Based on the importance of features, a simple and effective scheme to improve feature selection is proposed. By emphasizing the contribution of important features, the model accuracy and adaptability in different scenarios are improved. Attached Figure Description

[0020] Figure 1 This is the network framework of the technical solution of the present invention; Detailed Implementation

[0021] The technical solutions in the embodiments of the present invention have been clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0022] Please see Figure 1 This invention provides an unsupervised, untrained hyperspectral image change detection method, which includes a multi-expansion feature extraction module, a deformable feature extraction module, and a depth feature comparison module.

[0023] The steps of the unsupervised, untrained hyperspectral image change detection method are as follows:

[0024] Step 1: First, use a random initialization method to assign parameters to each layer of the entire network. Then, use a weight-sharing dual-branch network to extract depth features from the hyperspectral images before and after the change.

[0025] Step 2: The dual-branch deep learning network contains one or more combined modules of multi-expansion feature extraction and deformable feature extraction to obtain multi-scale depth spatial-spectral features at irregular sampling locations;

[0026] Step 3: Finally, the above features are input into the depth feature comparison module to obtain the discriminative difference features between the hyperspectral images before and after the change, and the changes of pixels representing ground features are classified by the threshold segmentation method.

[0027] like Figure 1 As shown, in step one, before the hyperspectral image is input into the MDFE module, it is first projected into a high-dimensional feature space through convolution. At this point, more non-trained parameters are randomly initialized for feature extraction by the convolutional layers, which helps increase the probability of generating discriminative features, thereby improving change detection accuracy. These features are then further processed by the MDFE and DFE modules to extract deep features. The combination of MDFE and DFE enables the network to extract local features in a larger receptive field (sampling area) and at any adjacent location. This combination can be considered a single combined module, and multiple combined modules can be stacked together. Furthermore, dense connection technology is applied between modules to further enhance local feature propagation and extract more reliable local features. After the combined MDFE and DFE module, convolutional layers are used to reduce the dimensionality of the feature channel number, thereby reducing computer memory requirements. Throughout the technical framework, the number of feature channels undergoes several transformations. However, increasing the number of feature channels may also increase the randomness of unreliable feature generation, thereby reducing the accuracy of the feature selection scheme. In this invention, the number of feature channels can be flexibly set to achieve a good balance between accuracy and randomness, meeting different application requirements.

[0028] The multi-dilated feature extraction module utilizes two branches of a dilated convolutional layer to extract local features with different receptive fields. Dilated convolution is a powerful tool for extracting local features within a relatively large receptive field. Compared to conventional convolution, it can exponentially increase the receptive field while the parameters only increase linearly. The dilated convolution operator is defined as follows:

[0029]

[0030] Where * represents the convolution operator, Represents the input features, y(x) l ) is the size determined by Convert to The output features of dilated convolution. This is achieved through a window size k and a dilation factor r. For a dilation factor r, the receptive field of a window size k is enlarged to k + (k-1)(r-1). When r = 1, the dilated convolution corresponds to a regular convolution. In this invention, the dilation factors of the two branches are set to 1 and 2, corresponding to 3×3 and 5×5 receptive fields, respectively. The outputs of the two branches of the dilated convolution layer are concatenated along the channel dimension and input into the next module, forming a multi-dilated feature extraction.

[0031] Furthermore, the deformable feature extraction module extracts local features from non-fixed grid sampling regions. This design is based on the idea of ​​using additional offsets to increase the sample position. For input feature f in In conventional convolution, the sampling positions fixed on a two-dimensional window (of size N×N) are represented as follows:

[0032] R={(0,0),(0,0),…,(N-1,N-1)}

[0033] Conventional convolution only covers a regular rectangular window and cannot cover other areas outside the window, which, because they convey surface structure information, also help in accurately detecting changes. To address this issue, this invention adds two steps: 1) randomly generating position offsets, and 2) convolution calculation of deformable features. In the design of the DFE module, the first step uses conventional convolution to generate position offsets and randomly initializes the network parameters using the He normalization method. The channel size of the offset is twice the input feature channel size, corresponding to a two-dimensional offset. The second step calculates the output feature f at position p0. out (p0):

[0034]

[0035] Where ω represents the enumeration position p in R. n The weights. The offset is {Δp}. n |n=1,2,…,N},N=|R|。Offset Δp n The values ​​are typically irregular and decimal. Therefore, a bilinear interpolation operation is performed to project the offset onto the full spatial location, as shown below:

[0036]

[0037] Where p represents p0 + p n +Δp n q represents the input feature f in The spatial location is given by ρ(a,b) = max(0,1-|ab|).

[0038] Furthermore, the depth feature comparison module identifies changes by comparing and analyzing the depth features extracted from the hyperspectral images before and after the change using the multi-expansion feature extraction and deformable feature extraction modules. Since the untrained network parameters are shared in the two branches, unchanged pixels should show similar feature representations, while changed pixels are transformed into completely different feature representations.

[0039] In step three, given the high dimensionality of hyperspectral images, the multi-channel depth features obtained through MDFE and DFE may be redundant. To extract the most discriminative change information, a feature selection scheme based on spatial segmentation and maximum variance is adopted. Among the difference features between the depth features of the images before and after the change, the top 20% of feature channels with the highest to lowest standard deviation are selected as discriminative feature channels. Then, given F1 and F2 as the selected feature channels of the depth features of the images before and after the change, respectively, for pixel (i,j), the difference feature (D(i,j)) between the selected feature channels can be quantified using the Euclidean law:

[0040]

[0041] Among them, C s To select feature channels, the conventional Euclidean norm in the formula assumes that each feature channel is equally important, which may limit its performance because different channels contribute differently to the final change detection result. This invention assumes that feature channels with greater differences provide more reliable change information than those with less differences, and proposes a weighting strategy to add it to the calculation of the image depth difference feature (D(i,j)) before and after the change:

[0042]

[0043]

[0044] Where std(·) represents the standard deviation of the eigenvalues ​​in a given feature channel, and D(i,j) will be C s The feature of the dimension is converted into a value representing the change information of pixel (i,j).

[0045] Finally, for the obtained image depth difference features D(i,j) before and after the change, the optimal threshold δ is generated using the Otsu method. Pixels with D(i,j)>δ are classified as changed pixels, while other pixels are classified as unchanged pixels.

[0046] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An unsupervised, untrained hyperspectral image change detection method, characterized in that, The unsupervised, untrained hyperspectral image change detection method includes a multi-dilation feature extraction (MDFE) module, a deformable feature extraction (DFE) module, and a deep feature comparison (DFC) module. The steps of the unsupervised, untrained hyperspectral image change detection method are as follows: Step 1: First, use a random initialization method to assign parameters to each layer of the entire network. Then, use a weight-sharing dual-branch network to extract depth features from the hyperspectral images before and after the change. Step 2: The dual-branch deep learning network contains one or more combined modules of multi-expansion feature extraction and deformable feature extraction to obtain multi-scale depth spatial-spectral features at irregular sampling locations; Step 3: Finally, the features obtained in Step 2 are input into the depth feature comparison module to obtain the discriminative difference features between the hyperspectral images before and after the change, and the change of pixels representing ground features is classified by the threshold segmentation method. The multi-dilated feature extraction module uses two branches of the dilated convolutional layer to extract local features with different receptive fields. Dilated convolution is a powerful tool for extracting local features in a relatively large receptive field. Compared with conventional convolution, it can exponentially increase the receptive field while the parameters only increase linearly. The outputs of the two branches of the dilated convolutional layer are connected together along the channel dimension and input into the next module to form multi-dilated feature extraction. The deformable feature extraction module can extract local features from non-fixed grid sampling areas. Based on adding sample positions with additional offsets, it generates position offsets using conventional convolution. Then, by calculating the convolution features of non-fixed grid sampling areas, it makes full use of surface structure information that helps to accurately detect changes. The depth feature comparison module identifies changes by analyzing and comparing the depth features extracted from hyperspectral images before and after the change using the multi-expansion feature extraction module and the deformable feature extraction module. It adopts a feature selection scheme based on spatial segmentation and maximum variance. Among the difference features between the depth features of the images before and after the change, some discriminative feature channels are selected according to the standard deviation from high to low, avoiding high-dimensional feature redundancy. Then, the differences between the selected feature channels are quantified by Euclidean norm, and the pixel changes are analyzed using a threshold segmentation method.

2. The unsupervised, non-training hyperspectral image change detection method according to claim 1, characterized in that: In step one, before the hyperspectral image is input into the MDFE module, it is first projected into a high-dimensional feature space through convolution operation. At this time, more non-training parameters are randomly initialized for feature extraction by the convolutional layer.

3. The unsupervised, non-training hyperspectral image change detection method according to claim 1, characterized in that: The improved feature selection scheme enhances model accuracy and adaptability across different scenarios by emphasizing the contribution of important features. Considering that the selected discriminative features contribute differently to the final change detection results, it is assumed that feature channels with greater differences can provide more reliable change information than feature channels with smaller differences. A weighting strategy is adopted in the Euclidean norm to further quantify the differences in image depth features before and after the change.

Images