Hyperspectral image change detection method based on change guidance

Abstract

The invention discloses a hyperspectral image change detection method based on change guidance, and belongs to the field of artificial intelligence. The method comprises the following steps: extracting features from a double-time-phase hyperspectral image by using a step-by-step extraction strategy; the ConvGRU module extracts difference information of the double-time-phase hyperspectral image in different stages; the difference information of each stage is used for generating a change weight so as to guide feature extraction of the next stage; and after the detection results of different stages are fused, a final detection result is obtained through a decision-making layer. According to the invention, the precision of hyperspectral image change detection can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a hyperspectral image change detection method. Background Art

[0002] In recent years, hyperspectral images have attracted wide attention in the field of earth observation because of their description of the spatial distribution of ground objects and their rich spectral information. Change detection algorithms based on hyperspectral images have become one of the hot topics in hyperspectral images. Today, hyperspectral image change detection has been widely used in land surveys, urban research, ecosystem monitoring, disaster monitoring and assessment, and military reconnaissance.

[0003] In the past few decades, a series of machine learning methods based on manual feature extraction have been widely proposed for change detection in hyperspectral images. Among them, transformation vector analysis (CVA) is an effective means for change detection, which allows us to obtain difference maps by calculating the difference of each pixel in the dual-phase image one by one. Another type of traditional change detection method is based on image transformation, such as principal component analysis (PCA), multivariate change detection (MAD), and iterative reweighted MAD (IR-MAD). These methods can facilitate the distinction between changed and unchanged areas by mapping the features in the original high-dimensional space into a new low-dimensional subspace representation. In recent years, with the popularity of deep learning, some neural networks, such as convolutional neural networks (CNN), have been widely used in hyperspectral image change detection tasks and achieved good detection accuracy.

[0004] Although the effectiveness of the above algorithms in hyperspectral image change detection has been proven, most of them rely on deep convolutional neural networks to extract hyperspectral images with low spatial resolution. The downsampling operation will further reduce the spatial resolution. On the other hand, the semantic gap between shallow and deep features will limit the effective use of features at different stages, and the impact of the difference information of dual-phase images on feature extraction is rarely considered. Summary of the invention

[0005] Purpose of the invention: In order to overcome the shortcomings of existing hyperspectral image change detection algorithms, a hyperspectral image change detection method based on change guidance is provided, which can effectively solve the shortcoming of insufficient utilization of difference information in hyperspectral image change detection and thus improve the algorithm performance.

[0006] Technical solution: To achieve the above object, the present invention provides a hyperspectral image change detection method based on change guidance, comprising the following steps:

[0007] S1: Extract features from dual-phase hyperspectral images using a step-by-step extraction strategy;

[0008] S2: ConvGRU module extracts the difference information of dual-phase hyperspectral images at different stages;

[0009] S3: The difference information at each stage is used to generate variation weights to guide feature extraction at the next stage;

[0010] S4: The detection results at different stages are integrated and passed through the decision layer to obtain the final detection result;

[0011] Furthermore, in step S1, a step-by-step extraction strategy is used to extract features from the dual-phase hyperspectral image, which is described by the formula:

[0012]

[0013] in Represents the spectral features extracted at the previous level. Conv3D means that a 3D convolution with a structure of 1×1×3 is used to extract the inter-band correlation features of the input cube in the spectral dimension. Represents features extracted based on the previous level features.

[0014] Furthermore, the ConvGRU module in step S2 extracts the difference information of the dual-phase hyperspectral image at different stages, which is described by the formula:

[0015]

[0016] and represents the feature maps from two images at the i-th stage, CGB represents the ConvGRU module, and D i Represents the spatial difference information obtained in the i-th stage.

[0017] Furthermore, the difference information of each stage in step S3 is used to generate change weights to guide feature extraction in the next stage, and the specific implementation method is as follows:

[0018] A1: The change information obtained by CGB at different stages is fed back to the spectral feature network before the next stage of spectral feature extraction, and the difference information D i Will be mapped by ReLU into a change guide weight matrix W i DG , described by the formula:

[0019] W i DG =δ(D i )

[0020] Here, δ(·) represents the ReLU function.

[0021] A2: Then, Wi DG Respectively and Point product to get the emphasized feature map and Right now

[0022]

[0023]

[0024] A3: To facilitate model fitting, attention feature map and Add back via skip connections and Described by formula:

[0025]

[0026]

[0027] in and It will be used as the input for feature extraction in the i+1th stage.

[0028] Furthermore, in step S4, the detection results of different stages are integrated and passed through the decision layer to obtain the final detection result, which is described by the formula:

[0029] V d =σ(FC(GAP([D1, D2, D3, D4])))

[0030] where operations FC(·), GAP(·), and [·,·] represent the fully connected layer, global average pooling layer, and concatenation, respectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0032] Figure 1 A schematic diagram of a method flow chart provided for an example of the present invention;

[0033] Figure 2 A schematic diagram of a change guidance model provided for an example of the present invention;

[0034] Figure 3 A schematic diagram of a ConvGRU module provided for an example of the present invention;

[0035] Figure 4 A schematic diagram of a feature enhancement with varying weights provided for an example of the present invention;

[0036] Figure 5 It is the pseudo color image of Farmland and River hyperspectral remote sensing images and the true value image of the corresponding changed pixels in the simulation experiment of the present invention;

[0037] Figure 6 The diagram is a graph showing the change detection results of the method of the present invention and other methods on the Farmland and River data sets in the simulation experiment of the present invention; DETAILED DESCRIPTION

[0038] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0039] like Figure 1 As shown, the present invention provides a hyperspectral image change detection method based on change guidance, comprising the following steps:

[0040] S1: Extract features from dual-phase hyperspectral images using a step-by-step extraction strategy;

[0041] S2: ConvGRU module extracts the difference information of dual-phase hyperspectral images at different stages;

[0042] S3: The difference information at each stage is used to generate variation weights to guide feature extraction at the next stage;

[0043] S4: The detection results at different stages are integrated and passed through the decision layer to obtain the final detection result;

[0044] In step S1 of this embodiment, a step-by-step extraction strategy is used to extract features from the dual-phase hyperspectral image, and the specific implementation method is as follows:

[0045] Extract hyperspectral image features step by step, such as Figure 2 As shown in the figure, the two main branches use a 3D convolution with a structure of 1×1×3 to extract the inter-band correlation features of the input cube in the spectral dimension, and the branch uses a 3×3 2D convolution to extract spatial features, which can be described by the formula:

[0046]

[0047] in Represents the spectral features extracted at the previous level. Conv3D means that a 3D convolution with a structure of 1×1×3 is used to extract the inter-band correlation features of the input cube in the spectral dimension. Represents features extracted based on the previous level features.

[0048] In this embodiment, the ConvGRU module in step S2 extracts the difference information of the dual-phase hyperspectral image at different stages, which is described by the formula:

[0049]

[0050] ConvGRU module Figure 3 As shown, and represents the feature maps from two images at the i-th stage, CGB represents the ConvGRU module, and D i Represents the spatial difference information obtained in the i-th stage.

[0051] In this embodiment, the difference information of each stage in step S3 is used to generate change weights to guide the feature extraction of the next stage, and the specific implementation method is as follows:

[0052] The change information obtained by CGB at different stages is fed back to the spectral feature network before the next stage of spectral feature extraction, and the difference information D i Will be mapped by ReLU into a change guide weight matrix Described by formula: W i DG =δ(D i ), where δ(·) represents the ReLU function. Then, W i DG Respectively and Point product to get the emphasized feature map and Right now The schematic diagram of the changing weight enhancement feature is as follows Figure 4 As shown. Finally, Figure 2 As shown, in order to facilitate the fitting of the model, the attention feature map and Add back via skip connections and Described by formula: in and It will be used as the input for feature extraction in the i+1th stage.

[0053] like Figure 2As shown, in step S4 of this embodiment, the detection results of different stages are integrated and passed through the decision layer to obtain the final detection result, which is described by the formula:

[0054] V d =σ(FC(GAP([D1, D2, D3, D4])))

[0055] where operations FC(·), GAP(·), and [·,·] represent the fully connected layer, global average pooling layer, and concatenation, respectively.

[0056] Based on the above scheme, comparative experiments are conducted to verify the beneficial effects of the method of the present invention. The method of the present invention is verified using the Farmland and River datasets. Figure 5 The pseudo-color images of the hyperspectral remote sensing images of Farmland and River and the true value images of the corresponding changed pixels are displayed. The transformation vector analysis CVA (method 1), principal component analysis PCA-CVA (method 2), recursive three-dimensional fully convolutional neural network Re3FCN (method 3) and change tensor three-way decomposition method TDSSC (method 4) are used to conduct comparative experiments with the method of the present invention. Among them, the method of the present invention needs to set the size of the training samples and test samples of each data set, and the experimental parameter settings are shown in Table 1:

[0057] Table 1 Experimental parameter settings

[0058]

[0059] Implementation Results

[0060] In the method of the present invention, the overall accuracy (OA) and Kappa coefficient (Kappa Coefficient, Kappa) are used to evaluate the performance of each method. Table 2 shows the OA value and Kappa coefficient comparison between the method of the present invention and other methods. Figure 6 The graphs show the change detection results using the method of the present invention and other methods on the Farmland and River datasets.

[0061] Table 2 Comparison of OA values ​​and Kappa coefficients of the method of the present invention and other methods

[0062]

[0063] According to the data in Table 2, the OA value and Kappa coefficient of the method of the present invention are higher than those of other methods, indicating that the detection accuracy of the method of the present invention is higher. Figure 6It can be seen from the detection image that the method of the present invention obtains a smoother and cleaner detection image compared with other algorithms, and is closest to the true value image, which shows that the method of the present invention can effectively use the difference information to guide feature extraction, gradually strengthen the model's extraction of change information, and make the model's recognition of the change area more accurate.

Claims

1. A hyperspectral image change detection method based on change guidance, characterized in that: The steps include: S1: Extract features from dual-phase hyperspectral images separately using a step-by-step extraction strategy; S2: ConvGRU module extracts the difference information of dual-phase hyperspectral images at different stages; S3: The difference information at each stage is used to generate variation weights to guide feature extraction at the next stage; S4: The detection results at different stages are integrated and passed through the decision layer to obtain the final detection result.

2. The hyperspectral image change detection method according to claim 1, characterized in that: In step S1, a step-by-step extraction strategy is used to extract features from the dual-phase hyperspectral image, which is described by the formula: in Represents the spectral features extracted at the previous level. Conv3D means that a 3D convolution with a structure of 1×1×3 is used to extract the inter-band correlation features of the input cube in the spectral dimension. Represents features extracted based on the previous level features.

3. The hyperspectral image change detection method according to claim 1, characterized in that: In step S2, the ConvGRU module extracts the difference information of the dual-phase hyperspectral image at different stages, which is described by the formula: and represents the feature maps from two images at the i-th stage, CGB represents the ConvGRU module, and D i Represents the spatial difference information obtained in the i-th stage.

4. The hyperspectral image change detection method according to claim 1, characterized in that: The difference information of each stage in step S3 is used to generate change weights to guide feature extraction in the next stage, and the specific implementation method is as follows: A1: The change information obtained by CGB at different stages is fed back to the spectral feature network before the next stage of spectral feature extraction, and the difference information D i Will be mapped by ReLU into a change guide weight matrix Described by formula: Here, δ(·) represents the ReLU function. A2: Then, Respectively and Point product to get the emphasized feature map and Right now A3: To facilitate model fitting, attention feature map and Add back via skip connections and Described by formula: in and It will be used as the input for feature extraction in the i+1th stage.

5. The hyperspectral image change detection method according to claim 1, characterized in that: In step S4, the detection results of different stages are integrated and passed through the decision layer to obtain the final detection result, which is described by the formula: V d =σ(FC(GAP([D1,D2,D3,D4]))) where operations FC(·), GAP(·), and [·,·] represent the fully connected layer, global average pooling layer, and concatenation, respectively.

Images