Hyperspectral image change detection method based on multi-scale convolution sparse coding

Abstract

The invention discloses a hyperspectral image change detection method based on multi-scale convolution sparse coding, and the method comprises the steps: inputting a dual-time-phase hyperspectral image into a deep interpretable network guided by the multi-scale convolution sparse coding, extracting multi-scale spatial spectrum features, reducing feature redundancy at the same time, and enhancing the interpretability of deep multi-scale feature extraction; putting forward a change-sensitive teacher-student self-distillation learning normal form, aligning dual-temporal features and difference features, and performing two-stage training on the network in combination with supervised learning; and extracting multi-scale features in the dual-temporal hyperspectral image by using the trained depth interpretable network, combining the multi-scale features with the dual-temporal hyperspectral image to obtain a difference feature map, and sending the difference feature map into a classifier to obtain a final change detection result. According to the method, the interpretability of the multi-scale feature extraction process in the deep neural network is improved, the dependence on labeled data is reduced, meanwhile, the sensitivity of a network model to changes is enhanced, and the generalization performance and the change detection precision of the model are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of hyperspectral remote sensing image change detection, and in particular relates to a hyperspectral image change detection method based on multi-scale convolutional sparse coding. Background Art

[0002] As a means of detection and identification, remote sensing technology collects surface information in a non-contact manner through sensors on aircraft or satellites to capture various surface features. Among them, change detection technology analyzes images of the same geographic area acquired at different times to identify and quantify changes in surface features. This technology has wide application value in urban planning, natural disaster assessment, and environmental protection. In recent years, hyperspectral images have become an important focus of change detection research because they can capture a wider spectral range than other remote sensing images and provide richer and more detailed spatial and spectral information.

[0003] Early change detection research includes image algebra, transformation-based, and classification-based methods. Change vector analysis (CVA) is a typical image algebra method that uses the Euclidean distance between pixels to identify changes, but it is sensitive to noise because it ignores spatial context. Transformation-based methods include principal component analysis (PCA), slow feature analysis (SFA), and iterative weighted multivariate change detection (IR-MAD). PCA concentrates important information in a few principal components, SFA extracts time-invariant components to enhance change detection, and IR-MAD uses a weighted method based on chi-square distance to maximize feature difference variance and focus on unchanged pixels. However, these methods may lose useful information during the dimensionality reduction process. Classification-based techniques include post-classification comparison and composite classification. Post-classification comparison classifies the two images separately and compares the results, while composite classification directly classifies the superimposed dual-phase hyperspectral images. Both of them rely heavily on classifier design and face the problem of cumulative or omitted errors.

[0004] In recent years, deep learning has been widely used in the fields of computer vision and remote sensing. In change detection, deep learning-based methods have dominated, and a series of neural network models such as GETNET, ReCNN, and ML-EDAN have emerged. Although these models show excellent detection performance, their design is empirical, without considering the prior characteristics of hyperspectral images, lacking clear mathematical theoretical guidance, and their "black box" nature limits the improvement and development of the model, making it difficult to extract more essential features, resulting in limited generalization performance of the network model.

[0005] In addition, deep learning techniques usually require a large amount of labeled data, which is unrealistic when processing complex hyperspectral images. When there is limited labeled data, how to effectively use unlabeled data for learning while enhancing the model's sensitivity to changes is an important challenge facing the hyperspectral image change detection task.

[0006] In summary, the current methods have the following main defects: (1) Traditional change detection methods rely on handcrafted feature extractors, which can only extract shallow features and are easily affected by noise. (2) The network architecture of deep learning methods is usually designed subjectively, lacks clear theoretical guidance, and is difficult to intuitively understand its internal mechanism, resulting in limited generalization performance. (3) Existing deep learning methods are highly dependent on labeled data, and lack solutions to effectively utilize a large amount of unlabeled data and enhance the model's sensitivity to changes. Summary of the invention

[0007] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a hyperspectral image change detection method based on multi-scale convolutional sparse coding, which not only enhances the interpretability of deep multi-scale feature extraction, but also reduces the annotation dependence of the network model, thereby effectively utilizing a large amount of unlabeled data while enhancing the sensitivity to changes.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is:

[0009] The hyperspectral image change detection method based on multi-scale convolutional sparse coding includes the following steps:

[0010] S101: Input a small number of labeled and a large number of unlabeled dual-phase hyperspectral images. Each image undergoes three convolutions and downsamplings in sequence to obtain feature maps x of different scales corresponding to each convolution and downsampling. 1 、x 2 、x 3 ;

[0011] S102: construct an iterative optimization update algorithm for a multi-scale convolutional sparse coding model;

[0012] S103: constructing a shared sparse coefficient update module and a private sparse coefficient update module based on an iterative optimization update algorithm of a multi-scale convolutional sparse coding model, and unfolding them into a deep interpretable neural network, and using two deep interpretable neural networks with the same structure but different weights as a student network and a teacher network respectively;

[0013] S104: Input three feature maps x of different scales 1 、x 2 、x 3 , repeatedly and alternately updating the shared sparse coefficient feature s in the forward propagation of the network sand private sparse coefficient features After multiple iterations, the fused sparse coefficient feature s is obtained f ;

[0014] S105: Input the sparse coefficient features of the student network into the classifier and use a small amount of labeled data to perform the first stage of supervised training;

[0015] S106: aligning the phase feature representation and the difference feature representation of the student network and the teacher network respectively, and performing mixed training using a small amount of labeled data and a large amount of unlabeled data;

[0016] S107: Input the dual-phase hyperspectral image into the trained student network to obtain the final change detection result.

[0017] As a further technical solution of the present invention, step S101 is specifically as follows:

[0018] In order to reduce the computational complexity, a small number of annotated and a large number of unannotated dual-phase hyperspectral images are segmented into overlapping input image blocks, and feature maps x of different scales are generated through three convolutions and downsampling. 1 、x 2 、x 3 , in order to fully consider the diversity and complexity of the size of the changing region. At the same time, in order to alleviate the information loss caused by downsampling, residual connections with convolutional downsampling are added between feature maps of different scales:

[0019] x 1 =ReLU(ReLU(Conv(x))+Conv(x))

[0020] x 2 =ReLU(ReLU(Conv(x 1 ))+Conv(x 1 ))

[0021] x 3 =ReLU(ReLU(Conv(x 2 ))+Conv(x 2 ))

[0022] Among them, Conv(·) represents the convolution downsampling operation, ReLU(·) represents the rectified linear unit activation function, is the input image block, C, h, w represent the number of bands, height and width respectively, thus obtaining feature maps of different scales As input to a deep interpretable network.

[0023] As a further technical solution of the present invention, step S102 is specifically as follows:

[0024] First, we get three features x of different scales. 1 、x 2 、x 3 Finally, the following multi-scale sparse coding model is constructed to capture the shared sparse coefficients and private sparse coefficients between different scales:

[0025]

[0026]

[0027]

[0028] in, They represent convolution operation, shared sparse coefficient, private sparse coefficient, shared sparse coefficient convolution layer, private sparse coefficient convolution layer, k is the convolution kernel size, P represents the number of channels in the sparse coefficient, the spatial dimension of the shared sparse coefficient and the input features of the intermediate scale. Figure 1 To ensure the balance between high-level semantic information and low-level details, after obtaining the shared sparse coefficients and the private sparse coefficients, they are fused into a comprehensive sparse coefficient to enhance the feature representation ability of the model, thereby improving the robustness of the model:

[0029]

[0030] in, They are fused sparse coefficients, shared sparse coefficient fused convolutional layers, and private sparse coefficient fused convolutional layers. In order to achieve convolutional sparse coding and obtain fused sparse coefficients, the goal is to freeze the convolutional layer when optimizing the objective function of the following sparse coefficients:

[0031]

[0032] Among them, λ is used as a regularization parameter to control the sparsity of the sparse coefficients. In order to enrich the spatial information in the sparse coefficients, based on the assumption that there is the same sparse support set between adjacent pixels, a structured row sparse constraint is imposed on them, namely, L 2,1 norm, in order to overcome this multi-objective optimization challenge, the alternating direction method of multipliers (ADMM) is used to solve the optimization problem of the objective function. By introducing auxiliary variables (AVs) and dual variables (DVs), the following augmented Lagrangian function is defined:

[0033]

[0034] Among them, ρ represents the penalty parameter that controls the consistency of constraints, and are auxiliary variables for shared and private sparse coefficients, and Denotes the dual variable of shared and private sparse coefficients, and F denotes the Frobenius norm expanded to a third-order tensor.

[0035] In order to optimize the augmented Lagrangian function, the ADMM algorithm is used to decompose it into six tractable sub-problems.

[0036] The specific iterative update process is as follows:

[0037] Update s: The subproblem of private sparse coefficients and its update process are as follows:

[0038]

[0039]

[0040]

[0041] Among them, α is the step size parameter that adjusts the update amplitude, represents the transposed convolution operation, and k represents the kth iteration.

[0042] Similarly, the subproblem of sharing sparse coefficients and its update process are as follows:

[0043]

[0044]

[0045]

[0046] in, Also represents the transposed convolution operation.

[0047] Update z: The subproblem of private auxiliary variables and their update process are as follows:

[0048]

[0049]

[0050] Similarly, the subproblem of sharing auxiliary variables and their update process are as follows:

[0051]

[0052]

[0053] Update μ: The update process of shared and private dual variables is as follows:

[0054]

[0055]

[0056] After K+1 iterations, the shared and private sparse coefficients are combined to obtain the fused sparse coefficients.

[0057] As a further technical solution of the present invention, step S103 is specifically as follows:

[0058] According to the derived multi-scale convolutional sparse coding model iterative update algorithm, a shared sparse coefficient update module and a private sparse coefficient update module are constructed; the information flow and internal working mechanism of these two modules are transparent and traceable, and each operation is model-driven and strictly aligned with the iterative algorithm to ensure that the output shared sparse coefficient feature s s and private sparse coefficient features It has intuitive physical meaning. According to the derived iterative optimization update algorithm, the convolution operators in the two update modules are replaced by the existing CNN layers, and the two update modules are alternately expanded to construct a deep interpretable network guided by multi-scale convolutional sparse coding.

[0059] Based on this, two networks with the same structure but different weights are constructed as the student network and the teacher network, which are used to extract the multi-scale feature representation of the student and the multi-scale feature representation of the teacher respectively.

[0060] As a further technical solution of the present invention, the step S104 is specifically as follows:

[0061] First, the three different scale feature maps of the hyperspectral image are input into the private sparse coefficient update module to extract and update the private sparse coefficient features. This captures private structural information and scale-dependent features at different scales, reduces redundancy, and accurately represents the key features in the input feature map.

[0062] After getting the updated private sparse coefficient feature Combine them with the feature map x 1 、x 2 、x 3 Input into the shared sparse coefficient update module together to extract and update the shared sparse coefficient feature s s , capturing general texture and structural patterns across scales;

[0063] After multiple iterations, the shared sparse coefficient feature s s and private sparse coefficient features Combine to get the fused sparse coefficient feature s f Output of a deep interpretable network guided by multi-scale convolutional sparse coding.

[0064] As a further technical solution of the present invention, step S105 is specifically as follows:

[0065] In the first stage of training, a small amount of labeled data is used to input the student network, and the obtained coefficient features are sent to the fully connected classifier to realize the supervised training of the binary classification of changes and invariants. Among them, the focal loss is used to guide the first stage of network training:

[0066]

[0067]

[0068]

[0069] Among them, B represents the batch size, y b is the true label of the b-th sample, is the weight factor for dealing with class imbalance, p b is the predicted probability. In addition, the focusing parameter γ f and modulation factor The contribution of easy-to-classify samples to the loss is reduced. In this way, the student network undergoes an effective “warm-up” phase, ensuring a robust initialization for subsequent training phases.

[0070] As a further technical solution of the present invention, step S106 is specifically as follows:

[0071] In the second phase of training, the weights of the student network are updated via standard gradient backpropagation, while the weights of the teacher network stop their extractive backpropagation and are updated using the exponential moving average (EMA) of the student network weights:

[0072] θ t ←βθ t +(1-β)θ s

[0073] Among them, θ t represents the weight of the teacher network, θ s represents the weight of the student network, and β is a smoothing factor that controls the update speed. In this way, the teacher network can slow down the random fluctuations of the student network and make the training process more stable.

[0074] By inputting a large amount of unlabeled data into the teacher and student networks, and inputting their phase features and difference features into the multi-layer perceptron respectively, projecting them into the same feature space, and combining L 2 Norm normalization and weight normalization ensure that features can be effectively aligned and uniformly optimized, thereby achieving self-supervised training in the teacher-student self-distillation paradigm. In addition, the centralization operation in the teacher network is used to prevent the network from crashing or degrading during training, ensuring the stability of training. The specific loss function of this process is as follows:

[0075] Lssl =ψL u +(1-ψ)L d

[0076]

[0077]

[0078]

[0079] in Represent the temporal feature representation of students and teachers respectively, O is the dimension of feature representation, and represents the difference feature representation between the student and the teacher, sim(·,·) represents the cosine similarity function, and γ is the penalty factor. Through the self-supervised training of the teacher-student self-distillation paradigm, useful feature representations can be learned from a large amount of unlabeled data, enhancing the network's sensitivity to changing information while also learning invariant information;

[0080] In addition, a small amount of labeled data is fed into the student network for supervised training. This hybrid training method ensures that the weights of the teacher network and the student network can be updated in the right direction.

[0081] As a further technical solution of the present invention, step S107 is specifically as follows:

[0082] After the initial supervised training in the first phase and the mixed training in the second phase, the trained student network and fully connected classifier are extracted and the parameters are frozen. The dual-phase hyperspectral image to be detected is input into the student network to obtain the coefficient features of the two phases. The difference features between the two are sent to the classifier to obtain the final change detection result:

[0083] D=G(M(X T1 )-M(X T2 ))

[0084] Among them, X T1 and X T2 represents the input dual-phase hyperspectral image, M(·) represents the deep interpretable network guided by multi-scale convolutional sparse coding, G(·) represents the fully connected classifier, and D is the final change map obtained.

[0085] Beneficial effects of the present invention:

[0086] First, the present invention designs a multi-scale convolutional sparse coding model that can efficiently capture the shared sparse coefficients and private sparse coefficients of the input image at different scales. While extracting rich multi-scale information, it significantly reduces feature redundancy and makes the feature distribution more compact.

[0087] The present invention combines a multi-scale convolutional sparse coding model with deep learning and designs a model-guided deep interpretable network, which not only enhances the interpretability of feature extraction, but also significantly improves the generalization performance and detection accuracy of the model.

[0088] The present invention designs a two-stage training strategy that combines supervised learning and change-sensitive teacher-student self-distillation learning paradigm. In the absence of labeled data, the teacher network is used to guide the student network to learn the changing and invariant features, which significantly reduces the model's dependence on labeled data, fully improves the utilization efficiency of unlabeled data, and effectively enhances the model's sensitivity to changes.

[0089] Second, the hyperspectral image change detection method based on multi-scale convolutional sparse coding proposed in the present invention can not only significantly improve the generalization performance and change detection accuracy of the model technically, but also reduce the dependence on labeled data, and has a wide range of commercial application potential. By reducing the dependence on labeled data, the present invention can still maintain efficient change detection capabilities in data-scarce environments, thereby reducing the cost and time investment of data labeling.

[0090] Existing change detection methods have many limitations in terms of interpretability and efficient feature extraction. The present invention innovatively combines multi-scale convolutional sparse coding with deep learning, significantly improving the performance and interpretability of change detection. By introducing multi-scale convolutional sparse coding, the present invention captures image features at different scales, making the extracted features more compact and rich in physical meaning. Compared with traditional "black box" deep learning models, the model of the present invention has higher interpretability, can clearly explain the physical and mathematical principles in the feature extraction process, and help users better understand the working mechanism of the model. In addition, by organically combining multi-scale convolutional sparse coding with deep learning methods, the present invention not only retains the advantages of sparse coding in feature extraction, but also utilizes the powerful representation ability of deep learning, significantly improving the accuracy and generalization ability of change detection.

[0091] Hyperspectral image change detection methods based on deep learning have always been deficient in terms of feature extraction accuracy, model interpretability and generalization performance, especially the difficulty in balancing model complexity and detection accuracy; in addition, most existing methods rely on a large amount of labeled data, resulting in a significant drop in performance when data is insufficient. The present invention combines multi-scale sparse coding technology with deep learning, which not only significantly improves the accuracy of feature extraction, but also enhances the interpretability of the model. The present invention can capture key features in images at different scales, and effectively reduce feature redundancy through sparse representation, so that the model can maintain efficient and accurate change detection in complex scenarios. At the same time, the present invention also reduces the dependence on labeled data, and can still show excellent detection performance in the absence of labeled data.

[0092] Previous technologies often rely on a single model or a specific algorithm. When faced with complex and changing scenarios, they are prone to problems such as incomplete feature extraction and insufficient model generalization. In addition, traditional methods have a high dependence on data, especially when there is insufficient labeled data, the performance of the model will drop significantly. This dependence and limitation form a technical bias, limiting the scope of application and effect of change detection methods. The present invention combines a multi-scale sparse coding model with deep learning to capture important features in images at different scales, avoiding the bias of traditional methods towards a specific scale or feature, and ensuring the comprehensiveness and accuracy of feature extraction. In addition, the introduction of sparse coding makes the feature representation more compact and reduces redundant information, thereby improving the interpretability and generalization ability of the model. At the same time, the present invention reduces the dependence on a large amount of labeled data through a two-stage semi-supervised learning strategy, so that the model can still maintain excellent detection performance when data is limited. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 It is a flow chart of a hyperspectral image change detection method provided by an embodiment of the present invention.

[0094] Figure 2 It is a schematic diagram of a network structure for extracting feature maps of different scales provided by an embodiment of the present invention.

[0095] Figure 3 It is a schematic diagram of the structure of a shared and private sparse coefficient update module provided in an embodiment of the present invention.

[0096] Figure 4 It is a schematic diagram of a deep interpretable network structure guided by multi-scale convolutional sparse coding provided by an embodiment of the present invention.

[0097] Figure 5 It is a schematic diagram of the student network supervision training structure provided by an embodiment of the present invention.

[0098] Figure 6 It is a schematic diagram of the self-supervised training structure of the teacher-student self-distillation paradigm provided in an embodiment of the present invention.

[0099] Figure 7 These are the visualization results of change detection using different methods provided by the embodiments of the present invention.

[0100] Figure 8 This is the effectiveness experimental result of applying different weights to the teacher-student self-distillation paradigm provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0101] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0102] Reference Figure 1-Figure 8 , a hyperspectral image change detection method based on multi-scale convolutional sparse coding, comprising the following steps; the present invention is described in detail below in conjunction with the accompanying drawings.

[0103] like Figure 1 As shown, a hyperspectral image change detection method based on multi-scale convolutional sparse coding provided by the present invention comprises the following steps:

[0104] S101: Input a small number of labeled and a large number of unlabeled dual-phase hyperspectral images, and after three convolutions and downsampling, obtain the feature maps x of different scales corresponding to each convolution and downsampling. 1 、x 2 、x 3 ;

[0105] S102: construct an iterative optimization update algorithm for a multi-scale convolutional sparse coding model;

[0106] S103: constructing a shared sparse coefficient update module and a private sparse coefficient update module, and expanding them into a deep interpretable neural network, and using two deep interpretable neural networks with the same structure but different weights as a student network and a teacher network respectively;

[0107] S104: Input three different scale feature maps corresponding to the dual-phase hyperspectral image, and repeatedly and alternately update the shared sparse coefficient feature s in the forward propagation of the network. s and private sparse coefficient features After multiple iterations, the fused sparse coefficient feature s is obtained f ;

[0108] S105: Input the sparse coefficient features of the student network into the classifier and use a small amount of labeled data to perform the first stage of supervised training;

[0109] S106: aligning the phase feature representation and the difference feature representation of the student network and the teacher network respectively, and performing mixed training using a small amount of labeled data and a large amount of unlabeled data;

[0110] S107: Input the dual-phase hyperspectral image into the trained student network to obtain the final change detection result.

[0111] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0112] (1) Figure 2 As shown in the figure, a small number of labeled and a large number of unlabeled dual-phase hyperspectral images are input, and after convolution and downsampling, the corresponding three different scale feature maps x are obtained. 1 、x 2 、x 3 ;

[0113] In order to reduce the computational complexity, a small number of annotated and a large number of unannotated dual-phase hyperspectral images are segmented into overlapping input image blocks, and feature maps x of different scales are generated through three convolutions and downsampling. 1 、x 2 、x 3 In order to fully consider the diversity and complexity of the size of the changing region, and at the same time, to alleviate the information loss caused by downsampling, residual connections with convolution downsampling are added between feature maps of different scales:

[0114] x 1 =ReLU(ReLU(Conv(x))+Conv(x))

[0115] x 2 =ReLU(ReLU(Conv(x 1 ))+Conv(x 1 ))

[0116] x 3 =ReLU(ReLU(Conv(x 2 ))+Conv(x 2 ))

[0117] Among them, Conv(·) represents the convolution downsampling operation, ReLU(·) represents the rectified linear unit activation function, is the input image block, C, h, w represent the number of bands, height and width respectively. Thus, three feature maps of different scales are obtained As input to a deep interpretable network.

[0118] (2) Construct an iterative optimization update algorithm for the multi-scale convolutional sparse coding model;

[0119] (2a) First, we obtain three features x of different scales. 1 、x 2 、x 3 Finally, the following multi-scale sparse coding model is constructed to capture the shared sparse coefficients and private sparse coefficients between different scales:

[0120]

[0121]

[0122]

[0123] in, Respectively represent convolution operation, shared sparse coefficient, private sparse coefficient, shared sparse coefficient convolution layer, private sparse coefficient convolution layer, k is the convolution kernel size, P represents the number of channels in the sparse coefficient. The spatial dimension of shared sparse coefficient and the input features of intermediate scale Figure 1 After obtaining the shared sparse coefficients and the private sparse coefficients, they are fused into a comprehensive sparse coefficient to enhance the feature representation ability of the model, thereby improving the robustness of the model:

[0124]

[0125] in, They are fused sparse coefficients, shared sparse coefficient fused convolutional layers, and private sparse coefficient fused convolutional layers. In order to implement convolutional sparse coding and obtain fused sparse coefficients, the goal is to freeze the convolutional layer when optimizing the objective function of the following sparse coefficients:

[0126]

[0127] Among them, λ is used as a regularization parameter to control the sparsity of sparse coefficients. In order to enrich the spatial information in the sparse coefficients, based on the assumption that there is the same sparse support set between adjacent pixels, a structured row sparse constraint is imposed, namely, L 2,1 norm. To overcome this multi-objective optimization challenge, we use the alternating direction method of multipliers (ADMM) to solve the optimization problem of the objective function. By introducing auxiliary variables (AVs) and dual variables (DVs), the following augmented Lagrangian function is defined:

[0128]

[0129] Among them, ρ represents the penalty parameter that controls the consistency of constraints, and are auxiliary variables for shared and private sparse coefficients, and Denotes the dual variable of shared and private sparse coefficients, and F denotes the Frobenius norm extended to a third-order tensor. In order to optimize the augmented Lagrangian function, the ADMM algorithm is used to decompose it into six tractable sub-problems. The specific iterative update process is as follows:

[0130] (2b) Update s: The subproblem of private sparse coefficients and its update process are as follows:

[0131]

[0132]

[0133]

[0134] Among them, α is the step size parameter that adjusts the update amplitude, represents the transposed convolution operation, and k represents the kth iteration.

[0135] Similarly, the subproblem of sharing sparse coefficients and its update process are as follows:

[0136]

[0137]

[0138]

[0139] in, Also represents the transposed convolution operation.

[0140] Update z: The subproblem of private auxiliary variables and their update process are as follows:

[0141]

[0142]

[0143] Similarly, the subproblem of sharing auxiliary variables and their update process are as follows:

[0144]

[0145]

[0146] Update μ: The update process of shared and private dual variables is as follows:

[0147]

[0148]

[0149] After K+1 iterations, the shared and private sparse coefficients are combined to obtain the fused sparse coefficients.

[0150] (3) Figure 3 As shown, a shared sparse coefficient update module and a private sparse coefficient update module are constructed and expanded into a deep interpretable neural network. Two interpretable networks with the same structure but different weights are used as the student network and the teacher network respectively.

[0151] According to the derived multi-scale convolutional sparse coding model iterative update algorithm, a shared sparse coefficient update module and a private sparse coefficient update module are constructed. The information flow and internal working mechanism of these two modules are transparent and traceable. Each operation is model-driven and strictly aligned with the iterative algorithm to ensure that the output shared sparse coefficient s sand private sparse coefficients It has intuitive physical meaning. According to the derived iterative optimization update algorithm, the convolution algorithm in the two update modules is replaced by the existing CNN layer, and the two update modules are alternately expanded to construct a deep interpretable network guided by multi-scale convolutional sparse coding.

[0152] Based on this, two networks with the same structure but different weights are constructed as the student network and the teacher network, which are used to extract the multi-scale feature representation of the student and the multi-scale feature representation of the teacher respectively.

[0153] (4) Figure 4 As shown in the figure, the three different scale feature maps corresponding to the input dual-phase hyperspectral image are repeatedly and alternately updated in the forward propagation of the network. s and private sparse coefficient features After multiple iterations, the fused sparse coefficient feature s is obtained f ;

[0154] First, the three different scale feature maps of the hyperspectral image are input into the private sparse coefficient update module to extract and update the private sparse coefficient features. This captures private structural information and scale-dependent features at different scales, reduces redundancy, and accurately represents the key features in the input feature map.

[0155] After getting the updated private sparse coefficient feature Combine them with the feature map x 1 、x 2 、x 3 Input into the shared sparse coefficient update module together to extract and update the shared sparse coefficient feature s s , capturing general texture and structural patterns across scales;

[0156] After multiple iterations, the shared sparse coefficient feature s s and private sparse coefficient features Combine to get the fused sparse coefficient feature s f Output of a deep interpretable network guided by multi-scale convolutional sparse coding.

[0157] (5) Figure 5 As shown, the sparse coefficient features of the student network are input into the classifier, and a small amount of labeled data is used for the first stage of supervised training;

[0158] In the first stage of training, a small amount of labeled data is used to input the student network, and the obtained coefficient features are sent to the fully connected classifier to implement supervised training of binary classification of changes and invariants. Among them, the focal loss is used to guide the first stage of network training:

[0159]

[0160]

[0161]

[0162] Among them, B represents the batch size, y b is the true label of the b-th sample, is the weight factor for dealing with class imbalance, p b is the predicted probability. In addition, the focusing parameter γ f and modulation factor The contribution of easy-to-classify samples to the loss is reduced. In this way, the student network undergoes an effective “warm-up” phase, ensuring a robust initialization for subsequent training phases.

[0163] (6) Figure 6 As shown, the phase feature representation and difference feature representation of the student network and the teacher network are aligned respectively, and a small amount of labeled data and a large amount of unlabeled data are used for mixed training;

[0164] In the second phase of training, the weights of the student network are updated via standard gradient backpropagation, while the weights of the teacher network stop their extractive backpropagation and are updated using the exponential moving average (EMA) of the student network weights:

[0165] θ t ←βθ t +(1-β)θ s

[0166] Among them, θ t represents the weight of the teacher network, θ s represents the weight of the student network, and β is a smoothing factor that controls the update speed. In this way, the teacher network can slow down the random fluctuations of the student network and make the training process more stable.

[0167] By inputting a large amount of unlabeled data into the teacher and student networks, and inputting their phase features and difference features into the multi-layer perceptron respectively, projecting them into the same feature space, and combining L 2 Norm normalization and weight normalization ensure that features can be effectively aligned and uniformly optimized, thereby achieving self-supervised training in the teacher-student self-distillation paradigm. In addition, the centralization operation in the teacher network is used to prevent the network from crashing or degrading during training, ensuring the stability of training. The specific loss function of this process is as follows:

[0168] L ssl =ψL u +(1-ψ)L d

[0169]

[0170]

[0171]

[0172] in Represent the temporal feature representation of students and teachers respectively, O is the dimension of feature representation, and represents the difference feature representation between the student and the teacher, sim(·,·) represents the cosine similarity function, and γ is the penalty factor. Through self-supervised training of the teacher-student self-distillation paradigm, useful feature representations can be learned from a large amount of unlabeled data, enhancing the network's sensitivity to changing information while also learning the invariant information.

[0173] In addition, a small amount of labeled data is fed into the student network for supervised training. This hybrid training method ensures that the weights of the teacher network and the student network can be updated in the right direction. (7) The dual-phase hyperspectral image is fed into the trained student network to obtain the final change detection result;

[0174] After the initial supervised training in the first phase and the mixed training in the second phase, the trained student network and fully connected classifier are extracted and the parameters are frozen. The dual-phase hyperspectral image to be detected is input into the student network to obtain the coefficient features of the two phases. The difference features between the two are sent to the classifier to obtain the final change detection result:

[0175] D=G(M(X T1 )-M(X T2 ))

[0176] Among them, X T1 and X T2 represents the input dual-phase hyperspectral image, M(·) represents the deep interpretable network guided by multi-scale convolutional sparse coding, G(·) represents the fully connected classifier, and D is the final change map obtained.

[0177] The technical effects of the present invention are described in detail below in conjunction with simulation experiments:

[0178] 1. Simulation Experiment Conditions and Datasets

[0179] The hardware platform of the simulation experiment of the present invention is NVIDIA GeForce RTX 3090 GPU and Intel (R) Core (TM) i9-10900X CPU @ 3.70 GHz.

[0180] The software platforms for the simulation experiment of the present invention are: Linux 20.04 operating system, Python 3.8 and PyTorch1.12.

[0181] The simulation experiment of the present invention uses the Santa Barbara dataset obtained by the Airborne Visible / Infrared Imaging Spectrometer (AVIRIS) sensor, which mainly covers the changes in mountainous areas and farmland near the Santa Barbara area in California, USA. The two hyperspectral images in the dataset were taken in 2013 and 2014, respectively. The spatial size of each image is 984×740 pixels, the spatial resolution is 20 meters, and it covers 224 spectral bands with a spectral range of 0.4-2.5μm. The true labels of the dataset are not fully annotated, including 80,418 unchanged pixels, 52,134 changed pixels, and 595,608 undetermined pixels. In the experiment, 1% of the determined pixels are randomly selected as label data, and combined with unlabeled data to form a training set, and the remaining determined pixels are used as a test set.

[0182] 2. Evaluation indicators

[0183] This experiment uses four indicators to evaluate the change detection results, namely overall accuracy OA, chi-square coefficient KC, F1 score and average accuracy AA. OA is the ratio of correctly classified samples to the total number of samples, KC represents the classification consistency between the predicted value and the target value, F1 is the harmonic mean of precision and recall, and AA is a measure of the average accuracy of all categories. The higher these four evaluation indicators are, the better, and the ideal value is 1.

[0184] 3. Comparison of algorithms and results analysis

[0185] In order to verify the effectiveness of the present invention, eight representative methods were selected for comparative experiments on the Santa Barbara dataset, including CVA, IR-MAD, SVM, D-LIEG, D 2 AGCN, MSDFFN, DA-Former, MFCEN.

[0186] The change detection visualization results obtained by different methods are shown in Figure 2. Figure 7 As shown, the visualization results of change detection of different methods. (a)-(j) are respectively the true label, CVA, IR-MAD, SVM, D-LIEG, D 2 AGCN, MSDFFN, DA-Former, MFECN, this method; CVA has the worst visualization results. Although IR-MAD and SVM have improved, they still have significant misclassification problems. D-LIEG and D 2AGCN shows significant advantages in reducing false positives and missed positives, with the former outperforming the latter due to computational overhead. MSDFFN, MFCEN, and DA-Former further improve performance, almost eliminating large-scale misclassifications. In addition, this method shows the best visualization results, further reduces noise in continuous regions, and optimizes boundaries.

[0187] Table 1 Objective evaluation results

[0188]

[0189] The quantitative evaluation results of different methods are shown in Table 1, where the bold font indicates the best performance. Among the traditional methods, unsupervised CVA and IR-MAD performed the worst, while supervised SVM improved. 2 AGCN further improves the effect of change detection. In addition, MSDFFN, MFCEN and DA-Former perform better, among which the semi-supervised DA-Former outperforms the supervised MSDFFN and MFCEN in the case of limited labeled data. Our proposed method outperforms these methods in all indicators, proving the effectiveness of combining deep learning with model-driven methods.

[0190] In summary, the present invention implements an innovative multi-scale convolutional sparse coding model and embeds it into a deep neural network to guide the optimization of network parameters, breaking through the traditional data-driven "black box" operation mode, enhancing the interpretability of deep multi-scale feature extraction, and significantly improving the generalization performance and detection accuracy of the network model. In addition, through a two-stage semi-supervised training strategy, the present invention further reduces the model's dependence on labeled data, fully improves the utilization efficiency of unlabeled data, and effectively enhances the model's sensitivity to changes.

[0191] The present invention can be widely used in multiple practical scenarios, covering fields including environmental monitoring, land use change analysis, disaster assessment, urban expansion research, forest resource management, ecological protection and restoration, etc. These application scenarios have high requirements on detection accuracy, generalization ability and sensitivity to regional changes. Through the method of the present invention, the accuracy and generalization of hyperspectral image change detection can be effectively improved, and the interpretability of the model decision process can be enhanced, thereby providing more accurate and reliable solutions for various fields.

[0192] The embodiments of the present invention have achieved some positive effects during the development or use process. The technical effects of the present invention are described in detail below in combination with experiments.

[0193] Evidence (1):

[0194] In order to verify the effectiveness of embedding the multi-scale convolutional sparse coding model into a deep neural network, and to simultaneously consider the impact of shared features and private features in multi-scale images on the model performance, the present invention is compared with its two variants. PSCUM represents the private sparse coefficient update module, while SSCUM represents the shared sparse coefficient update module. The experimental results are as follows:

[0195] Table 2 Objective evaluation results

[0196]

[0197] Evidence (2):

[0198] In order to reflect the effectiveness of the teacher-student self-distillation learning paradigm designed by the present invention, this experiment verifies the influence of the weight of the paradigm in training on the model performance. η represents the weight of the teacher-student self-distillation learning paradigm in calculating the loss function. The experimental results are shown in Figure 8 As shown in the figure, as the weight of the teacher-student self-distillation learning paradigm gradually increases, the model performance first reaches a peak and then gradually decays. When the weight is too small, the model is not able to fully utilize the unlabeled data; when the weight is too large, the model overemphasizes the learning of unlabeled data and ignores the learning of a small amount of labeled data. This proves that a suitable weight can enable the teacher-student self-distillation learning paradigm to significantly improve the detection accuracy of the model and effectively improve the utilization efficiency of unlabeled data.

[0199] The present invention designs an innovative multi-scale convolutional sparse coding model, which can effectively capture the shared sparse coefficients and private sparse coefficients at different scales of the input image, significantly reduce feature redundancy, and the extracted coefficient features have rich multi-scale information and a more compact feature distribution.

[0200] The present invention designs a deep interpretable network guided by multi-scale convolutional sparse coding, combines multi-scale sparse coding theory with deep learning, enhances the interpretability of deep multi-scale feature extraction, and improves the generalization performance and detection accuracy of the network model.

[0201] The present invention designs a two-stage training strategy that combines supervised learning and a change-sensitive teacher-student self-distillation learning paradigm, which reduces the annotation dependency of the network model and enhances the model's sensitivity to changes while effectively utilizing a large amount of unlabeled data.

[0202] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A hyperspectral image change detection method based on multi-scale convolutional sparse coding, characterized in that: The steps include: S101: Input a small number of labeled and a large number of unlabeled dual-phase hyperspectral images, and each image is sequentially convolved and down-sampled three times to obtain feature maps x1, x2, and x3 of different scales corresponding to each convolution and down-sampling; S102: construct an iterative optimization update algorithm for a multi-scale convolutional sparse coding model; S103: constructing a shared sparse coefficient update module and a private sparse coefficient update module based on an iterative optimization update algorithm of a multi-scale convolutional sparse coding model, and unfolding them into a deep interpretable neural network, and using two deep interpretable neural networks with the same structure but different weights as a student network and a teacher network respectively; S104: Input three feature maps of different scales x1, x2, and x3, and repeatedly and alternately update the shared sparse coefficient feature s in the forward propagation of the network. s and private sparse coefficient features After multiple iterations, the fused sparse coefficient feature s is obtained f ; S105: Input the sparse coefficient features of the student network into the classifier and use a small amount of labeled data to perform the first stage of supervised training; S106: aligning the phase feature representation and the difference feature representation of the student network and the teacher network respectively, and performing mixed training using a small amount of labeled data and a large amount of unlabeled data; S107: Input the dual-phase hyperspectral image into the trained student network to obtain the final change detection result.

2. The hyperspectral image change detection method based on multi-scale convolutional sparse coding according to claim 1 is characterized in that: The step S101 is specifically as follows: A small number of labeled and a large number of unlabeled dual-phase hyperspectral images are divided into overlapping input image blocks, and feature maps x1, x2, and x3 of different scales are generated through three convolutions and downsampling. At the same time, residual connections with convolution downsampling are added between feature maps of different scales: x1=ReLU(ReLU(Conv(x))+Conv(x)) x2=ReLU(ReLU(Conv(x1))+Conv(x1)) x3 = ReLU(ReLU(Conv(x2))+Conv(x2)) Among them, Conv(·) represents the convolution downsampling operation, ReLU(·) represents the rectified linear unit activation function, is the input image block, C, h, w represent the number of bands, height and width respectively, thus obtaining feature maps of different scales As input to a deep interpretable network.

3. The hyperspectral image change detection method based on multi-scale convolutional sparse coding according to claim 2 is characterized in that: The step S102 is specifically as follows: First, after obtaining the features x1, x2, and x3 of three different scales, the following multi-scale sparse coding model is constructed to capture the shared sparse coefficients and private sparse coefficients between different scales: in, They represent convolution operation, shared sparse coefficient, private sparse coefficient, shared sparse coefficient convolution layer, private sparse coefficient convolution layer respectively. k is the convolution kernel size. P represents the number of channels in the sparse coefficient. The spatial dimension of the shared sparse coefficient is consistent with the input feature map of the intermediate scale to ensure the balance between high-level semantic information and low-level details. After obtaining the shared sparse coefficient and the private sparse coefficient, they are fused into a comprehensive sparse coefficient: in, They are fused sparse coefficients, shared sparse coefficient fused convolutional layer, and private sparse coefficient fused convolutional layer respectively; when optimizing the objective function of the following sparse coefficients, freeze the convolutional layer, implement convolutional sparse coding and obtain fused sparse coefficients; Among them, λ is used as a regularization parameter to control the sparsity of the sparse coefficients. Based on the assumption that there is the same sparse support set between adjacent pixels, a structured row sparse constraint is imposed on it, that is, L 2,1 norm, the alternating direction method of multipliers (ADMM) is used to solve the optimization problem of the objective function. By introducing auxiliary variables (AVs) and dual variables (DVs), the following augmented Lagrangian function is defined: Among them, ρ represents the penalty parameter that controls the consistency of constraints, and are auxiliary variables for shared and private sparse coefficients, and Denotes the dual variable of shared and private sparse coefficients, and F denotes the Frobenius norm expanded to a third-order tensor.

4. The hyperspectral image change detection method based on multi-scale convolutional sparse coding according to claim 3 is characterized in that: In order to optimize the augmented Lagrangian function, the ADMM algorithm is used to decompose it into six tractable sub-problems; The specific iterative update process is as follows: Update s: The subproblem of private sparse coefficients and its update process are as follows: Among them, α is the step size parameter that adjusts the update amplitude, represents the transposed convolution operation, k represents the kth iteration; Similarly, the subproblem of sharing sparse coefficients and its update process are as follows: in, It also represents the transposed convolution operation; Update z: The subproblem of private auxiliary variables and their update process are as follows: Similarly, the subproblem of sharing auxiliary variables and their update process are as follows: Update μ: The update process of shared and private dual variables is as follows: After K+1 iterations, the shared and private sparse coefficients are combined to obtain the fused sparse coefficients.

5. The hyperspectral image change detection method based on multi-scale convolutional sparse coding according to claim 3 is characterized in that: The step S103 is specifically as follows: The information flow and internal working mechanism of the shared sparse coefficient update module and the private sparse coefficient update module are transparent and traceable. Each operation is model-driven and strictly aligned with the iterative algorithm to ensure the output shared sparse coefficient feature s s and private sparse coefficient features It has intuitive physical meaning. According to the derived iterative optimization update algorithm, the convolution operators in the two update modules are replaced by the existing CNN layers. By alternately expanding the two update modules, a deep interpretable network guided by multi-scale convolutional sparse coding is constructed. Based on this, two networks with the same structure but different weights are constructed as the student network and the teacher network, which are used to extract the multi-scale feature representation of the student and the multi-scale feature representation of the teacher respectively.

6. The hyperspectral image change detection method based on multi-scale convolutional sparse coding according to claim 5 is characterized in that: The step S104 is specifically as follows: First, the three different scale feature maps of the hyperspectral image are input into the private sparse coefficient update module to extract and update the private sparse coefficient features. In this way, private structural information and scale-dependent features are captured at different scales, and the updated private sparse coefficient features are obtained. Input them together with feature maps x1, x2, x3 into the shared sparse coefficient update module to extract and update the shared sparse coefficient feature s s , capturing general texture and structural patterns across scales; After multiple iterations, the shared sparse coefficient feature s s and private sparse coefficient features Combine to get the fused sparse coefficient feature s f Output of a deep interpretable network guided by multi-scale convolutional sparse coding.

7. The hyperspectral image change detection method based on multi-scale convolutional sparse coding according to claim 6 is characterized in that: The step S105 is specifically as follows: In the first stage of training, a small amount of labeled data is used to input the student network, and the obtained coefficient features are sent to the fully connected classifier to realize the supervised training of the binary classification of changes and invariants. Among them, the focal loss is used to guide the first stage of network training: Among them, B represents the batch size, y b is the true label of the b-th sample, is the weight factor for dealing with class imbalance, p b is the predicted probability.

8. The hyperspectral image change detection method based on multi-scale convolutional sparse coding according to claim 7 is characterized in that: The step S106 is specifically as follows: In the second phase of training, the weights of the student network are updated via standard gradient backpropagation, while the weights of the teacher network stop their extractive backpropagation and are updated using the exponential moving average (EMA) of the student network weights: i t ←bth t +(1-β)θ s Among them, θ t represents the weight of the teacher network, θ s represents the weight of the student network, and β is the smoothing factor that controls the update speed; By inputting a large amount of unlabeled data into the teacher and student networks, and inputting their phase features and difference features into the multi-layer perceptron respectively, projecting them into the same feature space, and combining L2 norm normalization and weight normalization, it is ensured that the features can be effectively aligned and uniformly optimized, thereby realizing the self-supervised training of the teacher-student self-distillation paradigm. In addition, the centralization operation in the teacher network is used to prevent the network from collapsing or degrading during the training process, ensuring the stability of the training. The specific loss function of this process is as follows: L ssl =ψL u +(1-ψ)L d in Represent the temporal feature representation of students and teachers respectively, O is the dimension of feature representation, and represents the difference feature representation between students and teachers, sim(·,·) represents the cosine similarity function, and γ is the penalty factor; In addition, a small amount of labeled data is fed into the student network for supervised training.

9. The hyperspectral image change detection method based on multi-scale convolutional sparse coding according to claim 8 is characterized in that: The step S107 is specifically as follows: After the initial supervised training in the first phase and the mixed training in the second phase, the trained student network and fully connected classifier are extracted and the parameters are frozen. The dual-phase hyperspectral image to be detected is input into the student network to obtain the coefficient features of the two phases. The difference features between the two are sent to the classifier to obtain the final change detection result: D=G(M(X T1 )-M(X T2 )) Among them, X T1 and X T2 represents the input dual-phase hyperspectral image, M(·) represents the deep interpretable network guided by multi-scale convolutional sparse coding, G(·) represents the fully connected classifier, and D is the final change map obtained.

Images