Remote sensing image change detection method based on DCT perception

Abstract

The invention discloses a remote sensing image change detection method based on DCT (Discrete Cosine Transformation) perception. The method comprises the following steps of: extracting and fusing double-time-phase characteristics by utilizing a four-layer double-branch network; the adaptive spectral feature extraction module generates a DCT filter, and extracts and synthesizes frequency domain features; the confidence coefficient weighted fusion device calculates the confidence coefficient through softmax, the confidence coefficient is used for the feature map after normalization in combination with the attention weight, and the image change is detected through the point convolution integral class. According to the method, an innovative network and an attention mechanism are adopted, extraction of detail features of the remote sensing image is enhanced, and the interpretability of the model is improved; the Siamese network structure ensures the consistency of dual-time-phase image feature extraction; the adaptive spectrum module selects a key frequency according to requirements, and optimizes the parameters of a DCT (Discrete Cosine Transform) filter; the confidence coefficient weighted fusion device improves the recognition capability of a change area through dynamic feature selection and weighting.

Description

Technical Field

[0001] The invention belongs to the field of change detection, and in particular relates to a remote sensing image change detection method based on DCT perception. Background Art

[0002] The task of change detection is to identify the differences in the state of a specific object or phenomenon by observing it at different points in time. Specifically, change detection aims to identify the changes between images taken at different points in time in the same geographic area. Change detection technology has been widely used in many fields, including disaster assessment, environmental monitoring, land use management, and urban development analysis. With the increasing frequency of extreme weather events (such as droughts, floods, hurricanes, and heat waves) caused by climate change, the dynamic changes in buildings and land use are becoming increasingly prominent.

[0003] In cognitive science, due to the bottleneck of information processing, humans will selectively focus on certain key information and ignore some unimportant information. Similarly, in order to improve the performance of change detection and refine deep features, attention modules are widely used in change detection networks. Change detection not only extracts global semantic features, but also enhances detailed features. Many studies have focused on how to design effective channel or spatial attention mechanisms, while ignoring a basic problem, that is, the channel attention mechanism uses scalars to represent channels, but it is difficult to implement due to a large amount of information loss.

[0004] In recent years, many deep learning-based change detection methods have been proposed, increasingly integrating traditional neural network architectures for semantic segmentation to facilitate the extraction of deeper representation features. A significant limitation of these methods is the lack of comprehensive supervision of each sub-layer during the training phase. Supervising only the last block will result in a lack of sufficient constraints on the intermediate layers. Summary of the invention

[0005] The purpose of the present invention is to provide a remote sensing image change detection method based on DCT perception, introduce an adaptive spectral feature extraction module, parameterize the discrete cosine transform filter as learnable weights, and allow the model to adaptively optimize frequency selection, which is an extension of the fixed properties of traditional DCT; in addition, the innovation of the confidence weighted feature fuser lies in the fusion of the outputs of multiple sub-networks, and through the confidence quantization and attention weighting mechanism, dynamic feature selection and weighting are realized, thereby enhancing the model's ability to recognize key change areas.

[0006] The technical solution to achieve the purpose of the present invention is as follows: In a first aspect, the present invention provides a remote sensing image change detection method based on DCT perception, comprising the following steps:

[0007] The first step is to construct a dual-branch network based on 4 different levels of UNet to extract bi-temporal features, concatenate features at the same level, and jump-connect the concatenated features to the features at a higher level to fuse features at different levels.

[0008] The second step is to design an adaptive spectral feature extraction module, select specific frequency components, and generate discrete cosine transform filters. In the forward propagation process, the input data is multiplied by the filter. The DCT filter corresponds to the response of different frequency components. Then, through the aggregation operation in the spatial dimension, the frequency domain representation of the feature is obtained, and the responses of each filter are integrated to generate a comprehensive frequency domain feature representation.

[0009] The third step is to design a confidence-weighted feature fuser, calculate the probability distribution for the output of each sub-network through the softmax layer, and select the change probability as the confidence; combine the confidence with the attention weight and normalize them to ensure that the sum of the weights is 1; classify the obtained feature map into background and foreground through a point convolution to achieve the change detection goal.

[0010] In a second aspect, the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect when executing the program.

[0011] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0012] In a fourth aspect, the present invention provides a computer program product, comprising a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0013] Compared with the prior art, the present invention has the following significant features: (1) The present invention designs an adaptive spectral feature extraction module, which realizes adaptive frequency domain feature extraction by integrating learnable parameters into discrete cosine transform. This method automatically optimizes frequency selection and enhances the sensitivity of the model to key features in the signal; the innovation of the module lies in its adaptive frequency domain feature extraction capability, which not only improves the generalization of the model, but also enhances the information integration capability through multi-spectral feature fusion; in addition, the module improves the computational and storage efficiency by compressing frequency domain features, and may bring additional regularization effects, which helps to control model complexity and reduce overfitting. (2) The core innovation of the confidence-weighted feature fuser is that it fuses the outputs of multiple sub-networks, and realizes dynamic feature selection and weighting through confidence quantization and attention weighting mechanisms, thereby enhancing the model's ability to identify key change areas. Through normalization processing, the balance of contributions from different sub-networks is ensured, while maintaining the adaptability and generalization capability of the model. This integrated learning method not only improves the accuracy of change detection, but also increases the interpretability of model decisions.

[0014] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of the present invention.

[0016] Figure 2 This is the change detection network structure of the present invention.

[0017] Figure 3 It is the adaptive spectral feature extraction module of the present invention.

[0018] Figure 4 (a) is the image ImageA of the change detection dataset at time T1.

[0019] Figure 4 (b) is the image ImageB of the change detection dataset at time T2.

[0020] Figure 4 (c) is the ground truth map of the change detection dataset.

[0021] Figure 4 (d) is the change graph of the change detection dataset using the FC-EF method.

[0022] Figure 4 (e) is the change graph of the change detection dataset using the FC-Siam-diff method.

[0023] Figure 4 (f) is the change graph of the change detection dataset using the FC-Siam-conc method.

[0024] Figure 4 (g) is the change graph of the change detection dataset using the SNUNet method.

[0025] Figure 4 (h) is the change graph of the change detection dataset using the change detection method of remote sensing images based on DCT perception. DETAILED DESCRIPTION

[0026] The present invention proposes a remote sensing image change detection method based on DCT perception. The deep supervision network based on UNet ensures that each layer of sub-network is supervised, so as to be closer to the real target. The multispectral attention module is used to extract detail features, and the channel attention module assigns differentiated weights to channels through different frequency components, which enables the network to focus limited attention on more critical areas during training and enhance the performance of detail features.

[0027] Combination Figure 1 , the implementation process of the present invention is described in detail below, the steps are as follows:

[0028] In the first step, a dual-branch network based on UNet at four different levels is constructed to extract bi-temporal features, and features at the same level are concatenated. The concatenated features are jump-connected to features at a higher level to fuse features at different levels. Specifically:

[0029] A dual-branch network based on UNet is constructed to extract bi-temporal features. It has a total of four sub-networks. The input of the network is two bi-temporal remote sensing images. The encoder obtains features of different scales. After the bi-temporal features are spliced, the high-resolution and fine-grained features in the encoder are directly passed to the corresponding level of the decoder through a dense skip connection mechanism.

[0030] The second step is to design an adaptive spectral feature extraction module, select specific frequency components, and generate discrete cosine transform filters. These filters correspond to specific frequency components. During the forward propagation process, the input data is multiplied by the filters. The DCT filters correspond to the responses of different frequency components. Subsequently, through aggregation operations in the spatial dimension, the frequency domain representation of the features is obtained. The responses of each filter are integrated to generate a comprehensive frequency domain feature representation.

[0031] like Figure 3 As shown, an adaptive spectral feature extraction module is designed to select specific frequency components and generate a discrete cosine transform filter. Usually, the basic function of the two-dimensional discrete cosine transform is:

[0032]

[0033] sth∈{0, 1, ..., H-1}, w∈{0, 1, ..., W-1}

[0034] in It represents the two-dimensional discrete cosine transform coefficient, H and W represent the height and width of the image respectively, C represents the channel, and i and j are the indices of the two-dimensional discrete cosine transform coefficient, indicating the amplitude of the signal at the image frequency.

[0035] First, we extract the feature map X∈R C×H×W , split into n parts according to the channel dimension, expressed as [X 0 , X 1 , ..., X n-1 ], where X i ∈R C′×H×W , Initialize weights W uv , where u and v represent the frequency component index in the two-dimensional space; thus, for each part, the corresponding two-dimensional discrete cosine transform frequency component is assigned, and the two-dimensional discrete cosine transform result can be used as the compression result of channel attention. In this way, we have:

[0036]

[0037] sti∈{0,1,...,n-1}

[0038] Where [u i , v i ] is X i The corresponding frequency component 2D index, is the pixel value of the original image data at position (h, w) on the i-th channel, is the discrete cosine transform basis function, is the learnable discrete cosine transform filter weight, F i ∈R C′ is the compressed C′-dimensional vector;

[0039] The entire compressed vector can be obtained by concatenation:

[0040] F=cat([F 0 , F 1 , ..., F n-1 ])

[0041] Where F = R C is the obtained multispectral vector. The entire multispectral channel attention framework can be written as:

[0042] MSCA_att = sigmoid(f(F))

[0043] Through the above process, we generalize the channel to a framework with multiple frequency components. These filters correspond to the corresponding frequency components. During the forward propagation process, the input data is multiplied by the filter. The DCT filter corresponds to the response of different frequency components. Then, through the aggregation operation in the spatial dimension, the frequency domain representation of the feature is obtained. The responses of each filter are integrated to generate a comprehensive frequency domain feature representation.

[0044] The third step is to design a confidence-weighted feature fuser, which calculates the probability distribution for the output of each sub-network through the softmax layer and selects the probability of change as the confidence. These confidences are combined with the attention weights and normalized to ensure that the sum of the weights is 1. This method enables the model to assign different weights when processing outputs at different levels, focusing on the most useful features, thereby improving performance. Finally, the obtained feature map is classified into background and foreground through a point convolution to achieve the change detection goal.

[0045] Define the confidence normalization attention module, the specific operations are as follows:

[0046] 1) First, for the four-layer subnetwork output, apply the softmax formula to obtain the probability distribution P of each pixel belonging to each change category n :

[0047]

[0048] in, is the probability of the jth category in the i-th batch, L n The logits matrix output by the nth subnetwork, corresponds to L n logits value in, K is the total number of categories, k∈{0, 1, ..., K-1}, and the denominator is the logits of all categories k The sum of ensures that the probability distribution is normalized, that is, the sum of the probabilities of all classes is 1.

[0049] 2) Use channel attention to generate feature maps C for each sub-network n Calculate the weight A n Multiply them together to get the weighted confidence W n :

[0050] W n =C n ⊙A n

[0051] where ⊙ represents element-wise multiplication,

[0052] 3) The weighted confidence W n Normalize so that the sum of the weights of all sub-networks is 1:

[0053]

[0054] 4) The weighted confidence W′ n Applied to the original feature map F n , and merge to get the final feature map F merged :

[0055]

[0056] Finally, the obtained feature map is classified into background and foreground through a point convolution to achieve the change detection goal.

[0057] The core features of the present invention are concentrated on its innovative network structure and attention weighting mechanism, which enables it to effectively extract and enhance detail features when processing remote sensing images, and also increases the interpretability of model decisions. The method utilizes the Siamese network structure to process dual-phase images, ensuring the consistency of feature extraction. The key innovation is the introduction of an adaptive spectral feature extraction module, which can adaptively select important frequency components based on data and task requirements, rather than relying on fixed, preset frequency selection. It parameterizes the discrete cosine transform filter as learnable weights, allowing the model to adaptively optimize frequency selection, which is an extension of the fixed properties of traditional DCT. In addition, the innovation of the confidence-weighted feature fuser lies in the fusion of the outputs of multiple subnetworks, and through confidence quantization and attention weighting mechanisms, dynamic feature selection and weighting are achieved, thereby enhancing the model's ability to recognize key change areas.

[0058] The effect of the present invention can be further illustrated by the following simulation experiment:

[0059] (1) Simulation conditions

[0060] The simulation experiment uses a change detection dataset. Our series of comparative experiments are conducted on CDD and LEVIR-CD, the most commonly used evaluation datasets in the field of change detection. The DD dataset consists of 11 pairs of multispectral (R, G, B) images acquired by Google Earth (Digital Globe) in different seasons, with spatial resolutions ranging from 3cm / px to 100cm / px. The present invention uses the Pytorch framework to implement SSUNet-CD. During the training process, we have tested and debugged many times, the batch size is set to 16, and Adam is used as the optimizer. The learning rate is set to 1e-3, and it decays 0.5 times every 8 epochs. We conducted experiments on an NVIDIA RTX3090 and trained 100 epochs to converge the model.

[0061] In terms of quantitative indicator evaluation, the present invention adopts three standard indicators: precision, recall and F1-Score.

[0062] (2) Simulation content

[0063] The present invention uses CDD dataset and LEVIR-CD dataset to test the performance of the algorithm. Four representative change detection algorithms are used for comparison, namely FC-EF (Fully Convolutional Early Fusion), FC-Siam-diff (Fully Convolutional Siamese-Difference), FC-Siam-conc (Fully Convolutional Siamese-Concatenation) and SNUNet (the combination of Siamese network and NestedUNet).

[0064] (3) Analysis of simulation experiment results

[0065] Table 1 shows the evaluation index results on the CDD dataset, and Table 2 shows the evaluation index results on the LEVIR-CD dataset.

[0066] Table 1 Quantitative evaluation of different change detection algorithms on CDD dataset

[0067] Methods Pre(%) Rec(%) F1(％) FC-EF 79.61 49.10 60.74 FC-Siam-diff 73.41 73.71 73.56 FC-Siam-conc 80.81 65.66 72.45 SNUNet 94.52 91.02 92.74 Ours 96.10 96.28 96.19

[0068] Table 2 Quantitative evaluation of different change detection algorithms on the LEVIR-CD dataset

[0069] Methods Pre(%) Rec(%) F1(％) FC-EF 89.08 79.62 84.08 FC-Siam-diff 91.22 82.85 86.83 FC-Siam-conc 89.95 84.21 86.99 SNUNet 91.58 88.79 90.17 Ours 93.13 92.50 92.78

[0070] As shown in Table 1, compared with the four change detection algorithms, the remote sensing image change detection method based on DCT perception proposed in the present invention shows excellent performance on the change detection dataset, and obtains the highest precision, recall rate and F1, which are 96.10%, 96.28% and 96.19% respectively. The reason why the method of the present invention performs well is that the deep supervision proposed in the present invention gradually brings the feature map closer to the ground truth during the training process. Our network also includes spatial and spectral attention modules, which effectively utilize complex input information, especially enhance the performance of detail features. The results of the method of the present invention on two change detection datasets are shown in the figure below. Figure 4 The simulation experiment results of the above real change detection data set demonstrate the effectiveness of the method of the present invention.

Claims

1. A remote sensing image change detection method based on DCT perception, characterized in that: The following steps are involved: The first step is to construct a dual-branch network based on 4 different levels of UNet to extract bi-temporal features, concatenate features at the same level, and jump-connect the concatenated features to the features at a higher level to fuse features at different levels. The second step is to design an adaptive spectral feature extraction module, select specific frequency components, and generate discrete cosine transform filters. In the forward propagation process, the input data is multiplied by the filter. The DCT filter corresponds to the response of different frequency components. Then, through the aggregation operation in the spatial dimension, the frequency domain representation of the feature is obtained, and the responses of each filter are integrated to generate a comprehensive frequency domain feature representation. The third step is to design a confidence-weighted feature fuser, which calculates the probability distribution for the output of each sub-network through the softmax layer and selects the change probability as the confidence; the confidence is combined with the attention weight and normalized to ensure that the sum of the weights is 1; The obtained feature map is classified into background and foreground through a point convolution to achieve the change detection goal.

2. The remote sensing image change detection method based on DCT perception according to claim 1 is characterized in that: The first step is to construct a dual-branch network based on UNet to extract bi-temporal features. There are four sub-networks in total. The input of the network is two bi-temporal remote sensing images. The encoder obtains features of different scales. After the bi-temporal features are spliced, the high-resolution and fine-grained features in the encoder are directly passed to the corresponding level of the decoder through a dense skip connection mechanism.

3. The remote sensing image change detection method based on DCT perception according to claim 1 is characterized in that: The second step is to design an adaptive spectral feature extraction module, select specific frequency components, and generate a discrete cosine transform filter. The specific steps are as follows: The basic function of the two-dimensional discrete cosine transform is: sth∈{0,1,…,H-1},w∈{0,1,…,W-1} in Represents the two-dimensional discrete cosine transform coefficient, H and W represent the height and width of the image respectively, i and j are the indices of the two-dimensional discrete cosine transform coefficient, indicating the amplitude of the signal at the image frequency; the feature map X∈R extracted by the backbone network C ×H×W , where C represents the channel, which is split into n parts according to the channel dimension, expressed as [X 0 ,X 1 ,…,X n-1 ], where X i ∈R C ′×H×W , Initialize weights W uv , where u and v represent the frequency component index in two-dimensional space; For each part, the corresponding 2D DCT frequency component is assigned, and the 2D DCT result can be used as the compression result of channel attention; thus, we have: sti∈{0,1,…,n-1} Where [u i ,v i ] is X i The corresponding frequency component 2D index, is the pixel value of the original image data at position (h, w) on the i-th channel, is the discrete cosine transform basis function, is the learnable discrete cosine transform filter weight, F i ∈R C′ is the compressed C′-dimensional vector; The entire compressed vector can be obtained by concatenation: F=cat([F 0 ,F 1 ,…,F n-1 ]) Where F = R C is the obtained multi-spectral vector; Through the above process, the channel is generalized to a framework with multiple frequency components. These filters correspond to the corresponding frequency components. During the forward propagation process, the input data is multiplied by the filter. The DCT filter corresponds to the response of different frequency components. Subsequently, the frequency domain representation of the feature is obtained through the aggregation operation in the spatial dimension. The responses of each filter are integrated to generate a comprehensive frequency domain feature representation.

4. The remote sensing image change detection method based on DCT perception according to claim 1 is characterized in that: The third step is to define the confidence weighted feature fuser. The specific operations are as follows: (1) For the output of the four-layer subnetwork, the softmax formula is applied to obtain the probability distribution P of each pixel belonging to each change category n : in, is the probability of the jth category in the i-th batch, L n The logits matrix output by the nth subnetwork, corresponds to L n The logits value in, K is the total number of categories, k∈{0,1,…,K-1}, and the denominator is the logits of all categories k The sum of ensures the normalization of the probability distribution, that is, the sum of the probabilities of all categories is 1; (2) Use channel attention to provide feature maps C for each sub-network n and calculate the weight A n Multiply them together to get the weighted confidence W n : W n =C n ⊙A n Where ⊙ represents element-by-element multiplication; (3) The weighted confidence W n Normalize so that the sum of the weights of all sub-networks is 1: (4) The weighted confidence W′ n Applied to the original feature map F n , and merge to get the final feature map F merged : Finally, the obtained feature map is classified into background and foreground through a point convolution to achieve the change detection goal.

5. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method described in any one of claims 1 to 4 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 4 are implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 4 are implemented.

Images