Remote sensing image change detection method based on tensor decomposition and self-attention mechanism

Abstract

The application provides a remote sensing image change detection method based on tensor decomposition and a self-attention mechanism, and mainly solves the problem that the existing detection technology has poor change classification effect on complex ground objects. The scheme comprises the following steps: 1) pre-processing input images to obtain three-channel image data; 2) constructing a detection model combining a tensor neural network and a Transformer, and taking the three-channel image data as the input of the model; 3) using the tensor neural network to retain the spatial structure information of the image and reduce the calculation amount, and using the self-attention mechanism to comprehensively capture global features and enhance the accuracy and robustness of change detection; 4) iteratively training the model to achieve optimization; and 5) performing pixel-level classification on the global feature tensor obtained by the optimized model through a classifier to generate a change detection map. The application can improve the precision and efficiency of change detection without significantly increasing the calculation overhead, and is suitable for change detection tasks of high-resolution remote sensing images.

Description

Technical Field

[0001] This invention belongs to the field of image detection technology, and further relates to remote sensing image change detection technology. Specifically, it is a remote sensing image change detection method based on tensor decomposition and self-attention mechanism, which can be used in the fields of urban expansion, ecological environment monitoring, and natural disaster response. Technical Background

[0002] Remote sensing image change detection is a key technology in remote sensing data processing, aiming to detect changes in the Earth's surface by comparing remote sensing images at different points in time. Change detection technology has been widely applied in fields such as urban expansion, ecological environment monitoring, and natural disaster response. With the acceleration of global urbanization and the increasing severity of environmental problems, accurate and efficient detection of surface changes has become an urgent need in urban planning, environmental management, and disaster assessment. However, with the increase in remote sensing image resolution and data dimensionality (such as multispectral and hyperspectral images), traditional change detection methods have gradually revealed many shortcomings when dealing with high-dimensional data. For example, early pixel-level change detection methods, such as Change Vector Analysis (CVA) and differential image methods, while performing well in low-resolution data, perform poorly in high-resolution data, mainly because they are too sensitive to noise and cannot fully utilize complex spatial structure information.

[0003] In recent years, the development of deep learning technology has provided new ideas for change detection in remote sensing images. In particular, models such as Convolutional Neural Networks (CNNs) and Transformers have shown outstanding performance in feature extraction and classification of large-scale image data. CNNs extract local features of images through convolutional kernels, achieving significant results in image classification and change detection tasks. For example, Daudt et al. proposed a change detection method based on deep convolutional networks, extracting image features through multiple convolutions and using difference images to determine surface changes. However, the local receptive field of CNNs limits their ability to capture global features, making it difficult to effectively combine local and global information when faced with complex surface changes. Transformer models, by introducing a self-attention mechanism, have shown great advantages in extracting long-range dependent features and have been gradually introduced into change detection tasks in remote sensing images. The Vision Transformer (ViT) model proposed by Dosovitskiy et al. achieves image classification and feature extraction by dividing the image into blocks and performing self-attention analysis on the relationships between these blocks. However, when processing images, Transformer needs to flatten two-dimensional images into one-dimensional vectors. This flattening process may lead to the loss of spatial information in the image, especially for high-resolution and multispectral images, which has a more significant impact and limits the application of Transformer in complex remote sensing tasks.

[0004] Current change detection algorithms often involve a large amount of computation and highly complex network structures when processing high-resolution remote sensing images, resulting in high computational costs and resource consumption, making it difficult to achieve efficient operation, especially in scenarios with limited hardware resources. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a remote sensing image change detection method based on tensor decomposition and self-attention mechanism. This method primarily addresses the problem of poor classification performance for complex ground feature changes in existing technologies. Traditional models typically suffer from high computational complexity, information loss, and limited classification effectiveness. This invention introduces a combination of tensor decomposition and self-attention mechanism to construct a novel remote sensing image change detection network. Tensor neural networks preserve the spatial structure information of the image while reducing computational load, while the self-attention mechanism comprehensively captures global features, enhancing the accuracy and robustness of change detection. This improves the accuracy and efficiency of change detection without significantly increasing computational overhead, making it suitable for change detection tasks in high-resolution remote sensing images.

[0006] To achieve the above objectives, the technical solution adopted by the present invention includes the following steps:

[0007] (1) Preprocess the input multi-temporal remote sensing images to reduce the illumination and geometric differences in the input images and obtain the preprocessed three-channel image data.

[0008] (2) Construct a tensor-transformer model that combines tensor neural networks and Transformer. Use the preprocessed three-channel image data as the model input data, and obtain global features through model processing. The implementation steps are as follows:

[0009] (2.1) The preprocessed three-channel image data is decomposed and optimized by tensor neural network to extract data features and obtain core tensor G;

[0010] (2.2) The self-attention mechanism of Transformer is used to perform global modeling of the features extracted by tensor decomposition network to mine long-distance dependencies and change features in the image; that is, by directly using the core tensor G of the tensor neural network as the input unit of Transformer to preserve the spatial and spectral structure of the image, and then performing calculations based on Transformer to obtain the global feature tensor F.

[0011] (3) Iteratively train the tensor Transformer model and optimize it by continuously adjusting the model parameters to obtain the final model for the change detection task;

[0012] (4) The global feature tensor obtained from the final model is classified at the pixel level by a classifier to generate a binary change detection map with the same size as the input image.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] First, by introducing tensor neural networks, this invention effectively extracts key features from remote sensing images, preserves spatial structure information, and avoids the loss of a large amount of structural information in traditional flattening operations. Compared with traditional high-dimensional data processing methods, tensor decomposition significantly improves the accuracy of change detection.

[0015] Secondly, this invention innovatively proposes a new network structure that combines tensor decomposition and the Transformer model. Tensor decomposition reduces the number of model parameters, while the self-attention mechanism of the Transformer captures global features in the image, overcoming the limitation of the limited receptive field of the convolutional neural network. In the change detection task, this network can extract features more comprehensively and effectively improve the classification accuracy of complex ground feature changes.

[0016] Third, because this invention uses a combination of efficient tensor decomposition and Transformer, it reduces computational steps and optimizes the complexity of the feature extraction process. Through the combination of effective feature dimensionality reduction and self-attention mechanism, the model exhibits higher efficiency and stronger robustness in change detection tasks of high-resolution, multispectral images, thereby achieving fast and accurate change detection under limited resources. Attached image description:

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention;

[0019] Figure 2 This is a schematic diagram of the tensor Transformer model structure constructed in this invention;

[0020] Figure 3 This is an example of the LEVIR-CD dataset used in this invention;

[0021] Figure 4 This is a simulation result diagram of scenario one in an embodiment of the present invention;

[0022] Figure 5 This is a simulation result diagram of scenario two in an embodiment of the present invention. Detailed implementation method:

[0023] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0024] Example 1: Refer to Figure 1-2 This example provides a remote sensing image change detection method based on tensor decomposition and self-attention mechanism. The specific implementation steps are as follows:

[0025] Step 1. Preprocess the input multi-temporal remote sensing images, including brightness normalization and image registration operations, to reduce illumination and geometric differences in the input images and obtain preprocessed three-channel image data.

[0026] The brightness normalization operation described above is as follows:

[0027] Let X1(t) and X2(t) be two multi-temporal remote sensing images. Standardize each image using a global normalization formula:

[0028]

[0029] Where, μ i and σ i Representing data X respectively i The mean and standard deviation of X i ′ represents the normalized data.

[0030] The image registration operation described above specifically aligns corresponding pixels in multi-temporal images to the same geographic coordinate system. The registration method includes the following steps:

[0031] (1.1) Use the Scale Invariant Feature Transform (SIFT) algorithm or the Orientation-Rotation (ORB) algorithm to extract key point features from the image;

[0032] (1.2) Key point features of multi-temporal images are matched using the nearest neighbor algorithm;

[0033] (1.3) Align the two images using geometric transformation methods, including at least affine transformation and projection transformation.

[0034] Step 2. Construct a TensorTransformer model that combines Tensor Neural Networks and Transformers, such as... Figure 2 As shown, the tensor neural network utilizes a tensor feature extraction layer to extract features. This feature extraction layer employs the inverse process of Tucker decomposition, that is, using a matrix group containing N matrices to extract features from the original data x, resulting in a tensor y. The preprocessed three-channel image data is used as the model input data, and the global features are obtained through model processing. The implementation steps are as follows:

[0035] (2.1) Feature decomposition and optimization of the preprocessed three-channel image data are performed using a tensor neural network to extract data features and obtain the core tensor G; the implementation steps are as follows:

[0036] (2.1.1) Design a tensor neural network that combines multi-layer tensor decomposition and nonlinear mapping, which includes an input layer, a multi-layer Tucker decomposition module, a ReLU activation function layer and a normalization layer.

[0037] (2.1.2) The input layer receives the preprocessed three-channel images and stacks them in time sequence to construct the input tensor X:

[0038] X∈R H×W×2C ,

[0039] Where H and W represent the height and width of the image, respectively; C is the number of channels in each image;

[0040] (2.1.3) The multi-layer Tucker decomposition module transforms the input tensor X into a feature representation through tensor decomposition operations, that is, it uses Tucker decomposition to decompose tensor X into a core tensor G and a factor matrix:

[0041] X≈G×1U1×2U2×3U3,

[0042] Wherein, 1U1 is the factor matrix of H and the local spatial features of the preserved image; 2U2 is the factor matrix of W and the local spatial features of the preserved image; 3U3 is the factor matrix of the channel dimension;

[0043] (2.1.4) After each decomposition layer, the ReLU activation function and batch normalization operation are applied to enhance the feature representation capability and accelerate training convergence, and the core tensor G is output.

[0044] (2.2) The self-attention mechanism of the Transformer is used to globally model the features extracted by the tensor decomposition network to mine long-distance dependencies and variation features in the image; that is, the core tensor G of the tensor neural network is directly used as the input unit of the Transformer to preserve the spatial and spectral structure of the image, and then the global feature tensor F is obtained based on the Transformer. The implementation steps are as follows:

[0045] (2.2.1) The core tensor G generated by the tensor decomposition network:

[0046]

[0047] Where r1 represents the spatial dimension of H after compression; r2 represents the spatial dimension of W after compression; and r3 is the channel dimension.

[0048] (2.2.2) Directly treating r3 as the number of tensor blocks, and using each tensor block as the input of a self-attention unit, the multi-head self-attention mechanism (MHSA) of the Transformer is used to operate on the tensor blocks to obtain the global feature tensor F:

[0049]

[0050] Where d is the number of output channels of the Transformer.

[0051] The above uses the Transformer's Multi-Head Self-Attention (MHSA) mechanism to operate on tensor blocks, where the calculation formula for each attention head is as follows:

[0052]

[0053] Where Q, K, and V represent the query, key, and value matrices, respectively; d kThis represents the normalization factor for attention weights.

[0054] Step 3. Iteratively train the Tensor Transformer model, optimizing it by continuously adjusting the model parameters to obtain the final model for the change detection task. In this embodiment, specifically, supervised learning is used to minimize the difference between the predicted change detection map Y and the true labeled change detection map Y0. true The error between them is optimized; the steps are as follows:

[0055] (3.1) Load the training dataset, which includes multi-temporal remote sensing image pairs [X1,X2] and corresponding change detection labels Y. true ;

[0056] (3.2) The input image pair [X1,X2] is passed through a tensor neural network and a Transformer in sequence, and finally a change detection map P is generated by a classifier;

[0057] (3.3) Through the cross-entropy loss function L BCE Calculate the predicted map P and the labeled map Y true The error between them; the cross-entropy loss function L BCE The binary cross-entropy loss (BCE) is used as the objective function, defined as follows:

[0058]

[0059] Where N represents the number of pixels; Y true (i) is the change detection label, which takes the value of 0 or 1; P(i) is the output probability of the model, which takes the value between [0,1].

[0060] (3.4) Backpropagate the error to the weights of the tensor neural network and the Transformer, and use the backpropagation algorithm to adjust the parameters of each layer;

[0061] (3.5) Update the trainable parameters of the network using the Adam optimizer or SGD optimizer to minimize the loss function.

[0062] Step 4. Perform pixel-level classification on the global feature tensor obtained from the final model using a classifier to generate a binary change detection map of the same size as the input image. The implementation is as follows:

[0063] (4.1) The feature tensor F is directly converted into a probability map P of the same size as the input image using a convolution classifier:

[0064] P=σ(Conv 1×1 (F))∈R H×W ,

[0065] Among them, Conv1×1 σ is a 1×1 convolution operation used to compress the number of channels d to 1; σ is the Sigmoid activation function used to map the output of the convolution to the interval [0,1].

[0066] (4.2) The probability map P is converted into a binary change detection map Y using a binarization operation:

[0067]

[0068] Where P(x,y) represents the probability of change of pixel (x,y) in the probability map, τ is a threshold; 1 represents the region of change, and 0 represents the region of no change.

[0069] Example 2: The overall implementation steps of the detection method provided in this example are the same as in Example 1. An example is given below to further describe the implementation process of this invention in detail:

[0070] Step 1. Preprocess the input multi-temporal remote sensing images. To ensure the consistency of the input image data and eliminate unnecessary noise interference, preprocessing operations are first performed on the two multi-temporal remote sensing data X1(t) and X2(t), including:

[0071] Brightness Normalization: Remote sensing images often exhibit brightness variations due to sensor conditions or changes in lighting. To reduce the impact of these variations on change detection results, a global normalization formula is used to standardize each image:

[0072]

[0073] Where, μ i and σ i Representing data X respectively i The mean and standard deviation of X i ′ represents the normalized data.

[0074] Image Registration: Multi-temporal remote sensing images may exhibit spatial misalignment due to factors such as sensor viewpoint, capture time, or terrain undulation. To ensure the accuracy of image change detection, multi-temporal images need to be registered. Image registration aims to align corresponding pixels in multi-temporal images to the same geographic coordinate system. Registration methods typically include the following steps: 1a) Feature point extraction: Extracting key point features from the image using algorithms such as SIFT (Scale-Invariant Feature Transform) or ORB (Oriented FAST and Rotated BRIEF). 1b) Feature point matching: Matching key points in the multi-temporal images using nearest neighbor algorithms or other matching strategies. 1c) Registration transformation: Aligning the two images using geometric transformation methods such as affine transformation and projection transformation.

[0075] By performing preprocessing steps such as brightness normalization and image registration, we can effectively reduce the illumination and geometric differences in the input image, ensuring that subsequent tensor decomposition and self-attention mechanisms can accurately extract the change features.

[0076] Step 2. Tensor Neural Network Extracts Data Features to Reduce Computational Load. In remote sensing image change detection tasks, three-channel data typically contains rich spatial and spectral information (e.g., RGB or near-infrared combinations), and direct dimensionality reduction may lead to information loss. Therefore, in this step, the core objective of the tensor decomposition network is not simply dimensionality reduction, but rather to extract key features and reduce computational complexity by efficiently decomposing and optimizing the representation of the three-channel data. The specific steps are as follows:

[0077] 1) Stack the preprocessed three-channel images X1(t) and X2(t) in time order to construct the input tensor X:

[0078] X∈R H×W×2C

[0079] Where: H and W represent the height and width of the image, respectively; C is the number of channels per image (e.g., C = 3 for RGB channels); 2C represents the number of channels in a dual-temporal image. This tensor X serves as the input to the tensor decomposition network, preserving the structural form of multi-channel and spatial information, thus avoiding the spatial correlation lost in traditional flattening operations.

[0080] 2) Tensor Decomposition. Tensor decomposition transforms the input tensor X into a more compact feature representation, rather than directly reducing dimensionality. The main goal is to extract key features across channels and in the spatial dimension. Tucker decomposition is used to decompose tensor X into a core tensor and a factor matrix:

[0081] X≈G×1U1×2U2×3U3

[0082] Here, G is the core tensor, representing the efficient and compact features of the image; 1U1 and 2U2 are the spatial dimension factor matrices, preserving the local spatial features of the image; and 3U3 is the channel dimension factor matrix, extracting relevant information between channels. This decomposition method can reduce redundant computation and storage requirements while maintaining image structural information.

[0083] 3) Tensor Neural Networks further optimize image features by combining multi-layer tensor decomposition and nonlinear mapping. The specific design includes: a) Input Layer: Receives a tensor representation X of a three-channel stacked image. b) Multi-layer Tensor Decomposition: Features are progressively optimized by stacking multiple Tucker decomposition modules. The core tensor G is compressed in each layer, extracting more abstract spatial and channel features. c) Activation and Normalization: After each decomposition layer, the ReLU activation function and batch normalization operation are applied to enhance feature representation capabilities and accelerate training convergence. The key advantage of Tensor Neural Networks is that they preserve the spatial-channel structure of the image while effectively reducing redundant information, providing high-quality feature representations for subsequent models.

[0084] 4) Although three-channel data does not require direct dimensionality reduction, tensor decomposition networks significantly reduce computational complexity by optimizing data representation. The main reasons include: 1) Compact feature representation: The core tensor G has a significantly smaller dimension than the original tensor X, reducing subsequent computational load. 2) Avoiding flattening operations: Traditional methods require flattening the image into a one-dimensional sequence, resulting in a computational complexity of O(n^2). 2 (where n is the number of pixels). The length of the flattened sequence is n, corresponding to the total number of pixels in the image. If a self-attention mechanism such as Transformer is used, its computational complexity is O(n^2). 2 This is because it requires calculating the correlation between each unit in the sequence. This is especially important for high-resolution images (e.g., 1024x1024 data points, n=10). 6 The processing of images by flattening them results in extremely high computational demands. Flattening loses spatial structural information (such as the relationship between adjacent pixels), requiring the model to relearn these relationships through additional computation, further increasing the computational burden. Tensor decomposition operates directly on tensor form, preserving structural information and reducing redundant computation. 3) Efficient feature extraction: By jointly modeling spatial and channel information, tensor decomposition extracts key variation features while significantly reducing unnecessary computation.

[0085] 5) Output. The output of the Tensor Decomposition Network is the core tensor G, which preserves the key spatial and spectral features of the image and has a dimensionality much smaller than the original input tensor X. The compact representation of the core tensor G not only effectively reduces computational complexity but also provides high-quality input for the subsequent Transformer module. Through tensor decomposition, the multidimensional features of the image are optimized into an efficient feature representation. Compared with traditional methods, the core tensor G reduces redundant information while avoiding the loss of structural information caused by flattening operations, laying the foundation for subsequent global feature extraction and change detection.

[0086] Step 3. Feature Processing Based on Tensor Units in Transformer. The core objective of this step is to leverage the self-attention mechanism of the Transformer to further global model the features extracted by the tensor decomposition network, thereby uncovering long-range dependencies and variation features in the image. To address the high computational complexity caused by the traditional Transformer flattening of images into a one-dimensional sequence, this invention directly uses the core tensor G of the tensor decomposition network as the input unit, preserving the spatial and spectral structure of the image, thus significantly reducing computational costs.

[0087] s1. Input data. In step 2, the input remote sensing image X is processed by a tensor decomposition network to generate the core tensor G:

[0088]

[0089] Here, r1 and r2 represent the compressed spatial dimensions, and r3 is the channel dimension (or the number of tensor blocks). This tensor G serves as the input to the Transformer model in this step. Unlike traditional Transformers that require flattening the image into a one-dimensional sequence, this invention directly performs block-level operations on G, preserving the spatial and channel information of the image and avoiding the high computational complexity caused by serialization.

[0090] s2. Transformer Self-Attention Mechanism. In the traditional Transformer, the computational complexity of the self-attention mechanism is O(n log n). 2 ), where n is the number of pixels after flattening. This high complexity leads to severe computational overhead in high-resolution images. To address this problem, the improvements in this invention are as follows: 1) Tensor block-based input design. Unlike traditional pixel serialization, tensor G preserves the tensor structure of the image. For dimension G∈R r1×r2×r3 Instead of flattening r1 and r2, the input is directly treated as the number of "tensor blocks," with each "tensor block" serving as the input to a self-attention unit. This way, the attention mechanism's operations are no longer O(n^2). 2 Instead of operating on one-dimensional sequences, this method operates on tensor blocks, significantly reducing computational complexity. 2) Multi-Head Self-Attention (MHSA). This invention employs a multi-head self-attention mechanism, using multiple independent attention heads to focus on different features in the image. The calculation formula for each attention head is as follows:

[0091]

[0092] Where Q, K, and V represent the query, key, and value matrices, respectively; d kThis represents the normalization factor for the attention weights; these operations are performed on the channel dimension r3 of the tensor block, reducing the computational complexity to O(r3). 2 This is significantly lower than the traditional O(n) 2 ).

[0093] S3. Output. The global feature tensor F is obtained through multi-head self-attention mechanism:

[0094]

[0095] Where r1 and r2 represent the spatial dimensions of the image, and d is the number of output channels of the Transformer (usually greater than r3).

[0096] Step 4. In this step, the global feature tensor F is classified at the pixel level by a classifier to generate a binary change detection map with the same size as the input image.

[0097] To improve the accuracy of detection, this invention also involves model optimization and training. By continuously adjusting the model parameters, the performance of the model in change detection tasks can be improved.

[0098] 1. Input. The global feature tensor F obtained in step 3 is used as the input for this step, and is expressed as:

[0099]

[0100] Where r1 and r2 represent the spatial dimensions of the image, and d represents the channel dimension of the feature, which is usually greater than the number of channels in the original input.

[0101] 2. The process of generating change detection maps:

[0102] Convolution classifier: Its purpose is to directly convert the feature tensor F into a change detection map P of the same size as the input image. Specifically, it performs channel compression on the feature tensor F using a 1×1 convolution operation.

[0103] P=σ(Conv 1×1 (F))∈R H×W

[0104] Among them, Conv 1×1 It is a 1×1 convolution operation, the purpose of which is to compress the number of channels d to 1; σ is the Sigmoid activation function, which maps the output of the convolution to the interval [0,1], representing the probability of change.

[0105] Thresholding generates a change detection map: The goal is to convert the probability map P into a binary change detection map Y to identify areas of change in the image. This is achieved using a simple binarization operation, defined as follows:

[0106]

[0107] Where P(x,y) represents the probability of change of pixel (x,y) in the probability map, τ is a threshold, usually set to 0.5; 1 represents the region of change, and 0 represents the region of no change.

[0108] 3. Model Training and Optimization. Model training and optimization are essential parts of generating detection results. The goal of model training is to minimize the difference between the predicted change detection map Y and the true labeled change detection map Y0 through supervised learning. true The error between them. 1) Definition of the loss function. This invention uses binary cross-entropy loss (BCE) as the main objective function to calculate the error between the detected map P and the labeled map Y predicted by the model. true The error between them. The loss function is defined as follows:

[0109]

[0110] Where N represents the number of pixels; Y true (i) is the change detection label, with a value of 0 or 1; P(i) is the output probability of the model, with a value between [0,1]. 2) Steps of the training process. The model training process usually consists of the following steps: loading the training dataset, including multi-temporal remote sensing image pairs [X1,X2] and corresponding change detection labels Y. true Forward propagation: The input image pair [X1,X2] is passed sequentially through a tensor decomposition network and a Transformer, and finally through a classifier to generate a change detection map P; Loss calculation: The cross-entropy loss function L defined earlier is used. BCE Calculate the predicted map P and the labeled map Y true Error between layers; Backpropagation: The error is backpropagated to the weights of the tensor network and the Transformer, and the parameters of each layer are adjusted using the backpropagation algorithm; Parameter update: The trainable parameters of the network are updated using the Adam optimizer or SGD optimizer to minimize the loss function.

[0111] 4. Output. The generated detection result image Y is a binary image with the same size as the input image:

[0112] Y∈{0,1} H×W

[0113] In this model, 1 represents a region that has changed, and 0 represents a region that has not changed. Furthermore, the output detection map Y can be saved as an image file (such as PNG or TIFF format) or used for visualization. In practical change detection tasks, the change detection map Y is often used as a comparative visualization result, placed alongside the original images X1 and X2 to visually show which regions have changed.

[0114] The effects of the present invention will be further explained below with reference to simulation experiments.

[0115] 1. Simulation conditions:

[0116] The simulation experiments of this invention were conducted in a hardware environment of GPU3070ti and a software environment of Python 3.8.

[0117] 2. Simulation content:

[0118] During the simulation, this invention employs a remote sensing image change detection method based on Tensor Transformer to simulate the task of detecting changes in remote sensing images under the same scene at different times. Specific steps include:

[0119] 1) Data preprocessing: Standardize the remote sensing images to adapt them to the input requirements of the model.

[0120] 2) Model training: The proposed Tensor Transformer model is trained using the training dataset.

[0121] 3) Model validation: Evaluate model performance using a test dataset.

[0122] 3. Simulation Results: Examples of simulation results are shown below. Figure 4 and Figure 5 .

[0123] See attached document Figure 2 Commonly used data includes graph A before the change, graph B after the change, and change labels, as shown below. Figure 2 As shown in (a)-(c). (a): Image before change, one of the input images in the change detection task of this invention, namely image A before change, showing the land cover information of a specific area. (b): Image after change, another input image in the change detection task of this invention, namely image B after change, showing the land cover information of the same area at different time periods. (c): Change annotation map, which is the change annotation map (Ground Truth) used to train the model, where white areas represent pixels that have changed and black areas represent pixels that have not changed.

[0124] See attached document Figure 2The diagram illustrates the structure of the proposed model, specifically including the following modules: 2a) Input module. The input image pairs include the image before change (Input A) and the image after change (Input B). These two image pairs represent the surface information of a certain area at different times, and are input into the model for change detection analysis. 2b) TBDM module. This module uses a Tensor Neural Network (TBDM) to extract features and perform differential analysis on the input image pairs. This module uses tensor operations to directly extract the differential features of the images in high-dimensional space, avoiding the flattening operation in traditional methods, effectively reducing computational complexity, and preserving the spatial and spectral information of the images. 2c) Encoder module. The encoder further extracts the features output by the TBDM module, compressing the high-dimensional features into a low-dimensional representation. The encoder captures local and global features of the images through convolution operations and a multi-layer network structure, providing support for the subsequent decoder to generate change detection maps. 2d) Decoder module. The decoder is responsible for reconstructing the low-dimensional features output by the encoder into a change detection map with the same size as the input image. Through progressive upsampling operations, the decoder can restore the spatial resolution of the image and generate a change detection map. 2e) Output Module. The final output includes a predicted change detection map (Predict) and a labeled change detection map (Label). The prediction results are used to compare and evaluate model performance, where white areas represent changed surface pixels and black areas represent unchanged surface pixels. Figure 2 The model structure of the present invention and the synergistic effect of each module are clearly demonstrated, intuitively reflecting the innovation and practical application value of the present invention in remote sensing image change detection.

[0125] Reference Figure 4 and Figure 5 These two figures illustrate the prediction results of this invention in remote sensing image change detection tasks, clearly demonstrating the model's ability to detect changed areas. Specifically, they include:

[0126] 1. Images before the change, such as Figure 4 (a) and Figure 5 As shown in (a):

[0127] Figure 4 and Figure 5 (a) shows the images before the change in the two scenes, demonstrating the surface information of the target area before the change.

[0128] 2. The altered image, such as Figure 4 (b) and Figure 5 As shown in (b):

[0129] Figure 4 and Figure 5(b) shows the changed images of the two scenes, reflecting the changed surface information of the same area.

[0130] 3. Change detection results predicted by the model, such as Figure 4 (c) and Figure 5 As shown in (c):

[0131] Figure 4 and Figure 5 In the diagram, (c) represents the change detection results predicted by the present invention for two different scenarios, where: the white area represents the detected change area; and the black area represents the area that has not changed.

[0132] 4. Marking actual changes, such as Figure 4 (d) in Figure 5 As shown in (d):

[0133] Figure 4 and Figure 5 In the diagram, (d) represents the change detection maps of the real-world annotations for two different scenarios, used to evaluate the model performance.

[0134] The network model proposed in this invention performs as follows in terms of performance metrics:

[0135] Performance metrics:

[0136] 1. Precision: 0.8850. The model effectively avoids false positives and accurately identifies pixels in changing areas.

[0137] 2. Recall: 0.9524. The model performs excellently in covering regions of change, capturing a large proportion of the change information.

[0138] 3. F1 score (F1): 0.9175. The F1 score, as a comprehensive indicator of precision and recall, reflects the model's excellent performance in balancing false positives and false negatives.

[0139] 4. Mean Intersection over Union (MIoU): 0.8475. The MIoU index reflects the degree of overlap between the predicted results and the actual area of ​​change, indicating that the model has a high adaptability in different scenarios.

[0140] Training metrics:

[0141] 1. Training time per epoch: 9 minutes and 32 seconds. This result demonstrates the model's efficient training performance on large-scale remote sensing datasets.

[0142] 2. Total model parameters: 0.9M. The model design is compact with a small number of parameters, making it suitable for resource-constrained embedded environments.

[0143] 3. Test images: 2048 pairs; total prediction time: 66 seconds. In practical applications, this model has the ability to perform fast inference and can efficiently complete change detection tasks for large-scale images.

[0144] The model achieved high performance in key metrics such as precision, recall, F1 score, and MIoU, demonstrating its superiority in change detection tasks. In particular, its low computational complexity and efficient inference capabilities make this model highly promising for real-world applications.

[0145] This invention can achieve the following main objectives:

[0146] (i) Effectively reduce spatial information loss and improve detection accuracy

[0147] Traditional Transformer models require flattening two-dimensional images into one-dimensional vectors, resulting in the loss of significant spatial structure information, especially in high-resolution and multispectral images. This information loss significantly impacts detection accuracy. This invention employs tensor neural networks, preserving the spatial and spectral information of the image and representing image features using tensors, effectively avoiding the information loss caused by the flattening operation. Therefore, compared to existing Transformer methods, this invention significantly improves the accuracy of change detection, particularly excelling in handling complex surface change scenarios.

[0148] (ii) Reduce computational complexity and improve computational efficiency

[0149] While Transformer models perform well in extracting local features from high-resolution images, they often require complex computations, resulting in high computational complexity and long computation time. This invention optimizes the feature extraction process by combining tensor decomposition with the Transformer's self-attention mechanism. Table 2 shows a comparison of different models in terms of parameter count and computational complexity under the same computing resources. Compared to other Transformer models, this invention significantly reduces the number of parameters and achieves lower computational complexity.

[0150] Table 1 shows a comparison of the computational complexity of the traditional Transformer model and the model of this invention, and Table 2 shows a comparison of the number of model parameters and training complexity.

[0151] Table 1. Comparison of computational complexity

[0152]

[0153] Table 2. Comparison of Model Parameter Count and Training Complexity

[0154]

[0155] Tensor decomposition is used to reduce the dimensionality of input data, thereby reducing the computational complexity of the network and ensuring lower computational costs. Compared with traditional deep learning methods, this invention can more efficiently complete change detection tasks and is suitable for resource-constrained environments, such as embedded remote sensing data processing systems.

[0156] (iii) Improve the ability to capture global features and enhance model robustness.

[0157] In existing technologies, convolutional neural networks (CNNs) suffer from limitations in their receptive field, making it difficult to effectively capture global information in images. This is particularly problematic in change detection tasks, where the lack of global information often leads to inaccurate identification of ground feature changes. This invention employs a self-attention mechanism, enabling the modeling of global dependencies in images during feature extraction, and combines this with a tensor neural network to ensure the comprehensive utilization of multi-dimensional information. This combination results in a more robust model when capturing the global features of complex ground features, effectively reducing false positives and false negatives caused by incomplete local features.

[0158] This invention, based on tensor decomposition and self-attention mechanisms, presents a promising remote sensing image change detection method with potential applications across multiple fields. Firstly, in urban expansion and environmental monitoring, this invention can be used to monitor real-time changes in urban development and the dynamic changes in ecological environments such as forest cover, providing timely data support for scientific planning and environmental decision-making. Secondly, this invention has significant application value in natural disaster assessment and emergency response, enabling rapid and accurate assessment of surface changes in disaster areas, providing a reliable basis for formulating rescue measures and post-disaster reconstruction. In agricultural management, by monitoring crop growth and pest and disease conditions, this invention helps farmers and agricultural managers take timely and effective measures, improving agricultural production efficiency and management precision. Furthermore, due to its low computational complexity and small parameter count, this invention is well-suited for deployment in embedded devices and edge computing scenarios, enabling real-time change detection in resource-constrained environments. Moreover, this invention has broad application potential in other remote sensing image processing tasks, such as image classification and target detection, providing efficient and economical solutions for fields such as Geographic Information Systems (GIS), military reconnaissance, and resource surveys, thus promoting the further development and application of remote sensing technology.

[0159] The above comparative analysis proves the correctness and effectiveness of the method proposed in this invention.

[0160] The parts of this invention not described in detail are common knowledge to those skilled in the art. The above descriptions are merely preferred embodiments of this invention and are not intended to limit the invention. Obviously, those skilled in the art, after understanding the content and principles of this invention, may make various modifications and changes in form and detail without departing from the principles and structure of this invention. However, these schemes based on the ideas of this invention and using a multi-agent reinforcement learning algorithm and a time slot allocation algorithm for joint design are all within the scope of protection of this invention.

Claims

1. A method for detecting changes in remote sensing images based on tensor decomposition and self-attention mechanism, characterized in that, Includes the following steps: (1) Preprocess the input multi-temporal remote sensing images to reduce the illumination and geometric differences in the input images and obtain the preprocessed three-channel image data. (2) Construct a tensor-transformer model that combines tensor neural networks and Transformer. Use the preprocessed three-channel image data as the model input data, and obtain global features through model processing. The implementation steps are as follows: (2.1) The preprocessed three-channel image data is decomposed and optimized by tensor neural network to extract data features and obtain core tensor G; (2.2) The self-attention mechanism of Transformer is used to perform global modeling of the features extracted by tensor decomposition network to mine long-distance dependencies and change features in the image; that is, by directly using the core tensor G of the tensor neural network as the input unit of Transformer to preserve the spatial and spectral structure of the image, and then performing calculations based on Transformer to obtain the global feature tensor F. (3) Iteratively train the tensor Transformer model and optimize it by continuously adjusting the model parameters to obtain the final model for the change detection task; (4) The global feature tensor obtained from the final model is classified at the pixel level by a classifier to generate a binary change detection map with the same size as the input image.

2. The method according to claim 1, characterized in that: The preprocessing described in step (1) includes brightness normalization and image registration operations.

3. The method according to claim 2, characterized in that: the brightness normalization operation is specifically as follows: Let X1(t) and X2(t) be two multi-temporal remote sensing images. Standardize each image using a global normalization formula: in, μ i and σ i Representing data X respectively i The mean and standard deviation of X i ′ represents the normalized data.

4. The method according to claim 2, characterized in that: The image registration operation specifically involves aligning corresponding pixels in multi-temporal images to the same geographic coordinate system. The registration method includes the following steps: (1.1) Use the Scale Invariant Feature Transform (SIFT) algorithm or the Orientation-Rotation (ORB) algorithm to extract key point features from the image; (1.2) Key point features of multi-temporal images are matched using the nearest neighbor algorithm; (1.3) Align the two images using geometric transformation methods, including at least affine transformation and projection transformation.

5. The method according to claim 1, characterized in that: The tensor neural network described in step (2) uses a tensor feature extraction layer to extract features. This feature extraction layer uses the inverse process of Tucker decomposition, that is, it uses a matrix group containing N matrices to extract features from the original data x to obtain tensor y.

6. The method according to claim 1, characterized in that: Step (2.1) involves using a tensor neural network to perform feature decomposition and optimization on the preprocessed three-channel image data, extracting data features, and obtaining the core tensor G; the implementation steps are as follows: (2.1.1) Design a tensor neural network that combines multi-layer tensor decomposition and nonlinear mapping, which includes an input layer, a multi-layer Tucker decomposition module, a ReLU activation function layer and a normalization layer. (2.1.2) The input layer receives the preprocessed three-channel images and stacks them in time sequence to construct the input tensor X: X∈R H×W×2C , Where H and W represent the height and width of the image, respectively; C is the number of channels in each image; (2.1.3) The multi-layer Tucker decomposition module transforms the input tensor X into a feature representation through tensor decomposition operations, that is, it uses Tucker decomposition to decompose tensor X into a core tensor G and a factor matrix: X≈G×1U1×2U2×3U3, Wherein, 1U1 is the factor matrix of H and the local spatial features of the preserved image; 2U2 is the factor matrix of W and the local spatial features of the preserved image; 3U3 is the factor matrix of the channel dimension; (2.1.4) After each decomposition layer, the ReLU activation function and batch normalization operation are applied to enhance the feature representation capability and accelerate training convergence, and the core tensor G is output.

7. The method according to claim 6, characterized in that: Step (2.2) involves computation based on Transformer to obtain the global feature tensor F. The implementation steps are as follows: (2.2.1) The core tensor G generated by the tensor decomposition network: Where r1 represents the spatial dimension of H after compression; r2 represents the spatial dimension of W after compression; and r3 is the channel dimension. (2.2.2) Directly treating r3 as the number of tensor blocks, and using each tensor block as the input of a self-attention unit, the multi-head self-attention mechanism (MHSA) of the Transformer is used to operate on the tensor blocks to obtain the global feature tensor F: Where d is the number of output channels of the Transformer.

8. The method according to claim 7, characterized in that: Step (2.2.2) describes using the Transformer's Multi-Head Self-Attention (MHSA) mechanism to operate on tensor blocks, where the calculation formula for each attention head is as follows: Where Q, K, and V represent the query, key, and value matrices, respectively; d k This represents the normalization factor for attention weights.

9. The method according to claim 1, characterized in that: Step (3) involves iteratively training the tensor Transformer model, specifically by minimizing the predicted change detection map Y and the true labeled change detection map Y through supervised learning. true The error between them is optimized; the steps are as follows: (3.1) Load the training dataset, which includes multi-temporal remote sensing image pairs [X1,X2] and corresponding change detection labels Y. true ; (3.2) The input image pair [X1,X2] is passed through a tensor neural network and a Transformer in sequence, and finally a change detection map P is generated by a classifier; (3.3) Through the cross-entropy loss function L BCE Calculate the predicted map P and the labeled map Y true The error between them; the cross-entropy loss function L BCE The binary cross-entropy loss (BCE) is used as the objective function, defined as follows: Where N represents the number of pixels; Y true (i) is the change detection label, which takes the value of 0 or 1; P(i) is the output probability of the model, which takes the value between [0,1]. (3.4) Backpropagate the error to the weights of the tensor neural network and the Transformer, and use the backpropagation algorithm to adjust the parameters of each layer; (3.5) Update the trainable parameters of the network using the Adam optimizer or SGD optimizer to minimize the loss function.

10. The method according to claim 1, characterized in that: The binary change detection map described in step (4) is obtained according to the following operation: (4.1) The feature tensor F is directly converted into a probability map P of the same size as the input image using a convolution classifier: P=σ(Conv 1×1 (F))∈R H×W , Among them, Conv 1×1 σ is a 1×1 convolution operation used to compress the number of channels d to 1; σ is the Sigmoid activation function used to map the output of the convolution to the interval [0,1]. (4.2) The probability map P is converted into a binary change detection map Y using a binarization operation: Where P(x,y) represents the probability of change of pixel (x,y) in the probability map, τ is a threshold; 1 represents the region of change, and 0 represents the region of no change.

Images