A remote sensing image change detection method based on edge perception and cross-time phase transformer

Abstract

The application belongs to the field of computer vision and new generation information technology, and in particular to a remote sensing image change detection method based on edge perception and cross-time phase Transformer. The method comprises the following steps: S1: feature extraction: obtaining a first feature map and a second feature map through a shallow feature extractor with shared weights; S2: adaptive physical edge binary mask generation; S3: sparse routing truncation; S4: asymmetric cross-attention calculation; S5: decoding output: mapping the difference features after sparse calculation back to the original two-dimensional space dimension, and outputting the change detection result through a decoder. The application realizes substantial technical progress in model convergence under extremely low computing power. The application screens the redundant background feature words that originally need to participate in operation, so that they are exempted from participating in the matrix multiplication operation of the subsequent attention mechanism, so that a consumer-level computing device can complete the training and deployment of an industrial-level remote sensing large model.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and next-generation information technology, specifically to a method for detecting changes in remote sensing images based on edge perception and cross-temporal Transformer. Background Technology

[0002] Remote sensing image change detection aims to extract information on changes in ground features from images of the same geographic area acquired at different times. Current technologies mainly fall into two categories, and their serious technical shortcomings are as follows: (1) Convolutional Neural Network (CNN) based methods: These methods rely on local receptive fields to extract features. Drawback: After multiple downsampling, high-frequency spatial structure information is inevitably lost. When processing dense small targets (such as small building clusters) or complex landforms commonly found in remote sensing images, problems such as blurred boundaries and target "sticking" are very likely to occur, making it impossible to achieve accurate contour cutting.

[0003] (2) Methods based on the conventional Transformer architecture: These methods establish global dependencies through a self-attention mechanism. Drawback: The computational complexity of the standard Transformer attention mechanism is quadratic with the sequence length of the input image (i.e., the number of pixels). The growth of high-resolution remote sensing images (such as those with a size of 1024×1024 or even larger) generates a massive number of feature tokens, which makes it extremely easy for memory overflow (OOM) or instantaneous computational overload to trigger hardware protection mechanisms on conventional computing devices. This results in extremely low computational efficiency and severely restricts the feasibility of its industrial application. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of the embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] To address the aforementioned technical problems, according to one aspect of the present invention, the present invention provides the following technical solution: a remote sensing image change detection method based on edge perception and cross-temporal Transformer, comprising the following steps: S1: Feature Extraction: Obtain the registered first-phase and second-phase remote sensing images, and obtain the first-phase feature map through a shallow feature extractor with shared weights. Compared with the second phase feature map ; S2: Adaptive physical edge binary mask generation: The first temporal feature map is used to generate a binary mask. The input is fed into an edge-aware operator with a dynamic adaptive threshold, which dynamically predicts the appropriate edge threshold for the current image through spatial global average pooling combined with a multilayer perceptron. and the gradient magnitude is greater than Region activation generates a physical edge binary mask; S3: Sparse route truncation: The feature map is divided into a sequence of multiple feature words. Instead of performing full sequence feature interaction, the physical edge binary mask is used as a "hardware-level routing switch". Only the "active word set" corresponding to the mask activation region is retained, and the computation flow and memory are stopped for the "redundant word set" corresponding to a large number of smooth backgrounds. S4: Asymmetric cross-attention calculation: The active word set of the first time phase is used as the query vector, and the active word set corresponding to the second time phase is input into the cross-time phase Transformer to perform calculation, extract the dual-time phase difference features, and the roles of the query vector and key-value pair can be interchanged between the feature words of the first and second time phases, or bidirectional symmetric cross-attention can be used to extract the dual-time phase collaborative difference features. S5: Decoding Output: Map the differential features after sparse computation back to the original two-dimensional spatial dimension, and output the change detection results through the decoder.

[0006] As a preferred embodiment of the remote sensing image change detection method based on edge perception and cross-temporal Transformer described in this invention, the shallow feature extractor with shared weights in S1 includes a lightweight convolutional network with BatchNorm and ReLU activation functions, which performs preliminary pattern feature mapping on the original image pixels to obtain a first temporal feature map. Compared with the second phase feature map .

[0007] As a preferred embodiment of the remote sensing image change detection method based on edge perception and cross-temporal Transformer described in this invention, the specific method of S2 is as follows: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] , set as The input is fed into an edge-aware operator with a dynamically adaptive threshold: Let the first phase feature map of the input be... The comprehensive gradient magnitude map is calculated using mutually orthogonal convolution kernels. A minimum constant is introduced in the gradient calculation. :

[0008] Subsequently, predict the dynamic adaptive edge threshold. :

[0009] GAP is a spatial global average pooling operation used to extract global contextual statistical features of the current remote sensing image. and Here is the learnable weight matrix for a multilayer perceptron network. It is a linear rectified activation function. This is the normalization function; Generate route mask :when If the value is 1, it indicates that the pixel belongs to the edge of a building or a high-frequency texture area, and the routing switch is set to 1; otherwise, it is set to 0.

[0010] As a preferred embodiment of the remote sensing image change detection method based on edge perception and cross-temporal Transformer described in this invention, the calculation method of S3 is as follows: [The text abruptly ends here, likely due to an incomplete sentence or missing information.] Based on this, a sparsity penalty loss is introduced. This forces the network to learn the optimal threshold cutoff rate.

[0011] in, The target sparsity is set.

[0012] Compared with existing technologies, the beneficial effects of this invention are: significantly improved edge detail segmentation accuracy: the attention mechanism of this invention occurs only between "active edge feature terms," ​​forcibly suppressing the interference of low-frequency noise such as background ambient lighting and seasonal changes on the change detection results. To verify the feasibility of the core mechanism of this invention under extreme computing power constraints, preliminary convergence verification experiments were conducted on a single consumer-grade graphics processor (such as an 8GB VRAM device). While existing conventional Transformer technologies cannot establish a basic computational graph due to memory overflow (OOM), this invention, benefiting from the sparsity mechanism, successfully established the backpropagation gradient flow and captured microscopic building change features with a very small miniaturized network capacity and only a single epoch (1) of initial training (achieving 7.34% initial IoU verification data). This initial verification data fully demonstrates that physical edge masks can accurately guide the network to focus on effective information, possessing the ability to eliminate adhesion phenomena in building detection both theoretically and in preliminary practice, verifying the substantial technical advancement of this invention in achieving model convergence under extremely low computing power.

[0013] This invention completely breaks through the bottlenecks of video memory and computing power: Through a sparse routing mechanism driven by physical edge binary masks, this invention masks redundant background feature words (often occupying more than 80% of the entire image area) that would otherwise need to participate in computation, thus exempting them from participating in the matrix multiplication operations of the subsequent attention mechanism. In a fair real-world comparative experiment (processing image slices with a resolution of 256×256, a batch size of 2, and all with underlying memory optimization algorithms enabled), the existing Transformer technology (100% feature word retention) would instantly exceed the physical video memory limit and cause system-level hardware crashes (OOM) or instantaneous computational overload triggering hardware protection mechanisms; while the method of this invention, by truncating the invalid computation flow, drastically compresses the peak video memory usage in the feature interaction stage to only 0.23GB. This revolutionary computing power advantage enables consumer-grade computing devices to complete the training and deployment of industrial-grade remote sensing large-scale models. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a flowchart of a remote sensing image change detection method based on edge perception and cross-temporal Transformer according to the present invention. Detailed Implementation

[0015] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0016] Secondly, the present invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not according to the usual scale. Furthermore, the schematic diagrams are merely examples and should not limit the scope of protection of the present invention. In addition, actual fabrication should include three-dimensional spatial dimensions of length, width, and depth.

[0017] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0018] This invention provides a remote sensing image change detection method based on edge perception and cross-temporal Transformer, aiming to solve the contradiction between "accurate preservation of microscopic high-frequency boundaries" and "massive memory consumption caused by global attention mechanism" in the processing of high-resolution remote sensing images by existing models. It achieves high-precision, low memory consumption, and fast response change detection: by introducing image gradient calculation to extract physical edge priors and using edge binary masks as routing switches for token sparsification, it effectively solves the contradiction between "accurate preservation of microscopic high-frequency boundaries" and "massive memory consumption of Transformer" in high-resolution image processing, and achieves accurate recognition and pixel-level classification of complex patterns such as dense buildings.

[0019] The known conditions (i.e., input variables) of this invention: a geo-registered first temporal remote sensing image. Second phase remote sensing images In this embodiment, the input is high-resolution multispectral or RGB image data of the same city area taken one year apart.

[0020] Undetermined variable: Building change prediction mask (i.e., a pixel-level binarized image that identifies newly built or demolished buildings).

[0021] The specific workflow of a remote sensing image change detection method based on edge perception and cross-temporal Transformer is as follows: To further illustrate the specific application of this invention in the field of image data processing and pattern recognition, this embodiment uses "urban building expansion monitoring" as an example for detailed explanation.

[0022] Step 1: Image Feature Extraction and Pattern Recognition Initialization Acquire registered first- and second-phase remote sensing image data. Input both images into a shallow feature extractor with shared weights (e.g., a lightweight convolutional network containing BatchNorm and ReLU activation functions). This step performs preliminary pattern feature mapping on the original image pixels to obtain the first-phase feature map. Compared with the second phase feature map .

[0023] Step 2: Physical edge detection and binary mask generation based on image gradient (core image processing process): To accurately identify building structural patterns in complex urban contexts (such as roads, vegetation, and shadows), the first-phase feature map is used. (set as) The input is fed into the edge-aware operator with a dynamically adaptive threshold.

[0024] Let the first phase feature map of the input be... The composite gradient magnitude map is calculated using mutually orthogonal convolution kernels. To ensure the numerical stability of gradient calculation during network backpropagation, this invention introduces a minimal constant in gradient calculation. (like ):

[0025] Subsequently, predict the dynamic adaptive edge threshold. :

[0026] GAP is a spatial global average pooling operation used to extract global contextual statistical features of the current remote sensing image. and Here is the learnable weight matrix for a multilayer perceptron network. It is the linear rectified activation function (ReLU). It is the normalization function (Sigmoid).

[0027] Generate route mask :when If the pixel is located at a certain time, it indicates that the pixel belongs to a building edge or a high-frequency texture region, and the routing switch is set to 1 (preserving image terms); otherwise, it is set to 0 (discarding redundant terms in smooth background regions to free up computing resources). This step achieves accurate extraction of image regions of interest (ROIs) based on physical priors.

[0028] Step 3: Sparse routing of image tokens based on edge priors: The feature map is divided into a sequence of multiple image feature terms. The physical edge binary mask generated in step two is used as a "hardware-level routing switch" to truncate invalid image processing computation flows. Only a portion of the original sequence length, approximately [percentage missing], is collected in the hardware memory. (e.g., 10%) of active edge words are used for subsequent calculations.

[0029] In this embodiment, the model training process employs a joint loss function optimization. This is based on the traditional segmentation cross-entropy loss... Based on this, the present invention introduces a sparsity penalty loss. This forces the network to learn the optimal threshold cutoff rate.

[0030] in, The target sparsity is set (e.g., 0.10, which means that only about 10% of the high-frequency core region is expected to participate in subsequent large Transformer calculations).

[0031] Step 4: Cross-temporal feature interaction and pixel-level change decoding (pattern classification output): The active word set from the first time phase is used as the query vector, and the corresponding active word set from the second time phase is input into the cross-temporal Transformer to perform asymmetric cross-attention computation. This architecture enables the network to focus on visual morphological changes occurring at high-frequency edges between different time phases, extracting bi-temporal difference features. Finally, the sparsely computed difference features are re-distributed and mapped back to the original two-dimensional image space dimension. After upsampling by the decoder, a binarized prediction mask with the same resolution as the input image is output, thereby accurately identifying the geometric outlines of newly constructed buildings in urban expansion.

[0032] Comparative experiments and verification of beneficial effects under different parameter configurations: To further verify the beneficial effects of the edge-prior-based lexical sparse routing mechanism proposed in this invention and to determine the optimal range of process parameters, ablation comparison experiments were conducted on a unified hardware platform and software environment.

[0033] Experimental environment description: Hardware specifications: AMD Ryzen 98945HX processor, NVIDIA GeForce RTX 5060 graphics processor (8GB VRAM).

[0034] Software environment: Operating system is Windows 11, deep learning framework is PyTorch 2.8.0, CUDA version is 12.8.

[0035] Comparison of process condition changes: This experiment focuses on the core hyperparameter—target sparsity. The proportion of feature terms retained in the masked activation region was transformed. Three examples with different values ​​were used to verify the balance between memory overhead and prediction accuracy. Experiments were conducted using... Verification was performed using remote sensing slices of varying resolution.

[0036] Table 1: Sparsity of Different Objectives Model performance vs. hardware consumption comparison table

[0037] Experimental conclusions and analysis of the critical significance of parameters: Comparing Example C (target sparsity of 0.30) and the comparative example (full graph computation), it can be observed that due to the quadratic complexity of Transformer cross-attention, even with the sparse mechanism of this invention and the significant discarding of 70% of the background words, the remaining 30% of words still cause a sudden surge in memory usage during backpropagation, leading to system crashes and restarts. This demonstrates that simply introducing the concept of sparse routing is insufficient; if the feature retention rate is not precisely lowered and locked within an extremely narrow critical range of around 10%, the physical barrier of memory overflow cannot be truly overcome.

[0038] Through extensive calculations and experiments, this invention determined a critical threshold of 0.10. In Example B, the memory usage experienced a non-linear, precipitous drop, locking at 0.17GB; this is superior to the poor performance of Example A, which suffered from feature loss due to extreme sparsity (IoU of approximately 4%). This fully demonstrates that the sparsity rate around 0.10 is not arbitrarily chosen, but rather the only feasible solution on this constrained heterogeneous hardware that simultaneously balances "avoiding physical crashes and triggering hardware protection mechanisms due to instantaneous computational overload" with "preserving high-frequency feature characteristics," exhibiting strong irreplaceability and significant substantial progress.

[0039] Although the present invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the disclosed embodiments can be combined with each other in any manner. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A remote sensing image change detection method based on edge perception and cross-temporal Transformer, characterized in that, Includes the following steps: S1: Feature Extraction: Obtain the registered first-phase and second-phase remote sensing images, and obtain the first-phase feature map through a shallow feature extractor with shared weights. Compared with the second phase feature map ; S2: Adaptive physical edge binary mask generation: The first temporal feature map is used to generate a binary mask. The input is fed into an edge-aware operator with a dynamic adaptive threshold, which dynamically predicts the appropriate edge threshold for the current image through spatial global average pooling combined with a multilayer perceptron. and the gradient magnitude is greater than Region activation generates a physical edge binary mask; S3: Sparse route truncation: The feature map is divided into a sequence of multiple feature words. Instead of performing full sequence feature interaction, the physical edge binary mask is used as a "hardware-level routing switch". Only the "active word set" corresponding to the mask activation region is retained, and the computation flow and memory are stopped for the "redundant word set" corresponding to a large number of smooth backgrounds. S4: Asymmetric cross-attention calculation: The active word set of the first time phase is used as the query vector, and the active word set corresponding to the second time phase is input into the cross-time phase Transformer to perform calculation, extract the dual-time phase difference features, and the roles of the query vector and key-value pair can be interchanged between the feature words of the first and second time phases, or bidirectional symmetric cross-attention can be used to extract the dual-time phase collaborative difference features. S5: Decoding Output: Map the differential features after sparse computation back to the original two-dimensional spatial dimension, and output the change detection results through the decoder.

2. The remote sensing image change detection method based on edge perception and cross-temporal Transformer according to claim 1, characterized in that, The shallow feature extractor with shared weights in S1 comprises a lightweight convolutional network using BatchNorm and ReLU activation functions to perform preliminary pattern feature mapping on the original image pixels, thereby obtaining the first temporal feature map. Compared with the second phase feature map .

3. The remote sensing image change detection method based on edge perception and cross-temporal Transformer according to claim 1, characterized in that, The specific method of S2 is to convert the first phase feature map , set as The input is fed into an edge-aware operator with a dynamically adaptive threshold: Let the first phase feature map of the input be... The comprehensive gradient magnitude map is calculated using mutually orthogonal convolution kernels. A minimum constant is introduced in gradient calculation. : ； Subsequently, predict the dynamic adaptive edge threshold. : ； GAP is a spatial global average pooling operation used to extract global contextual statistical features of the current remote sensing image. and Here is the learnable weight matrix for a multilayer perceptron network. It is a linear rectified activation function. This is the normalization function; Generate route mask :when If the value is 1, it indicates that the pixel belongs to the edge of a building or a high-frequency texture area, and the routing switch is set to 1; otherwise, it is set to 0.

4. The remote sensing image change detection method based on edge perception and cross-temporal Transformer according to claim 1, characterized in that, The calculation method for S3 is as follows: based on the traditional splitting cross-entropy loss... Based on this, a sparsity penalty loss is introduced. This forces the network to learn the optimal threshold cutoff rate. ； in, The target sparsity is set.

Images