Remote sensing cultivated land change detection method

Abstract

The present application relates to the technical field of remote sensing image processing and deep learning change detection, and discloses a remote sensing cultivated land change detection method. The method maps the front and rear time phase remote sensing images into feature vectors embedded with position coding, extracts features by using a 24-layer visual Transformer with low-rank matrix fine-tuning, selects the 12th, 16th, 20th and 24th layer features, obtains double time phase fusion features through a space-time interaction unit composed of a global module and a Mamba2 time sequence module in parallel, channel attention and cross-level deep fusion, and generates a pixel-level cultivated land change detection map through multi-level decoding. The present application can strengthen the expression of spatial homogeneity and time sequence difference, reduce the loss of cross-scale restoration information, and reduce the risk of cultivated land change omission.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing and deep learning change detection technology, and in particular to a remote sensing method for detecting changes in cultivated land. Background Technology

[0002] Farmland is a crucial target for food production and land use control. Farmland change detection requires identifying areas of change, such as new construction encroachment and changes in land cover, from remote sensing images acquired at different times. In the context of routine dynamic monitoring of farmland protection, remote sensing change detection not only demands accurate positioning but also requires minimizing the risk of missed detections to meet the regulatory requirement of "investigating all that should be investigated."

[0003] The paper "Remote Sensing Image Change Detection with Transformers" (DOI:10.1109 / TGRS.2021.3125522, arXiv:2103.00208), published by Hao Chen, Zhenwei Qi, and Zhengxia Shi in IEEE Transactions on Geoscience and Remote Sensing, Volume 60, discloses a Bi-temporal Image Transformer (BIT) remote sensing change detection scheme. This scheme receives two remote sensing images of the same area at different times, constructs a dual-temporal image Transformer module, performs contextual modeling and spatiotemporal relationship representation of the dual-temporal features through a Transformer encoder, and then uses a Transformer decoder to back-feature the semantic information to pixel-level features for outputting change detection results.

[0004] The aforementioned BIT scheme can enhance feature interaction between two-temporal remote sensing images using the Transformer structure, and has good applicability to general remote sensing change detection. However, in the scenario of farmland change detection, crop growth status changes with seasons and phenological stages, and differences in spectral, texture, and local morphology between images from different time periods may appear as pseudo-changes in similar change areas; at the same time, real changes in farmland non-agriculturalization may be similar to these pseudo-changes in local texture, boundary morphology, or spectral differences. Therefore, when the BIT scheme is applied to farmland change detection, if the model responds too strongly to suspected change areas, it is easy to misclassify crop phenological changes as farmland changes; if the response to suspected changes is reduced overall in order to suppress such pseudo-changes, real farmland changes that are similar to pseudo-changes may also be suppressed simultaneously, resulting in missed detections and reduced recall.

[0005] Furthermore, the BIT scheme primarily relies on a general dual-temporal Transformer interaction and decoding structure. However, under conditions of strong interference from spurious changes in farmland phenology, it still struggles to adequately distinguish the spatial homogeneity of stable features, the temporal dependencies between farmland and its surrounding environment, and mid-to-high-level semantic change features within the same detection process. With limited farmland samples, scene adaptation of the visual feature extraction network faces pressures related to training parameter scale and scene transfer stability. Moreover, when deep semantic features are restored to pixel-level change maps, the large scale recovery span can lead to the loss of boundary and local change information in subtle change areas. Therefore, the remaining technical challenges for BIT are: how to enhance the model's adaptability to farmland change scenarios under the interference of spurious changes in farmland phenology, and how to improve the recall rate of real farmland changes, reduce the risk of missed detections, and improve the recovery effect of change boundaries while maintaining a certain detection accuracy. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a remote sensing method for detecting changes in cultivated land, so as to overcome the shortcomings of the prior art.

[0007] The technical solution of this invention is as follows: a remote sensing method for detecting changes in cultivated land includes: acquiring registered and identically sized pre-temporal and post-temporal remote sensing images of the same area; mapping the pre-temporal and post-temporal remote sensing images into feature vector embeddings with location encoding; extracting features from the feature vector embeddings using a visual Transformer basic network pre-tuned with a low-rank matrix based on cultivated land change samples, wherein the visual Transformer basic network includes multiple Transformer layers; selecting pre-temporal and post-temporal remote sensing image features output by multiple target Transformer layers; and constructing a spatiotemporal intersection of the pre-temporal and post-temporal remote sensing image features for each target Transformer layer. The spatiotemporal interaction unit includes a global module and a temporal module set in parallel. The global module extracts spatial features based on a multi-head attention mechanism, and the temporal module extracts temporal features based on a selective state-space model. The spatial features and temporal features are fused into pre-temporal and post-temporal spatiotemporal features through a channel attention mechanism. Cross-level deep fusion is performed on the pre-temporal and post-temporal spatiotemporal features corresponding to each target Transformer layer to obtain pre-temporal fused features and post-temporal fused features. The pre-temporal fused features and post-temporal fused features are connected by channels and then decoded at multiple levels to generate a probability map of each pixel belonging to the farmland change category. Based on a preset binarization threshold, a farmland change detection map including farmland change category and no farmland change category is obtained.

[0008] Further, the step of mapping the preceding and following time-phase remote sensing images into feature vector embeddings with position encoding includes: dividing a 1024×1024 remote sensing image with 3 bands into 64×64 remote sensing image blocks of size 16×16; flattening each remote sensing image block into a one-dimensional vector of length 768 according to band order; mapping the one-dimensional vector to a 768-dimensional embedding space through a fully connected layer; and adding the position encoding to the mapped high-dimensional embedding vector to obtain the feature vector embedding.

[0009] Furthermore, the visual Transformer base network, pre-tuned with a low-rank matrix based on farmland change samples, is obtained as follows: For the query projection parameter matrix, key projection parameter matrix, value projection parameter matrix, and output projection parameter matrix in each Transformer layer, a first low-rank matrix and a second low-rank matrix are introduced respectively. The original projection parameter matrix is ​​then added to the low-rank update amount obtained by multiplying the first low-rank matrix and the second low-rank matrix to form the fine-tuned parameter matrix. The farmland change samples include pre- and post-temporal remote sensing images and pixel-level farmland change labels. The rank of the low-rank update amount is set to 16. The first low-rank matrix is ​​initialized according to a Gaussian distribution, and the second low-rank matrix is ​​initialized to a zero matrix. During the training phase of fine-tuning using the farmland change samples, the original projection parameter matrix is ​​frozen, and only the first low-rank matrix and the second low-rank matrix are updated.

[0010] Furthermore, each Transformer layer of the visual Transformer base network includes layer normalization, query projection, key projection, value projection, attention weight calculation, output projection, and feedforward neural network processing to obtain the features of the preceding and subsequent time-phase remote sensing images, respectively; wherein, the feedforward neural network includes two fully connected layers, two Dropout layers, and a GELU activation function.

[0011] Further, the construction of the spatiotemporal interaction unit includes: connecting the features of the preceding and following temporal remote sensing images by channel to form a dual-temporal connected feature; performing multi-head attention processing on each spatial location in the dual-temporal connected feature after spatial location coding by the global module to obtain spatial features; performing selective state-space modeling on the dual-temporal connected feature by the temporal module to obtain temporal features; splitting the spatial features and the temporal features by channel into preceding temporal spatial features, following temporal spatial features, preceding temporal temporal features, and following temporal temporal temporal features; connecting the preceding temporal spatial features with the preceding temporal temporal features by channel, and connecting the following temporal spatial features with the following temporal temporal features by channel; restoring the connected features to the original channel dimension through a channel attention mechanism to obtain the preceding spatiotemporal features and the following spatiotemporal features.

[0012] Furthermore, the temporal module adopts a selective state-space model with a Mamba2 structure and includes: expanding the dual-temporal connection features into sequence features with spatial positions as sequence elements and channel dimensions containing preceding and following temporal information; performing linear projection on the sequence features to obtain sequence modeling branch features and gated branch features; performing one-dimensional convolution, normalization, and linear projection on the sequence modeling branch features to generate state-space parameters, which include sampling interval, state transition matrix, input-to-state mapping matrix, and state-to-output mapping matrix; performing negative exponential mapping on each learnable state transition parameter in the state transition matrix to make the state transition parameter negative, and discretizing the state transition matrix and the input-to-state mapping matrix; performing state recursion based on the discretized state transition matrix, the discretized input-to-state mapping matrix, and the convolved sequence modeling branch features to obtain the state output; and using the gated branch features to perform sigmoid-gated modulation on the state output, and obtaining the temporal features through normalization and output projection.

[0013] Furthermore, the channel attention mechanism processes the channel-connected features as follows: global average pooling is performed on the channel-connected features to obtain global features corresponding to each channel; importance weights for each channel are learned through a fully connected layer; the importance weights are multiplied by the channel-connected features channel by channel, and the number of channels is restored to the original channel dimension through a 1×1 convolution layer to obtain the spatiotemporal features after the number of channels is restored.

[0014] Further, the visual Transformer base network includes 24 Transformer layers, and the multiple target Transformer layers are layers 12, 16, 20, and 24. The cross-layer deep fusion includes: performing recursive fusion for each temporal spatiotemporal feature in the preceding and following temporal spatiotemporal features; setting the initial fusion term corresponding to layer 12 to zero, and for each target layer in layers 16, 20, and 24, concatenating the temporal spatiotemporal features corresponding to the previous selected layer of the target layer with the temporal spatiotemporal features corresponding to the target layer by channel, performing a 1×1 convolution, and adding it to the temporal spatiotemporal features corresponding to the target layer to obtain intermediate fusion features; then adding the intermediate fusion features to the fusion features corresponding to the previous selected layer to obtain the fusion features corresponding to the target layer; wherein, the fusion features corresponding to layer 24 are used as the final fusion features for the corresponding temporal phase.

[0015] Further, the multi-level decoding includes: concatenating the pre-temporal fusion features and the post-temporal fusion features by channel to obtain decoding input features; performing first-level upsampling and second-level upsampling sequentially on the decoding input features to gradually restore the feature scale of the decoding input features, and obtaining upsampled features corresponding to the original sizes of the pre-temporal remote sensing images and the post-temporal remote sensing images when the input remote sensing image size is 1024×1024; performing pixel-level mapping on the upsampled features through a 1×1 convolution with pre-trained and determined parameters to obtain a probability map with pixel values ​​normalized to the interval between 0 and 1; assigning a value of 1 to pixels greater than or equal to 0.5 in the probability map as the category of cultivated land change, and assigning a value of 0 to pixels less than 0.5 as the category of no cultivated land change, where a pixel value of 1 represents that the corresponding pixel has undergone cultivated land change, and a pixel value of 0 represents that the corresponding pixel has not undergone cultivated land change.

[0016] Through the above technical solution, the present invention has at least the following beneficial effects.

[0017] First, compared to the existing Bi-temporal Image Transformer (BIT) technology, which mainly relies on a general bi-temporal Transformer interaction and decoding structure for change detection, this invention employs a visual Transformer base network pre-tuned from farmland change samples using a low-rank matrix. This distinguishing feature allows for scene adaptation for farmland change detection with fewer trainable parameters while preserving the existing visual representation capabilities of the visual Transformer base network. This alleviates the training pressure and scene migration stability issues caused by directly adjusting a large-scale visual feature extraction network under limited farmland sample conditions, providing a more stable base feature for distinguishing between pseudo-crop phenological changes and real farmland changes.

[0018] Secondly, addressing the issue that the general two-phase Transformer interaction in BIT struggles to adequately distinguish between the spatial homogeneity of stable features and the temporal dependence of cultivated land, this invention constructs a spatiotemporal interaction unit comprising a parallel global module and a temporal module, fusing spatial and temporal features through a channel attention mechanism. The global module aggregates spatial consistency information of stable features based on a multi-head attention mechanism, while the temporal module uses a selective state-space model with a Mamba2 structure to model the temporal dependence in the two-phase features. The channel attention mechanism further strengthens channels related to actual cultivated land changes and suppresses noisy channels related to pseudo-phenological changes. Thus, spatial homogeneity information and temporal difference information can synergistically work in the same spatiotemporal feature expression, which helps suppress misjudgments caused by pseudo-phenological changes, while simultaneously preventing the simultaneous suppression of actual cultivated land changes that are similar to pseudo-changes.

[0019] Furthermore, addressing the issue that BIT struggles to fully utilize mid-to-high-level semantic change features under the interference of spurious changes in farmland phenology, this invention selects the dual-temporal features output from layers 12, 16, 20, and 24 of the Visual Transformer base network and performs cross-layer deep fusion on the corresponding spatiotemporal features. This distinguishing technique reduces the transmission of spurious change information by noise features such as shallow textures and spectra, while preserving the mid-to-high-level semantic change information expressed by layers 12 to 24. Through channel connections, 1×1 convolutions, and residual addition, step-by-step fusion is achieved, enabling a continuous expression of change semantics across different layers, thereby improving the recall capability of real farmland changes and reducing the risk of missed detections. In ablation experiments, removing the spatiotemporal feature deep fusion processor resulted in a decrease in recall, F1 score, and intersection-over-union ratio.

[0020] Finally, addressing the issue that BIT-based solutions often lose subtle change region boundaries and local change information due to large scale recovery spans when restoring deep semantic features to pixel-level change maps, this invention performs multi-level decoding after concatenating pre- and post-temporal fusion features by channel. It gradually restores the spatial scale of the decoded input features through first-level and second-level upsampling, restoring the original image size when the input remote sensing image size is 1024×1024. Then, it uses 1×1 convolution to generate a pixel-level probability map and obtains the farmland change detection map based on a threshold. This reduces information loss during cross-scale restoration and is beneficial for restoring change region boundaries and subtle change features. Combined with experimental results from the CLCD publicly available remote sensing farmland non-agriculturalization change dataset, the method of this invention achieves a precision of 90.51%, a recall of 97.54%, an F1 score of 93.89%, and a mean intersection-over-union (MIU) ratio of 88.49%. Figure 4 and Figure 5As shown, this invention improves the suppression of false changes and boundary continuity while maintaining a high recall rate. Furthermore, the recall rate, F1 score, and crossover ratio decrease after removing multi-level decoding, further demonstrating that multi-level decoding and deep fusion of spatiotemporal features together correspond to the technical effects of reducing the risk of missed detections, suppressing false positives, and improving boundary recovery. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall process of the method of the present invention;

[0022] Figure 2 This is a schematic diagram of the spatiotemporal interaction unit of the present invention;

[0023] Figure 3 This is a schematic diagram of the spatiotemporal feature deep fusion process of the present invention;

[0024] Figure 4 This is a comparison chart of the visualization detection results of the present invention and the comparative model on the publicly available remote sensing data set of farmland non-agriculturalization change in CLCD;

[0025] Figure 5 This is a comparison chart of the visualization detection results of different module removal schemes and different fusion depth strategies in the ablation experiment of this invention. Detailed Implementation

[0026] The technical solution of the present invention will be further described below with reference to the accompanying drawings. It should be understood that the following specific embodiments are used to illustrate the present invention and are not intended to limit the scope of protection of the present invention; in the absence of conflict, the technical features of each embodiment can be combined with each other. This specific embodiment takes pre-temporal and post-temporal remote sensing images of the same area as the processing objects, and describes the technical features such as feature mapping, low-rank matrix fine-tuning, spatiotemporal interaction, cross-level deep fusion, and multi-level decoding in combination with parameters, model structure, and experimental verification results.

[0027] Example 1:

[0028] like Figure 1 As shown in the figure, this embodiment provides a remote sensing method for detecting farmland change based on deep fusion of spatiotemporal features. This method takes registered and identically sized pre- and post-temporal remote sensing images of the same area as input, and sequentially performs image feature mapping, low-rank matrix fine-tuning of the visual Transformer network based on farmland change samples, target layer feature selection, spatiotemporal interaction, cross-level deep fusion, and multi-level decoding. Finally, it outputs a binary classification detection map indicating whether farmland change has occurred for each pixel.

[0029] S101: Acquire registered and identical-sized pre-temporal and post-temporal remote sensing images of the same region, and map each image into a feature vector embedding with positional encoding. In one specific implementation, both the pre-temporal and post-temporal remote sensing images are 1024×1024 pixels in size and have 3 bands. For any image, divide it into 16×16 blocks to obtain 64×64 remote sensing image blocks; each remote sensing image block is unfolded into a one-dimensional vector in band order, and the length of this one-dimensional vector is 16×16×3=768; then, the one-dimensional vector is mapped to a 768-dimensional embedding space through a fully connected layer, and the positional encoding with the same dimension as the mapped embedding vector is added to obtain the feature vector embedding with positional encoding. The pre-temporal and post-temporal images are processed in the same way, thus forming the input of the subsequent visual Transformer basic network.

[0030] S102, a visual Transformer base network, pre-tuned with a low-rank matrix based on farmland change samples, is used to extract features from the embedded feature vectors. This visual Transformer base network includes multiple Transformer layers, with 24 layers in one specific implementation. A low-rank matrix is ​​introduced for the query projection parameter matrix, key projection parameter matrix, value projection parameter matrix, and output projection parameter matrix in each Transformer layer. and low-rank matrix The fine-tuned query projection parameters, key projection parameters, value projection parameters, and output projection parameters are generated according to the following dimensional and parameter update relationships. During the training phase, which uses farmland change samples for fine-tuning, the original projection parameters are frozen, and training is performed only. and The farmland change samples include front- and back time-phase remote sensing images and pixel-level farmland change labels.

[0031] In one specific implementation, the visual Transformer basic network adopts a ViT (Vision Transformer) basic network backbone architecture with 24 Transformer layers, which is used to extract features of pre-temporal remote sensing images and post-temporal remote sensing images, respectively.

[0032] In low-rank matrix fine-tuning, the relationship between the low-rank matrix dimension and parameter update is as follows:

[0033]

[0034]

[0035]

[0036]

[0037]

[0038]

[0039] in, This represents the dimension of the feature vector embedding. This represents the rank of the low-rank matrix, and in this embodiment it is set to 16; and The two matrices representing low-rank micro-calls are respectively used. and Multiplying yields a low-rank update quantity with the same dimension as the original projection parameter matrix; , , and These represent the original parameters of the query projection parameter matrix, key projection parameter matrix, value projection parameter matrix, and output projection parameter matrix, respectively. Parameters marked with an asterisk represent fine-tuning parameters after superimposing low-rank update amounts.

[0040] In each Transformer layer, the input feature vector embedding is first standardized. The standardized result is then multiplied by the fine-tuned query projection parameter matrix, key projection parameter matrix, and value projection parameter matrix to obtain the query matrix Q, key matrix K, and value matrix V. An attention output is then calculated based on Q, K, and V, and a nonlinear transformation is performed using the output projection parameter matrix fine-tuned by a low-rank matrix and a feedforward neural network layer. The feedforward neural network layer may include a fully connected layer, a Dropout layer, and a GELU activation function. Through the layer-by-layer processing of the above 24 Transformer layers, the features of the preceding and subsequent temporal remote sensing images are obtained, respectively.

[0041] Specifically, the feedforward neural network layer includes two fully connected layers, two Dropout layers, and the activation function GELU, which is used to perform nonlinear mapping on the output of the linear layer and combine the nonlinearly mapped output with the attention output to obtain the pre-temporal remote sensing image features and post-temporal remote sensing image features extracted by a single Transformer layer.

[0042] The layer normalization, query key-value generation, and attention output calculations in the Transformer layer described above are as follows:

[0043]

[0044]

[0045]

[0046] in, This represents the embedding of the input feature vector. The result after the presentation layer is standardized. and These represent the mean and standard deviation of the corresponding features, respectively. and These represent the learnable scaling and translation parameters, respectively. , and These represent the query matrix, key matrix, and value matrix, respectively. This represents the normalized exponent operation. Key matrix transpose, This indicates scaling the feature dimension in attention, and Attention Weight represents the attention output.

[0047] S103, select pre-temporal and post-temporal remote sensing image features from the outputs of multiple target Transformer layers in the visual Transformer base network as input to the spatiotemporal feature deep fusion unit. In one specific embodiment, the visual Transformer base network includes 24 Transformer layers, and the multiple target Transformer layers are layers 12, 16, 20, and 24. Since shallow features mainly carry basic information such as texture and spectrum, they are prone to confusion between pseudo-changes in crop phenology and real changes in cultivated land. This embodiment selects mid-layer and mid-to-high-layer features for fusion to reduce the propagation of shallow noise and enhance the semantic information of real changes.

[0048] S104, construct a spatiotemporal interaction unit for each target Transformer layer. For any target Transformer layer, let the preceding temporal remote sensing image features of that layer be... The characteristics of the later-phase remote sensing images are The tensor dimensions of both and the bi-temporal connectivity features after channel-based concatenation. The tensor dimension is shown in the following equation. The spatiotemporal interaction unit includes a global module and a temporal module configured in parallel. The global module extracts spatial features based on a multi-head attention mechanism, while the temporal module extracts temporal features based on a selective state-space model. The spatial and temporal features are then fused into pre-temporal and post-temporal spatiotemporal features, respectively, through a channel attention mechanism. In a specific implementation, the temporal module adopts a selective state-space model with a Mamba2 structure.

[0049] The dimensional relationship between the two-phase features and their channel connectivity features is as follows:

[0050]

[0051]

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058]

[0059] in, Indicates the characteristics of previous time-phase remote sensing images. Indicates the characteristics of later-phase remote sensing images. Indicates by and Dual-phase connection characteristics formed by channel connection; Indicates batch size, Indicates the height of the remote sensing image feature map. Indicates the width of the remote sensing image feature map. The number of channels or feature dimensions that represent the characteristics of a remotely sensed image; According to the channel, it can be divided into and , According to the channel, it can be divided into and ; express and Based on the characteristics after channel connection, express and Features after connecting by channel.

[0060] Specifically, the global module will be with Dimensionally consistent spatial location encoding added Spatial features were obtained by following a processing flow of layer normalization, multi-head attention, and feedforward neural networks. The timing module will The sequence is expanded according to spatial location into a sequence feature where spatial location is the sequence element and the channel dimension includes preceding and following temporal information. This sequence feature is then subjected to linear projection, convolution, normalization, state space parameter generation, state recursion, and output projection to obtain the temporal feature. Subsequently, spatial features Decomposed into pre-temporal spatial features according to channels and post-temporal spatial characteristics Time series features Separate into pre-phase time series characteristics according to channels and subsequent phase timing characteristics ;Will and Connect by channel, and Connect the channels and restore them to their original channel dimension d using a channel attention mechanism to obtain the spatiotemporal features of the preceding time phase. and post-temporal spatiotemporal characteristics .

[0061] The tensor dimensions output by the global module and the temporal module are as follows:

[0062]

[0063]

[0064]

[0065] in, Indicates the connection features with dual temporal phases Dimensionally consistent positional encoding; This represents the spatial characteristics output by the global module after adding positional encoding. and These represent the spatial characteristics of the preceding and subsequent time phases, respectively. This indicates the timing characteristics output by the timing module. and These represent the timing characteristics of the preceding and subsequent phases, respectively.

[0066] S105 performs cross-level deep fusion on the pre-temporal and post-temporal spatiotemporal features corresponding to layers 12, 16, 20, and 24, respectively. (Reference) Figure 3 The spatiotemporal features of the 12th, 16th, 20th, and 24th layers of the preceding and following time phases were collected and extracted. Deep fusion of shallow, medium, and high-level features was carried out to obtain the fused features of the preceding time phases. and post-temporal fusion features Taking the deep fusion of past temporal and spatial features as an example, the specific details are as follows:

[0067] Shallow-level fusion: pre-temporal spatiotemporal features Features of the previous time period Features are obtained by connecting channels and then performing a 1×1 convolution. ,feature Then compare with the characteristics of the previous time phase Adding them together yields the phase fusion features before layer 16. ;

[0068] Mid-level fusion: Spatiotemporal features of pre-temporal phases Features of the previous time period Features are obtained by connecting channels and then performing a 1×1 convolution. ,feature Then compare with the characteristics of the previous time phase Adding them together yields the phase fusion features before layer 20. ;

[0069] High-level fusion: pre-temporal spatiotemporal characteristics Features of the previous time period Features are obtained by connecting channels and then performing a 1×1 convolution. ,feature Then compare with the characteristics of the previous time phase Adding them together yields the previous phase fusion features. .

[0070] S105 appears , and It can be uniformly understood as The specific notation for i taking values ​​of 16, 20, and 24 represents the pre-temporal intermediate fusion features obtained by channel-wise concatenation and 1×1 convolution at the corresponding level, which is consistent with the following formulas. The meaning is consistent; , and It can be uniformly understood as The specific notation for i when it takes 16, 20, and 24 represents the preceding temporal fusion feature at the corresponding level. The subsequent temporal phase is obtained using the same fusion method, and the corresponding intermediate and subsequent temporal fusion features are denoted as follows: and .

[0071] S106, fusing features from previous time phases and post-temporal fusion features Multi-level decoding is performed after channel concatenation. This multi-level decoding includes two-stage upsampling and pixel-level mapping: the first and second stages of upsampling are executed sequentially to preserve deep semantic information and gradually restore the edge details of farmland changes during scale restoration. With an input remote sensing image size of 1024×1024, the two-stage upsampling gradually restores the decoded input features to the original 1024×1024 image size. The upsampled features are then subjected to pixel-level mapping using a pre-trained 1×1 convolution with predetermined parameters. This mapping transforms the pixel features of the feature map into change probabilities, resulting in a probability map with pixel values ​​normalized to the 0-1 range. Each pixel value in the probability map represents the predicted probability of farmland change for that pixel. Pixels with a value greater than or equal to 0.5 are assigned a value of 1 and are classified as farmland change categories, while pixels with a value less than 0.5 are assigned a value of 0 and are classified as no farmland change categories, resulting in a binary classification prediction map of farmland change. A pixel value of 1 indicates that the corresponding pixel has experienced farmland change, and a pixel value of 0 indicates that the corresponding pixel has not experienced farmland change.

[0072] Example 2:

[0073] This embodiment further explains the spatiotemporal interaction unit in Embodiment 1. For example... Figure 2 As shown, the spatiotemporal interaction unit uses the bi-temporal connectivity feature of any selected Transformer layer. As input, spatial feature extraction paths and temporal feature extraction paths are constructed simultaneously, so that both homogeneous and heterogeneous information of the same cultivated land area between two time phases can be expressed.

[0074] In the spatial feature extraction path, the global module incorporates spatial location encoding into the bi-temporal connectivity features. Furthermore, a multi-head attention mechanism is used to model the global spatial positional relationships, resulting in spatial features. This spatial feature can aggregate spatial consistency information of stable features or background areas, and can be used to reduce the interference of spurious changes caused solely by crop growth, seasonal variations, or differences in image texture.

[0075] In this embodiment, the global module is constructed based on a multi-head attention mechanism and performs attention modeling on each spatial location in the dual-temporal connectivity features to capture spatial location information in remote sensing image features and improve homogeneity representation.

[0076] In the temporal feature extraction path, the temporal module adopts a selective state-space model with a Mamba2 structure. First, it extracts the bi-temporal connectivity features... Expanded into sequence features based on spatial location, these sequence features use spatial location as sequence elements, and their channel dimensions include both preceding and following temporal information; then, a linear projection matrix is ​​used... and Projecting the sequence features respectively yields... and After that, regarding Performing one-dimensional convolution yields The convolution result is then normalized and linearly projected to generate state-space parameters. These state-space parameters include the sampling interval. State transition matrix The mapping matrix from input to state and the mapping matrix from state to output To improve training stability, the state transition matrix is ​​adjusted. Each learnable state transition parameter is subjected to a negative exponential mapping element by element, so that the state transition parameter has a negative value.

[0077] The relationships between projection, convolution, and state space parameter generation in the selective state-space model of the Mamba2 structure are as follows:

[0078]

[0079]

[0080]

[0081]

[0082]

[0083]

[0084]

[0085] in, Indicates by Sequence modeling branch features obtained through linear projection Indicates by Gated branch features obtained by linear projection and These represent the corresponding linear projection matrices; This represents a one-dimensional convolution operation. Indicates to Features after performing a one-dimensional convolution; This indicates a normalization operation. This represents the linear projection matrix used to generate the state-space parameters. Indicates the state-space sampling interval. The matrix representing the mapping from input to state. This represents the mapping matrix from state to output.

[0086] The negative value constraint and discretization relationship of the state space parameters are as follows:

[0087]

[0088]

[0089]

[0090]

[0091] in, Represents a continuous state transition matrix. Indicates used for The learnable parameter matrix is ​​log-parameterized for each learnable state transition parameter in the matrix. Represents the identity matrix; and These represent the discretized state transition matrix and input mapping matrix, respectively. They are represented by subscripts. Distinguishing between state-space parameters and low-rank fine-tuning matrices , . The exponential operation is represented; 128 in the formula represents the state dimension of the selective state-space model in this embodiment.

[0092] The state recursion, output, and gating relationships of the selective state-space model of the Mamba2 structure are as follows:

[0093]

[0094]

[0095]

[0096]

[0097] in, This indicates the hidden state at the current moment. Indicates the hidden state at the previous moment; Represents one-dimensional convolutional features; This represents the state output obtained from the hidden state mapping; This indicates the state output after gating and branching modulation; Indicates activation operation; Indicates the gating branch feature; Indicates the output projection matrix; This represents the final temporal characteristics obtained.

[0098] In the stage of fusion of spatial and temporal features, the channel attention mechanism first performs global average pooling on the features concatenated by channels to obtain the global features corresponding to each channel; then, it learns the importance weights of each channel through a fully connected layer; then, it multiplies the importance weights with the concatenated features channel by channel to enhance the channels related to real farmland changes and suppress noisy channels related to crop phenological pseudo-changes; finally, in order to save computational costs, a 1×1 convolution layer is used to restore the number of channels to the original number of channels, and the spatiotemporal features of the previous and subsequent phases are obtained respectively.

[0099] The relevant output dimensions of spatial and temporal features in the fusion stage are as follows:

[0100]

[0101]

[0102]

[0103]

[0104] in, Representing the spatial characteristics of the preceding time phase Timing characteristics of previous phases Based on the characteristics after channel connection, Representing the spatial characteristics of the later phase Timing characteristics of subsequent phases Features after channel concatenation; SE channel attention mechanism and 1×1 convolution are used to complete channel recalibration and channel number recovery; Indicates the spatiotemporal characteristics of the preceding phase. It represents the spatiotemporal characteristics of the later phase.

[0105] Example 3:

[0106] This embodiment further explains the cross-level deep fusion and multi-level decoding in Embodiment 1. The cross-level deep fusion uses the features of layers 12, 16, 20 and 24 as the backbone, and does not use the features of shallower layers 1 to 11 as the main fusion objects, thereby reducing the probability of shallow texture noise and crop phenological pseudo-change information being transmitted to the final change map.

[0107] In cross-level deep fusion, the 12th-layer features provide the mid-level semantic foundation, the 16th and 20th-layer features progressively supplement stronger semantic variations, and the 24th-layer features provide the top-level semantic expression. The deep fusion strategy relies on asynchronous modeling and cross-layer collaboration to gradually amplify the differences between features, achieving heterogeneous representation. Each fusion level employs a combination of channel concatenation, 1×1 convolution, and residual addition, enabling the current layer features to retain their own semantics while receiving the fusion results from the previous level. This fusion process is executed separately for the preceding and following phases, resulting in preceding-phase and following-phase fusion features used for difference discrimination.

[0108] Taking the previous and subsequent time phases as examples, the recursive relationship of cross-level deep fusion is as follows:

[0109]

[0110]

[0111]

[0112]

[0113]

[0114]

[0115]

[0116] in, and They represent the first The spatiotemporal characteristics of the preceding and subsequent phases corresponding to the layer; and They represent the first The preceding and following temporal fusion features corresponding to the layer; and These represent intermediate fusion features; Indicates connection by channel. This represents a 1×1 convolutional layer. Take 16, 20 and 24; , .

[0117] Note: The above cross-level deep fusion is performed in the same way for both the preceding and subsequent time phases; and These are used as the initial terms for fusion in the 12th layer for the preceding and following phases, respectively, and are both set to zero.

[0118] In multi-level decoding, pre-temporal fusion features and post-temporal fusion features The features are concatenated by channel to form the decoding input features. Compared to restoring to the original image scale in one step, two-stage upsampling can preserve more deep semantics and gradually restore change boundaries during scale restoration. The first and second stages of upsampling are performed sequentially to alleviate the scale gap between deep features and pixel-level output, and to gradually restore the edge details of farmland changes during scale restoration. With an input remote sensing image size of 1024×1024, two-stage upsampling restores the feature size to the 1024×1024 size of the input remote sensing image. Finally, the restored features are mapped to a single-channel probability map using a 1×1 convolution with pre-trained parameters. A binary classification farmland change detection map is generated using a 0.5 threshold, where a pixel value of 1 represents farmland change and a pixel value of 0 represents no farmland change.

[0119] Example 4:

[0120] In one experimental embodiment, the publicly available CLCD remote sensing dataset on farmland non-agriculturalization change (hereinafter referred to as the CLCD dataset) was selected for training, validation, and testing. This dataset contains 600 pairs of farmland change sample images, of which 360 pairs were used for training, 120 pairs for validation, and 120 pairs for testing. The training process employed the AdamW optimizer with a learning rate of 0.00004. The AdamW optimizer is an adaptive moment estimator with weight decay, and a cosine annealing strategy was used to adjust the learning rate. The batch size was set to 4, the maximum number of training epochs was set to 100, and the binary cross-entropy loss function was used.

[0121] The binary cross-entropy loss function used during training is as follows:

[0122]

[0123] in, Represents the binary cross-entropy loss. Represents a pixel or sample index. Indicates the first The true change label for each pixel or sample, with a value of 0 or 1; The first output of the model represents the... The predicted probability of farmland change for a given pixel or sample; It represents logarithmic operations.

[0124] Additional explanation: To avoid confusion between the predicted probability sign and the true label sign, in the aforementioned loss function... The label represents the true change label of the i-th pixel or sample, and the predicted probability output by the model is denoted by the capped label. That is, the "predicted probability of farmland change for the i-th pixel or sample output by the model" in the above explanatory paragraph corresponds to .

[0125] Under the same dataset and training conditions, the method of this embodiment is compared with the SNUNET change detection model, the BIT change detection model, the ChangeFormer change detection model, and the pre-improvement method disclosed in Chinese patent application number CN202510554150.5. The experimental results are shown in Table 1 below.

[0126] Table 1. Comparison results of different models on the CLCD dataset

[0127] Model Accuracy Recall rate F1 score Average crossover ratio SNUNET 87.87 89.70 88.78 79.82 BIT 83.60 88.44 85.95 75.37 ChangeFormer 65.61 87.63 75.04 60.04 Before the improvement (CN202510554150.5) 82.05 98.30 89.44 80.90 This method 90.51 97.54 93.89 88.49

[0128] As shown in Table 1, the method in this embodiment achieves a precision of 90.51%, a recall of 97.54%, an F1 score of 93.89%, and a mean intersection-over-union (MIU) of 88.49%. Compared to the previous method, this embodiment maintains a high recall while significantly improving precision, F1 score, and MIU, indicating that it can reduce false alarms and lower the burden of subsequent field and office verification in farmland protection scenarios. Figure 4 As shown, the comparison of prediction results from previous and subsequent temporal images, labels, SNUNET change detection model, BIT change detection model, ChangeFormer change detection model, the method before improvement, and the proposed method shows that the change region generated by the proposed method is closer to the label, and the boundary continuity and pseudo-change suppression effect are better.

[0129] To verify the effectiveness of multi-level decoding, spatiotemporal feature deep fusion, and the fusion strategies for layers 12, 16, 20, and 24, ablation experiments were further conducted on the CLCD dataset. The experimental results are shown in Table 2 below.

[0130] Table 2 Ablation Experiment Results

[0131] Model Accuracy Recall rate F1 score Average crossover ratio This method 90.51 97.54 93.89 88.49 Remove multi-level decoding 91.60 95.56 93.54 87.86 Remove spatiotemporal feature deep fusion 90.82 96.34 93.50 87.79 Simultaneously remove the multi-level decoding and spatiotemporal feature deep fusion processor. 92.07 94.52 93.28 87.40 Deep Integration Strategy 1 89.49 95.62 92.45 85.96 Deep Integration Strategy 2 89.48 98.22 93.65 88.05

[0132] Table 2 shows that while removing multi-level decoding, spatiotemporal feature deep fusion, or both individually increases precision in some schemes, it decreases recall, F1 score, and mean intersection-over-union (MIU). This indicates that multi-level decoding and spatiotemporal feature deep fusion play a crucial role in reducing missed detections of farmland change. Deep fusion strategy 1 uses feature fusion at layers 6, 12, 18, and 24, while deep fusion strategy 2 uses feature fusion at layers 15, 18, 21, and 24. Compared to these two strategies, the scheme using feature fusion at layers 12, 16, 20, and 24 in this embodiment is superior in overall performance. Figure 5As shown, the visualization results of different module removal schemes and different depth fusion strategies further demonstrate that the method in this embodiment can reduce missed detections of cultivated land changes while suppressing misjudgments caused by false changes in crop phenology.

Claims

1. A remote sensing method for detecting changes in cultivated land, characterized in that, include: Acquire registered and identically sized pre- and post-temporal remote sensing images of the same region. Map the pre- and post-temporal remote sensing images into feature vector embeddings with location encoding. Extract features from the feature vector embeddings using a visual Transformer basic network, which includes multiple Transformer layers, based on farmland change samples pre-tuned with a low-rank matrix. Select pre- and post-temporal remote sensing image features output from multiple target Transformer layers. For each target Transformer layer, a spatiotemporal interaction unit is constructed based on the pre-temporal and post-temporal remote sensing image features. The spatiotemporal interaction unit includes a global module and a temporal module set in parallel. The global module extracts spatial features based on a multi-head attention mechanism, and the temporal module extracts temporal features based on a selective state space model. The spatial features and the temporal features are then fused into pre-temporal and post-temporal spatiotemporal features through a channel attention mechanism. Cross-level deep fusion is performed on the pre-temporal and post-temporal spatiotemporal features corresponding to each target Transformer layer to obtain pre-temporal fusion features and post-temporal fusion features; The preceding and following time-phase fusion features are connected by channels and then decoded at multiple levels to generate a probability map of each pixel belonging to a farmland change category. Based on a preset binarization threshold, a farmland change detection map including farmland change categories and no farmland change categories is obtained.

2. The remote sensing method for detecting changes in cultivated land according to claim 1, characterized in that, The step of mapping the preceding and following time-phase remote sensing images into feature vector embeddings with positional encoding includes: dividing a 1024×1024 remote sensing image with three bands into 64×64 remote sensing image blocks of size 16×16; flattening each remote sensing image block into a one-dimensional vector of length 768 according to band order; mapping the one-dimensional vector to a 768-dimensional embedding space through a fully connected layer; and adding the positional encoding to the mapped high-dimensional embedding vector to obtain the feature vector embedding.

3. The remote sensing method for detecting changes in cultivated land according to claim 1, characterized in that, The visual Transformer base network, pre-tuned with a low-rank matrix based on farmland change samples, is obtained as follows: For the query projection parameter matrix, key projection parameter matrix, value projection parameter matrix, and output projection parameter matrix in each Transformer layer, a first low-rank matrix and a second low-rank matrix are introduced respectively. The original projection parameter matrix is ​​then added to the low-rank update amount obtained by multiplying the first low-rank matrix and the second low-rank matrix to form the fine-tuned parameter matrix. The farmland change samples include pre- and post-temporal remote sensing images and pixel-level farmland change labels. The rank of the low-rank update amount is set to 16. The first low-rank matrix is ​​initialized according to a Gaussian distribution, and the second low-rank matrix is ​​initialized to a zero matrix. During the training phase using the farmland change samples for fine-tuning, the original projection parameter matrix is ​​frozen, and only the first low-rank matrix and the second low-rank matrix are updated.

4. The remote sensing method for detecting changes in cultivated land according to claim 1, characterized in that, Each Transformer layer of the visual Transformer base network includes layer normalization, query projection, key projection, value projection, attention weight calculation, output projection, and feedforward neural network processing to obtain the features of the preceding and following time-phase remote sensing images, respectively; wherein, the feedforward neural network includes two fully connected layers, two Dropout layers, and a GELU activation function.

5. The remote sensing method for detecting changes in cultivated land according to claim 1, characterized in that, The construction of the spatiotemporal interaction unit includes: connecting the features of the preceding and following temporal remote sensing images by channel to form a dual-temporal connected feature; performing multi-head attention processing on each spatial location in the dual-temporal connected feature after spatial location encoding by the global module to obtain spatial features; performing selective state-space modeling on the dual-temporal connected feature by the temporal module to obtain temporal features; splitting the spatial features and the temporal features by channel into preceding temporal spatial features, following temporal spatial features, preceding temporal temporal features, and following temporal temporal temporal features; connecting the preceding temporal spatial features with the preceding temporal temporal features by channel, and connecting the following temporal spatial features with the following temporal temporal features by channel; restoring the connected features to the original channel dimension through a channel attention mechanism to obtain the preceding spatiotemporal features and the following spatiotemporal features.

6. The remote sensing method for detecting changes in cultivated land according to claim 5, characterized in that, The temporal module employs a selective state-space model based on the Mamba2 structure and includes: expanding the dual-temporal connection features into sequence features with spatial positions as sequence elements and channel dimensions containing preceding and following temporal information; performing linear projection on the sequence features to obtain sequence modeling branch features and gated branch features; performing one-dimensional convolution, normalization, and linear projection on the sequence modeling branch features to generate state-space parameters, which include sampling interval, state transition matrix, input-to-state mapping matrix, and state-to-output mapping matrix; performing negative exponential mapping on each learnable state transition parameter in the state transition matrix to make the state transition parameter negative, and discretizing the state transition matrix and the input-to-state mapping matrix; performing state recursion based on the discretized state transition matrix, the discretized input-to-state mapping matrix, and the convolved sequence modeling branch features to obtain the state output; and performing sigmoid gating modulation on the state output using the gated branch features, followed by normalization and output projection to obtain the temporal features.

7. The remote sensing method for detecting changes in cultivated land according to claim 5, characterized in that, The channel attention mechanism processes the channel-connected features as follows: global average pooling is performed on the channel-connected features to obtain the global features corresponding to each channel; importance weights for each channel are learned through a fully connected layer; the importance weights are multiplied by the channel-connected features channel by channel, and the number of channels is restored to the original channel dimension through a 1×1 convolution layer to obtain the spatiotemporal features after the number of channels is restored.

8. The remote sensing method for detecting changes in cultivated land according to claim 1, characterized in that, The visual Transformer base network includes 24 Transformer layers, and the multiple target Transformer layers are layers 12, 16, 20, and 24. The cross-layer deep fusion includes: performing recursive fusion for each temporal spatiotemporal feature in the preceding and following temporal spatiotemporal features; setting the initial fusion term corresponding to layer 12 to zero, and for each target layer in layers 16, 20, and 24, concatenating the spatiotemporal features corresponding to the previous selected layer of the target layer with the spatiotemporal features corresponding to the target layer by channel, performing a 1×1 convolution, and adding it to the spatiotemporal features corresponding to the target layer to obtain intermediate fusion features; then adding the intermediate fusion features to the fusion features corresponding to the previous selected layer to obtain the fusion features corresponding to the target layer; wherein, the fusion features corresponding to layer 24 are used as the final fusion features for the corresponding temporal phase.

9. The remote sensing method for detecting changes in cultivated land according to claim 1, characterized in that, The multi-level decoding includes: concatenating the pre-temporal fusion features and the post-temporal fusion features by channel to obtain the decoding input features; performing first-level upsampling and second-level upsampling sequentially on the decoding input features to gradually restore the feature scale of the decoding input features, and obtaining upsampled features corresponding to the original sizes of the pre-temporal and post-temporal remote sensing images when the input remote sensing image size is 1024×1024; performing pixel-level mapping on the upsampled features through a 1×1 convolution with pre-trained and determined parameters to obtain a probability map with pixel values ​​normalized to the interval between 0 and 1; assigning a value of 1 to pixels greater than or equal to 0.5 in the probability map as the category of farmland change, and assigning a value of 0 to pixels less than 0.5 as the category of no farmland change, where a pixel value of 1 represents that the corresponding pixel has undergone farmland change, and a pixel value of 0 represents that the corresponding pixel has not undergone farmland change.

Images