Time sequence remote sensing land coverage change detection method and device, equipment and medium

Abstract

The invention relates to the technical field of remote sensing image processing, and discloses a time sequence remote sensing land coverage change detection method and device, equipment and a medium. The method comprises the following steps: acquiring dense time sequence remote sensing image data containing continuous original time steps, and constructing a time sequence data sequence to carry out multi-label coarse labeling; generating a time sequence dimension original class activation diagram based on the multi-label coarse annotation and the time sequence data sequence, and generating a plurality of enhanced class activation diagrams through various time sequence enhancement so as to screen weak labels; training a time sequence semantic segmentation model; inputting to-be-detected dense time sequence remote sensing image data into the trained model to obtain a surface feature category prediction result of each original time step; and identifying pixel positions and corresponding time points of ground feature category mutation, and generating a spatial change graph and a change time graph. According to the method, the marking cost is greatly reduced, the change detection robustness and the space-time positioning precision are improved, deployment is efficient, and the method is suitable for large-scale refined land coverage dynamic monitoring scenes.

Description

Technical Field

[0001] This application relates to the field of remote sensing image processing technology, specifically to a method, apparatus, equipment, and medium for detecting time-series remote sensing land cover change. Background Technology

[0002] Land cover change detection is one of the core tasks in the field of remote sensing applications. Its purpose is to identify spatial and temporal changes in land cover types by analyzing remote sensing images from different periods. With the rapid development of satellite remote sensing technology, the acquisition of dense temporal remote sensing image data (such as satellite image sequences) has become increasingly convenient.

[0003] However, current methods for detecting land cover change using satellite remote sensing time series face the fundamental problem of extremely high pixel-level time series annotation costs.

[0004] Traditional methods rely on dense pixel-level temporal annotation. Professional remote sensing analysts need to annotate pixel by pixel and time step by time. Annotating a complete dataset containing 48 time steps typically takes months, with the workload increasing exponentially. Land cover classification involves complex remote sensing interpretation rules and land use expertise, which ordinary annotators cannot handle, leading to a shortage of annotators and further increasing annotation costs. Different annotators have different professional backgrounds and interpretation experience, and even the same person's annotations at different times may have deviations. This inconsistency in annotation directly affects the accuracy of subsequent model training. For long-term dynamic monitoring of large-scale areas (such as urban agglomerations and watersheds), the cost of pixel-by-pixel and time-step annotation has become a major bottleneck for technology implementation, making traditional methods difficult to apply to real-world business scenarios.

[0005] Currently, some improved solutions for detecting time-series changes have emerged in the field, as follows: (a) CCDC (Continuous Change Detection and Classification) and its derivative algorithms. Although these methods can complete basic change detection tasks, they are limited by high classification costs, loss of detailed information over time during model initialization, and error propagation in the two-stage processing flow. In long-term scenarios, their change detection accuracy is limited and it is difficult to meet the high-precision requirements such as detailed dynamic mapping of cities.

[0006] (II) Temporal Semantic Change Detection Model (TSSCD). This model uses time-series data labeled with each time phase and a fully supervised temporal semantic segmentation deep learning model to learn the semantic association between observations at each time point and land cover types end-to-end, enabling simultaneous detection of changes in time and type. Its advantage lies in avoiding error propagation in two-stage processing, but its core drawback is its heavy reliance on a large amount of time-series labeled data, resulting in high labeling costs and making it difficult to apply to long-term time series real-world scenarios.

[0007] (III) Patent CN120451664A. This method employs a three-step strategy of "weakly supervised training → SAM (Segmentation Model) optimization → fully supervised training," utilizing the zero-shot segmentation capability of the SAM model to transform coarse class activation maps (CAMs) into high-quality pixel-level pseudo-labels, thereby improving the accuracy and boundary accuracy of change detection. However, it suffers from two major drawbacks: the entire process comprises three independent stages and heavily relies on a large external general-purpose model (SAM), leading to compatibility issues and high computational demands during deployment and application; model performance is highly dependent on the quality of pseudo-labels, and if the initial CAM quality is poor, resulting in incorrect prompts, SAM will produce erroneous segmentation results, and this error will propagate to the final fully supervised model, affecting detection accuracy.

[0008] (iv) Patent CN120708084A. This method generates pseudo-labels based on prototype-aware learning and trains a robust change detection model by combining wavelet transform and dual-view regularization strategy. Its main drawbacks include: the prototype learning effect depends on the quality of the initial CAM; if the initial CAM activation region has a large deviation, the learned prototype will be inaccurate and mislead the subsequent CAM enhancement process; the model structure is complex, and the introduced wavelet transform module and dual-view regularization strategy increase the training difficulty and computational cost, reducing the practicality of the model.

[0009] (v) Patent CN116912698A. The proposed temporal semantic segmentation method guides the model to learn semantic mapping through dense time series annotation, realizing land cover change detection and arbitrary temporal classification mapping. Theoretically, it can detect changes of any number and type. However, this method requires a large amount of manual annotation pixel by pixel and temporal phase, which consumes huge human resources and has high annotation costs, making it difficult to promote and apply.

[0010] A comprehensive analysis of existing technical solutions reveals four key limitations: Fixed annotation patterns: Most methods still rely on pixel-level or time-phase-based fine annotation, failing to break through the bottleneck of "reliability for fine annotation", and the annotation cost problem has not been fundamentally solved; Insufficient information utilization: The dual-temporal change detection method loses the process information between the two temporal phases, making it difficult to capture gradual changes, and is susceptible to noise interference from seasonal changes, differences in imaging conditions, etc. The detection dimension is limited: it can only output spatial change maps, and cannot accurately locate the time point when the change occurred. It lacks the detection capability of "integrated spatiotemporal" and is difficult to meet the actual needs of scenarios such as disaster emergency assessment and crop phenology monitoring. Insufficient model practicality: Some methods rely on large external models or complex modules, resulting in high deployment difficulty and low computational efficiency, making them unsuitable for large-scale long-term time-series data processing scenarios.

[0011] Therefore, developing a temporal remote sensing land cover change detection method that can reduce annotation costs, make full use of temporal information, achieve accurate spatiotemporal detection, and be deployed efficiently has become an urgent technical problem to be solved in this field. Summary of the Invention

[0012] To address the aforementioned issues, this application provides a method, apparatus, device, and medium for detecting temporal remote sensing land cover change. Through weakly supervised learning, it transforms difficult-to-obtain fine-grained temporal annotations into easily obtainable coarse-grained sequence annotations, significantly reducing annotation costs. Simultaneously, it fully utilizes the temporal dimension information of dense temporal data to achieve accurate synchronous detection of the spatial location and time points of land cover changes, improving the robustness and practicality of change detection and meeting the needs of large-scale, high-precision dynamic monitoring of land cover.

[0013] The embodiments of this application adopt the following technical solutions: Firstly, this application provides a method for detecting land cover change using time-series remote sensing, including: Acquire dense time-series remote sensing image data containing continuous raw time steps and construct a time-series data sequence, then perform multi-label coarse annotation on the time-series data sequence; wherein, the multi-label coarse annotation marks the land cover categories included in the time-series data sequence; Based on multi-label coarse annotation and time-series data sequences, a raw class activation graph in the time-series dimension is generated; Multiple time-series enhancement processes are applied to the time-series data sequence to generate multiple enhanced activation maps. Weak labels are then filtered based on the original activation maps and the enhanced activation maps. A temporal semantic segmentation model is trained based on temporal data sequences and weak labels; The dense temporal remote sensing image data to be detected is input into the trained temporal semantic segmentation model to obtain the land cover category prediction results for each original time step; By analyzing the prediction results of land cover categories, the pixel locations and corresponding time points of land cover category abrupt changes are identified, and spatial change maps and change time maps are generated.

[0014] Secondly, this application also provides a time-series remote sensing land cover change detection device, comprising: The data annotation module is used to acquire dense time-series remote sensing image data containing continuous raw time steps and construct time-series data sequences, and to perform multi-label coarse annotation on the time-series data sequences; wherein, the multi-label coarse annotation marks the land cover categories included in the time-series data sequences; The original class activation graph generation module is used to generate original class activation graphs in the time series dimension based on multi-label coarse annotation and time series data sequences. The weak label filtering module is used to perform various time-series enhancement processes on time-series data sequences, generate multiple enhanced class activation maps, and filter weak labels based on the original class activation maps and enhanced class activation maps; The model training module is used to train a temporal semantic segmentation model based on temporal data sequences and weak labels; The change detection module is used to input the dense temporal remote sensing image data to be detected into the trained temporal semantic segmentation model to obtain the land cover category prediction results for each original time step; The mapping module is used to analyze the prediction results of land cover categories, identify the pixel locations and corresponding time points of abrupt changes in land cover categories, and generate spatial change maps and temporal change maps.

[0015] Thirdly, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method for detecting time-series remote sensing land cover change.

[0016] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for detecting time-series remote sensing land cover change.

[0017] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: Significantly reduces annotation costs. Through weakly supervised learning, time-series fine-grained annotation is transformed into sequence-level coarse-grained annotation, reducing the annotation workload by more than 95%. At the same time, it reduces the professional knowledge requirements for annotators, solving the core problem of high annotation costs in traditional methods.

[0018] Improve the robustness of change detection. By fully utilizing the temporal dimension information of dense time-series data, it can effectively distinguish between periodic seasonal changes and sudden permanent changes, significantly reduce the detection of false changes, and has better robustness than dual-temporal change detection methods and traditional time-series methods.

[0019] Achieve precise spatiotemporal detection. This method enables simultaneous and precise detection of the spatial location and time of changes in ground features, overcoming the limitation of existing methods that can only detect spatial changes but cannot pinpoint the time of change.

[0020] Improve model usability. The model architecture is closed-loop and efficient, does not depend on large external models, has high computational efficiency, is easy to deploy, and is suitable for large-scale, long-term remote sensing data processing scenarios. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1A flowchart illustrating a method for detecting temporal remote sensing land cover change according to an embodiment of this application is shown; Figure 2 A schematic diagram of the structure of a time-series remote sensing land cover change detection device according to an embodiment of this application is shown; Figure 3 A schematic diagram of the structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Figure 1 A schematic flowchart illustrating a method for detecting temporal remote sensing land cover change according to an embodiment of this application is shown. (Refer to...) Figure 1 As shown, this embodiment includes steps S110 to S160: Step S110: Acquire dense time-series remote sensing image data containing continuous original time steps and construct a time-series data sequence, and perform multi-label coarse annotation on the time-series data sequence; wherein, the multi-label coarse annotation marks the land cover categories included in the time-series data sequence.

[0024] This embodiment first acquires dense temporal remote sensing image data and performs multi-label coarse annotation. The core objective of this step is to construct a temporal data sequence to obtain a supervision signal with low annotation cost.

[0025] In some optional implementations, step S110, acquiring dense time-series remote sensing image data containing continuous original time steps and constructing a time-series data sequence, and performing multi-label coarse annotation on the time-series data sequence, includes: acquiring satellite remote sensing image data containing continuous original time steps as dense time-series remote sensing image data; extracting time-series features from the dense time-series remote sensing image data to obtain a time-series data sequence; and labeling the land cover categories included in the time-series data sequence as multi-label coarse annotations.

[0026] Satellite remote sensing image data containing continuous raw time steps are acquired as dense time-series remote sensing image data.

[0027] Satellite remote sensing imagery data can be, but is not limited to, satellite imagery. Satellite imagery has the advantages of high spatial resolution, high temporal resolution, and multispectral bands, and can fully reflect the temporal spectral characteristics of ground objects.

[0028] The number of initial time steps can be flexibly adjusted according to monitoring needs. Preferably, there can be 48 initial time steps.

[0029] Temporal features are extracted from dense temporal remote sensing image data to obtain a temporal data sequence.

[0030] The temporal feature extraction process can include preprocessing of dense temporal remote sensing image data and encoding of temporal features of dense temporal remote sensing image data.

[0031] Preprocessing of dense temporal remote sensing image data may include, but is not limited to, performing radiometric correction, atmospheric correction, geometric correction, and cloud removal on dense temporal remote sensing image data to eliminate the impact of differences in imaging conditions on subsequent analysis.

[0032] Temporal feature encoding of dense temporal remote sensing image data can arrange the preprocessed dense temporal remote sensing image data in chronological order, with each pixel corresponding to a spectral temporal curve containing continuous original time steps, so that the spectral temporal curves of all pixels constitute a temporal data sequence.

[0033] The land cover categories contained in the time series data are labeled as coarse multi-label annotations.

[0034] Unlike traditional time-series pixel-level labeling, this step only needs to label the land cover categories (such as vegetation, built-up land, bare land, water bodies, etc.) contained in the entire time series data sequence, without needing to label the specific category for each original time step. For example, for a time series data sequence containing the change from "vegetation to built-up land", only the category labels "vegetation" and "built-up land" need to be labeled.

[0035] This step reduces the dimensionality of the annotation granularity, transforming "time-series" annotation into "sequence-sequence" annotation, and replacing fine-grained time-phase annotation with coarse multi-label annotation. This significantly reduces annotation costs and the professional threshold without losing supervisory information. For example, for dense time-series remote sensing image data with 48 original time steps, the annotation workload is reduced from 48 times in traditional methods to only 1 time.

[0036] Step S120: Based on the multi-label coarse annotation and the time-series data sequence, generate the original class activation graph in the time-series dimension.

[0037] This embodiment then utilizes Class Activation Mapping (CAM) technology to backpropagate the sequence-level supervision signal to each location in the time series. By analyzing the feature activation patterns within the deep network, the most likely land cover category for each original time step is automatically inferred, thereby achieving knowledge transfer from global supervision to local prediction.

[0038] In some optional implementations, step S120, generating the original class activation map in the time-series dimension based on multi-label coarse annotation and time-series data sequence, includes: extracting the core features encoded by the last layer of the ResNet50-1D model from the time-series data sequence; performing global average pooling on the core features to obtain a global feature vector; inputting the global feature vector into a fully connected layer to obtain class recognition weights; back-projecting the class recognition weights to each downsampling time step of the time-series data sequence through weighted summation to generate an initial class activation map; performing temporal interpolation on the initial class activation map using linear interpolation to restore the downsampling time step to the original time step, thereby obtaining the original class activation map.

[0039] Extract the core features of the time-series data sequence encoded by the last layer of the ResNet50-1D model.

[0040] The time-series data sequence is input into the ResNet50-1D model, and the core features encoded by the last layer (Stage4) of the ResNet50-1D model are extracted.

[0041] The core features of the Stage4 layer were chosen because it has the following four major advantages.

[0042] Sufficient semantic information: After multiple rounds of convolution and pooling operations, the features of Stage4 contain high-level semantic information, which can effectively distinguish different land cover categories; Moderate temporal resolution: It retains 12 downsampling time steps, which can capture the temporal change patterns of ground objects while avoiding the loss of location information due to oversampling; Sufficient number of channels: 2048 feature channels provide a rich feature representation space, which can comprehensively characterize the spectral temporal features of ground objects; Direct connection: It is directly connected to the subsequent fully connected layer, which facilitates the reverse tracing of the basis for classification decisions and provides convenience for the generation of class activation graphs.

[0043] Global average pooling is performed on the core features to obtain the global feature vector.

[0044] Global average pooling refers to calculating the average value of each feature channel across the entire time dimension, using a single numerical value to represent the global feature of that channel. Its advantages are as follows:

[0045] Small number of parameters: The global feature vector has a dimension of 2048×1. If the number of coarse labels for the land cover category is 6, the number of parameters for the subsequent fully connected layer is only 2048×6, which is much lower than that of the traditional fully connected layer, thus reducing the model complexity. Strengthen global features: Force each channel to learn global temporal features, avoid the model from focusing too much on local noise, and facilitate the subsequent calculation of the original class activation map and the transmission of global supervision signals.

[0046] The global feature vector is input into the fully connected layer to obtain the category recognition weights.

[0047] The global feature vector is input into a fully connected layer, and the category recognition weights are obtained through linear transformation and activation function operations. These category recognition weights reflect the contribution of each feature channel to different land cover categories. For example, if a feature channel has a high weight for the "vegetation" category, it indicates that the features of that channel play an important role in identifying vegetation categories.

[0048] The category recognition weights are back-projected onto each downsampling time step of the time series data sequence by weighted summation to generate an initial class activation map.

[0049] The initial class activation graph has a dimension of 6×12 (number of coarse multi-label annotations for each land cover category × number of downsampling time steps), with each element representing the probability that the corresponding downsampling time step belongs to a certain land cover category. Essentially, this process transforms the global classification knowledge learned by the fully connected layers into the local category probability for each downsampling time step, achieving a leap from sequence-level supervision to temporal-level annotation.

[0050] Linear interpolation is used to perform temporal interpolation on the initial class activation map, and the downsampling time step is restored to the original time step to obtain the original class activation map.

[0051] Linear interpolation is used to perform temporal interpolation on the initial class activation map, restoring 12 downsampled time steps to 48 original time steps, thus obtaining the original class activation map.

[0052] The reasons for choosing linear interpolation are as follows:

[0053] Simple and efficient: It has low computational complexity, can quickly complete the recovery of time series dimensions, and is suitable for large-scale data processing; Maintaining relative relationships: It can better preserve the changing trends and relative magnitudes of class probabilities in the initial class activation graph, avoiding the introduction of additional distortion; Avoid overfitting: Compared to complex interpolation methods, linear interpolation is less likely to overfit local noise, thus ensuring the stability of the original class activation map.

[0054] This step extends the class activation graph technique from the traditional 2D image domain to the 1D temporal domain, proposes a temporal dimension class activation graph generation method, realizes the transformation of sequence-level supervision signals into temporal-level pseudo-labels, and provides key technical support for weakly supervised learning.

[0055] Step S130: Perform various time-series enhancement processes on the time-series data sequence to generate multiple enhanced activation maps, and filter weak labels based on the original activation maps and enhanced activation maps.

[0056] Weak label generation is a crucial bridge between continuous classifiers and segmentation networks. While the original class activation map can generate preliminary temporal labels, a single original class activation map contains noise and uncertainty, and directly using it as a supervision signal can affect the model's training accuracy. This embodiment then uses multiple enhancement strategies and consistency constraints to screen out high-quality weak supervision labels, providing reliable supervision signals for subsequent temporal segmentation network training.

[0057] In some optional implementations, step S130 involves performing various time-series enhancement processes on the time-series data sequence to generate multiple enhanced activation maps, and filtering weak labels based on the original activation maps and the enhanced activation maps. This includes: performing time-series flipping on the time-series data sequence to generate a first enhanced activation map of the flipped time-series data sequence; performing time-series translation on the time-series data sequence to generate a second enhanced activation map of the translated time-series data sequence; and performing noise perturbation on the time-series data sequence to generate a third enhanced activation map of the perturbated time-series data sequence. Weak labels that are completely consistent with the coarse annotations of the multi-labels in the original class activation graph and the three enhanced class activation graphs are selected through a consistency voting mechanism.

[0058] The time series data sequence is time-reversed to generate the first enhanced class activation graph of the reversed time series data sequence.

[0059] The time sequence of the time series data is reversed. For example, if the original time series is t1→t2→…→t48, the time series of the reversed time series data is t48→t47→…→t1. Step S120 is performed on the reversed time series data sequence to generate the first enhanced class activation graph.

[0060] The significance of time-series reversal lies in the fact that the change process of land cover categories is essentially independent of the time direction. That is, the change process itself remains unchanged; only the time direction is reversed. For example, the process of vegetation being cleared and converted into building land, and then, after reversing the time sequence, the process of building land being restored to vegetation, does not change its core characteristics. The role of time-series reversal is to: test the model's invariance to the time direction, eliminate possible time direction bias, and improve the bidirectional consistency of change detection.

[0061] A temporal shift is performed on the time-series data sequence to generate a second enhanced class activation graph of the shifted time-series data sequence.

[0062] The time series data sequence is shifted in time. For example, the time series data sequence is shifted backward by two original time steps, with the first two original time steps filled with the last two original time steps, resulting in the shifted time series data sequence. Step S120 is then performed on the shifted time series data sequence to generate the second enhanced class activation map.

[0063] The significance of time shifting lies in the fact that the attributes of land cover categories do not change due to slight variations in the start time. For example, while there may be subtle temporal differences in vegetation growth cycles across different years, the spectral temporal characteristics of vegetation remain consistent. That is, the land cover category remains the same, only the start time differs. The purpose of time shifting is to: verify the model's invariance to the temporal start point, test the model's ability to identify periodic patterns, and eliminate potential start time biases.

[0064] The time series data sequence is subjected to noise perturbation to generate a third-enhanced activation map of the perturbed time series data sequence.

[0065] Gaussian noise is added to the time-series data sequence to generate a perturbed time-series data sequence. For example, the mean of the Gaussian noise can be set to 0, and the variance can be adjusted according to the actual noise level. Step S120 is then performed on the perturbed time-series data sequence to generate a third enhanced class activation map.

[0066] The significance of noise perturbation lies in the fact that observation noise is unavoidable in real remote sensing data, but slight noise should not change the category attributes of land features. That is, slight noise should not alter the land feature category. The role of noise perturbation is to: test the robustness of the model to noise, simulate observation errors in real remote sensing data, and filter out unreliable labels that are sensitive to slight perturbations.

[0067] Weak labels that are completely consistent with the coarse annotations of the multi-labels in the original class activation graph and the three enhanced class activation graphs are selected through a consistency voting mechanism.

[0068] The design principle of consistency constraints is that if the labeling of an original time step is accurate, then regardless of any reasonable transformation (flipping, shifting, adding noise) applied to the input time series data sequence, the predicted labels given by the model should remain consistent. Conversely, if the predicted labels obtained by different transformations are inconsistent, it indicates that the labeling of the original time step is not robust enough and is unreliable.

[0069] The consistency requirement is 4 / 4 consistency, meaning that the land cover category labeling at a given original time step must be identical across the original class activation map, the first enhanced class activation map, the second enhanced class activation map, and the third enhanced class activation map. Only land cover category labels that are completely identical are retained as weak labels. If there is any inconsistency in the land cover category labeling across any of the class activation maps, the land cover category will not be included in the weak labeling.

[0070] The reason for using consistency constraints is: Weakly supervised learning is sensitive to noise, and mislabels can seriously affect the training effect of the model. Therefore, it is necessary to prioritize the accuracy of weak labels. It's better to have fewer annotations than to have incorrect ones. By strictly adhering to consistency constraints, uncertain annotations are filtered out.

[0071] The 4 / 4 consistency requirement aims to achieve the best balance between accuracy and reliability, ensuring the quality of weak labels while providing sufficient supervision signals to support model training.

[0072] This step effectively improves the quality and reliability of weak labels by using three temporal enhancement methods—temporal flipping, temporal translation, and noise perturbation—combined with a strict consensus voting mechanism. It solves the noise and uncertainty problems existing in a single original class activation graph, laying a solid foundation for the training of subsequent temporal semantic segmentation models.

[0073] Step S140: Train a temporal semantic segmentation model based on temporal data sequences and weak labels.

[0074] This embodiment then trains a temporal semantic segmentation model based on time-series data sequences and weak labels, so that the temporal semantic segmentation model learns the mapping relationship between the time-series features of the time-series data sequences and the weak labels of land cover categories, thereby achieving accurate prediction of land cover categories at each original time step.

[0075] In some optional implementations, step S140, training a temporal semantic segmentation model based on temporal data sequences and weak labels, includes: the temporal semantic segmentation model includes a ResNet50-1D backbone network and an FPN-like multi-scale feature fusion module; the ResNet50-1D backbone network includes an initial convolutional layer and four feature extraction levels; multi-level feature extraction is performed on the temporal data sequence through the ResNet50-1D backbone network; multi-level features are fused through the FPN-like multi-scale feature fusion module in a top-down path; and the temporal semantic segmentation model is trained using weak labels as supervision signals.

[0076] The temporal semantic segmentation model adopts a combined architecture of ResNet50-1D backbone network and FPN-like multi-scale feature fusion module.

[0077] The ResNet50-1D backbone network consists of an initial convolutional layer and four feature extraction levels (Stage 1 to Stage 4). The ResNet50-1D backbone network is used to perform multi-level feature extraction on time-series data sequences.

[0078] The initial convolutional layer uses a 1D convolutional kernel to transform the input temporal data sequence into a feature map (dimension B×64×24, where B represents the batch size). Stage 1 contains 3 residual blocks, each consisting of a 1×1 convolution, a 3×3 convolution, and a 1×1 convolution. Batch normalization and ReLU activation function are used, and the output feature map dimension is B×256×24. Stage 2 contains 8 residual blocks, and the output feature map has a dimension of B×512×12; Stage 3 contains 12 residual blocks, and the output feature map has a dimension of B×1024×12; Stage 4 contains 3 residual blocks, and the output feature map has a dimension of B×2048×12.

[0079] Multi-level features are fused in a top-down manner using an FPN-like multi-scale feature fusion module.

[0080] Layer 4 dimensionality reduction: C4 = Conv1D(F4:2048→6, kernel=1), output C4(B×6×12). That is, the dimensionality of the feature map output by Stage 4 is reduced, and the number of channels is reduced from 2048 to 6 to obtain the C4 feature map.

[0081] Layer 3 dimensionality reduction and fusion: C3 = Conv1D (F3: 1024 → 6, kernel = 1), output C3 (B × 6 × 12); Fuse1 = C4 + C3, output Fuse1 (B × 6 × 12). That is, the dimensionality of the feature map output from Stage 3 is reduced, the number of channels is reduced from 1024 to 6 to obtain the C3 feature map, and the C4 feature map is added element-wise to the C3 feature map to obtain the Fuse1 feature map.

[0082] Layer 2 dimensionality reduction and fusion: C2 = Conv1D(F2: 512 → 6, kernel = 1), output C2 (B × 6 × 12); Fuse2 = Fuse1 + C2, output Fuse2 (B × 6 × 12); Fuse2_up = Interpolate(Fuse2, size = 24, mode = 'linear'), output Fuse2_up (B × 6 × 24). That is, the dimensionality of the feature map output from Stage 2 is reduced from 512 to 6 to obtain the C2 feature map. The Fuse1 feature map and the C2 feature map are added element-wise to obtain the Fuse2 feature map. Linear interpolation upsampling is then performed on the Fuse2 feature map, increasing the time step from 12 to 24 to obtain the Fuse2_up feature map.

[0083] Layer 1 Dimensionality Reduction + Fusion: C1 = Conv1D(F1:256→6, kernel=1), output C1 (B×6×24); Fuse3 = Fuse2_up + C1, output Fuse3 (B×6×24); Output = Interpolate(Fuse3, size=48, mode='linear'), output Output (B×6×48). That is, the feature map output from Stage 1 is dimensionality reduced, decreasing the number of channels from 256 to 6 to obtain the C1 feature map. The Fuse2_up feature map and the C1 feature map are element-wise added to obtain the Fuse3 feature map. Linear interpolation upsampling is then performed on the Fuse3 feature map, increasing the time step from 24 to 48 to obtain the output.

[0084] The reason for adopting an FPN-like multi-scale feature fusion module is to address the problem caused by traditional segmentation networks only applying the deepest features. The traditional method is: F4(2048,12) → Conv1D → (6,12) → Upsample → (6,48). The traditional method only uses Stage4 features (2048×12) for dimensionality reduction and upsampling, which has the problems of low resolution (12 steps), loss of details (4 times upsampling), blurred change boundaries, and only containing high-level semantics without low-level spatial details.

[0085] The FPN-like multi-scale feature fusion module integrates multi-scale features: the low-level features of F1 (256,24) are rich in spatial details, the mid-level features of F2 (512,12) contain local patterns, the mid-to-high-level features of F3 (1024,12) contain local patterns and semantic information, and the high-level features of F4 (2048,12) have global semantics.

[0086] By adopting a top-down fusion approach, a combination of high-level semantic guidance and low-level detail supplementation is achieved, which not only ensures the accurate classification of land cover categories but also enables precise positioning of change boundaries.

[0087] Weak labels are used as supervision signals to train the temporal semantic segmentation model.

[0088] The cross-entropy loss function can be used to calculate the loss between the predicted result and the weak label. The cross-entropy loss function can effectively measure the difference between the predicted probability distribution and the true probability distribution corresponding to the weak label.

[0089] The stochastic gradient descent optimizer or the adaptive moment estimation (Adam) optimizer is used. The Adam optimizer is preferred. The learning rate can be set to 0.001, the weight decay can be set to 0.0001, and the batch size can be set to 32.

[0090] This step designs an efficient network architecture suitable for time-series data sequences. It achieves deep extraction of time-series features through the ResNet50-1D backbone network and solves the problems of detail loss and boundary blurring in traditional models by combining an FPN-like multi-scale feature fusion module. At the same time, it uses weak labels as supervision signals to realize model training under weak supervision, which balances training accuracy and computational efficiency.

[0091] Step S150: Input the dense temporal remote sensing image data to be detected into the trained temporal semantic segmentation model to obtain the land cover category prediction results for each original time step.

[0092] This embodiment continues to use the trained temporal semantic segmentation model to predict the land cover category of new dense temporal remote sensing image data to be detected.

[0093] In some optional implementations, step S150 involves inputting the dense temporal remote sensing image data to be detected into the trained temporal semantic segmentation model to obtain the land cover category prediction results for each original time step. This includes: acquiring satellite remote sensing image data to be detected containing consecutive original time steps as the dense temporal remote sensing image data to be detected; extracting temporal features from the dense temporal remote sensing image data to be detected to obtain the temporal data sequence to be detected; inputting the temporal data sequence to be detected into the trained temporal semantic segmentation model, and outputting the land cover category prediction results corresponding to each original time step based on the temporal semantic segmentation model.

[0094] The satellite remote sensing image data to be detected, containing continuous raw time steps, is acquired as the dense time-series remote sensing image data to be detected.

[0095] The satellite type, number of original time steps, etc. of the dense time-series remote sensing image data to be detected are the same as those of the dense time-series remote sensing image data with continuous original time steps obtained in step S110, so as to ensure the effectiveness of the model prediction.

[0096] Temporal features are extracted from the dense temporal remote sensing image data to be detected, resulting in the temporal data sequence to be detected.

[0097] The same temporal features as those in step S110 are extracted from the dense temporal remote sensing image data to be detected, such as radiometric correction, atmospheric correction, geometric correction, cloud removal, and chronological arrangement.

[0098] The time-series data sequence to be detected is input into the trained temporal semantic segmentation model, and the prediction results of the land cover category corresponding to each original time step are output based on the temporal semantic segmentation model.

[0099] The time-series data sequence to be detected is input into the trained temporal semantic segmentation model. The model extracts features through the ResNet50-1D backbone network, and after fusion through the FPN-like multi-scale feature fusion module, it outputs the predicted probability of land cover category corresponding to each original time step. The predicted probability of land cover category for each original time step is calculated to obtain the predicted result of land cover category for each original time step.

[0100] Step S160: By analyzing the prediction results of land cover categories, identify the pixel locations and corresponding time points of land cover category abrupt changes, and generate spatial change maps and change time maps.

[0101] Finally, this embodiment extracts change information from the prediction results to achieve "spatiotemporal integrated" change detection.

[0102] In some optional implementations, step S160, by analyzing the land cover category prediction results, identifies the pixel locations and corresponding time points of land cover category mutations, and generates a spatial change map and a change time map, includes: performing sequence analysis on the land cover category prediction results of each pixel at each original time step, recording the time point of land cover category mutation and the spatial coordinate position of the corresponding pixel; generating a spatial change map based on the spatial position of the pixel, and generating a change time map based on the time point of the mutation.

[0103] Sequence analysis is performed on the land cover category prediction results of each pixel at each original time step, and the time points of land cover category changes and the spatial coordinates of the corresponding pixels are recorded.

[0104] For a single pixel's land cover category prediction sequence, such as c1, c1, ..., c1, c2, c2, ..., c2, c2 (c1 and c2 represent different land cover categories), identify the mutation points in the prediction sequence, and then record the time point of the mutation and the spatial coordinates of the corresponding mutated pixel.

[0105] Spatial change maps are generated based on the spatial location of pixels, and temporal change maps are generated based on the time points of abrupt changes.

[0106] Based on the spatial resolution of dense temporal remote sensing image data, pixels that undergo abrupt changes in land cover categories are marked as changed pixels, while pixels that do not undergo changes in land cover categories are marked as unchanged pixels, thereby generating a spatial change map that visually displays the spatial distribution of abrupt changes.

[0107] Based on the spatial resolution of dense temporal remote sensing image data, each changed pixel is labeled with the time point of its land cover category change, thereby generating a change time map that clearly shows the time of change of each changed pixel.

[0108] This embodiment achieves "spatiotemporal integration" information display, which can be applied in many fields such as disaster emergency assessment, crop phenology monitoring, and dynamic monitoring of illegal construction, and is more valuable than a single spatial change map.

[0109] In addition to the ResNet50-1D architecture used in this embodiment, similar effects can be achieved by replacing it with deep learning time modeling models such as Transformer, LSTM, and FCN.

[0110] Figure 2 A schematic diagram of a temporal remote sensing land cover change detection device according to an embodiment of this application is shown. (Refer to...) Figure 2 As shown, the time-series remote sensing land cover change detection device 200 includes: The data annotation module 210 is used to acquire dense time-series remote sensing image data containing continuous raw time steps and construct a time-series data sequence, and to perform multi-label coarse annotation on the time-series data sequence; wherein, the multi-label coarse annotation marks the land cover categories included in the time-series data sequence; The original class activation graph generation module 220 is used to generate the original class activation graph in the time series dimension based on multi-label coarse annotation and time series data sequence; The weak label filtering module 230 is used to perform various time-series enhancement processing on the time-series data sequence, generate multiple enhanced class activation maps, and filter weak labels based on the original class activation map and the enhanced class activation map; Model training module 240 is used to train a temporal semantic segmentation model based on temporal data sequences and weak labels; The change detection module 250 is used to input the dense temporal remote sensing image data to be detected into the trained temporal semantic segmentation model to obtain the land cover category prediction results for each original time step; The mapping module 260 is used to identify the pixel locations and corresponding time points of abrupt changes in land cover categories by analyzing the prediction results of land cover categories, and to generate spatial change maps and change time maps.

[0111] In some optional embodiments, in the above apparatus, the data labeling module 210 is used to: acquire satellite remote sensing image data containing continuous raw time steps as dense time-series remote sensing image data; extract time-series features from the dense time-series remote sensing image data to obtain a time-series data sequence; and label the land cover categories contained in the time-series data sequence as multi-label coarse annotations.

[0112] In some optional embodiments, in the above apparatus, the original class activation map generation module 220 is used to: extract the core features of the time-series data sequence encoded by the last layer of the ResNet50-1D model, perform global average pooling on the core features to obtain a global feature vector; input the global feature vector into a fully connected layer to obtain class recognition weights, and back-project the class recognition weights to each downsampling time step of the time-series data sequence by weighted summation to generate an initial class activation map; perform temporal interpolation on the initial class activation map using linear interpolation, restore the downsampling time step to the original time step, and obtain the original class activation map.

[0113] In some optional embodiments, in the above apparatus, the weak label screening module 230 is used to: perform time-series flipping on the time-series data sequence to generate a first enhanced class activation map of the flipped time-series data sequence; perform time-series translation on the time-series data sequence to generate a second enhanced class activation map of the translated time-series data sequence; perform noise perturbation on the time-series data sequence to generate a third enhanced class activation map of the perturbed time-series data sequence; and screen weak labels whose coarse multi-label annotations are all consistent with those in the original class activation map and the three enhanced class activation maps through a consensus voting mechanism.

[0114] In some optional embodiments, in the above apparatus, the model training module 240 is used for: the temporal semantic segmentation model including a ResNet50-1D backbone network and an FPN-like multi-scale feature fusion module, wherein the ResNet50-1D backbone network includes an initial convolutional layer and four feature extraction levels; performing multi-level feature extraction on the temporal data sequence through the ResNet50-1D backbone network; fusing multi-level features through the FPN-like multi-scale feature fusion module in a top-down path; and training the temporal semantic segmentation model using weak labels as supervision signals.

[0115] In some optional embodiments, in the above apparatus, the change detection module 250 is used to: acquire satellite remote sensing image data to be detected containing continuous original time steps as dense time-series remote sensing image data to be detected; extract temporal features from the dense time-series remote sensing image data to be detected to obtain a time-series data sequence to be detected; input the time-series data sequence to be detected into a trained temporal semantic segmentation model, and output the land cover category prediction results corresponding to each original time step based on the temporal semantic segmentation model.

[0116] In some optional embodiments, in the above apparatus, the mapping module 260 is used to: perform sequence analysis on the land cover category prediction results of each pixel at each original time step, record the time point of land cover category change and the spatial coordinate position of the corresponding pixel; generate a spatial change map based on the spatial position of the pixel, and generate a change time map based on the time point of the change.

[0117] It should be noted that the aforementioned time-series remote sensing land cover change detection device 200 can realize the aforementioned time-series remote sensing land cover change detection methods, which will not be elaborated further.

[0118] Figure 3 This invention illustrates a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 3 As shown, the electronic device includes a processor, internal memory, a network interface, and a non-volatile storage medium connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for communication with external devices via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a time-series remote sensing land cover change detection method.

[0119] In one embodiment, the electronic device provided in this application includes a memory and a processor. The memory stores a database and a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of a method for detecting time-series remote sensing land cover change.

[0120] The above is as stated in this application. Figure 2 The method executed by the time-series remote sensing land cover change detection device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The steps of the method disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0121] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of a method for detecting time-series remote sensing land cover change.

[0122] It should be noted that the functions or steps that the above-mentioned electronic devices or computer-readable storage media can achieve can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0123] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0124] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0125] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for detecting land cover change using time-series remote sensing, characterized in that, include: Acquire dense time-series remote sensing image data containing continuous raw time steps and construct a time-series data sequence, then perform multi-label coarse annotation on the time-series data sequence; wherein, the multi-label coarse annotation marks the land cover categories included in the time-series data sequence; Based on multi-label coarse annotation and time-series data sequences, a raw class activation graph in the time-series dimension is generated; Multiple time-series enhancement processes are applied to the time-series data sequence to generate multiple enhanced activation maps. Weak labels are then filtered based on the original activation maps and the enhanced activation maps. A temporal semantic segmentation model is trained based on temporal data sequences and weak labels; The dense temporal remote sensing image data to be detected is input into the trained temporal semantic segmentation model to obtain the land cover category prediction results for each original time step; By analyzing the prediction results of land cover categories, the pixel locations and corresponding time points of land cover category abrupt changes are identified, and spatial change maps and change time maps are generated.

2. The method for detecting time-series remote sensing land cover change according to claim 1, characterized in that, The process of acquiring dense temporal remote sensing image data containing continuous raw time steps and constructing a temporal data sequence, followed by multi-label coarse annotation of the temporal data sequence, includes: Acquire satellite remote sensing image data containing continuous raw time steps as dense time-series remote sensing image data; Temporal features are extracted from dense temporal remote sensing image data to obtain a temporal data sequence; The land cover categories contained in the time series data are labeled as coarse multi-label annotations.

3. The method for detecting time-series remote sensing land cover change according to claim 1, characterized in that, The process of generating the original class activation map in the time series dimension based on multi-label coarse annotation and time series data sequences includes: Extract the core features of the time series data sequence after the last layer of the ResNet50-1D model encoding, and perform global average pooling on the core features to obtain the global feature vector; The global feature vector is input into the fully connected layer to obtain the class recognition weights. The class recognition weights are then back-projected to each downsampling time step of the time series data sequence by weighted summation to generate the initial class activation map. Linear interpolation is used to perform temporal interpolation on the initial class activation map, and the downsampling time step is restored to the original time step to obtain the original class activation map.

4. The method for detecting time-series remote sensing land cover change according to claim 1, characterized in that, The process involves performing various time-series enhancement processes on the time-series data sequence to generate multiple enhanced activation maps. Weak labels are then filtered based on the original activation maps and the enhanced activation maps, including: Perform time-series flipping on the time-series data sequence to generate the first enhanced class activation graph of the flipped time-series data sequence; A temporal shift is performed on the time-series data sequence to generate a second enhanced class activation graph of the shifted time-series data sequence; Noise perturbation is applied to the time series data sequence to generate a third-enhanced activation map of the perturbed time series data sequence; Weak labels that are completely consistent with the coarse annotations of the multi-labels in the original class activation graph and the three enhanced class activation graphs are selected through a consistency voting mechanism.

5. The method for detecting time-series remote sensing land cover change according to claim 1, characterized in that, The time-series semantic segmentation model trained based on time-series data sequences and weak labels includes: The temporal semantic segmentation model includes a ResNet50-1D backbone network and an FPN-like multi-scale feature fusion module. The ResNet50-1D backbone network includes an initial convolutional layer and four feature extraction layers. Multi-level feature extraction of time-series data sequences is performed using the ResNet50-1D backbone network; Multi-level features are fused in a top-down path using an FPN-like multi-scale feature fusion module. Weak labels are used as supervision signals to train the temporal semantic segmentation model.

6. The method for detecting time-series remote sensing land cover change according to claim 1, characterized in that, The process involves inputting the dense temporal remote sensing image data to be detected into a trained temporal semantic segmentation model to obtain the land cover category prediction results for each original time step, including: Acquire satellite remote sensing image data containing continuous raw time steps as dense time-series remote sensing image data to be detected; Temporal features are extracted from the dense temporal remote sensing image data to be detected to obtain the temporal data sequence to be detected; The time-series data sequence to be detected is input into the trained temporal semantic segmentation model, and the prediction results of the land cover category corresponding to each original time step are output based on the temporal semantic segmentation model.

7. The method for detecting time-series remote sensing land cover change according to claim 1, characterized in that, The process of analyzing the land cover category prediction results, identifying the pixel locations and corresponding time points of land cover category abrupt changes, and generating spatial change maps and temporal change maps includes: Sequence analysis was performed on the land cover category prediction results of each pixel at each original time step, and the time points of land cover category changes and the spatial coordinates of the corresponding pixels were recorded. Spatial change maps are generated based on the spatial location of pixels, and temporal change maps are generated based on the time points of abrupt changes.

8. A time-series remote sensing device for detecting land cover change, characterized in that, include: The data annotation module is used to acquire dense time-series remote sensing image data containing continuous raw time steps and construct time-series data sequences, and to perform multi-label coarse annotation on the time-series data sequences; wherein, the multi-label coarse annotation marks the land cover categories included in the time-series data sequences; The original class activation graph generation module is used to generate original class activation graphs in the time series dimension based on multi-label coarse annotation and time series data sequences. The weak label filtering module is used to perform various time-series enhancement processes on time-series data sequences, generate multiple enhanced class activation maps, and filter weak labels based on the original class activation maps and enhanced class activation maps; The model training module is used to train a temporal semantic segmentation model based on temporal data sequences and weak labels; The change detection module is used to input the dense temporal remote sensing image data to be detected into the trained temporal semantic segmentation model to obtain the land cover category prediction results for each original time step; The mapping module is used to analyze the prediction results of land cover categories, identify the pixel locations and corresponding time points of abrupt changes in land cover categories, and generate spatial change maps and temporal change maps.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for detecting time-series remote sensing land cover change as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for detecting time-series remote sensing land cover change as described in any one of claims 1 to 7.

Images