Mangrove forest dense time sequence detection method and device

Abstract

This invention provides a method and apparatus for detecting dense mangrove forests over time, relating to the field of target recognition technology. The method includes: obtaining radar images and optical remote sensing images of the area to be tested; using a pre-constructed dual-branch feature extraction network to independently extract features from the radar images and optical remote sensing images, obtaining radar features and optical remote sensing features; fusing the radar features and optical remote sensing features to obtain a first fused feature; calculating SSMI features based on the reflectivity data of the radar and optical remote sensing images; fusing the first fused feature and the SSMI feature to obtain a second fused feature; using a pre-constructed target network to process the second fused feature to obtain candidate detection results, wherein the target network is used to simultaneously identify the changing location and type of change of the target object; and performing spatiotemporal consistency correction on the candidate detection results to obtain the detection results for output.

Description

Technical Field

[0001] This invention relates to the field of image data detection technology, and in particular to a method and apparatus for detecting dense mangrove forests over time. Background Technology

[0002] The existing dynamic monitoring technology for mangroves has three problems: (1) It relies too much on optical remote sensing data, but the availability of optical images is affected by clouds and fog. Not only is the temporal resolution low, but the spectral responses of mangroves and coastal vegetation are also highly similar, making it difficult to accurately identify mangroves by relying solely on optical data. (2) The temporal resolution is too low, making it difficult to meet the needs of high-frequency monitoring of mangroves in specific areas. (3) Tidal inundation seriously affects the spectral reflectance of mangroves in remote sensing images, making it difficult to effectively distinguish between mangroves and coastal vegetation, and between high tide mangroves and water bodies.

[0003] There are various existing methods for mangrove distribution extraction and change detection. For example, for distribution extraction, existing methods can generally be divided into two categories. The first category is sample-based supervised classification, which uses input data from raw bands or vegetation indices from remote sensing images and employs traditional machine learning or deep learning classifiers to map mangroves. However, both machine learning and deep learning methods require a large number of samples to train the classifier, so the quality of the samples and the choice of classifier have a significant impact on the classification results. The second category is unsupervised classification based on exponential thresholding, which identifies target objects based on exponential thresholding of exponential amplitude. Since each index is composed of different spectral bands, it can express the characteristics of the target land cover better than a single band, but it requires different thresholds to be set in different regions and has weak transferability. For mangrove change detection, existing methods such as CCDC and BFAST cannot provide detailed information on mangrove changes, such as the land cover type before and after the change.

[0004] Some scholars have proposed a novel spectral and SAR mangrove index (SSMI). This index utilizes the band spectral sensitivity analysis of optical and radar images, leveraging the unique characteristics of mangroves in terms of optical properties (greenness and humidity) and SAR backscattering coefficients to effectively combine optical and SAR images. This achieves complementary spectral and spatial structure information, accurately capturing mangrove features. However, this method has limitations. Its core objective is the mapping of static mangroves, failing to capture temporal changes and only outputting "single-time-point" results. It lacks change detection capabilities, unable to pinpoint the "timing and type of change." Its temporal resolution is extremely low, failing to meet high-frequency monitoring requirements. The feature fusion method is coarse, failing to fully utilize the complementarity of multi-source data. Furthermore, its simple model architecture results in poor generalization ability and adaptability to complex scenarios.

[0005] Some scholars have proposed a continuous detection method for mangrove conditions based on time-series Landsat images. This method first uses the Identification and Characterization of Tidal Wetland Changes (DECODE) algorithm for disturbance detection. This algorithm is an existing dense time-series model designed to detect disturbances adapted to tidal fluctuations in tidal wetlands. While this algorithm is well-suited for detecting tidal wetland disturbances, it cannot provide satisfactory post-disturbance monitoring results due to the significant differences in Landsat observations after disturbance. To better monitor post-disturbance conditions, a novel time-series fitting method, DECODER (DECODE and Recovery), is proposed for the recovery phase. Furthermore, a random forest classifier is constructed for time periods divided by disturbance events, incorporating temporal spectral variables derived from the time-series model to characterize mangrove conditions. However, a drawback is that this method is overly dependent on optical imagery, making it unsuitable for effective mangrove monitoring during cloudy and rainy seasons or weather conditions. Summary of the Invention

[0006] To address the aforementioned technical problems, embodiments of the present invention provide a method for detecting dense mangrove forests over time, comprising:

[0007] The radar and optical remote sensing images of the area to be measured are obtained, and the area to be measured includes mangroves, which are the main objects of detection.

[0008] The pre-built dual-branch feature extraction network is invoked to extract features from the radar image and the optical remote sensing image independently, respectively, to obtain radar features and optical remote sensing features. The dual-branch feature extraction network has two independent feature extraction branches, and different feature extraction methods are used in different feature extraction branches.

[0009] The radar features and optical remote sensing features are fused to obtain the first fused feature;

[0010] The SSMI characteristics are calculated based on the radar and optical remote sensing image reflectance data.

[0011] The first fusion feature and the SSMI feature are fused to obtain the second fusion feature;

[0012] The pre-built target network is invoked to process the second fusion feature to obtain candidate detection results. The target network is used to simultaneously identify the change location and change type of the target object.

[0013] Spatiotemporal consistency correction is performed on the candidate detection results to obtain the detection results used for output.

[0014] In one embodiment, the dual-branch feature extraction network includes a first feature extraction branch and a second feature extraction branch;

[0015] Constructing the dual-branch feature extraction network includes:

[0016] The first feature extraction branch for extracting the radar features is prepared based on a network model with long-term processing capability, wherein the network model includes the Mamba model.

[0017] A second feature extraction branch for extracting the optical remote sensing features is prepared based on a lightweight convolution model, wherein the lightweight convolution model includes a one-dimensional convolution model.

[0018] In one embodiment, the feature fusion of the radar features and optical remote sensing features to obtain the first fused feature includes:

[0019] The radar features and optical remote sensing features are fused together using a target mechanism to obtain the first fused feature. The target mechanism includes an attention mechanism and a gating fusion mechanism.

[0020] In one embodiment, the feature fusion of the radar features and optical remote sensing features using the target-combining mechanism includes:

[0021] High-dimensional features are extracted from the radar features and optical remote sensing features respectively to obtain feature vectors X and Y;

[0022] The attention weight matrix W of the feature vectors X and Y is calculated using a scaled dot product attention mechanism;

[0023] The attention weight matrix W is multiplied by the feature vector Y, and the result of the multiplication is superimposed on the feature vector X to obtain the first fused feature.

[0024] In one embodiment, fusing the first fusion feature with the SSMI feature to obtain the second fusion feature includes:

[0025] Align the first fusion feature with the SSMI feature;

[0026] The SSMI features after feature alignment are converted into a target dimension through a mapping process, and the target dimension is the same as the dimension of the first fused feature after feature alignment.

[0027] The SSMI features with the same dimension are concatenated with the first fusion feature;

[0028] Determine the adaptive fusion weights for each time step;

[0029] The adaptive fusion weights are separated to obtain a first weight corresponding to the SSMI feature and a second weight corresponding to the first fusion feature;

[0030] The first weight, the second weight, and the spliced ​​SSMI feature of each time step are dynamically fused with the first fusion feature to obtain the second fusion feature.

[0031] In one embodiment, the target network includes at least one of a temporal semantic segmentation network, a dual-branch contrast network, and a temporal change detection network based on an attention mechanism.

[0032] In one embodiment, the target network is a temporal semantic segmentation network;

[0033] The process of calling the pre-built target network to process the second fused feature and obtain candidate detection results includes:

[0034] The second fused feature is downsampled by multi-order one-dimensional convolution operations, and the first feature map is generated accordingly.

[0035] Gradually increase the receptive field and compress the feature dimension, and for the feature map output by downsampling at a specified stage, use one-dimensional transpose convolution with different sampling multiples to perform scale recovery and feature preservation.

[0036] The saved multi-scale features are spliced ​​together using a skip connection method, and then upsampled and convolutionally processed to generate the candidate monitoring results.

[0037] In one embodiment, the loss function in the temporal semantic segmentation network is generated by fusing the cross-entropy loss function and the Dice loss, and the loss value of the temporal semantic segmentation network is determined by the sum of the cross-entropy loss value and the Dice loss value.

[0038] In one embodiment, performing spatiotemporal consistency correction on the candidate detection results to obtain the detection results for output includes:

[0039] For the overall time series of a pixel in the candidate detection result, the center point type within the window area is modified to the target type using a first sliding window. The target type is the land cover type that appears most frequently in the current window area, and the frequency of the land cover type meets a preset value range.

[0040] Sequences whose land cover type changes last for less than a preset time range are merged into the nearest sequence with the longest duration of stable land cover type.

[0041] The distribution of mangroves within a single time phase in the candidate detection results is extracted and then filtered using a second sliding window.

[0042] Another embodiment of the present invention also provides a mangrove dense temporal detection device, comprising:

[0043] The acquisition module is used to acquire radar images and optical remote sensing images of the area to be measured, wherein the area to be measured includes mangroves as the main detection object;

[0044] The first calling module is used to call a pre-built dual-branch feature extraction network to independently extract features from the radar image and the optical remote sensing image, thereby obtaining radar features and optical remote sensing features. The dual-branch feature extraction network has two independent feature extraction branches, and different feature extraction methods are used in different feature extraction branches.

[0045] The first fusion module is used to fuse the radar features and the optical remote sensing features to obtain the first fused features;

[0046] The calculation module is used to calculate the SSMI features based on the radar and optical remote sensing image reflectance data;

[0047] The second fusion module is used to fuse the first fusion feature with the SSMI feature to obtain the second fusion feature;

[0048] The second invocation module is used to invoke the pre-built target network to process the second fusion feature and obtain candidate detection results. The target network is used to simultaneously identify the change position and change type of the target object.

[0049] The correction module is used to perform spatiotemporal consistency correction on the candidate detection results to obtain the detection results for output.

[0050] Based on the above, this application solves the problem of unstable optical images by introducing an optical and radar fusion mechanism. By combining supervised classification and exponential thresholding methods, it identifies the semantic boundaries between different land cover types in the time series, thereby achieving synchronous classification and change detection of land cover at each time point. This not only improves efficiency but also significantly improves the accuracy of detection.

[0051] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0052] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0053] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0054] Figure 1 This is a flowchart illustrating the mangrove dense temporal detection method in an embodiment of the present invention.

[0055] Figure 2 This is a schematic diagram illustrating the application process of the mangrove dense temporal detection method in this embodiment of the invention.

[0056] Figure 3 This is a schematic diagram of the overall application process of feature extraction and fusion in an embodiment of the present invention.

[0057] Figure 4 This is a schematic diagram of the mangrove dense temporal detection device in an embodiment of the present invention. Detailed Implementation

[0058] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but these are not intended to limit the scope of the invention.

[0059] It should be understood that various modifications can be made to the embodiments disclosed herein. Therefore, the following description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope of this disclosure will be apparent to those skilled in the art.

[0060] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present disclosure and, together with the general description of the disclosure given above and the detailed description of the embodiments given below, serve to explain the principles of the disclosure.

[0061] These and other features of the invention will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0062] It should also be understood that although the invention has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of the invention, which have the features described in the claims and are therefore all within the scope of protection defined herein.

[0063] The above and other aspects, features and advantages of this disclosure will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0064] Specific embodiments of the present disclosure are described thereafter with reference to the accompanying drawings; however, it should be understood that the disclosed embodiments are merely examples of the present disclosure and can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the present disclosure. Therefore, the specific structural and functional details disclosed herein are not intended to be limiting, but merely to serve as the basis and representative basis for the claims to teach those skilled in the art to use the present disclosure in a variety of substantially any suitable detailed structures.

[0065] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in still another embodiment,” all of which may refer to one or more of the same or different embodiments according to this disclosure.

[0066] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0067] like Figure 1 and Figure 2 As shown, this embodiment of the invention provides a method for detecting dense mangrove forests over time, including:

[0068] S1: Obtain radar and optical remote sensing images of the area to be tested, wherein the area to be tested includes mangroves as the main detection target;

[0069] S2: The pre-built dual-branch feature extraction network is invoked to extract features from the radar image and the optical remote sensing image independently, so as to obtain radar features and optical remote sensing features. The dual-branch feature extraction network has two independent feature extraction branches, and different feature extraction methods are used in different feature extraction branches.

[0070] S3: Perform feature fusion on the radar features and optical remote sensing features to obtain the first fused feature;

[0071] S4: SSMI characteristics are calculated based on the radar and optical remote sensing image reflectance data;

[0072] S5: The first fusion feature is fused with the SSMI feature to obtain the second fusion feature;

[0073] S6: Call the pre-built target network to process the second fusion feature and obtain candidate detection results. The target network is used to simultaneously identify the change location and change type of the target object.

[0074] S7: Perform spatiotemporal consistency correction on the candidate detection results to obtain the detection results for output.

[0075] In this embodiment, the detection process described above is implemented based on a detection model, namely an Optical-Synthetic Aperture Radar (SAR) Time Series Change Detection (OSTCD) model. This model integrates the extraction and detection of dense time series mangrove distribution within a unified framework. The OSTCD model fully leverages the complementary advantages of SAR data and optical remote sensing data. Specifically, the model uses SAR images as the primary time data source because SAR images are acquired at high frequencies, are stable under various atmospheric conditions, and can better characterize vegetation structure. Simultaneously, optical images are used as supplementary spectral information through a dual-branch feature extraction network. Furthermore, the model introduces a novel multi-level adaptive feature fusion method for fusing optical-radar features and the SAR Feature Mangrove Index (SSMI). This method effectively mitigates the impact of tidal inundation on mangrove identification without requiring paired high and low tide images or increasing data complexity. In addition, a network is proposed to analyze one-dimensional time series data, enabling simultaneous land cover classification at each time point and detecting changes by identifying semantic boundaries between different land cover types in the time series. This integrated approach enables end-to-end dense time-series mangrove detection, where the transitions between adjacent land cover types in the detected pixel-level time profiles naturally reveal the timing and nature of the changes.

[0076] Specifically, the OSTCD model architecture in one embodiment consists of three core components: (1) a dual-branch feature extraction network, (2) a multi-level adaptive feature fusion network, and (3) a temporal semantic segmentation network. The inherent differences between optical remote sensing images and SAR images in terms of geometric features, radiometric properties, and imaging mechanisms can lead to serious errors if their time-series data are directly used for change detection without proper feature processing. To overcome this challenge, the framework in this embodiment implements different feature extraction paths for each data modality—processing optical remote sensing data and SAR data separately through specially designed network branches to capture their respective unique features. The extracted features are then initially fused through our adaptive network, which intelligently combines them based on the complementary advantages of optical and SAR features. Subsequently, the fused representation is further fused with features derived from SSMI, utilizing spectral-hydrological-structural information to enhance the combined feature space. Finally, the rich feature set is processed by the temporal semantic segmentation network, which analyzes complete spatiotemporal patterns to achieve accurate detection of mangrove density changes.

[0077] Furthermore, in this embodiment, the dual-branch feature extraction network includes a first feature extraction branch and a second feature extraction branch; constructing the dual-branch feature extraction network includes:

[0078] S8: Prepare the first feature extraction branch for extracting the radar features based on a network model with long-term processing capability, wherein the network model includes the Mamba model;

[0079] S9: Prepare a second feature extraction branch for extracting the optical remote sensing features based on a lightweight convolution model, wherein the lightweight convolution model includes a one-dimensional convolution model.

[0080] For example, the network model may include not only the Mamba model, but also improved Transformers (such as a temporally adapted version of the Swin Transformer, which reduces computational complexity through window attention), variants of Long Short-Term Memory (LSTM) networks (such as BiLSTM, which captures temporal dependencies bidirectionally), or temporal convolutional networks (TCNs, which process long sequences through causal convolution). These temporal convolutional networks can still efficiently extract the temporal structure features of mangrove forests by simply adjusting the network input dimension (such as adapting the state space dimension of Mamba to the attention head dimension of the Transformer) and training parameters (such as the number of hidden layer nodes of the LSTM), thus avoiding the gradient vanishing problem of traditional RNNs.

[0081] The lightweight convolutional model may also include a 1D (1-dimensional) adapted version of MobileNet, which reduces parameters through depthwise separable convolutions, a multilayer perceptron (MLP, suitable for low-dimensional spectral features), or an attention-enhanced CNN (such as SE-CNN, which strengthens the weights of key bands such as red edges and SWIR through channel attention). Only by adjusting the convolutional kernel size (such as MobileNet using a 3×1 convolutional kernel instead of the 5×1 kernel in 1D-CNN) and the activation function (such as MLP using LeakyReLU instead of ReLU) can the local extraction capability of spectral features be maintained.

[0082] In this embodiment, the Mamba model is used for feature extraction from SAR time series. The Mamba model incorporates a state-space model (SSM) and combines it with hardware optimization algorithms to establish long-range dependencies with linear computational complexity. Furthermore, this embodiment introduces the HiPPO matrix into the Mamba model, enabling it to optimally preserve historical information while deeply integrating recent observation data, achieving linear time complexity and significantly improving training and inference efficiency. This architectural innovation not only facilitates efficient processing of long remote sensing time series but also enhances the model's ability to capture trends and periodic patterns. For optical images, this embodiment utilizes CNNs to effectively extract local spatial and spectral features while maintaining model simplicity. The convolutional structure of CNNs significantly reduces the number of learnable parameters, thereby achieving excellent training performance with limited hardware resources. This dual-architecture approach combines the superior long-sequence modeling capabilities of the Mamba model for SAR time analysis with the efficient local feature extraction using CNNs for spectral characterization, forming the computationally efficient and powerful mangrove feature extraction network framework in this embodiment.

[0083] Furthermore, the feature fusion of the radar features and optical remote sensing features to obtain the first fused feature includes:

[0084] S301: Combine the target mechanism to perform feature fusion of the radar features and optical remote sensing features to obtain the first fused feature. The target mechanism includes an attention mechanism and a gating fusion mechanism.

[0085] Specifically, in this embodiment, the feature fusion of radar features and optical remote sensing features using the target-based mechanism includes:

[0086] S302: Extract high-dimensional features from the radar features and optical remote sensing features respectively to obtain feature vectors X and Y;

[0087] S303: Calculate the attention weight matrix W of the feature vectors X and Y using a scaled dot product attention mechanism;

[0088] S304: Multiply the attention weight matrix W with the feature vector Y, and superimpose the multiplication result with the feature vector X to obtain the first fused feature.

[0089] For example, to perform feature fusion more effectively, this embodiment introduces an attention mechanism into the feature fusion process, constructing an attention-based feature fusion module. This module is used to integrate heterogeneous radar and optical time series data into a high-dimensional feature vector Z. The fusion process is as follows (see the following formula): First, high-dimensional features are extracted from the optical time series and the synthetic aperture radar (SAR) time series respectively to obtain the feature vector. X andY ,in and Representing optical feature vectors and radar feature vectors, i and j Indicates the order of the matrix (e.g., optical data has 10 bands). i Use numbers 0-9; radar has two frequency bands: j (Take 0-1). Score ( X , Y The function is used to calculate the attention score; subsequently, a scaled dot product attention mechanism is used to calculate... X and Y Attention weight matrix W ,in W [ i [j] characterizes the optical data of the first... j The first order matrix for SAR data i The focus is on the weights of the eigenvectors and weight matrices. After obtaining the eigenvectors and weight matrices, the weight matrix is... W and Y Multiply and then combine the results with... X By superimposing the features, a fused feature vector can be obtained. Z Optical radar fusion feature vector Z = W·Y + X It should be noted that, X and Y Let D be the input feature vector, and D represent the dimension of the hidden layer of the input data.

[0090]

[0091]

[0092] The softmax() function is used to normalize the attention score to obtain the attention weight matrix W. i , j ].

[0093] The aforementioned feature fusion process achieves the fusion of optical and radar data, preserving sufficient terrain feature information at each moment while maintaining high temporal resolution.

[0094] In another embodiment, the step of fusing the first fusion feature with the SSMI feature to obtain the second fusion feature includes:

[0095] S501: Align the first fusion feature with the SSMI feature;

[0096] S502: The SSMI features after feature alignment are converted into a target dimension through mapping processing, and the target dimension is the same as the dimension of the first fused feature after feature alignment;

[0097] S503: The SSMI features with the same dimension are concatenated with the first fusion feature;

[0098] S504: Determine the adaptive fusion weights for each time step;

[0099] S505: Separate the adaptive fusion weights to obtain a first weight corresponding to the SSMI feature and a second weight corresponding to the first fusion feature;

[0100] S506: Dynamically fuse the first weight, the second weight, and the spliced ​​SSMI feature of each time step with the first fusion feature to obtain the second fusion feature.

[0101] For example, since SSMI raster data (SSMI features) is a single-dimensional index calculated based on optical-SAR data, the purpose of using this data is to highlight the specificity of mangroves in the optical-SAR joint features and enhance the distinction between mangroves and other land cover types such as coastal vegetation and water bodies. The high-dimensional, multimodal feature set obtained by fusing SSMI features with optical-SAR (the first fusion feature) has significant differences. Direct splicing or simple weighted fusion would lead to an imbalance in modal weights. Furthermore, as a "secondary processing product" of both, the "importance" of SSMI features compared to the optical / radar features in the original feature vector cannot be balanced by fixed weights. Moreover, SSMI features themselves are calculated from optical and radar bands and have strong redundancy with some core features in the feature vector (such as near-infrared reflectivity and radar VH polarization). Direct fusion would amplify this redundancy, causing the model to overemphasize repetitive information and mask key differences (such as the subtle spectral / structural differences between mangroves and tidal flat vegetation).

[0102] To solve this problem, such as Figure 3As shown in the figure (ASFF represents feature fusion), the Adaptive Spatial Feature Fusion (ASFF) proposed in this embodiment is a data-driven pyramid feature fusion strategy. Its core lies in suppressing feature inconsistency by learning a spatial-level conflict information filtering mechanism, thereby improving the scale invariance of features with almost no increase in inference overhead. Therefore, this embodiment proposes an adaptive weight fusion strategy suitable for different data sources at the same scale to achieve efficient fusion of optical-SAR features and SSMI raster features. In the fusion strategy proposed in this embodiment, firstly, the SSMI time-series raster data and optical-SAR feature data (the first fused feature) are feature-aligned. Then, the SSMI features are transformed to the same dimension as the fused feature through mapping. Subsequently, the two types of features after feature alignment are concatenated, and an adaptive fusion weight for the concatenated feature is generated at each time step using a pre-built weight generation network. Finally, the weights are separated, and the dynamic fusion of the two features is achieved based on the weights corresponding to different data sources.

[0103] Based on the two-stage fusion scheme in this embodiment, as an alternative to feature fusion, in the first-stage fusion process, alternative solutions can be other attention mechanisms (such as multi-head attention, which captures different modal associations through multiple sets of attention heads), gated fusion mechanisms (such as GRU gate, which dynamically adjusts the weights of optical / SAR features through update gates and reset gates), or feature stitching + convolutional fusion (stitching optical and SAR features together and then compressing the dimensions through 1D convolution). Only by adjusting the weight calculation logic of the fusion layer (such as setting the number of heads of multi-head attention to 8) and the feature dimension matching method (such as mapping optical features to SAR feature dimensions through a fully connected layer before gated fusion), complementary integration of multimodal features can still be achieved.

[0104] In the secondary fusion process, alternative solutions include feature pyramid fusion (using SSMI features as an independent feature layer and fusing them with optical-SAR fusion features at different resolution levels), weighted summation fusion (directly superimposing the three types of features through dynamically learned weight coefficients), or attention-guided feature addition (such as using SSMI features as attention weights to weight the optical-SAR fusion features). Only the fusion level (such as using three-layer resolution matching for feature pyramids) and the weight learning method (such as using the Adam optimizer to learn the weights for weighted summation) need to be adjusted to avoid redundant conflicts between SSMI features and the original features.

[0105] After feature fusion is completed, the system needs to generate candidate detection results using the target network. In this embodiment, the target network includes at least one of the following: a temporal semantic segmentation network, a two-branch comparison network, and a temporal change detection network based on an attention mechanism. For example, this embodiment uses a temporal semantic segmentation network. Alternative solutions could include a two-branch comparison network (such as a temporally adapted version of the Siamese network, which locates changes by comparing differences between adjacent temporal features), a Transformer-based temporal change detection network (such as TimeSformer, which uses temporal images as sequence input to capture changes), or a two-step method of classification followed by comparison (first classifying a single temporal image using ResNet, then comparing the classification results to locate changes). Only adjustments to the network output dimension (e.g., replacing the classification map with a change probability map from the Siamese network) and the comparison logic (e.g., using IoU to compare differences in classification results in the two-step method) are needed to still achieve the output of three-dimensional change information: space-time-type.

[0106] Specifically, in this embodiment, the target network is a temporal semantic segmentation network. The process of calling the pre-built target network to process the second fused feature and obtain candidate detection results includes:

[0107] S601: The second fused feature is downsampled through multi-order one-dimensional convolution operation, and the first feature map is generated respectively;

[0108] S602: Gradually increase the receptive field and compress the feature dimension, and for the feature map output by downsampling at a specified stage, use one-dimensional transpose convolution with different sampling multiples to perform scale recovery and feature preservation.

[0109] S603: The stored multi-scale features are spliced ​​together using a skip connection method, and then upsampled and convolutionally processed to generate the candidate monitoring results.

[0110] Specifically, remote sensing image semantic segmentation tasks, through pixel-level category labeling, provide a mature paradigm for "analyzing land cover types from a spatial dimension." On the other hand, temporal semantic segmentation research focuses on how to perform fine-grained category classification of sequence data over time. Inspired by this dual approach, this embodiment proposes an improved temporal semantic segmentation network, aiming to simultaneously achieve accurate localization of changing locations and refined identification of change types. The network's specific workflow includes: First, time-series data after feature fusion is input into the network for end-to-end prediction; then, the network performs downsampling through multiple stages, such as four stages, using one-dimensional convolutional operations to gradually increase the receptive field and compress the feature dimension. While achieving feature abstraction and dimensionality reduction, for the feature maps output from the downsampling in stages 2, 3, and 4, 2x, 4x, and 8x one-dimensional transposed convolutions are used respectively for scale recovery and feature preservation to reduce the loss of key information and align features at different levels (shallow features containing detailed information and deep features containing semantic information) in the spatial dimension. Subsequently, these multi-scale features are stitched together through skip connections, thereby integrating multi-level information from local to global perspectives. The concatenated and fused features are then processed through a final upsampling operation and convolutional layers to generate prediction results with the same length as the input time-series data. The output at each time step corresponds to the land cover category label at that moment. During model training, a multi-scale loss function is used to calculate the difference between the predicted labels and the ground truth labels to assist in optimizing the model parameters.

[0111] Furthermore, regarding the loss function, this embodiment uses a multi-scale loss function fusion method. The cross-entropy loss function can evaluate the proximity of the classification result of each pixel to the ground truth label. However, in the mangrove change detection task, most time series in the time-series remote sensing data are usually unchanged, with only a small portion being changed. To make the model pay more attention to changed time series, this study introduces Dice loss on top of cross-entropy loss. An additional label is added to the time series to indicate whether the time series has changed (0 indicates no change, 1 indicates a change). This loss function uses the 0 / 1 label as a supervision signal, focusing on samples that are incorrectly detected during training. The total loss value of the model is defined as the sum of the two loss functions, as shown in the following formula:

[0112] ;

[0113]

[0114]

[0115] in, The result is the cross-entropy loss calculation. This is the result of the Dice loss calculation. and For the first i Label values ​​and predicted values ​​of land cover types at each time series point N The total number of time series points. and For the first i The label values ​​and predicted values ​​for whether the land cover type has changed over time. B This represents the number of samples used to update the model parameters during a single training session.

[0116] Although time-series semantic segmentation models excel in pixel-level remote sensing temporal change detection, their predictions often suffer from temporal jitter and spatial noise due to data noise and inherent uncertainties in remote sensing imagery. Specifically, this manifests as frequent changes in land cover type within a short period and spatial salt-and-pepper noise. These issues not only reduce the actual reliability of land cover change detection results but also hinder the accurate understanding of the overall change trend in the study area. Therefore, this embodiment introduces a temporal and spatial scale consistency correction strategy to perform secondary optimization on the model's output change detection results, thereby improving the spatial consistency and temporal continuity of the final results.

[0117] The step of performing spatiotemporal consistency correction on the candidate detection results to obtain the detection results for output includes:

[0118] S701: For the overall time series of a pixel in the candidate detection result, use a first sliding window to modify the center point type within the window area to the target type. The target type is the land cover type that appears most frequently in the current window area, and the frequency of the land cover type meets a preset value range.

[0119] S702: Merge sequences whose land cover type changes last for less than a preset time range into the closest sequence with the longest duration of stable land cover type;

[0120] S703: Extract the mangrove distribution results within a time phase from the candidate detection results, and perform mode filtering using a second sliding window.

[0121] Specifically, for the entire time series of a pixel, time consistency correction is used to correct misclassification and eliminate false changes. The time consistency correction consists of two steps: (1) Misclassification correction: For example, using a sliding window of length 5 (or other values), the type of the center point within the window area is modified to the land cover type that appears most frequently in the current window and has an occurrence count greater than or equal to 3 (or other values). If the condition is not met, no correction is performed. (2) False change elimination: A threshold of 4 (or other values) is defined. For time series where the duration of land cover type change is less than this threshold, they are merged into the sequence that is closest to the one with the longest duration of land cover type change.

[0122] For mangrove distribution extraction results at a given time phase, spatial consistency correction is used to remove salt-and-pepper noise, thereby improving the overall spatial consistency of land cover change detection results. Specifically, a sliding window of size 5×5 (or other values) can be used to perform mode filtering on candidate detection results.

[0123] Based on the above embodiments, it can be seen that this application solves the problem of unstable optical images by introducing an optical and radar fusion mechanism. At the same time, by combining supervised classification and exponential thresholding methods, it achieves synchronous classification and change detection of land cover at each time point by identifying the semantic boundaries between different land cover types in the time series. This not only improves efficiency but also greatly improves the accuracy of detection.

[0124] like Figure 4 As shown, another embodiment of the present invention also provides a mangrove dense temporal detection device, comprising:

[0125] The acquisition module is used to acquire radar images and optical remote sensing images of the area to be measured, wherein the area to be measured includes mangroves as the main detection object;

[0126] The first calling module is used to call a pre-built dual-branch feature extraction network to independently extract features from the radar image and the optical remote sensing image, thereby obtaining radar features and optical remote sensing features. The dual-branch feature extraction network has two independent feature extraction branches, and different feature extraction methods are used in different feature extraction branches.

[0127] The first fusion module is used to fuse the radar features and the optical remote sensing features to obtain the first fused features;

[0128] The calculation module is used to calculate the SSMI features based on the radar and optical image reflectivity data;

[0129] The second fusion module is used to fuse the first fusion feature with the SSMI feature to obtain the second fusion feature;

[0130] The second invocation module is used to invoke the pre-built target network to process the second fusion feature and obtain candidate detection results. The target network is used to simultaneously identify the change position and change type of the target object.

[0131] The correction module is used to perform spatiotemporal consistency correction on the candidate detection results to obtain the detection results for output.

[0132] In one embodiment, the dual-branch feature extraction network includes a first feature extraction branch and a second feature extraction branch;

[0133] Constructing the dual-branch feature extraction network includes:

[0134] The first feature extraction branch for extracting the radar features is prepared based on a network model with long-term processing capability, wherein the network model includes the Mamba model.

[0135] A second feature extraction branch for extracting the optical remote sensing features is prepared based on a lightweight convolution model, wherein the lightweight convolution model includes a one-dimensional convolution model.

[0136] In one embodiment, the feature fusion of the radar features and optical remote sensing features to obtain the first fused feature includes:

[0137] The radar features and optical remote sensing features are fused together using a target mechanism to obtain the first fused feature. The target mechanism includes an attention mechanism and a gating fusion mechanism.

[0138] In one embodiment, the feature fusion of the radar features and optical remote sensing features using the target-combining mechanism includes:

[0139] High-dimensional features are extracted from the radar features and optical remote sensing features respectively to obtain feature vectors X and Y;

[0140] The attention weight matrix W of the feature vectors X and Y is calculated using a scaled dot product attention mechanism;

[0141] The attention weight matrix W is multiplied by the feature vector Y, and the result of the multiplication is superimposed on the feature vector X to obtain the first fused feature.

[0142] In one embodiment, fusing the first fusion feature with the SSMI feature to obtain the second fusion feature includes:

[0143] Align the first fusion feature with the SSMI feature;

[0144] The SSMI features after feature alignment are converted into a target dimension through a mapping process, and the target dimension is the same as the dimension of the first fused feature after feature alignment.

[0145] The SSMI features with the same dimension are concatenated with the first fusion feature;

[0146] Determine the adaptive fusion weights for each time step;

[0147] The adaptive fusion weights are separated to obtain a first weight corresponding to the SSMI feature and a second weight corresponding to the first fusion feature;

[0148] The first weight, the second weight, and the spliced ​​SSMI feature of each time step are dynamically fused with the first fusion feature to obtain the second fusion feature.

[0149] In one embodiment, the target network includes at least one of a temporal semantic segmentation network, a dual-branch contrast network, and a temporal change detection network based on an attention mechanism.

[0150] In one embodiment, the target network is a temporal semantic segmentation network;

[0151] The process of calling the pre-built target network to process the second fused feature and obtain candidate detection results includes:

[0152] The second fused feature is downsampled by multi-order one-dimensional convolution operations, and the first feature map is generated accordingly.

[0153] Gradually increase the receptive field and compress the feature dimension, and for the feature map output by downsampling at a specified stage, use one-dimensional transpose convolution with different sampling multiples to perform scale recovery and feature preservation.

[0154] The saved multi-scale features are spliced ​​together using a skip connection method, and then upsampled and convolutionally processed to generate the candidate monitoring results.

[0155] In one embodiment, the loss function in the temporal semantic segmentation network is generated by fusing the cross-entropy loss function and the Dice loss, and the loss value of the temporal semantic segmentation network is determined by the sum of the cross-entropy loss value and the Dice loss value.

[0156] In one embodiment, performing spatiotemporal consistency correction on the candidate detection results to obtain the detection results for output includes:

[0157] For the overall time series of a pixel in the candidate detection result, the center point type within the window area is modified to the target type using a first sliding window. The target type is the land cover type that appears most frequently in the current window area, and the frequency of the land cover type meets a preset value range.

[0158] Sequences whose land cover type changes last for less than a preset time range are merged into the nearest sequence with the longest duration of stable land cover type.

[0159] The distribution of mangroves within a single time phase in the candidate detection results is extracted and then filtered using a second sliding window.

[0160] Another embodiment of the present invention also provides an electronic device, comprising:

[0161] One or more processors;

[0162] Memory, configured to store one or more programs;

[0163] When the one or more programs are executed by the one or more processors, the one or more processors implement the mangrove dense temporal detection method as described above.

[0164] Furthermore, one embodiment of the present invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the mangrove dense temporal detection method described above. It should be understood that the various solutions in this embodiment have the corresponding technical effects in the above-described method embodiments, and will not be repeated here.

[0165] Furthermore, embodiments of the present invention also provide a computer program product, which is tangibly stored on a computer-readable medium and includes computer-readable instructions that, when executed, cause at least one processor to perform a mangrove dense temporal detection method as described in the embodiments above.

[0166] It should be noted that the computer storage medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program configured for use by or in connection with an instruction execution system, system, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, antenna, optical fiber, RF, etc., or any suitable combination thereof.

[0167] Furthermore, those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0168] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0169] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0170] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of protection of this application is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of this application as described above, which are not provided in detail for the sake of brevity.

Claims

1. A method for temporal detection of dense mangrove forests, characterized in that, include: The radar and optical remote sensing images of the area to be measured are obtained, and the area to be measured includes mangroves, which are the main objects of detection. The pre-built dual-branch feature extraction network is invoked to extract features from the radar image and the optical remote sensing image independently, respectively, to obtain radar features and optical remote sensing features. The dual-branch feature extraction network has two independent feature extraction branches, and different feature extraction methods are used in different feature extraction branches. The radar features and optical remote sensing features are fused to obtain the first fused feature; The SSMI characteristics are calculated based on the radar and optical remote sensing image reflectance data. The first fusion feature and the SSMI feature are fused to obtain the second fusion feature; The pre-built target network is invoked to process the second fusion feature to obtain candidate detection results. The target network is used to simultaneously identify the change location and change type of the target object. Spatiotemporal consistency correction is performed on the candidate detection results to obtain the detection results for output; The target network is a temporal semantic segmentation network; The process of calling the pre-built target network to process the second fused feature and obtain candidate detection results includes: The second fused feature is downsampled by multi-order one-dimensional convolution operations, and the first feature map is generated accordingly. Gradually increase the receptive field and compress the feature dimension, and for the feature map output by downsampling at a specified stage, use one-dimensional transpose convolution with different sampling multiples to perform scale recovery and feature preservation. The stored multi-scale features are concatenated using a skip connection method, and then upsampled and convolutionally processed to generate the candidate detection results. The loss function in the temporal semantic segmentation network is generated by fusing the cross-entropy loss function and the Dice loss. The loss value of the temporal semantic segmentation network is determined by the sum of the cross-entropy loss value and the Dice loss value. ； ； in, The result is the cross-entropy loss calculation. The result of the Dice loss calculation. and For the first i Label values ​​and predicted values ​​of land cover types at each time series point N The total number of time series points. and For the first i The label values ​​and predicted values ​​for whether the land cover type has changed over time. B This represents the number of samples used to update the model parameters during a single training session.

2. The method for detecting dense mangrove forests in a temporal sequence according to claim 1, characterized in that, The dual-branch feature extraction network includes a first feature extraction branch and a second feature extraction branch; Constructing the dual-branch feature extraction network includes: The first feature extraction branch for extracting the radar features is prepared based on a network model with long-term processing capability, wherein the network model includes the Mamba model. A second feature extraction branch for extracting the optical remote sensing features is prepared based on a lightweight convolution model, wherein the lightweight convolution model includes a one-dimensional convolution model.

3. The method for detecting dense mangrove forests over time according to claim 1, characterized in that, The feature fusion of the radar features and optical remote sensing features to obtain the first fused feature includes: The radar features and optical remote sensing features are fused together using a target mechanism to obtain the first fused feature. The target mechanism includes an attention mechanism and a gating fusion mechanism.

4. The method for detecting dense mangrove forests in a time sequence according to claim 3, characterized in that, The feature fusion of radar features and optical remote sensing features using the target-combining mechanism includes: High-dimensional features are extracted from the radar features and optical remote sensing features respectively to obtain feature vectors X and Y; The attention weight matrix W of the feature vectors X and Y is calculated using a scaled dot product attention mechanism; The attention weight matrix W is multiplied by the feature vector Y, and the result of the multiplication is superimposed on the feature vector X to obtain the first fused feature.

5. The method for detecting dense mangrove forests in a time sequence according to claim 1, characterized in that, The step of fusing the first fusion feature with the SSMI feature to obtain the second fusion feature includes: Align the first fusion feature with the SSMI feature; The SSMI features after feature alignment are converted into a target dimension through a mapping process, and the target dimension is the same as the dimension of the first fused feature after feature alignment. The SSMI features with the same dimension are concatenated with the first fusion feature; Determine the adaptive fusion weights for each time step; The adaptive fusion weights are separated to obtain a first weight corresponding to the SSMI feature and a second weight corresponding to the first fusion feature; The first weight, the second weight, and the spliced ​​SSMI feature of each time step are dynamically fused with the first fusion feature to obtain the second fusion feature.

6. The method for detecting dense mangrove forests in a time sequence according to claim 1, characterized in that, The target network includes at least one of the following: a temporal semantic segmentation network, a dual-branch contrast network, and a temporal change detection network based on an attention mechanism.

7. The method for detecting dense mangrove forests in a time sequence according to claim 1, characterized in that, The step of performing spatiotemporal consistency correction on the candidate detection results to obtain the detection results for output includes: For the overall time series of a pixel in the candidate detection result, the center point type within the window area is modified to the target type using a first sliding window. The target type is the land cover type that appears most frequently in the current window area, and the frequency of the land cover type meets a preset value range. Sequences whose land cover type changes last for less than a preset time range are merged into the nearest sequence with the longest duration of stable land cover type. The distribution of mangroves within a single time phase in the candidate detection results is extracted and then filtered using a second sliding window.

8. A mangrove dense temporal detection device, characterized in that, include: The acquisition module is used to acquire radar images and optical remote sensing images of the area to be measured, wherein the area to be measured includes mangroves as the main detection object; The first calling module is used to call a pre-built dual-branch feature extraction network to independently extract features from the radar image and the optical remote sensing image, thereby obtaining radar features and optical remote sensing features. The dual-branch feature extraction network has two independent feature extraction branches, and different feature extraction methods are used in different feature extraction branches. The first fusion module is used to fuse the radar features and the optical remote sensing features to obtain the first fused features; The calculation module is used to calculate SSMI features based on the reflectance values ​​of the radar image and the optical remote sensing image; The second fusion module is used to fuse the first fusion feature with the SSMI feature to obtain the second fusion feature; The second invocation module is used to invoke the pre-built target network to process the second fusion feature and obtain candidate detection results. The target network is used to simultaneously identify the change position and change type of the target object. The correction module is used to perform spatiotemporal consistency correction on the candidate detection results to obtain the detection results for output; The target network is a temporal semantic segmentation network; The process of calling the pre-built target network to process the second fused feature and obtain candidate detection results includes: The second fused feature is downsampled by multi-order one-dimensional convolution operations, and the first feature map is generated accordingly. Gradually increase the receptive field and compress the feature dimension, and for the feature map output by downsampling at a specified stage, use one-dimensional transpose convolution with different sampling multiples to perform scale recovery and feature preservation. The stored multi-scale features are concatenated using a skip connection method, and then upsampled and convolutionally processed to generate the candidate detection results. The loss function in the temporal semantic segmentation network is generated by fusing the cross-entropy loss function and the Dice loss. The loss value of the temporal semantic segmentation network is determined by the sum of the cross-entropy loss value and the Dice loss value. ； ； in, The result is the cross-entropy loss calculation. The result of the Dice loss calculation. and For the first i Label values ​​and predicted values ​​of land cover types at each time series point N The total number of time series points. and For the first i The label values ​​and predicted values ​​for whether the land cover type has changed over time. B This represents the number of samples used to update the model parameters during a single training session.

Images