Remote sensing data time sequence seamless reconstruction method and device with sar data actively integrated

Abstract

The application discloses a kind of SAR data active integration's remote sensing data timing seamless reconstruction method and device, belong to remote sensing image processing and space-time information analysis technical field.Sentinel-2 waveband and Sentinel-1 dual polarization data are screened and time-space alignment, radiation normalization and cloud mask construction are completed;Parallel operation dual-branch deep network optical branch extracts multi-scale space-time characteristics, SAR branch uses deformable convolution+texture self-attention to mine multi-scale directional texture;Then optical characteristics are used as Query, in cloud mask marked area active search SAR Key-Value complementary information, realize deep fusion by cross-modal multi-head attention;Fusion result is jointly corrected by relative time distance coding, time attenuation factor and cloud perception mask, and long-time sequence attention drift is inhibited;Final level fusion multi-level feature and adaptive weighting are outputted, and full-time seamless image is outputted.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing and spatiotemporal information analysis technology, specifically a method and apparatus for seamless temporal reconstruction of remote sensing data by actively integrating SAR data. Background Technology

[0002] With the urgent need for global environmental change monitoring and precision agriculture development, continuous, high-quality remote sensing time-series data has become a crucial foundation for dynamic monitoring of the Earth's surface. In agriculture, crop growth monitoring, yield estimation, and pest and disease early warning all require intensive time-series observation data. Specifically, crop phenological stage identification requires an observation frequency of 5-10 days to accurately capture key growth stage transitions; crop growth assessment requires time-series data covering the entire growing season, and missing data for any critical period can lead to yield prediction deviations of 15-20%; precision fertilization and irrigation decisions require real-time, continuous vegetation index changes, and data interruptions directly impact agricultural management efficiency. In ecological environment assessment, applications such as vegetation phenological change detection, land use change monitoring, and forest fire monitoring also rely on complete time-series data. Furthermore, in natural disaster emergency response, dynamic monitoring and loss assessment of disasters such as floods, droughts, and landslides place extremely high demands on data timeliness and continuity.

[0003] However, optical remote sensing imagery is severely affected by cloud and rainy weather. According to global cloud cover statistics, tropical rainforest regions have an average annual cloud cover exceeding 80%, while the Southeast Asian monsoon region can reach over 90% during the rainy season. Even in temperate regions, the average annual cloud cover is between 40% and 60%. Taking southern China as an example, it is not uncommon for effective optical imagery to be unavailable for a full month during the plum rain season (June to August). This lack of data severely restricts the effectiveness of remote sensing technology in practical applications, becoming a critical technical bottleneck that urgently needs to be addressed.

[0004] Optical and SAR data exhibit significant complementarity in terms of physical mechanisms, information content, and spatiotemporal characteristics. In the temporal dimension, optical data is constrained by weather, resulting in discontinuous time series, while SAR provides stable temporal sampling. In the information dimension, optical data offers rich spectral information, while SAR provides unique structural and moisture information. In the application dimension, combining the two can achieve a synergistic effect greater than the sum of its parts. Studies have shown that fusing Sentinel-1 and Sentinel-2 data for crop classification can improve accuracy by 15-25%; in flood monitoring, SAR can penetrate clouds to monitor the extent of water bodies in real time, while optical data can assess water quality and suspended solids concentration.

[0005] Early temporal reconstruction primarily relied on mathematical interpolation methods. Linear interpolation assumes that missing values ​​change linearly over time, offering simplicity but neglecting the nonlinear variations of the land surface. Cubic spline interpolation uses piecewise polynomials to ensure curve smoothness but is prone to oscillations. Kriging interpolation considers spatial autocorrelation but has high computational complexity and struggles with large-scale data. A common limitation of these methods is their inability to handle long-term continuous missing data, relying solely on the spatiotemporal correlation of a single data source, leading to a sharp decline in interpolation accuracy with increasing missing duration. Experiments show that interpolation errors can reach 30-40% when continuous missing data exceeds 30 days. STARFM pioneered a spatiotemporal fusion framework, fusing the high temporal resolution of MODIS and the high spatial resolution of Landsat through a weighted function. This model is based on three assumptions: a linear relationship between high- and low-resolution images, stable ground reflectance over short periods, and similar temporal variation patterns among neighboring pixels. STARFM introduces the concept of a transformation coefficient, better handling heterogeneous landscapes; FSDAF employs unmixing and super-resolution reconstruction techniques to enhance edge preservation capabilities. However, these models have fundamental limitations: they rely heavily on at least one pair of concurrent high- and low-resolution images and cannot work under complete cloud cover; they assume slow surface changes and are difficult to capture sudden events; the computation process involves a large amount of neighborhood search and weight calculation, which is inefficient, and it usually takes 2-3 hours to process a single Landsat image.

[0006] Breakthroughs in deep learning technology have brought revolutionary progress to temporal reconstruction. CNN architectures automatically extract spatial features through convolutional operations, avoiding tedious feature engineering. SpaceNet uses an encoder-decoder structure, preserving multi-scale information through skip connections; DSeN2-CR introduces residual learning, focusing on learning the residuals between cloud areas and clear skies, resulting in more stable training; CTGAN uses generative adversarial networks to improve reconstruction quality through adversarial training, but training is unstable, and modality collapse is a prominent issue. Other algorithms have implemented end-to-end fusion learning. FusionNet uses dual-stream CNNs to process different modalities separately, fusing them through feature concatenation; MDFNet designs multi-scale dense connections, fully utilizing features at different levels; CrossNet introduces cross connections to achieve information exchange between modalities. These networks have achieved significant improvements in fusion performance, but their architectural designs are still relatively simple and do not fully consider the differences in modal characteristics. The introduction of attention mechanisms has brought new breakthroughs. Channel attention learns the importance of different feature channels; spatial attention focuses on key spatial regions; self-attention models long-distance dependencies; and cross attention enables selective information transfer between modalities. However, how to design an attention mechanism suitable for heterogeneous data, especially optical-SAR fusion with huge differences in physical properties, remains an open question.

[0007] However, existing technologies still have the following problems:

[0008] 1. Inefficient fusion mechanisms of multispectral and SAR data: Existing methods typically use simple stitching of optical imagery and SAR data as input. For example, after 11-band Sentinel-2 multispectral data (retaining B2-B8A, B8, and B11-B12), dual-band SAR (VV / VH) channels are directly appended to form a 13-channel tensor input deep learning model. This operation has significant drawbacks:

[0009] Conflicting physical properties: Optical bands reflect surface reflectivity, while SAR bands characterize the dielectric properties and roughness of ground objects. These two bands differ inherently in their physical meaning and numerical distribution. Direct stitching exacerbates the heterogeneity of the feature space, making it difficult for the model to interpret the coupling relationship between scattering mechanisms and spectral responses.

[0010] Feature redundancy and interference: Redundant stitching not only easily weakens the penetration advantage of SAR, but also easily introduces noise, such as SAR spot noise contaminating optical features, leading to a reduction in the purity of fused information.

[0011] Loss of structural information: SAR's geometric structural features, such as the linear contour of the dam body and the specular reflection of the water body, lack cross-modal alignment mechanisms with optical textures, and simple channel superposition cannot achieve synergistic enhancement of structural semantics.

[0012] Therefore, simple channel splicing cannot take full advantage of the characteristics of SAR data. At the same time, since optical bands contain more information, they are very likely to have too little weight in the final model prediction stage, and thus have little impact on the results.

[0013] 2. Insufficient temporal continuity in multi-period data reconstruction: Current temporal reconstruction of multi-cloud areas largely relies on single-data interpolation or fusion prediction of multiple periods of optical data to obtain high-quality images from a single period. Single-source data temporal interpolation model

[0014] Representative methods include: Spatiotemporal Adaptive Reflectance Fusion Model (STARFM), Enhanced Spatiotemporal Adaptive Reflectance Fusion (ESTARFM), and Sparse Representation Time Series Model (STRUM). These models rely on high-quality reference images and use spatiotemporal weighting functions or dictionary learning mechanisms to interpolate and reconstruct missing pixels. However, their nature is limited by the dependence on single-source data, which has problems including (1) cumulative spectral drift: in long-term continuous cloud-covered areas, multiple interpolations will transmit and amplify spectral errors, resulting in nonlinear radiation distortion in the reconstructed sequence; (2) delayed response of abrupt land cover changes: for sudden surface changes affected by precipitation, irrigation, etc., it is difficult to capture short-term hydrological abrupt changes due to the lack of contemporaneous auxiliary data constraints.

[0015] Common approaches to multi-source data substitution fusion strategies include: direct spatiotemporal replacement: directly filling missing periods with optical images from adjacent time periods, but easily ignoring the evolution of land cover and phenology; multi-period fusion method: applying methods such as temporal averaging filtering to multi-period images, reconstructing one period using multi-period data, weakening local noise but blurring transient surface processes, requiring high data quality from multi-period data. Summary of the Invention

[0016] To address the problems existing in the prior art, this invention provides a method and apparatus for seamless temporal reconstruction of remote sensing data by actively integrating SAR data, effectively solving the problems of inefficient fusion of multispectral and SAR data and insufficient temporal continuity in multi-period data reconstruction. The specific technical solution is as follows:

[0017] A method for seamless temporal reconstruction of remote sensing data by actively incorporating SAR data includes the following steps:

[0018] Step 1: Acquire and filter multi-source remote sensing observation data of the target area, and extract effective band data of Sentinel-2 optical image and Sentinel-1 dual-polarization SAR data;

[0019] Step 2: Spatiotemporally align the effective band data of the Sentinel-2 optical image with the SAR data, spatially resample to a uniform resolution, and perform radiometric normalization.

[0020] Step 3: Generate cloud masks using effective band data from Sentinel-2 optical images, and construct a standardized time-series sample set containing cloud masks;

[0021] Step 4: Construct a dual-branch network, in which the optical branch extracts multi-scale spatial features using a U-Net encoder-decoder structure and captures temporal dependencies to obtain optical features using a Transformer temporal encoder, and the SAR branch extracts multi-scale directional texture features to obtain SAR features through deformable convolution and texture self-attention.

[0022] Step 5: Input the optical and SAR features obtained in Step 4 into the unified embedding space. First, use the optical features as the query and the SAR features as the key-value pair. Then, actively retrieve complementary information in the cloud mask marking area through cross-modal multi-head attention to achieve deep fusion. The fusion result is then used as the input feature for the next step.

[0023] Step 6: Based on the fused features from Step 5, an attention shift correction mechanism is introduced. Combining relative temporal distance encoding, temporal decay factor, and cloud-aware mask, the long-term attention weights are dynamically corrected. The corrected weights are then directly used to reweight and fuse the features.

[0024] Step 7: Using the corrected weighted fusion features obtained in Step 6, after hierarchical fusion and adaptive weight integration, the sequence-to-sequence output complete time-series reconstruction result is obtained.

[0025] A device for seamless temporal reconstruction of remote sensing data by actively incorporating SAR data includes the following modules:

[0026] The data extraction module acquires and filters multi-source remote sensing observation data of the target area, and extracts effective band data of Sentinel-2 optical image and Sentinel-1 dual-polarization SAR data.

[0027] The data processing module performs spatiotemporal alignment, spatial resampling to a uniform resolution, and radiometric normalization on the effective band data of the Sentinel-2 optical image and the SAR data.

[0028] The sample set generation module uses Sentinel-2 optical image effective band data to generate cloud masks and constructs a standardized time-series sample set containing cloud masks.

[0029] The mesh construction module constructs a dual-branch network. The optical branch extracts multi-scale spatial features using a U-Net encoder-decoder structure and captures temporal dependencies to obtain optical features using a Transformer temporal encoder. The SAR branch extracts multi-scale directional texture features to obtain SAR features through deformable convolution and texture self-attention.

[0030] The fusion module uses optical and SAR features as inputs into a unified embedding space. It first uses optical features as queries and SAR features as key-value pairs. Through cross-modal multi-head attention, it actively retrieves complementary information in the cloud mask marking region to achieve deep fusion. The fusion result is then used as the input feature for the next step.

[0031] The correction module introduces an attention shift correction mechanism based on the fused features. It combines relative temporal distance encoding, temporal decay factor and cloud perception mask to dynamically correct long-term attention weights. The corrected weights are directly used to reweight and fuse features.

[0032] The reconstruction module utilizes corrected weighted fusion features, and after hierarchical fusion and adaptive weight integration, outputs complete time-series reconstruction results from sequence to sequence.

[0033] An electronic device includes: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method.

[0034] A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to implement the method described thereon.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] (1) This invention addresses the attention drift problem in long-term time series reconstruction by introducing an attention shift correction mechanism. Relative temporal distance encoding (RPE) captures the relative relationships between time steps, a time decay factor balances the contributions of different time steps, and a cloud-aware attention mask adaptively adjusts weight allocation based on cloud coverage. This design effectively solves the problem of excessive attention focusing on recent time steps and ignoring earlier information, ensuring the effective utilization of long-term time series data.

[0037] (2) This invention uses a learnable deep texture network to replace traditional handcrafted features such as GLCM, and automatically learns the texture orientation and pattern of SAR images through deformable convolution. A texture self-attention module is designed to capture global texture relationships, and the original SAR data and deep texture features are fused through a multi-source feature integration mechanism. This end-to-end learning approach can not only adaptively extract texture information related to the reconstruction task, but also preserve the original scattering characteristics of SAR data, giving full play to the unique advantages of SAR under cloud cover conditions.

[0038] (3) This invention abandons the traditional simple channel stitching method and proposes a cross-modal deep fusion strategy based on an attention mechanism. By projecting heterogeneous features, optical and SAR features are mapped to a unified embedding space, solving the incompatibility problem caused by the difference in physical characteristics of the two types of data. Optical features are used as queries to actively retrieve complementary information in SAR features, and multi-head attention is used to capture complex intermodal relationships from multiple angles, truly realizing the active integration of SAR data rather than passive superposition.

[0039] (4) This application adopts a dual-branch architecture to process optical and SAR data separately, and achieves deep fusion through a cross-modal attention mechanism. Optical features serve as complementary information in the actively retrieved SAR features, while the texture and land cover information extracted by the SAR branch can effectively assist in the reconstruction of optical images. This design allows SAR data to truly leverage its unique advantages of penetrating clouds and providing structural information, achieving active integration of SAR rather than passive overlay.

[0040] (5) This application proposes a sequence-to-sequence processing method, outputting the reconstruction result of the entire time series at once, rather than interpolating period by period. An attention shift correction mechanism is introduced, which captures the relative relationship of time steps through relative time distance encoding, balances the contributions of near and far time steps through a time decay factor, and adaptively adjusts the weights through a cloud-aware mask. This effectively solves the problem of excessive attention focusing on recent time steps, ensures the effective use of long-term time series data, and avoids error accumulation and spectral drift. Attached Figure Description

[0041] Figure 1 This is a technical flowchart of the method according to an embodiment of the present invention;

[0042] Figure 2 This is a network structure framework diagram of the dual-input fusion temporal declouding algorithm provided in an embodiment of the present invention;

[0043] Figure 3 This is a structural framework diagram of the SAR data texture feature extraction module provided in an embodiment of the present invention;

[0044] Figure 4 This is a structural framework diagram of the cross-modal multi-head attention module provided in an embodiment of the present invention;

[0045] Figure 5 This is a flowchart of the dual-branch strategy provided in an embodiment of the present invention;

[0046] Figure 6 This is a visual comparison chart of the input and reconstruction results provided in an embodiment of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other. To achieve the above objectives, this invention adopts the following technical solution.

[0048] To achieve the above objectives, this invention provides a method for seamless temporal reconstruction of remote sensing data by actively incorporating SAR data, specifically including the following steps:

[0049] like Figure 1 As shown, this invention mainly includes two stages: data preprocessing and model processing. The data preprocessing stage aims to construct a standardized multimodal time-series dataset, and the specific steps are as follows:

[0050] Step 1: Acquisition and feature screening of multi-source remote sensing data.

[0051] First, multi-source remote sensing data of the target reconstruction area are acquired. For optical data, 11 bands are extracted from Sentinel-2 data, i.e., B1 and B10 are removed from the original 13 bands, retaining the remaining 11 effective bands. For radar data, dual-polarized Sentinel-1 data (VV and VH polarization) is acquired, utilizing its all-weather imaging capability to supplement the missing optical information. Sentinel-2 data refers to high-resolution multispectral optical remote sensing imagery, covering 13 spectral bands from visible light and near-infrared to shortwave infrared, with a spatial resolution of up to 10 meters. It accurately reflects the spectral reflectance characteristics of surface vegetation, soil, and water bodies. Preferably, Level-2A products with atmospheric correction are used, which physically represent the surface reflectance of the lower atmosphere and are a key data source for land cover classification and surface parameter inversion. However, considering that Sentinel-2 imagery, as passive optical remote sensing data, is highly susceptible to atmospheric conditions such as clouds, fog, and rain, leading to the loss of crucial surface information, this invention also introduces SAR data as a supplement.

[0052] Sentinel-1 data refers to active microwave remote sensing imagery acquired by C-band synthetic aperture radar (SAR). As an active microwave sensor, Sentinel-1 is not limited by lighting conditions or cloud cover, providing all-weather, all-day data acquisition capabilities. This data records the backscattering coefficient of ground features to radar waves, primarily reflecting the dielectric constant, roughness, and geometric structure characteristics of the surface. GRD products in wide-swath interferometric (IW) mode are preferred, and the VV and VH dual-polarization channels are selected to fully utilize their physical properties of penetrating clouds and their rich textural information as key auxiliary features for under-cloud reconstruction in optical imagery. In the data screening and integration stage, for Sentinel-2 data, bands B1 (aerosol observation) and B10 (cirrus cloud observation), which contributed weakly to surface texture reconstruction, were removed from the original 13 bands. The remaining 11 effective bands, along with the Sentinel-1 dual-polarization data, were used to construct a multimodal input source.

[0053] Step 2: Spatiotemporal alignment and radiometric normalization of multi-source heterogeneous data.

[0054] Considering the spatial resolution differences of 10m, 20m, and 60m across different bands in Sentinel-2 imagery, this spatial heterogeneity hinders feature extraction by deep convolutional neural networks. Therefore, this invention resamples all bands of Sentinel-2 data to a 10m resolution to unify the spatial scale: for the 20m resolution bands (B5, B6, B7, B8A, B11, and B12), bilinear interpolation is used for upsampling to increase spatial detail while maintaining image smoothness; while for the 60m resolution band (B9, water vapor), nearest-neighbor interpolation is used to preserve the original physical properties of atmospheric water vapor content to the greatest extent possible, avoiding false texture information introduced by interpolation. Regarding radiometric correction, the quantization value of the L2A product (typically 10000) is verified, and the surface reflectance value is divided by this quantization value to map to the [0, 1] interval. Simultaneously, invalid values ​​(NoData) and saturation values ​​in the image are uniformly set to 0 to eliminate numerical dimension differences and ensure correct physical meaning. For Sentinel-1 data, this invention first trims it to the range of Sentinel-2 data, then performs spatiotemporal alignment with the Sentinel-2 data with the closest pairing times, and establishes a time series index to record multi-temporal observations for each geographical location, thus achieving precise spatiotemporal alignment of the "optical-radar" data. In numerical processing, normalization mapping is performed on both optical and radar data, mapping the original values ​​to a standard interval to eliminate dimensional differences.

[0055] Step 3: High-precision mask generation and temporal sample construction based on scene classification.

[0056] To provide the model with accurate "regions to be repaired" supervision signals, this invention utilizes the Scene Classification Layer (SCL) built into the Sentinel-2 Level-2A product to create cloud masks. The mask is extracted by combining SCL bands with cloud detection, obtaining cloud, cloud shadow, and invalid pixel categories from the SCL, and merging them to generate a binarized cloud mask (0 represents clear sky, 1 represents the region to be reconstructed). Subsequently, a cropping size of 256 is performed. The system performs standardized tile processing using 256 optical data points, SAR data, and corresponding cloud masks. To avoid edge artifacts and support seamless stitching of subsequent large images, a certain overlap area is set during the tile processing, and a unique sample ID is generated for each tile, recording its row and column index and spatial coordinates in the original image. During the data organization phase, the system automatically generates metadata files (JSON format) that record detailed tile prefixes, time steps, and effective pixel ratios. Finally, cloudless frame samples are extracted as ground truth, and then cloud masks from clouded frames are extracted as composite masks and superimposed on the cloudless samples. Through the above process, a time-series dataset is finally established, and the processed image sequences are packaged into standardized time-series samples as input data for subsequent dual-branch deep learning networks.

[0057] The specific implementation method for the model processing stage is as follows:

[0058] Step 4: Construct a dual-input fusion time-series cloud-free network architecture.

[0059] After data preparation, this invention constructs a deep learning network based on an encoder-decoder structure, employing a dual-branch parallel processing strategy to fully utilize multimodal information, including synthesized masked temporal data, temporal information at each time step, unmasked temporal data, and corresponding SAR temporal data. The optical backbone branch, based on the U-Net architecture, extracts multi-scale spatial features through a hierarchical spatial encoder, learns temporal dependencies using a temporal encoder, and restores the spatial resolution of the original image through a hierarchical spatial decoder. In the SAR processing branch, a texture feature extraction module extracts directionality and multi-scale textures, which are then fused with the original features and texture features in the SAR deep feature processing module. Finally, the features from both branches converge in a cross-modal attention fusion module, where deep feature interaction occurs through a cross-modal multi-head attention mechanism (QKV structure), ultimately outputting the reconstructed temporal image.

[0060] Furthermore, the specific steps of step 4 are as follows: Figure 2 As shown:

[0061] Sentinel-2 optical image processing branch:

[0062] ① Input preprocessing module: The format of the multi-temporal Sentinel-2 image sequence with cloud contamination is B×T×H×W, where B=11 (number of bands), T is the time series length, and H×W is the spatial size. The cloud mask (T×H×W) is applied to the input image X to mark the area to be reconstructed. Each band is normalized to ensure numerical stability.

[0063] ② Hierarchical Spatial Encoder: Based on the U-Net architecture, the encoder extracts multi-scale spatial features using a two-layer 3×3 convolution with BatchNorm+ReLU activation. For the i-th layer encoding block, the feature extraction process can be represented as:

[0064] ;

[0065] in, This represents the feature map output by the i-th layer. This represents the feature map of the (i-1)th layer, which is the input data of the current module. For convolution operations, the kernel size is 3. 3. BN stands for batch normalization. is the activation function for the linear rectified unit, and MaxPool is the maximum pooling operation.

[0066] ③ Temporal Attention Module: The Transformer self-attention mechanism is used to process time series relationships. Spatial features are reshaped into a sequence format, T×N×D, where N=H'×W' is the number of spatial tokens and D is the feature dimension. Temporal position encoding adds learnable position embeddings to each time step, and spatial position encoding maintains the relative relationship of spatial positions. A temporal attention mask is generated, and the contribution weight of each time step to the target time is learned to represent the dependency relationship between time steps. Finally, the features of different time steps are weighted and summed according to the attention weights to integrate multi-temporal information.

[0067] In addition, an attention shift correction mechanism was added to handle the attention drift problem in long time series, and relative temporal distance encoding was introduced, where k is the maximum relative distance:

[0068] ;

[0069] in, This represents the relative positional encoding value between positions i and j. Represents the position index of two distinct elements in a sequence or feature map. This represents the difference between two position indices, where k is the truncation threshold (hyperparameter). This is a truncation function.

[0070] Add a time decay factor, where The attenuation rate, For time scale parameters:

[0071] ;

[0072] in, This represents the time decay weight between time i and time j. Represents an exponential function. Represents the timestamps of the i-th and j-th samples. Represents the absolute value of the time difference. Indicates the attenuation rate. Indicates the time scale parameter.

[0073] Finally, the weights are adjusted based on cloud coverage, and the corrected attention calculation is as follows:

[0074] ;

[0075] in, The final corrected attention weight matrix, Let Q be the normalized exponential function, and let Q and K be the query matrix and key matrix, respectively. Let K be the transpose of matrix K. Where is the dimension of the key vector, and RPE is the relative position encoding. The time decay matrix is... For element-wise multiplication of matrices, Cloud mask matrix

[0076] ④ Hierarchical Spatial Decoder: The decoder part based on the U-Net architecture, its function is to progressively restore the spatial resolution. For the i-th layer decoded block:

[0077] ;

[0078] in, This represents the output feature map of the i-th layer of the decoder. This represents the feature map of the (i+1)th layer (deeper layers, lower resolution) of the decoder. This represents the feature map of the i-th layer of the encoder. Indicates an upsampling operation. This indicates a splicing operation. Indicates 3 3. Convolution operation; finally, the reconstruction results of 11 bands are output through the output layer.

[0079] SAR Data Processing Branch: For the SAR data (VV+VH) branch, the model first extracts spatial features from radar images using a variable convolutional layer, leveraging its offset learning capability to adapt to geometric deformations of ground features. Subsequently, it enters a multi-scale texture feature extraction module, capturing rich surface structures and texture details from radar images under different receptive fields. Finally, a texture self-attention module enhances the saliency of the extracted features, suppressing random speckle noise while highlighting key texture information, providing high-quality radar feature input for cross-modal fusion.

[0080] Heterogeneous Feature Projection and Adaptive Fusion Output: In the multimodal fusion stage, the model first maps heterogeneous features from the optical and radar branches to a consistent high-dimensional feature space through heterogeneous feature projection (unified feature embedding space). Next, a cross-modal multi-head attention mechanism is used to achieve deep interaction and information completion between optical temporal features and radar texture features via a QKV structure. The fused features are then dynamically weighted by an adaptive feature fusion module to ensure that the reconstructed image maintains spectral fidelity while possessing clear structural details. Finally, the temporal reconstruction result is calculated and output.

[0081] Furthermore, the detailed steps of the SAR data processing branch are as follows: Figure 3 :

[0082] ①SAR data preprocessing: The input data is VV and VH polarized SAR data. The dB values ​​are converted into linear values ​​through data transformation to facilitate feature extraction and spatial registration to ensure spatial alignment with Sentinel-2 data.

[0083] ② Texture-aware deformable convolutional network: This network consists of an offset learning branch and a modulation learning branch, which automatically learns the texture patterns of SAR images. The texture direction is extracted as follows:

[0084] ;

[0085] in, On the output feature map Pixel value at the location, This represents the total number of sampling points in the convolution kernel, where k represents the index of the sampling point. This represents the weight parameters of the convolution kernel at the k-th position. Indicates the input feature map, This indicates the center coordinates of the current convolution window. This represents a fixed offset of the standard convolutional grid. This represents the learned coordinate offset. This represents the modulation scalar, ranging from [0, 1], and indicates the importance (weight) of the sampling point.

[0086] ③ Multi-scale directional texture extraction: After deformable convolution processing, the system enters the multi-scale directional texture extraction stage, extracting features in three dimensions: Scale 1 (3×3) captures the microstructure of the land surface for detailed texture. Scale 2 (5×5) extracts medium-scale land cover features for medium texture. Scale 3 (7×7) extracts large-scale background texture for coarse texture. All features are passed through a directional filter bank to capture texture patterns in different directions. Finally, multi-scale and multi-directional texture feature fusion is performed to form a comprehensive texture feature map.

[0087] ④ The texture self-attention module is as follows:

[0088] ;

[0089] ;

[0090] in, It is the feature map of the original input SAR image. , , is a learnable linear projection matrix (weight parameters) used to map input features to query, key, and value spaces, respectively, and d is a scaling factor. This is used to prevent the softmax gradient from vanishing due to an excessively large dot product result. It is a normalization function. It is a texture feature map enhanced by a self-attention mechanism, capturing global contextual information.

[0091] The texture self-attention module maps the fused texture features to three vector spaces: Query, Key, and Value. It captures the spatial correlation between textures through attention calculation, thereby enhancing the output of texture features.

[0092] ⑤ SAR Depth Feature Processing Module: This module receives multiple input sources, including raw SAR data and depth texture features, forming a comprehensive feature processing pipeline. A feature integration module performs initial fusion of features from different sources. Raw SAR data undergoes preliminary feature extraction via a lightweight convolutional layer, preserving details of the original scattering information. Depth texture features are spatially weighted using a spatial attention module, emphasizing regions with significant texture.

[0093] ;

[0094] ;

[0095] in, The fusion weights are generated separately for "original features" and "texture features". This is the original feature map. The texture feature map is output by the aforementioned attention module, where GAP represents global average pooling. Here are the weight matrix and bias terms for a fully connected layer (MLP). The Sigmoid activation function restricts the output to the range (0, 1).

[0096] The network first calculates global statistics for each feature, then predicts fusion weights using a small fully connected network. These weights are used to weighted combine different feature maps, generating a unified feature representation as input for subsequent processing. The fused features are as follows:

[0097] ;

[0098] in, This represents the unified feature representation after final fusion. The normalized weights This is a feature transformation function used to align the number of feature channels or perform preliminary processing to facilitate weighted summation.

[0099] Cross-modal attention fusion module such as Figure 4 :

[0100] Instead of simple cat-and-stick (CAT) concatenation, deep fusion of optical and SAR features is achieved through an attention mechanism. First, heterogeneous feature projection maps features from different modalities to a unified embedding space. Then, feature projection and normalization transform the optical and SAR feature projections into representations with similar statistical properties, laying the foundation for subsequent interactions.

[0101] ;

[0102] ;

[0103] in, The features representing optical and SAR modes serve as input for the subsequent attention module. The feature map representing the original input. This represents the weight matrix of the linear projection layer, used to adjust the channel dimensions. This represents the bias term of the linear projection layer. Representation layer normalization standardizes the features of individual samples (subtracting the mean and dividing the variance) to stabilize the training process.

[0104] Secondly, the cross-modal multi-head attention mechanism allows optical features to actively focus on useful information in SAR features. Head 1 handles general feature retrieval, consisting of queries (optical requirements) carrying the needs of optical images, keys providing the index (SAR index), and values ​​storing the actual content (SAR content). Head 2 focuses on texture interaction, extracting details through texture queries, texture keys, and texture values. Heads 3-8 cover different semantic levels, such as geometric structure, edge information, global context, and temporal correlation. After parallel computation of each attention head, their outputs are first concatenated along the feature dimension to form a comprehensive multi-head feature representation. Subsequently, this concatenated result is fused and dimensionality-reduced through a learnable output projection layer (a linear transformation matrix), integrating the multi-head information into a unified context vector. Finally, this context vector is added to and layer-normalized with the original optical features through a residual connection and layer normalization module. This process enables the model to dynamically calibrate attention: in cloud-contaminated areas, the model assigns high weights to SAR information, while reducing its weights in areas with clear optical images, thereby achieving accurate extraction and fusion of effective information.

[0105] ;

[0106] ;

[0107] Where h represents the head index in the multi-head attention mechanism. This represents a query vector derived from an optical image. Represents the keys and values ​​from SAR images. This represents the learnable projection weight matrix corresponding to the h-th head. This represents a cross-modal attention weight map. This represents the model bias term, used to introduce positional encoding. Represents the normalization function. This represents the scaling factor to prevent gradient vanishing.

[0108] Query : To meet the need for carrying target optical images, used for actively retrieving relevant information in SAR. : Characteristic identifiers representing SAR, providing an index for matching queries, values Stores the original SAR information and outputs the actual feature content based on the matching results.

[0109] Next, multiple complementary loss terms are used to ensure the effectiveness of training, and the reconstruction loss guarantees pixel-level accuracy and perceptual quality.

[0110] ;

[0111] in, Indicates the losses incurred during reconstruction. L1 loss (mean absolute error) ensures pixel-level accuracy, making the color values ​​of the generated image close to the true values. This represents perceptual loss, ensuring that visual features (such as texture and edges) are consistent with the real image and conform to human visual perception. This represents the spectral loss, calculated in the frequency domain to ensure image sharpness and high-frequency details. This represents the hyperparameters used to adjust the weights of each item.

[0112] Attention regularization loss prevents the model from over-relying on specific time steps or spatial locations and promotes a uniform distribution of attention;

[0113] ;

[0114] in, This represents the attention regularization loss. This represents the value of the element in the i-th row and j-th column of the attention weight matrix.

[0115] Modality consistency loss ensures that the fused features remain relevant to the target, avoiding the loss or distortion of information between modalities.

[0116] ;

[0117] ;

[0118] in, This represents the modal consistency loss. This represents the function for calculating the correlation matrix. This represents the input optical image feature map. This represents the feature map of the input SAR image. This represents the feature map generated after fusion from the intermediate layers of the network. This represents the characteristics of a cloudless, true optical image. Indicates the total loss. , This represents the hyperparameter used to adjust the weights;

[0119] The enhanced optical features (including SAR supplemental information) perform dynamic weight allocation through an adaptive feature selection mechanism, and use complementary information to output optimized feature representations.

[0120] The following example, taken in Zouping City, Shandong Province, shows an experiment on time-series cloud removal from fused SAR multispectral remote sensing images, which will be further explained with reference to the accompanying drawings.

[0121] The experimental area is located in Zouping City, Shandong Province, China, in the transitional zone between the plains and mountains in north-central Shandong. Geographically, it spans 117°30′E–117°59′E and 36°41′N–37°06′N, covering a total area of ​​approximately 1252 square kilometers. The region has a temperate monsoon climate, characterized by hot and rainy summers and cold, dry winters. Annual precipitation is approximately 600-700 mm, with over 70% occurring from June to September. Influenced by the monsoon climate, the study area experiences frequent cloud and rainy weather, especially during the critical crop growth period (May-September), when cloud cover often reaches 60-80%. This results in discontinuous optical remote sensing imagery, severely limiting the timeliness and accuracy of applications such as agricultural monitoring and urban sprawl analysis.

[0122] Zouping City boasts diverse topography, with low mountains and hills in the south, a sloping plain in the central region, and an alluvial plain of the Yellow River in the north, exhibiting an overall topographical characteristic of higher elevations in the south and lower elevations in the north. Land use types are complex, with various surface cover types, including arable land, construction land, forest land, and water areas, interspersed throughout the region. Agricultural land accounts for over 65% of the total area, primarily used for winter wheat and summer corn, exhibiting significant seasonal variations. The urban built-up area is mainly concentrated in the central plain area. In recent years, rapid urbanization has led to a rapid expansion of construction land and drastic changes in surface cover.

[0123] Under cloudy and rainy conditions, traditional monitoring methods based on single optical remote sensing struggle to acquire continuous and complete surface observation data. While Sentinel-2 boasts a 5-day revisit period and a spatial resolution of 10-20 meters, its effective observation frequency drops significantly during periods of severe cloud cover, with data gaps exceeding 70% in some key time phases. This temporal discontinuity poses serious challenges to applications such as crop growth monitoring, urban change detection, and flood disaster assessment. Particularly during critical agricultural production periods and flood seasons, data gaps caused by cloud cover make it difficult to accurately capture dynamic changes in the land surface, impacting the timeliness and decision support capabilities of remote sensing monitoring.

[0124] SAR data offers all-weather, 24 / 7 observation capabilities, unaffected by cloud cover or rain. Sentinel-1's VV and VH dual-polarization data effectively penetrates cloud layers to capture surface scattering information. Through proactive SAR data integration, the surface texture, structure, and backscattering features contained in SAR images can be utilized to provide crucial supplementary information for optical image reconstruction of cloud-covered areas, enabling seamless reconstruction of dense temporal sequences and ensuring the continuity and integrity of surface monitoring.

[0125] The detailed steps of the SAR data active integration remote sensing data intensive temporal seamless reconstruction technology process are demonstrated as follows: Figure 5 :

[0126] 1. Optical Image Branch - Spatial Encoder: The input data consists of Sentinel-2 temporal data and cloud masks. It is processed by a 4-layer cascaded encoder based on the U-Net architecture. Each layer uses two-layer 3×3 convolutions for feature extraction. The first layer has 11 input channels (corresponding to 11 bands) and 64 output channels. The number of channels in subsequent layers doubles successively (128, 256, 512). After each encoding layer, 2×2 max pooling is applied for downsampling to halve the spatial dimension of the feature map. At the same time, the feature maps of each layer are saved for subsequent skip connections. To enhance the feature representation capability, residual connections are added in the deep encoder (layers 3 and 4) to alleviate the gradient vanishing problem. Spatial pyramid pooling (SPP) is applied at the bottom layer of the encoder to capture multi-scale contextual information and enhance global feature representation.

[0127] 2. Optical Image Branch - Temporal Encoder: The feature map output by the spatial encoder is reshaped into a sequence format. For time step T and spatial dimension H×W, the features are reshaped into (T,N,D) format, where N=H×W is the number of spatial tokens and D=512 is the feature dimension. Learnable temporal and spatial location codes are added to enhance the model's ability to model spatiotemporal relationships. A 4-layer Transformer encoder is constructed, with each layer containing a multi-head self-attention mechanism and a feedforward network. Layer normalization and residual connections are applied to ensure training stability.

[0128] 3. Optical Image Branch - Attention Correction: A relative temporal distance coding matrix is ​​introduced to model the relative positional relationship between different time steps. For time steps i and j, the relative distance coding is defined as:

[0129] ;

[0130] Where k = T / 2 is the maximum relative distance. For learnable relative position embeddings.

[0131] A time decay factor was designed to decay the attention weights at long time steps, thereby reducing attention drift in long time series:

[0132] ;

[0133] in, =0.95 is the attenuation rate. =5 represents the time scale parameter. , This is the actual timestamp.

[0134] The attention weights are adjusted based on cloud mask information to enhance the contribution of clear time relative to cloud coverage time phase. All correction terms are integrated, and the final corrected attention is calculated. The output of the time encoder is fused with the cloud mask information at the feature level. First, the cloud mask is feature-encoded. Then, the channel attention mechanism is applied to adaptively adjust the contribution weights of different time steps. Finally, the time-enhanced feature fusion result is output.

[0135] 4. Optical Image Branch - Spatial Decoder: After passing through 4 layers of U-Net decoder, upsampling is performed using transposed convolutions, with the number of channels in each layer being halved sequentially (512→256→128→64). At this stage, a time-weighted summation mechanism is introduced, and an attention mask generated by the attention module is used to dynamically assign weights to the multi-temporal features of the corresponding layer of the encoder, achieving feature integration in the temporal dimension. Then, skip connections are implemented to concatenate the features of the corresponding layer of the encoder with the upsampled features, preserving spatial detail information. Two-layer 3×3 convolutions are applied for feature refinement, restoring spatial resolution while maintaining feature quality. In the last layer, a 1×1 convolution is applied to generate optical feature outputs in 11 bands.

[0136] 5. SAR Image Branch - Data Preprocessing: First, the input SAR data is registered, that is, the SAR time series data is cropped to the Sentinel-2 data range, and the Sentinel-2 data with the closest time is matched. Then, normalization, time series indexing and other data preprocessing operations are performed. Multi-temporal observations of each geographical location are recorded to complete the spatiotemporal accurate alignment of "optical-radar" data.

[0137] 5. SAR Image Branch - Texture Processing: After passing through deformable convolutional layers, additional convolutional branches learn the position offset and modulation weight of each sampling point. Based on the learned offset, deformable sampling is performed on the input features to enhance the ability to model irregular textures, extract multi-directional texture features, and use directional filtering to capture texture patterns in different directions for feature enhancement.

[0138] 6. SAR Image Branch - Multi-scale Analysis: Multi-granularity texture features are extracted using convolutional kernels of three different scales. Adaptive pooling is applied to the features at each scale, the feature map size is unified, and multi-scale features are adaptively fused through a channel attention mechanism. The feature pyramid network (FPN) is applied to further enhance the multi-scale representation.

[0139] 7. SAR Image Branch - Texture Self-Attention: The texture features are projected into three representation spaces: Query, Key, and Value. The self-attention weights between textures are calculated, the correlation between textures at different spatial locations is modeled, the attention weights are applied to the texture features for weighted aggregation, and the original texture information is preserved through residual connections to generate enhanced SAR feature output.

[0140] Cross-modal fusion stage:

[0141] In the cross-modal fusion stage, the model maps heterogeneous optical and SAR features to a unified alignment space through a feature projection network. The cross-modal attention mechanism uses optical features as the query and SAR features as the key and value, accurately completing the data using SAR information in cloud-covered areas. Subsequently, the adaptive fusion module perceives the global cloud cover distribution through a fully connected network and performs dynamic feature selection: the more severe the cloud cover, the higher the weight of the SAR features.

[0142] The fused features enter a hierarchical fusion stage, refined in three dimensions according to feature depth: shallow fusion focuses on basic texture information such as edges and colors; mid-level fusion focuses on the shape and composition of ground objects; and deep fusion is responsible for interpreting the meaning and function of the overall scene. Features at each level undergo non-linear combination and dimensionality reduction during feature aggregation to eliminate residual biases between modalities. Finally, the model outputs a temporal reconstructed image through a reconstruction network, achieving high-quality, all-weather coverage of Shandong Province from 2022 to 2025, with a 10-meter resolution and a 5-day revisit cycle.

[0143] A comparative experiment was conducted in Zouping City during the rainy season from June to September 2024 to verify the effectiveness of the SAR data active integration strategy. The results are as follows: Figure 6As shown in (a), (b), and (c), the experiment selected Sentinel-2 images from eight time phases, with an average cloud coverage of 65.3%, and cloud coverage exceeding 80% in three of these phases. This method significantly improves the temporal consistency of the reconstructed sequence. The texture feature extraction module of the SAR branch demonstrates excellent detail preservation capabilities, making the reconstructed images more realistic and natural in visual quality. The cross-modal attention mechanism achieves efficient fusion of optical and SAR features. Experiments show that the 8-head attention design can simultaneously capture cross-modal associations at multiple levels, including texture, structure, and semantics. In cloud-covered areas, the average attention weight between optical Query and SARKey-Value reaches 0.73, indicating that the model has successfully learned the ability to retrieve useful information from SAR features.

[0144] Despite the introduction of a dual-branch architecture, computational efficiency remains within acceptable limits thanks to optimized network design. On an NVIDIA RTX 3090 GPU, processing a single 256×256×11 image patch takes only 45ms, a limited increase compared to the single-branch method (28ms). In batch processing mode (batch_size=8), the average processing time for each complete Sentinel-2 image (10980×10980) is 3.2 minutes, meeting the requirements of commercial applications. In terms of memory usage, the dual-branch network has a peak memory consumption of 11.3GB. This method demonstrates good robustness under various weather conditions. Under thin cloud conditions (cloud optical thickness <5), the reconstruction accuracy PSNR reaches 40.6dB; under thick cloud conditions (cloud optical thickness >20), it still maintains a reconstruction quality of 37.5dB. Even under prolonged data loss due to continuous rainy weather (no effective observations for 5 consecutive time phases), this method can still achieve reasonable reconstruction using historical clear time phases and SAR data.

[0145] As described in the above embodiments, those skilled in the art will understand that the present invention provides a method for seamless temporal reconstruction of remote sensing data with active SAR data integration. This method first performs comprehensive data preprocessing, creating a comprehensive dataset containing 11-band Sentinel-2 optical data, cloud mask data, and Sentinel-1 SAR data for algorithm training and feature learning. Next, a dual-branch deep learning network architecture is designed. The first branch uses a Transformer temporal encoder with attention shift correction to process the optical temporal data, effectively solving the long-term attention drift problem. The second branch fully mines the texture features of the SAR data through deformable convolution and texture self-attention mechanisms, achieving effective extraction of all-weather surface information.

[0146] The two branches interact with each other through a cross-modal attention fusion module. Optical features act as a query to actively retrieve complementary information from SAR features. An adaptive fusion mechanism dynamically adjusts the fusion weights based on cloud cover, ensuring optimal feature combinations under different weather conditions. The hierarchical fusion strategy integrates features at multiple levels, from texture and structure to semantics, guaranteeing a balance between detail fidelity and semantic consistency in the reconstruction results.

[0147] Experimental results show that this method achieves seamless reconstruction of dense temporal sequences in cloudy and rainy areas, effectively solving the problem of discontinuous temporal sequences in traditional optical remote sensing under cloud pollution conditions. It provides reliable data support for applications such as agricultural monitoring, urban change detection, and disaster assessment, and improves the timeliness and accuracy of remote sensing monitoring.

Claims

1. A method for seamless temporal reconstruction of remote sensing data by actively incorporating SAR data, characterized in that, Includes the following steps: Step 1: Acquire and filter multi-source remote sensing observation data of the target area, and extract effective band data of Sentinel-2 optical image and Sentinel-1 dual-polarization SAR data; Step 2: Spatiotemporally align the effective band data of the Sentinel-2 optical image with the SAR data, spatially resample to a uniform resolution, and perform radiometric normalization. Step 3: Generate cloud masks using effective band data from Sentinel-2 optical images, and construct a standardized time-series sample set containing cloud masks; Step 4: Construct a dual-branch network, in which the optical branch extracts multi-scale spatial features using a U-Net encoder-decoder structure and captures temporal dependencies to obtain optical features using a Transformer temporal encoder, and the SAR branch extracts multi-scale directional texture features to obtain SAR features through deformable convolution and texture self-attention. Step 5: Input the optical features and SAR features obtained in Step 4 into the unified embedding space. First, use the optical features as the query and the SAR features as the key-value pair. Then, actively retrieve complementary information in the cloud mask marking area through cross-modal multi-head attention to achieve deep fusion. The fusion result is then used as the input feature for the next step. Step 6: Based on the fused features from Step 5, an attention shift correction mechanism is introduced. Combining relative temporal distance encoding, temporal decay factor, and cloud-aware mask, the long-term attention weights are dynamically corrected. The corrected weights are then directly used to reweight and fuse the features. Step 7: Using the corrected weighted fusion features obtained in Step 6, after hierarchical fusion and adaptive weight integration, the sequence-to-sequence output complete time-series reconstruction result is obtained.

2. The method for seamless temporal reconstruction of remote sensing data by actively integrating SAR data according to claim 1, characterized in that, All bands in the Sentinel-2 optical image were resampled to 10m resolution.

3. The method for seamless temporal reconstruction of remote sensing data by actively integrating SAR data according to claim 1, characterized in that, The U-Net encoding feature extraction process is represented as follows: ； in, This represents the feature map output by the i-th layer. This represents the feature map of the (i-1)th layer, which is the input data of the current module. For convolution operations, the kernel size is 3.

3. BN stands for batch normalization. is the activation function for the linear rectified unit, and MaxPool is the maximum pooling operation.

4. The method for seamless temporal reconstruction of remote sensing data by actively integrating SAR data according to claim 1, characterized in that, Decoding includes: For the i-th layer decoding block: ； in, This represents the output feature map of the i-th layer of the decoder. This represents the feature map of the (i+1)th layer (deeper, lower resolution) of the decoder. This represents the feature map of the i-th layer of the encoder. Indicates an upsampling operation. This indicates a splicing operation. Indicates 3 3. Convolution operation.

5. The method for seamless temporal reconstruction of remote sensing data by actively integrating SAR data according to claim 1, characterized in that, The optical branch described in step 4 introduces learnable temporal and spatial position codes into the Transformer time encoder.

6. The method for seamless temporal reconstruction of remote sensing data by actively integrating SAR data according to claim 1, characterized in that, The SAR branch described in step 4 captures micro, intermediate, and background texture patterns through 3×3, 5×5, and 7×7 multi-scale convolutions and directional filter banks, and models global texture correlation through texture self-attention.

7. The method for seamless temporal reconstruction of remote sensing data by actively integrating SAR data according to claim 1, characterized in that, The hierarchical fusion described in step 7 performs nonlinear aggregation based on three levels of features: shallow edge-color, mid-layer shape-landscape, and deep scene-semantic features. Residual connections and layer normalization are used to maintain spectral fidelity.

8. A device for seamless temporal reconstruction of remote sensing data by actively integrating SAR data, characterized in that, Includes the following modules: The data extraction module acquires and filters multi-source remote sensing observation data of the target area, and extracts effective band data of Sentinel-2 optical image and Sentinel-1 dual-polarization SAR data. The data processing module performs spatiotemporal alignment, spatial resampling to a uniform resolution, and radiometric normalization on the optical data and SAR data. The sample set generation module uses Sentinel-2 optical image effective band data to generate cloud masks and constructs a standardized time-series sample set containing cloud masks. The mesh construction module constructs a dual-branch network. The optical branch extracts multi-scale spatial features using a U-Net encoder-decoder structure and captures temporal dependencies to obtain optical features using a Transformer temporal encoder. The SAR branch extracts multi-scale directional texture features to obtain SAR features through deformable convolution and texture self-attention. The fusion module uses optical and SAR features as inputs into a unified embedding space. It first uses optical features as queries and SAR features as key-value pairs. Through cross-modal multi-head attention, it actively retrieves complementary information in the cloud mask marking region to achieve deep fusion. The fusion result is then used as the input feature for the next step. The correction module introduces an attention shift correction mechanism based on the fused features. It combines relative temporal distance encoding, temporal decay factor and cloud perception mask to dynamically correct long-term attention weights. The corrected weights are directly used to reweight and fuse features. The reconstruction module utilizes corrected weighted fusion features, and after hierarchical fusion and adaptive weight integration, outputs complete time-series reconstruction results from sequence to sequence.

9. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed by a processor, cause the processor to perform the method described in any one of claims 1 to 7.

Images