Model optimization method, image processing method and system for remote sensing dynamic monitoring
By constructing a framework for a temporal fusion convolutional model and employing dimension alignment and bidirectional information exchange modules, the problems of low processing efficiency and poor image quality in remote sensing dynamic monitoring are solved, achieving efficient remote sensing image processing and enhancing global feature correlation and noise filtering capabilities.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHANGCHUN INST OF OPTICS FINE MECHANICS & PHYSICS CHINESE ACAD OF SCI
- Filing Date
- 2026-04-15
- Publication Date
- 2026-05-15
AI Technical Summary
Existing temporal fusion convolutional models have low processing efficiency and poor image processing effect in remote sensing dynamic monitoring, making it difficult to meet the requirements of high precision, especially in terms of noise processing.
The framework for constructing a temporal fusion convolutional model includes spatial feature branches and temporal feature branches. Feature alignment and complementary fusion are performed through a dimension alignment module and a bidirectional information exchange module. A gating fusion mechanism is used to adaptively calculate the fusion weights, and inter-stream and intra-stream information exchange mechanisms are embedded in each layer to achieve global information exchange and complementary fusion.
It improves the quality of remote sensing image processing, reduces errors in real-world scenes, enhances the correlation of global features, accurately filters noise and invalid pixel interference, while maintaining processing efficiency and improving the processing effect of remote sensing images.
Smart Images
Figure CN122048700A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of remote sensing data processing, and in particular to model optimization methods, image processing methods, and systems for remote sensing dynamic monitoring. Background Technology
[0002] Remote sensing dynamic monitoring refers to a technical means of quantitatively analyzing and continuously tracking the spatiotemporal changes of surface resources and environmental elements using remote sensing data from different time phases. Its core lies in identifying and quantifying the changing processes of geographical phenomena such as land use, vegetation cover, urban expansion, and arable land changes by comparing remote sensing images of the same area acquired at different times, thereby providing a scientific basis for natural resource management, ecological protection, and policy formulation.
[0003] In remote sensing dynamic monitoring, temporal fusion convolutional models are commonly used for image processing. These models are a core technology for processing temporal data (such as event streams and video frames), aiming to generate effective representations with both spatiotemporal correlation by fusing spatial detail features and temporal dependency features from different branches. This provides crucial support for computer vision tasks such as event stream super-resolution, video reconstruction, and target tracking. Temporal data (especially event streams) possesses unique properties such as asynchronicity, sparsity, and global structure dominance, making it difficult for traditional image / video processing methods to directly adapt. While temporal fusion convolution, with its ability to capture local features through convolution operations, is widely used in feature extraction and fusion of temporal data, it is less effective for image processing (including noise reduction) in remote sensing dynamic monitoring—the processed image still has significant errors compared to the real scene, and the image processing results affect subsequent tasks, failing to meet the requirements of high-precision tasks.
[0004] With the widespread application of remote sensing satellites such as the Gaofen series, Sentinel, and Landsat, multi-temporal remote sensing data has experienced explosive growth. As the scale and complexity of time-series data continue to increase, higher demands are placed on image processing performance, while also maintaining processing efficiency. Therefore, it is necessary to optimize the temporal fusion convolutional model to reduce the error between the processed image and the real scene, while simultaneously ensuring processing efficiency. Summary of the Invention
[0005] Therefore, it is necessary to provide model optimization methods, image processing methods, and systems for remote sensing dynamic monitoring to address the problem of how to improve the quality of remote sensing image processing without reducing processing efficiency.
[0006] To solve the above problems, the present disclosure adopts the following technical solution:
[0007] Secondly, this disclosure provides a model optimization method for remote sensing dynamic monitoring, including the following steps:
[0008] A framework for constructing a temporal fusion convolutional model is provided. The framework includes a spatial feature branch and a temporal feature branch. The spatial feature branch is used to extract the land texture features of the current frame remote sensing image, and the temporal feature branch is used to stitch together the features of the current frame remote sensing image and the features of historical remote sensing images from the same period, and extract the temporal dependency features of land surface changes based on the stitching results.
[0009] A dimension alignment module is constructed in the framework, which is used to align the texture features of land features and the temporal dependency features in dimensions.
[0010] In the framework, a bidirectional information exchange module is constructed to realize global information exchange and complementary fusion of multi-temporal remote sensing data to obtain fused features. The dimension-aligned land texture features and the dimension-aligned temporal dependency features are used as bidirectional inputs to the bidirectional information exchange module. The cross-layer interactive representation is initialized as a zero matrix with the same dimension as the features. The bidirectional information exchange module adopts a gated fusion mechanism to adaptively calculate the fusion weights.
[0011] In the temporal fusion convolutional model, each layer used for feature extraction embeds an inter-flow information exchange mechanism and an intra-flow information exchange mechanism. The inter-flow information exchange mechanism is responsible for cross-branch information exchange between spatial feature branches and temporal feature branches, while the intra-flow information exchange mechanism is responsible for the interaction of features at different levels within a single branch.
[0012] In a preferred embodiment, the spatial feature branch is specifically used to extract the texture features of the ground features in the current frame of remote sensing imagery through successive convolutional residual blocks.
[0013] In a preferred embodiment, the bidirectional information exchange module is specifically used to update cross-layer interaction representations, generate query key values, calculate global relevance, determine complementary features, and calculate gating coefficients.
[0014] In a preferred embodiment, the update formula for the cross-layer interaction representation is:
[0015]
[0016] in, This indicates the layer number in the bidirectional information exchange module. This represents the cross-layer interaction characteristics of the previous layer. This represents a 1×1 convolution processing function. express Activation function This represents the cross-layer interaction characteristics of this layer.
[0017] In a preferred embodiment, generating the query key specifically involves: concatenating the dimension-aligned land feature texture features and the dimension-aligned temporal dependency features with the current cross-layer interaction representation, and generating a query vector corresponding to the positive event through layer normalization and 1×1 convolution. Query vectors corresponding to negative events The input features of positive and negative events are represented by stacked convolutional layers. Convert to the key vector corresponding to the positive event Value vector corresponding to positive events The input features of negative events are represented by stacked convolutional layers. Convert to the key vector corresponding to the negative event value vectors corresponding to negative events .
[0018] In a preferred embodiment, calculating global relevance involves calculating the relevance between the query vector of a negative event and the key vector of a positive event, using the following formula:
[0019]
[0020] in, express and The correlation matrix Represents the normalized exponential function, This indicates the layer number in the bidirectional information exchange module. The bidirectional information exchange module is represented by the first one. The query vector corresponding to negative events in the layer. The bidirectional information exchange module is represented by the first one. The key vector corresponding to positive events in the layer. Indicates the scaling factor;
[0021] The complementary features are determined by weighting and aggregating the value vectors using attention scores. and The gating coefficient is calculated using the Sigmoid function, satisfying the following:
[0022]
[0023]
[0024]
[0025] in, The bidirectional information exchange module is represented by the first one. The features passed from the spatial feature branch obtained from the layer to the temporal feature branch. This represents a 1×1 convolution processing function. The bidirectional information exchange module is represented by the first one. The value vector corresponding to positive events in the layer. The bidirectional information exchange module is represented by the first one. The fused features of negative events in the layer. The bidirectional information exchange module is represented by the first one. The gating factor of the layer, The bidirectional information exchange module is represented by the first one. Features extracted from the temporal feature branches obtained from the layers. express Linear transformation weights, express Linear transformation weights, The sigmoid function is represented by ⊙, which is the element-wise product, and the features are complementary. The bidirectional information exchange module is represented by the first one. The features obtained from the temporal feature branch are passed to the spatial feature branch.
[0026] Secondly, this disclosure provides a model optimization system for remote sensing dynamic monitoring, including:
[0027] The first construction module is used to construct the framework of the temporal fusion convolutional model. The framework includes a spatial feature branch and a temporal feature branch. The spatial feature branch is used to extract the land texture features of the current frame remote sensing image, and the temporal feature branch is used to stitch the features of the current frame remote sensing image with the features of historical remote sensing images from the same period, and extract the temporal dependency features of land surface changes based on the stitching results.
[0028] The second building module is used to build a dimension alignment module in the framework, which is used to align the texture features of the ground features and the temporal dependency features in dimensions.
[0029] The third construction module is used to build a bidirectional information exchange module in the framework to realize global information exchange and complementary fusion of multi-temporal remote sensing data to obtain fused features. The dimension-aligned land texture features and the dimension-aligned temporal dependency features are used as bidirectional inputs of the bidirectional information exchange module. The cross-layer interaction representation is initialized as a zero matrix with the same dimension as the features. The bidirectional information exchange module adopts a gated fusion mechanism to adaptively calculate the fusion weights.
[0030] The embedding module is used to embed inter-flow information exchange mechanism and intra-flow information exchange mechanism in each layer used for feature extraction in the temporal fusion convolutional model. The inter-flow information exchange mechanism is responsible for cross-branch information exchange between spatial feature branches and temporal feature branches, and the intra-flow information exchange mechanism is responsible for the interaction of features at different levels within a single branch.
[0031] Thirdly, this disclosure provides an image processing method for remote sensing dynamic monitoring, including:
[0032] Acquire remote sensing images;
[0033] The model obtained by the model optimization method for remote sensing dynamic monitoring described in the first aspect is used to process the remote sensing image to obtain fused features;
[0034] The deep features obtained by the bidirectional information exchange module are mapped into feature maps using pixel rearrangement technology. The remote sensing image sequence is obtained based on the feature maps and the shooting time of the remote sensing images.
[0035] In a preferred embodiment, the step of using pixel rearrangement technology to map the deep features obtained by the bidirectional information exchange module into a feature map specifically involves: converting the channel dimension of the fused features into a spatial dimension using pixel rearrangement technology to obtain the feature map.
[0036] Fourthly, this disclosure provides an image processing system for remote sensing dynamic monitoring, including:
[0037] The acquisition module is used to acquire remote sensing images;
[0038] The processing module is used to process the remote sensing image using the model obtained by the model optimization method for remote sensing dynamic monitoring described in the first aspect to obtain fused features;
[0039] The mapping module is used to map the deep features obtained by the bidirectional information exchange module into a feature map using pixel rearrangement technology, and to obtain a remote sensing image sequence based on the feature map and the shooting time of the remote sensing image.
[0040] The aforementioned model optimization method, image processing method, and system for remote sensing dynamic monitoring replace the traditional fusion component with a two-way information exchange module. The two types of features to be fused provide complementary information to each other, realizing global information exchange and complementary fusion of multi-temporal remote sensing data to obtain fused features. Specifically, inter-flow information exchange mechanisms and intra-flow information exchange mechanisms are embedded in each layer of feature extraction, introducing cross-layer interactive representations and using a gated fusion mechanism to adaptively calculate fusion weights. This design increases the correlation of global features, making the fused features contain both local details and global consistency. It can accurately filter out interference from noise and invalid pixels. Based on the optimized model, the processed remote sensing images have small errors with the real scene, are closer to the real scene, and can also maintain processing efficiency. Attached Figure Description
[0041] Figure 1 This is a flowchart illustrating a model optimization method for remote sensing dynamic monitoring in one embodiment of the present disclosure.
[0042] Figure 2 This is a flowchart illustrating an image processing method for remote sensing dynamic monitoring in one embodiment of the present disclosure.
[0043] Figure 3 This is a flowchart illustrating the bidirectional information exchange module in one embodiment of the present disclosure;
[0044] Figure 4 This is a schematic diagram of the structure of an image processing system for remote sensing dynamic monitoring in one embodiment of the present disclosure. Detailed Implementation
[0045] The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and preferred embodiments.
[0046] See Figure 1 This disclosure provides a model optimization method for remote sensing dynamic monitoring, wherein the model is a temporal fusion convolutional model, and the method includes:
[0047] A framework for constructing a temporal fusion convolutional model is provided. The framework includes a spatial feature branch and a temporal feature branch. The spatial feature branch is used to extract the land texture features of the current frame remote sensing image, and the temporal feature branch is used to stitch together the features of the current frame remote sensing image and the features of historical remote sensing images from the same period, and extract the temporal dependency features of land surface changes based on the stitching results.
[0048] A dimension alignment module is constructed in the framework, which is used to align the texture features of land features and the temporal dependency features in dimensions.
[0049] In the framework, a bidirectional information exchange module is constructed to realize global information exchange and complementary fusion of multi-temporal remote sensing data to obtain fused features. The dimension-aligned land texture features and the dimension-aligned temporal dependency features are used as bidirectional inputs to the bidirectional information exchange module. The cross-layer interactive representation is initialized as a zero matrix with the same dimension as the features. The bidirectional information exchange module adopts a gated fusion mechanism to adaptively calculate the fusion weights.
[0050] In the temporal fusion convolutional model, each layer used for feature extraction embeds an inter-flow information exchange mechanism and an intra-flow information exchange mechanism. The inter-flow information exchange mechanism is responsible for cross-branch information exchange between spatial feature branches and temporal feature branches, while the intra-flow information exchange mechanism is responsible for the interaction of features at different levels within a single branch.
[0051] After the above steps, the optimized temporal fusion convolutional model is obtained, which is a new temporal fusion convolutional model.
[0052] See Figure 2This disclosure provides an image processing method for remote sensing dynamic monitoring. The image processing method is implemented using a model obtained by the model optimization method. The method includes:
[0053] Acquire remote sensing images;
[0054] The model obtained by the aforementioned model optimization method is used to process the remote sensing image to obtain fused features;
[0055] The deep features obtained by the bidirectional information exchange module are mapped into feature maps using pixel rearrangement technology. The remote sensing image sequence is obtained based on the feature maps and the shooting time of the remote sensing images.
[0056] Preferably, the deep features mapped to the feature map are directly adopted from the final output of the bidirectional information exchange module, that is, the fused features are upsampled and mapped to the feature map using pixel rearrangement technology.
[0057] Understandably, in some embodiments, in addition to upsampling the fused features in the deep features, other deep features are also upsampled depending on the specific circumstances.
[0058] In one specific embodiment, the model optimization method includes:
[0059] Step 1: Construct the basic temporal fusion convolutional model network framework. This framework is specifically designed for multi-temporal remote sensing images and includes a spatial feature branch and a temporal feature branch. The spatial feature branch extracts the land cover texture features of the current frame remote sensing image through 3×3 convolutional residual blocks. The temporal feature branch extracts the temporal dependency features of surface changes by stitching together the features of the current frame and historical remote sensing images from the same period, providing two types of core feature inputs for subsequent fusion. It is specifically optimized for the asynchronous acquisition characteristics of remote sensing images. The land cover texture features are multispectral or hyperspectral land cover texture features.
[0060] In step 1, the spatial feature branch of the temporal fusion convolutional model extracts spatial features from low to high levels through continuous (3×3) convolutional residual blocks (i.e., a structure formed by stacking multiple residual blocks continuously). The low-level spatial features contain pixel-level details (such as edges and textures), while the high-level spatial features cover global structural information. The temporal feature branch captures the dynamic change patterns of adjacent frames based on inter-frame feature splicing.
[0061] The temporal fusion convolutional model extracts spatial and temporal features through convolutional residual blocks and inter-frame concatenation, respectively, providing differentiated and high-quality feature inputs for subsequent bidirectional complementary fusion. The independent extraction of the two types of branch features lays the foundation for subsequent bidirectional complementarity.
[0062] Step 2: A dimension alignment module is constructed within the framework to perform dimension alignment preprocessing on the bi-branch features (texture features and temporal dependency features) of the temporal fusion convolutional model. Channel alignment involves unifying the output channel numbers of the spatial and temporal feature branches using a 1×1 convolutional layer. This dimension alignment module is used to dimensionally align the texture features and temporal dependency features, including unifying the output channel numbers of the spatial and temporal feature branches using a 1×1 convolutional layer. After dimension alignment, both the dimension-aligned texture features and the dimension-aligned temporal dependency features undergo atmospheric correction and radiometric normalization, specific to the remote sensing field, to eliminate radiometric differences caused by different time phases and sensors, preserving the physical meaning and geometric accuracy of the remote sensing data. If the spatial dimensions of the bi-branch features differ (e.g., satellite images of different resolutions), bilinear interpolation is used to upsample the low-resolution features to the target size, ensuring that the feature dimensions input to the bidirectional information exchange module are completely consistent, while preserving the geometric accuracy of the remote sensing image.
[0063] In step 2, to address the potential inconsistency between the number of channels and spatial dimensions in temporal fusion convolutional bi-branch features, dimensionality standardization is achieved through a 1×1 convolutional layer and bilinear interpolation. This ensures that the bidirectional information exchange module can efficiently receive and process bi-branch features, avoiding performance degradation due to dimensionality mismatch.
[0064] Step 3 introduces a bidirectional information exchange module to replace the original fusion part of the temporal fusion convolution. The dimension-aligned land texture features (spatial features) and the dimension-aligned temporal dependency features (temporal features) are used as the two inputs of the bidirectional information exchange module. At the same time, the cross-layer interactive representation is initialized as a zero matrix with the same dimension as the features. Through this module, global information exchange and complementary fusion of multi-temporal remote sensing data are realized, especially enhancing the feature reconstruction capability of cloud-covered areas.
[0065] Step 3, based on the core fusion mechanism of the bidirectional information exchange module, addresses the limitations of local visibility and noise sensitivity in traditional temporal and convolutional fusion. It achieves bidirectional complementarity of dual-branch features through channel-dimensional global correlation modeling, gated adaptive fusion, and cross-layer interactive representation context accumulation. Simultaneously, it filters out invalid information, significantly improving the effectiveness and robustness of the fused features. Specifically, the bidirectional information exchange module is used to update cross-layer interactive representations, generate query keys, calculate global correlation, determine complementary features, and calculate gating coefficients. The specific implementation process of the bidirectional information exchange module in Step 3 is as follows:
[0066] Initialization and updating of cross-layer interaction representations: The initial cross-layer interaction representation is set as an all-zero matrix. In each layer (each layer of the bidirectional information exchange module, the bidirectional information exchange module neural network is composed of multiple stacked layers, starting from the input layer and passing through several hidden layers) = 1, 2, ..., Finally, it reaches the output layer. Each layer of the bidirectional information exchange module refers to each hidden layer, as described below. Hidden layer Or called the first In the iteration of the hidden layer, the query vector output from the previous layer is concatenated and then processed by a 1×1 convolution. The result is then element-wise added to the original cross-layer interaction representation to achieve the update. The update formula for the cross-layer interaction representation is as follows:
[0067]
[0068] in, This indicates the layer number in the bidirectional information exchange module. Indicates the previous level (the first) The cross-layer interaction representation features (i.e., the original cross-layer interaction representation, which is the concatenation result of the query vector output by the previous layer) are the features of the previous layer. This represents a 1×1 convolution processing function. express Activation function This indicates this layer (the first layer) The cross-layer interaction representation features (i.e., the updated cross-layer interaction representation) are obtained. This process accumulates useful contextual information across layers and frames, alleviating the sparsity problem of time-series data.
[0069] Query key generation: The dimension-aligned land cover texture features and dimension-aligned temporal dependency features are concatenated with the current cross-layer interaction representation, and then normalized and convolved by layers to generate the query vector. , By stacking convolutional layers Convert to key vector Sum value vector By stacking convolutional layers Convert to key vector Sum value vector This ensures that features are distinguishable in high-dimensional space. This represents the query vector corresponding to a positive event. This represents the query vector corresponding to the negative event. The input feature representation of a positive event. Input feature representation representing positive and negative events, This represents the key vector corresponding to a positive event. This represents the key vector corresponding to a negative event. This represents the value vector corresponding to a positive event. This represents the value vector corresponding to a negative event. Here, features extracted by the spatial feature branch are defined as positive events, and features extracted by the temporal feature branch are defined as negative events.
[0070] Global relevance calculation: This involves calculating the relevance between the query vector of a negative event and the key vector of a positive event. Based on a scaled dot product attention mechanism, it calculates the relevance matrix between the query vector of a negative event and the key vector of a positive event. The formula is as follows:
[0071]
[0072] in, Represents the normalized exponential function, Equivalent to the above This indicates the layer number in the bidirectional information exchange module. The bidirectional information exchange module is represented by the first one. The query vector corresponding to negative events in the layer. The bidirectional information exchange module is represented by the first one. The key vector corresponding to positive events in the layer. express and The correlation matrix This is the scaling factor, specifically... , Scale parameters representing the height direction, The scale parameter in the width direction is used to avoid attention score saturation caused by excessive dimensionality. This process models global structural correlation in the channel dimension to avoid interference from spatial noise.
[0073] Complementary Feature Aggregation and Gated Fusion: Attention scores are used to weight and aggregate value vectors to obtain complementary features. and complementary features The gating coefficients are calculated using the Sigmoid function to dynamically adjust the fusion weights of the original and complementary features, as shown in the following formula:
[0074]
[0075]
[0076]
[0077] in, The bidirectional information exchange module is represented by the first one. The fused features of negative events in the layer. The bidirectional information exchange module is represented by the first one. The features extracted from the temporal feature branches obtained from the layer are the original features. The bidirectional information exchange module is represented by the first one. The features passed from the spatial feature branch obtained from the layer to the temporal feature branch. The bidirectional information exchange module is represented by the first one. The features passed from the temporal feature branch to the spatial feature branch obtained from the layer. The bidirectional information exchange module is represented by the first one. The value vector corresponding to positive events in the layer. The bidirectional information exchange module is represented by the first one. The gating factor of the layer, Representation of features Linear transformation weights, Indicating complementary features Linear transformation weights, The sigmoid function is used, and ⊙ is the element-wise product. The gating mechanism adaptively filters out invalid information, thereby improving the robustness of the fused features.
[0078] As one example, see Figure 3 The process of the two-way information exchange module is as follows:
[0079] Feature initialization stage: For negative event features, extract key vectors and value vectors as feature carriers for querying and providing information; for positive event features, combine CIR (Committed Information Rate) to generate query vectors as the initiator of active feature query.
[0080] Attention calculation: Perform global relevance calculation to obtain attention-weighted features;
[0081] Aggregating complementary information: Aggregating attention-weighted features allows positive and negative event features to complement each other's semantic information, uncovering cross-feature correlation value;
[0082] Feature optimization stage: Preserve detailed information, such as retaining the original feature details of negative events separately to avoid losing key information during feature fusion; Perform gated fusion: Dynamically adjust the fusion ratio of positive and negative event features through a gating mechanism to finally obtain updated positive event features and achieve optimized feature representation.
[0083] Step 4: In each layer of the temporal fusion convolutional model used for feature extraction, inter-flow information exchange mechanisms and intra-flow information exchange mechanisms are embedded. The inter-flow information exchange mechanism is responsible for cross-branch information exchange between spatial and temporal feature branches, focusing on the global structural complementarity of spatial and temporal branches. The intra-flow information exchange mechanism is responsible for information exchange of features extracted from different layers within a single branch. Combined with the iterative update of cross-layer interactive representations, this achieves effective propagation of hierarchical spatiotemporal context, enabling the effective propagation and accumulation of multi-scale ground feature features from remote sensing images. It is understandable that both inter-flow and intra-flow information exchange mechanisms are bidirectional. The synergistic effect of these two types of information exchange enables the temporal fusion convolutional model to simultaneously capture cross-branch global correlations and intra-branch spatiotemporal dynamics, significantly improving the richness of feature representation. It is also understandable that exchange implies sharing, not the deletion of extracted features after transmission.
[0084] It is understood that the bidirectional information exchange module in this embodiment is a layer-wise stacking bidirectional information exchange module.
[0085] In this embodiment, targeting the hierarchical spatiotemporal correlation characteristics of time-series data, inter-stream information exchange and intra-stream information exchange through layer-wise stacking are used to achieve cross-branch global structural complementarity and intra-branch spatiotemporal feature interaction, respectively. Combined with the iterative update of cross-layer interaction representations, a hierarchical information propagation path is constructed to fully explore the deep correlation information of time-series data. The bidirectional information exchange module includes specific feature interaction and fusion logic. Specifically, through global correlation calculation and gated fusion update, accurate correlation calculation and adaptive fusion of bi-branch features are achieved, ensuring effective complementarity of global structural information while suppressing noise interference.
[0086] In one specific embodiment, the image processing method for remote sensing dynamic monitoring includes:
[0087] Step 1: Acquire remote sensing images;
[0088] Step 2: Use the model obtained by the model optimization method to process the remote sensing image to obtain fused features;
[0089] Step three involves upsampling the fused features. Pixel rearrangement technology is used to map the upsampled fused features into a high-resolution feature map. This involves converting the channel dimension of the fused features into a spatial dimension through pixel rearrangement, achieving resolution improvement without introducing additional parameters. Combined with resampling methods, the original format of the temporal data is restored, ensuring the output data can be directly used for downstream tasks (such as feature input in target recognition or frame generation in video reconstruction). After obtaining the high-resolution feature map, a temporal resampling method based on geographic grid alignment is used to map the features back to the spatial resolution and geographic coordinate system of the original remote sensing image, restoring a high-resolution remote sensing image sequence. This accurately presents key information such as land cover changes, water boundary migration, and vegetation growth status, completing the entire remote sensing dynamic monitoring and processing workflow. It is understood that "high resolution" does not limit the specific resolution size, but only indicates a higher resolution compared to deep features.
[0090] This disclosure provides a model optimization system for remote sensing dynamic monitoring, including:
[0091] The first construction module is used to construct the framework of the temporal fusion convolutional model. The framework includes a spatial feature branch and a temporal feature branch. The spatial feature branch is used to extract the land texture features of the current frame remote sensing image, and the temporal feature branch is used to stitch the features of the current frame remote sensing image with the features of historical remote sensing images from the same period, and extract the temporal dependency features of land surface changes based on the stitching results.
[0092] The second building module is used to build a dimension alignment module in the framework, which is used to align the texture features of the ground features and the temporal dependency features in dimensions.
[0093] The third construction module is used to build a bidirectional information exchange module in the framework to realize global information exchange and complementary fusion of multi-temporal remote sensing data to obtain fused features. The dimension-aligned land texture features and the dimension-aligned temporal dependency features are used as bidirectional inputs of the bidirectional information exchange module. The cross-layer interaction representation is initialized as a zero matrix with the same dimension as the features. The bidirectional information exchange module adopts a gated fusion mechanism to adaptively calculate the fusion weights.
[0094] The embedding module is used to embed inter-flow information exchange mechanism and intra-flow information exchange mechanism in each layer used for feature extraction in the temporal fusion convolutional model. The inter-flow information exchange mechanism is responsible for cross-branch information exchange between spatial feature branches and temporal feature branches, and the intra-flow information exchange mechanism is responsible for the interaction of features at different levels within a single branch.
[0095] In specific implementation, the model optimization system for remote sensing dynamic monitoring can be implemented by referring to the model optimization method for remote sensing dynamic monitoring in any of the above embodiments. The specific implementation steps will not be repeated.
[0096] An electronic device can be implemented according to the method of this disclosure, the electronic device comprising: a memory; one or more processors; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing a model optimization method for remote sensing dynamic monitoring according to any of the above embodiments.
[0097] This disclosure provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the steps of the model optimization method for remote sensing dynamic monitoring described in any of the above embodiments.
[0098] See Figure 4 This disclosure provides an image processing system for remote sensing dynamic monitoring, comprising:
[0099] The acquisition module is used to acquire remote sensing images;
[0100] The processing module is used to process the remote sensing image using the model obtained by the model optimization method to obtain fused features;
[0101] The mapping module is used to map the deep features obtained by the bidirectional information exchange module into a feature map using pixel rearrangement technology, and to obtain a remote sensing image sequence based on the feature map and the shooting time of the remote sensing image.
[0102] In specific implementation, the image processing system can be implemented by referring to the image processing method in any of the above embodiments, and the specific implementation steps will not be repeated.
[0103] An electronic device can be implemented according to the method of this disclosure, the electronic device comprising: a memory; one or more processors; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing an image processing method for remote sensing dynamic monitoring according to any of the above embodiments.
[0104] This disclosure also provides a storage medium including instructions that, when executed on a computer, cause the computer to perform the various steps of the image processing method for remote sensing dynamic monitoring described in any of the above embodiments.
[0105] This disclosure discloses a model optimization method, image processing method, and system for remote sensing dynamic monitoring. By replacing the traditional fusion component with a two-way information exchange module, it achieves global information exchange and complementary fusion of multi-temporal remote sensing data to obtain fused features. The two types of features to be fused provide complementary information to each other. Inter-flow information exchange mechanisms and intra-flow information exchange mechanisms are embedded in each layer of feature extraction, introducing cross-layer interactive representations and using a gated fusion mechanism to adaptively calculate fusion weights. This design can increase the correlation of global features, making the fused features contain both local details and global consistency. It can accurately filter out interference from noise and invalid pixels. Based on the optimized model, the processed remote sensing images have small errors with the real scene and are closer to the real scene, while also taking into account computational complexity, i.e., processing efficiency.
[0106] Specifically, the model optimization method, image processing method, and system for remote sensing dynamic monitoring have the following advantages:
[0107] Overcoming the limitations of local receptive fields and enhancing global feature correlation: Existing temporal fusion convolutional models rely on traditional convolution or simple concatenation for the fusion part. Limited by the local receptive field of the convolution kernel, they can only capture the local correlation between adjacent pixels and cannot effectively model the long-range dependencies and global structural information of different feature branches (such as spatial and temporal features, positive and negative event features) in temporal data. This results in a lack of overall consistency in the fused features and incomplete detail recovery. This disclosure replaces the traditional fusion part with a bidirectional information exchange module, innovatively modeling global structural correlation at the channel dimension. Each channel is treated as an independent structural representation, and the global correlation of different feature branches is calculated through a scaled dot product attention mechanism, achieving effective transmission of long-distance feature information. This design aligns with the "global edge structure dominance" characteristic of temporal data such as event streams and videos, and can fully explore the complementary relationships between positive and negative events and spatiotemporal features. The fused features contain both local details and global consistency, achieving performance improvement in event stream super-resolution tasks compared to traditional methods.
[0108] Enhancing noise robustness and filtering out invalid information interference: Temporal data (especially event streams) commonly suffers from sensor noise and data sparsity. Traditional fusion methods in existing temporal fusion convolutional models lack effective noise suppression mechanisms, easily amplifying invalid information and leading to blurred output features and distorted edges, severely impacting downstream task performance. This disclosed bidirectional information exchange module addresses this problem through a dual design: first, it introduces cross-layer interactive representations to accumulate useful contextual information from previous layers and frames, effectively compensating for information loss due to data sparsity; second, it employs a gated fusion mechanism (using the Sigmoid function) to adaptively calculate fusion weights, dynamically adjusting the ratio of original features to complementary features, and accurately filtering out interference from noise and invalid pixels.
[0109] Optimizing feature complementarity efficiency and balancing performance and computational complexity: Existing optimization schemes for temporal fusion convolutions either enhance the fusion effect by deepening the network layers and increasing the number of channels, leading to an exponential increase in computational complexity; or they employ a one-way fusion paradigm, which fails to fully utilize the complementary information of multi-source features, resulting in limited feature representation capabilities. This disclosure's bidirectional information exchange module adopts a bidirectional information exchange paradigm, enabling the two types of features to be fused to provide complementary information to each other, rather than through one-way transmission, significantly improving the richness of feature representation; simultaneously, the computational complexity of the bidirectional information exchange module is... Compared to traditional 3D convolutional fusion ( , (for kernel size) and cross-attention fusion ( This significantly reduces computational and processing costs while improving processing efficiency and real-time performance, all while maintaining performance. Used to describe the time or space complexity of an algorithm or module. Indicates the number of feature channels. Indicates the height of the feature map. Indicates the width of the feature map. This indicates the kernel size.
[0110] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0111] The embodiments described above are merely illustrative of several implementations of this disclosure, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this disclosure, and these all fall within the protection scope of this disclosure. Therefore, the protection scope of this patent should be determined by the appended claims.
Claims
1. A model optimization method for remote sensing dynamic monitoring, characterized in that, Includes the following steps: A framework for constructing a temporal fusion convolutional model is provided. The framework includes a spatial feature branch and a temporal feature branch. The spatial feature branch is used to extract the land texture features of the current frame remote sensing image, and the temporal feature branch is used to stitch together the features of the current frame remote sensing image and the features of historical remote sensing images from the same period, and extract the temporal dependency features of land surface changes based on the stitching results. A dimension alignment module is constructed in the framework, which is used to align the texture features of land features and the temporal dependency features in dimensions. In the framework, a bidirectional information exchange module is constructed to realize global information exchange and complementary fusion of multi-temporal remote sensing data to obtain fused features. The dimension-aligned land texture features and the dimension-aligned temporal dependency features are used as bidirectional inputs to the bidirectional information exchange module. The cross-layer interactive representation is initialized as a zero matrix with the same dimension as the features. The bidirectional information exchange module adopts a gated fusion mechanism to adaptively calculate the fusion weights. In the temporal fusion convolutional model, each layer used for feature extraction embeds an inter-flow information exchange mechanism and an intra-flow information exchange mechanism. The inter-flow information exchange mechanism is responsible for cross-branch information exchange between spatial feature branches and temporal feature branches, while the intra-flow information exchange mechanism is responsible for the interaction of features at different levels within a single branch.
2. The model optimization method for remote sensing dynamic monitoring according to claim 1, characterized in that, The spatial feature branch is specifically used to extract the texture features of the ground features in the current frame of remote sensing imagery through continuous convolutional residual blocks.
3. The model optimization method for remote sensing dynamic monitoring according to claim 1, characterized in that, The bidirectional information exchange module is specifically used to update cross-layer interaction representations, generate query key values, calculate global relevance, determine complementary features, and calculate gating coefficients.
4. The model optimization method for remote sensing dynamic monitoring according to claim 3, characterized in that, The update formula for the cross-layer interaction representation is: in, This indicates the layer number in the bidirectional information exchange module. This represents the cross-layer interaction characteristics of the previous layer. This represents a 1×1 convolution processing function. express Activation function This represents the cross-layer interaction characteristics of this layer.
5. The model optimization method for remote sensing dynamic monitoring according to claim 3, characterized in that, The specific process for generating query keys involves concatenating dimension-aligned land texture features and dimension-aligned temporal dependency features with the current cross-layer interaction representation, followed by layer normalization and 1×1 convolution to generate query vectors corresponding to positive events. Query vectors corresponding to negative events The input features of positive and negative events are represented by stacked convolutional layers. Convert to the key vector corresponding to the positive event Value vector corresponding to positive events The input features of negative events are represented by stacked convolutional layers. Convert to the key vector corresponding to the negative event value vectors corresponding to negative events .
6. The model optimization method for remote sensing dynamic monitoring according to claim 5, characterized in that, The calculation of global relevance involves calculating the relevance between the query vector of a negative event and the key vector of a positive event, using the following formula: in, express and The correlation matrix Represents the normalized exponential function, This indicates the layer number in the bidirectional information exchange module. The bidirectional information exchange module is represented by the first one. The query vector corresponding to negative events in the layer. The bidirectional information exchange module is represented by the first one. The key vector corresponding to positive events in the layer. Indicates the scaling factor; The complementary features are determined by weighting and aggregating the value vectors using attention scores. and The gating coefficient is calculated using the Sigmoid function, satisfying the following: in, The bidirectional information exchange module is represented by the first one. The features passed from the spatial feature branch obtained from the layer to the temporal feature branch. This represents a 1×1 convolution processing function. The bidirectional information exchange module is represented by the first one. The value vector corresponding to positive events in the layer. The bidirectional information exchange module is represented by the first one. The fused features of negative events in the layer. The bidirectional information exchange module is represented by the first one. The gating factor of the layer, The bidirectional information exchange module is represented by the first one. Features extracted from the temporal feature branches obtained from the layers. express Linear transformation weights, express Linear transformation weights, The sigmoid function is represented by ⊙, which is the element-wise product, and the features are complementary. The bidirectional information exchange module is represented by the first one. The features obtained from the temporal feature branch are passed to the spatial feature branch.
7. A model optimization system for remote sensing dynamic monitoring, characterized in that, include: The first construction module is used to construct the framework of the temporal fusion convolutional model. The framework includes a spatial feature branch and a temporal feature branch. The spatial feature branch is used to extract the land texture features of the current frame remote sensing image, and the temporal feature branch is used to stitch the features of the current frame remote sensing image with the features of historical remote sensing images from the same period, and extract the temporal dependency features of land surface changes based on the stitching results. The second building module is used to build a dimension alignment module in the framework, which is used to align the texture features of the ground features and the temporal dependency features in dimensions. The third construction module is used to build a bidirectional information exchange module in the framework to realize global information exchange and complementary fusion of multi-temporal remote sensing data to obtain fused features. The dimension-aligned land texture features and the dimension-aligned temporal dependency features are used as bidirectional inputs of the bidirectional information exchange module. The cross-layer interaction representation is initialized as a zero matrix with the same dimension as the features. The bidirectional information exchange module adopts a gated fusion mechanism to adaptively calculate the fusion weights. The embedding module is used to embed inter-flow information exchange mechanism and intra-flow information exchange mechanism in each layer used for feature extraction in the temporal fusion convolutional model. The inter-flow information exchange mechanism is responsible for cross-branch information exchange between spatial feature branches and temporal feature branches, and the intra-flow information exchange mechanism is responsible for the interaction of features at different levels within a single branch.
8. An image processing method for remote sensing dynamic monitoring, characterized in that, include: Acquire remote sensing images; The remote sensing image is processed using the model optimization method for remote sensing dynamic monitoring as described in any one of claims 1 to 6 to obtain fused features; The deep features obtained by the bidirectional information exchange module are mapped into feature maps using pixel rearrangement technology. The remote sensing image sequence is obtained based on the feature maps and the shooting time of the remote sensing images.
9. The image processing method for remote sensing dynamic monitoring according to claim 8, characterized in that, The process of using pixel rearrangement technology to map the deep features obtained by the bidirectional information exchange module into a feature map specifically involves: converting the channel dimension of the fused features into a spatial dimension using pixel rearrangement technology to obtain the feature map.
10. An image processing system for remote sensing dynamic monitoring, characterized in that, include: The acquisition module is used to acquire remote sensing images; The processing module is used to process the remote sensing image using the model obtained by the model optimization method for remote sensing dynamic monitoring as described in any one of claims 1 to 6 to obtain fused features; The mapping module is used to map the deep features obtained by the bidirectional information exchange module into a feature map using pixel rearrangement technology, and to obtain a remote sensing image sequence based on the feature map and the shooting time of the remote sensing image.