A local-global collaborative feature encoding and adaptive fusion method for multi-source heterogeneous remote sensing data

Abstract

The application discloses a local-global collaborative feature coding and adaptive fusion method for multi-source heterogeneous remote sensing data, and belongs to the technical field of remote sensing information processing and intelligent prediction. The method realizes direct input of multivariate remote sensing data by performing channel consistency mapping on a pre-training feature extraction network; a local convolution coding unit and a global sequence dependence modeling unit are constructed to respectively extract local fine-grained spatial features and global context features; adaptive context clipping is performed on the global features based on geographic positioning information, and a longitude wrap mechanism is combined to process the spherical boundary problem; a local-driven cross-modal selective interactive fusion is adopted at the multi-scale feature layer to realize local and global information collaborative modeling; finally, a prediction result is output through a step-by-step decoding structure, and a grouping differentiated learning rate strategy is adopted to improve the training stability. The application improves the cross-scale expression capability and fusion precision of multi-source remote sensing data, and enhances the prediction robustness in a complex geographical environment.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing information processing and intelligent prediction technology, and in particular relates to a local-global collaborative feature encoding and adaptive fusion method for multi-source heterogeneous remote sensing data. It can be applied to forest fire risk prediction, land cover classification, flood disaster monitoring, and other remote sensing analysis tasks that require the simultaneous use of local fine features and global macro background. Background Technology

[0002] Remote sensing technology plays a crucial role in disaster monitoring, resource surveys, and environmental assessments. With the diversification of remote sensing observation methods, single data sources are no longer sufficient to meet the demands of high-precision prediction, making the fusion processing of multi-source heterogeneous remote sensing data a research hotspot. In tasks such as fire risk prediction, input data typically includes multi-temporal and multi-variable observation information, such as surface temperature, air pressure, vegetation index, soil moisture, precipitation, and wind speed, while also requiring the integration of static auxiliary information such as geographical location. Existing technologies mainly suffer from the following problems: (1) Inconsistency of channels in multi-source heterogeneous data. Most existing pre-trained models are designed based on three-channel natural images, while remote sensing data often has multivariable and high-channel characteristics. Existing methods usually adapt the input through dimensionality reduction and compression, resulting in serious information loss.

[0003] (2) Insufficient fusion of local and global information. Existing methods often use simple splicing or interpolation fusion to process global and local features, which makes it difficult to effectively model the semantic relationship between the two and easily loses global structural information.

[0004] (3) Discontinuity problem of spherical longitude boundary. Global remote sensing data has periodic characteristics in the longitude direction. When crossing the date line, feature breaks are prone to occur, which affects the spatial consistency modeling.

[0005] (4) Instability problem in training heterogeneous networks. Local and global branch structures are different. If a uniform learning rate is used for training, it is easy to cause the pre-trained parameters to be corrupted or the new module to have difficulty converging. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes a local-global collaborative feature encoding and adaptive fusion method for multi-source heterogeneous remote sensing data. This method can simultaneously model local details and global context, and supports unified fusion of multi-source heterogeneous data, thereby improving the modeling capability and prediction accuracy of remote sensing data under complex spatial structures.

[0007] To achieve the above objectives, the technical solution adopted by this invention is: a local-global collaborative feature encoding and adaptive fusion method for multi-source heterogeneous remote sensing data, comprising the following steps: Step 1: Acquire multi-source heterogeneous remote sensing data containing multivariate environmental information, temporal information, and spatial location information, and construct local input data and global input data according to the target area range; Step 2: Based on the number of channels in the local input data and the global input data, perform channel adaptation mapping on the input parameters of the pre-trained feature extraction network so that the pre-trained feature extraction network can process multivariate remote sensing data input. Step 3: Input the local input data into the local spatial feature encoding unit, and generate local feature representations at different spatial scales through the hierarchical feature extraction process; Step 4: Input the global input data into the global dependency modeling unit, and generate global feature representations at different scales through the spatial relationship modeling process; Step 5: Based on the geospatial positioning information of the local area, determine the corresponding spatial location in the global feature representation and extract the context feature region containing the surrounding environment information; Step 6: Adaptive interactive fusion is performed in multiple scale feature layers based on the correlation between local features and context features to generate a fused feature representation; Step 7: Input the fused features into the multi-scale feature recovery structure, and generate the final prediction result through step-by-step reconstruction.

[0008] Furthermore, channel adaptation mapping of input parameters includes the following steps: Statistical processing is performed on the first-layer convolutional parameters of the pre-trained feature extraction network along the input dimension; The statistical results are expanded and mapped based on the number of target remote sensing data channels, and the expansion parameters are adjusted according to the ratio between the original number of channels and the target number of channels.

[0009] Furthermore, the processing of the local spatial feature encoding unit includes the following steps: extracting the internal texture structure information of the remote sensing area step by step through multi-layer spatial convolution operations, and outputting local feature maps corresponding to multiple spatial scales.

[0010] Furthermore, the processing steps of the global dependency modeling unit include: converting global remote sensing data into spatial correlation sequences, establishing long-distance dependencies between different locations through multi-directional spatial relationship calculations, and recovering and forming a multi-scale global feature map.

[0011] Furthermore, extracting contextual feature regions based on geospatial positioning information includes the following steps: Establish a mapping relationship between latitude and longitude coordinates and global feature space, and generate global feature index locations based on the mapping results; The feature extraction window is dynamically determined based on the global feature index location, the local area range, and the preset context expansion ratio. When the extraction window crosses the global longitude boundary, the global features are subjected to boundary compensation processing based on the longitude cycle continuity to obtain a continuous context feature region.

[0012] Furthermore, the adaptive interactive fusion process includes the following steps: generating feature selection guidance information using local features, selecting relevant regional information from global context features based on the guidance information, and jointly fusing the selected context features with local features to obtain a multi-scale fusion representation.

[0013] Furthermore, the multi-scale feature restoration structure processing includes: performing stepwise spatial restoration of fused features at different scales, jointly reconstructing high-level semantic information and low-level spatial information, and outputting prediction results corresponding to the spatial resolution of the input remote sensing data.

[0014] The beneficial effects of adopting this technical solution are: This invention constructs a dual-path feature encoding architecture consisting of a local spatial feature extraction branch and a global dependency modeling unit. An input channel adaptation mechanism enables the pre-trained feature extraction network to handle multivariate remote sensing data compatiblely. Based on geospatial positioning information, it dynamically extracts contextual information corresponding to local regions from the global feature representation. A selective interactive fusion mechanism achieves the collaborative representation of local spatial information and global background information. Furthermore, a parameter grouping optimization strategy enhances the stability of the model training process. This invention improves the cross-scale representation capability and fusion accuracy of multi-source remote sensing data, and enhances prediction robustness in complex geographical environments.

[0015] This invention utilizes a channel consistency mapping mechanism to enable pre-trained models to directly adapt to multivariate remote sensing inputs, avoiding information compression loss. It achieves collaborative modeling of fine-grained spatial information and global contextual information through a local-global dual-branch structure. A longitude-encircling mechanism addresses the discontinuity problem at the boundaries of spherical remote sensing data, improving global modeling consistency. A locally driven selective interactive fusion mechanism achieves adaptive semantic alignment of local and global features. Finally, a multi-scale decoding structure enhances spatial reconstruction capabilities, improving prediction accuracy and robustness. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the process of a local-global collaborative feature encoding and adaptive fusion method for multi-source heterogeneous remote sensing data according to the present invention; Figure 2 This is a diagram of the channel consistency mapping structure in an embodiment of the present invention; Figure 3 This is a partial convolutional coding structure diagram in an embodiment of the present invention; Figure 4 This is a diagram illustrating the global dependency modeling structure in an embodiment of the present invention. Figure 5 This is a diagram illustrating the spatial indexing and context pruning mechanism in an embodiment of the present invention; Figure 6 This is a selective interactive fusion structure diagram in an embodiment of the present invention; Figure 7 This is a diagram of the multi-scale feature decoding structure in an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings and specific embodiments. This embodiment uses forest fire prediction in Sichuan Province as an application scenario to illustrate the specific implementation process of this invention. Sichuan Province is only one example of the target area; this invention can also be applied to forest fire prediction in other provincial, national, cross-administrative region, or arbitrary latitude and longitude ranges, and can also be applied to drought, flood, vegetation change, and other remote sensing prediction tasks.

[0018] This embodiment selects a forest area in Liangshan Yi Autonomous Prefecture, Sichuan Province as a local target area. Using multi-source environmental data within the current eight-day time window as input, it predicts the probability of fire or burned area occurring in each target pixel within the next eight-day time window. The following descriptions use samples from the same target area in sequence. Figures 1 to 7 The processing steps are shown below.

[0019] In this embodiment, see Figure 1 As shown, this invention proposes a local-global collaborative feature encoding and adaptive fusion method for multi-source heterogeneous remote sensing data, including the following steps: Step 1: Acquire multi-source heterogeneous remote sensing data containing multivariate environmental information, temporal information, and spatial location information, and construct local input data and global input data according to the target area range; Step 2: Based on the number of channels in the local input data and the global input data, perform channel adaptation mapping on the input parameters of the pre-trained feature extraction network so that the pre-trained feature extraction network can process multivariate remote sensing data input. Step 3: Input the local input data into the local spatial feature encoding unit, and generate local feature representations at different spatial scales through the hierarchical feature extraction process; Step 4: Input the global input data into the global dependency modeling unit, and generate global feature representations at different scales through the spatial relationship modeling process; Step 5: Based on the geospatial positioning information of the local area, determine the corresponding spatial location in the global feature representation and extract the context feature region containing the surrounding environment information; Step 6: Adaptive interactive fusion is performed in multiple scale feature layers based on the correlation between local features and context features to generate a fused feature representation; Step 7: Input the fused features into the multi-scale feature recovery structure, and generate the final prediction result through step-by-step reconstruction.

[0020] As an optimization of the above embodiments, such as Figure 2 As shown, the channel adaptation mapping of input parameters includes the following steps: To acquire multivariate remote sensing input, the pre-trained convolutional weights are obtained as follows: The local feature encoding unit uses a convolutional network pre-trained with natural images. Since the pre-trained network processes RGB three channels, while the local input has ten environmental variables and four spatial location channels, channel adaptation is required for the first-layer weights. The weights of each channel in the first layer are extracted, and the base weights are calculated by averaging along the channel direction. These base weights are then replicated according to the actual number of channels, ensuring that the first-layer convolution can receive all channels.

[0021] Input channel expansion mapping and weight scaling calibration: The expanded weight amplitude is scaled according to the ratio of the original to the target number of channels to maintain a similar output activation range and avoid abnormal amplification of the response. The first-layer bias remains unchanged, and subsequent network parameters are loaded with pre-trained weights. The same adaptation applies to the block embedding layer or the first-layer convolution of the global feature encoding unit: when the global input only contains channels corresponding to the local input, it is expanded according to the number; when it contains large-scale variables such as climate indices, the weight replication and amplitude scaling are redone according to the actual number of channels.

[0022] Output adaptation features: Through channel adaptation, the basic feature extraction capabilities of the pre-trained network can be used to process multivariate remote sensing data both globally and locally.

[0023] As an optimization of the above embodiments, such as Figure 3 As shown, the processing procedure of the local spatial feature encoding unit includes the following steps: Local remote sensing input: Local input of the target area in Liangshan Yi Autonomous Prefecture, Sichuan Province, is sent to the local spatial feature coding unit.

[0024] Convolutional Feature Extraction and Stepwise Scale Encoding: Local encoding units extract basic spatial features through convolution, normalization, and nonlinear activation. Then, multi-level convolutional layers and downsampling layers are used to progressively reduce spatial resolution and increase the number of feature channels. In one example setup, the local branch outputs four levels of local features sequentially, with spatial sizes of 40x40, 20x20, 10x10, and 5x5, and channel numbers of 64, 128, 256, and 512, respectively. The first and second level local features preserve forest edges, vegetation patches, valleys, areas near roads, residential areas, and local temperature and moisture gradients. The third and fourth level local features express combined risks such as continuous high temperatures, low soil moisture, high saturated vapor pressure differentials, vegetation aridity, and insufficient precipitation. Shallow features distinguish between mountain forests, non-forested areas, and areas near human activity, while deep features form a comprehensive representation of local drought background, fuel status, and thermal conditions. The four levels of local features are retained and passed to subsequent interactive fusion units at corresponding scales.

[0025] Multi-scale local feature output.

[0026] As an optimization of the above embodiments, such as Figure 4 As shown, the processing steps of the global dependency modeling unit include: Global input data: Global inputs from the same time location as the local samples are fed into the global dependency modeling unit.

[0027] Spatial feature mapping: Global coding units reduce spatial size and increase feature dimension through block embedding, and then generate global features at different scales through multi-level state space coding layers.

[0028] Sequence dependency modeling and long-distance information aggregation: In each coding layer, the two-dimensional global features are sequentially expanded in the directions of west to east, east to west, north to south, and south to north. After long-distance dependency modeling, the sequences in each direction are restored to a two-dimensional spatial structure and fused to express the long-distance environmental associations in the latitudinal and longitudinal directions. The fused global features are downsampled stepwise to form four levels, which respectively express multi-scale environmental information from finer regional background to coarser global background, and establish scale correspondence with the four-level features of the local branches.

[0029] Finally, multi-scale global features are output.

[0030] For the target area in Sichuan Province, global features provide large-scale environmental backgrounds of the Tibetan Plateau, the western Pacific Ocean, the Indian Ocean, and East Asia, supplementing long-distance environmental conditions that cannot be fully represented within a local 80-kilometer radius. This invention selects relevant global information through a data-driven approach, without pre-defining that a particular region will necessarily have a specific impact on forest fires in Sichuan Province.

[0031] As an optimization of the above embodiments, such as Figure 5As shown, the steps for extracting contextual feature regions based on geospatial location information include: The data module obtains geospatial positioning information, and stores the center latitude and longitude of the target area, local boundaries, and global raster size for local samples in Sichuan Province.

[0032] Global Feature Spatial Index: Based on geospatial positioning information, determine the spatial location corresponding to the target area in Liangshan Yi Autonomous Prefecture, Sichuan Province, in each level of the global feature map.

[0033] The context window is dynamically determined: the clipping window expands around the target location according to a preset context ratio. In addition to covering the corresponding location of the target area, the clipping range also preserves the environmental context of the southwestern edge of the Sichuan Basin, the Panxi River Valley, adjacent plateaus, and other surrounding areas. The context ratio can be adjusted based on the prediction lead time, global feature resolution, and target area scale. Because the four levels of global features have different spatial dimensions, location mapping and context clipping are performed on the feature maps at each level. After clipping, the spatial dimensions are adjusted to match the corresponding level's local features using bilinear interpolation, adaptive pooling, or other scale alignment methods.

[0034] Longitude cycle compensation processing: When the clipping window crosses a global longitude boundary, boundary compensation processing is performed on global features based on the continuity of the longitude cycle to obtain a continuous contextual feature region. When the context window crosses a longitude boundary, the corresponding portions of the right and left ends of the global feature map are extracted separately and stitched along the longitude direction to maintain spatial continuity; no wrapping processing is performed in the latitude direction, and window truncation, copy filling, or other boundary processing methods are used. The target area of ​​Sichuan Province usually does not trigger longitude wrapping processing, but clipping is still completed according to the unified spatial indexing process.

[0035] Finally, the context feature block is output.

[0036] As an optimization of the above embodiments, such as Figure 6 As shown, the adaptive interactive fusion process includes the following steps: Selective interactive computation: Using local features as the query (Q) and global context features as the retrieved information including keys (K) and values ​​(V), attention computation is performed within the corresponding spatial window. Based on the current temperature, moisture, vegetation and precipitation status of the local area, information related to the target area of ​​Sichuan Province is selected from the global context to avoid indiscriminately adding all global features.

[0037] Selective interactive computation: Using local features as the query and global context features as the retrieved information, attention computation is performed within the corresponding spatial window. Based on the current temperature, moisture, vegetation, and precipitation status of the local area, information relevant to the target area of ​​Sichuan Province is selected from the global context, avoiding the indiscriminate inclusion of all global features. Taking the second-level features as an example, the local features and aligned global context features are divided into multiple spatial windows. Within each window, a correlation is established between the local location and the global context location. The attention result is used to represent information related to the large-scale environmental background. Simultaneously, a convolution enhancement path is set to perform spatial convolution and channel mapping on the local features, preserving forest boundaries, areas adjacent to roads, residential activity areas, and fine-grained temperature and humidity gradients.

[0038] Feature residual fusion: Attention path output, convolutional enhanced features, and original local features are fused through gating weights and residual connections. When local environmental signals are sufficient, the fusion unit enhances the contribution of local detail features; when local information is insufficient but the global background is strongly correlated with the target region, the contribution of global context features is enhanced. The above selective interaction process is applied to four feature scales, and can be executed once or multiple times at each scale depending on model capacity and computational resources.

[0039] Finally, the fused data is output.

[0040] As an optimization of the above embodiments, such as Figure 7 As shown, the multi-scale feature recovery structure processing procedure includes: Obtain multi-scale fusion features: Input the four-level fusion features into the multi-scale features to restore the structure.

[0041] Hierarchical upsampling and cross-scale feature fusion: Multi-scale context extraction is performed on the highest-level fused features, followed by hierarchical upsampling and weighted merging with fused features at corresponding scales to gradually restore spatial resolution. During decoding, high-level fused features provide regional drought, large-scale environmental background, and comprehensive risk semantics, while low-level fused features restore spatial details such as forest edges, valleys, roadside areas, residential activity areas, and local vegetation patches. The fusion weights are automatically adjusted during training to ensure the model selects appropriate high-level semantic information and low-level localization information.

[0042] Predictive features are generated, and the final prediction results are output: After the features are restored to 80x80, they are input into the feature refinement module. This module performs calibration from both channel and spatial directions. Channel calibration highlights relevant environmental information such as surface temperature, saturated vapor pressure difference, soil moisture, vegetation index, wind speed, and precipitation; spatial calibration highlights continuous dry forest areas, forest edges, and areas adjacent to human activities. After feature refinement, a fire probability map with the same size as the local input space is generated through the output layer. Each target pixel outputs both the non-fire probability and the fire probability. During training, supervised loss is calculated only on pixels with valid burn area labels and that meet the target forest conditions; during inference, a forest fire risk distribution map of the target area can be generated by combining forest masks. The model is trained using eight-day synthetic data of Sichuan Province from 2003 to 2018, model parameters are selected using data from 2019 to 2020, and independent testing is conducted using data from 2021 to 2023. The model is trained using cross-entropy loss and the Adam optimizer, and the model with the lowest validation loss is selected for testing. When evaluating the results, indicators such as the area under the precision-recall curve, the area under the receiver operating characteristic curve, precision, recall, and F1 score can be used to assess the prediction results.

[0043] The above-described Sichuan Province forest fire prediction embodiment illustrates the data transfer and collaborative processing between modules. The target area can be replaced with other provincial regions, watersheds, ecological function zones, forest management areas, or any latitude and longitude range; the data sources for local and global inputs can also be replaced according to the target task. In other embodiments, environmental variables can include topographic elevation, slope, aspect, land cover, climate index, or human activity indicators, or some unavailable variables can be removed. The local area size, spatial resolution, time window, prediction lead time, global input resolution, number of encoding layers, number of feature channels, context pruning ratio, number of interactions, and decoding structure can all be adjusted according to data conditions and computing resources. The Sichuan Province, Liangshan Yi Autonomous Prefecture, time range, spatial resolution, input variables, local area size, global grid size, and network parameters mentioned in this embodiment are specific examples set for ease of explanation and do not constitute a limitation on the application area, application object, data source, or protection scope of this invention.

[0044] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for local-global collaborative feature encoding and adaptive fusion of multi-source heterogeneous remote sensing data, characterized in that, Including the following steps: Step 1: Acquire multi-source heterogeneous remote sensing data containing multivariate environmental information, temporal information, and spatial location information, and construct local input data and global input data according to the target area range; Step 2: Based on the number of channels in the local input data and the global input data, perform channel adaptation mapping on the input parameters of the pre-trained feature extraction network so that the pre-trained feature extraction network can process multivariate remote sensing data input. Channel adaptation mapping of input parameters includes the following steps: Statistical processing is performed on the first-layer convolutional parameters of the pre-trained feature extraction network along the input dimension; The statistical results are extended and mapped based on the number of target remote sensing data channels, and the extension parameters are adjusted according to the ratio between the original number of channels and the target number of channels. Step 3: Input the local input data into the local spatial feature encoding unit, and generate local feature representations at different spatial scales through the hierarchical feature extraction process; Step 4: Input the global input data into the global dependency modeling unit, and generate global feature representations at different scales through the spatial relationship modeling process; Step 5: Based on the geospatial positioning information of the local area, determine the corresponding spatial location in the global feature representation and extract the context feature region containing the surrounding environment information; Extracting contextual feature regions based on geospatial location information includes the following steps: Establish a mapping relationship between latitude and longitude coordinates and global feature space, and generate global feature index locations based on the mapping results; The feature extraction window is dynamically determined based on the global feature index location, combined with the local region range and the preset context expansion ratio. When the extraction window crosses the global longitude boundary, the global feature is subjected to boundary compensation processing based on the longitude cycle continuity to obtain a continuous context feature region. Step 6: Adaptive interactive fusion is performed in multiple scale feature layers based on the correlation between local features and context features to generate a fused feature representation; Step 7: Input the fused features into the multi-scale feature recovery structure, and generate the final prediction result through step-by-step reconstruction.

2. The method for local-global collaborative feature encoding and adaptive fusion of multi-source heterogeneous remote sensing data according to claim 1, characterized in that, The processing steps of the local spatial feature encoding unit include: extracting the internal texture structure information of the remote sensing area step by step through multi-layer spatial convolution operations, and outputting local feature maps corresponding to multiple spatial scales.

3. The method for local-global collaborative feature encoding and adaptive fusion of multi-source heterogeneous remote sensing data according to claim 1, characterized in that, The processing steps of the global dependency modeling unit include: converting global remote sensing data into spatial correlation sequences, establishing long-distance dependencies between different locations through multi-directional spatial relationship calculations, and recovering and forming a multi-scale global feature map.

4. The method for local-global collaborative feature encoding and adaptive fusion of multi-source heterogeneous remote sensing data according to claim 1, characterized in that, The adaptive interactive fusion process includes the following steps: generating feature selection guidance information using local features, selecting relevant regional information from global context features based on the guidance information, and jointly fusing the selected context features with local features to obtain a multi-scale fusion representation.

5. The method for local-global collaborative feature encoding and adaptive fusion of multi-source heterogeneous remote sensing data according to claim 1, characterized in that, The multi-scale feature restoration structure processing includes: performing stepwise spatial restoration of fused features at different scales, jointly reconstructing high-level semantic information and low-level spatial information, and outputting prediction results corresponding to the spatial resolution of the input remote sensing data.

Images