Remote sensing image intelligent surveying and mapping method and system based on multi-source data fusion

Abstract

The invention relates to the field of image scene recognition, in particular to a remote sensing image intelligent surveying and mapping method and system based on multi-source data fusion. The method includes acquiring image data; analyzing the image data through a parallel neural network processing framework, and extracting a geological scene representation map by using a first deep neural network encoder; using a second deep neural network encoder to extract an image quality assessment map; based on the image quality evaluation graph, quantifying the reliability degree, and generating a confidence graph; inputting the geological scene representation map and the confidence map into a gating semantic fusion model; the gating semantic fusion model takes the confidence map as a spatial gating signal, and performs adaptive weighting on the geological scene representation map to generate a single fusion scene map; and processing by using the single fusion scene graph to generate a surveying and mapping product. Through a parallel neural network processing framework, fusion precision reduction caused by data quality uncertainty is relieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image scene recognition, in particular to a remote sensing image intelligent mapping method and system based on multi-source data fusion. BACKGROUND

[0002] Remote sensing technology has entered the era of big data, providing massive amounts of multi-source heterogeneous images. The core of effectively utilizing these data is to intelligently extract geographic spatial information from the original images, which is essentially an image recognition and understanding problem. Multi-source data fusion is the key to improving the accuracy of pattern recognition, and deep learning technology provides the core driving force for the automation of feature extraction and other steps.

[0003] The prior art mainly faces some defects when processing multi-source remote sensing images. The geometric and radiometric differences of multi-source images result in insufficient image registration accuracy. The error of the initial alignment is directly transmitted and amplified, which seriously affects the accuracy of subsequent processing. Traditional methods rely on manually designed feature operators, which have weak generalization ability when facing multi-source remote sensing images with varying content, and cannot adaptively learn and extract deep and robust features for describing the boundaries and attributes of geographic objects. Existing fusion algorithms mostly use fixed mathematical models, lack dynamic perception of image content and data quality, and cannot adaptively adjust the fusion strategy according to the reliability of local features, resulting in information loss or accuracy reduction. When the general process of image recognition or understanding is applied to intelligent mapping, the internal links (such as preprocessing, feature extraction, and fusion) are fragmented and rigid. When there are local quality differences or information conflicts in multi-source images, the fixed processing flow is difficult to effectively cope with.

[0004] In summary, there is a need for a method that can intelligently and differentially evaluate the reliability of data points from different sources and different locations according to the data quality and content characteristics of multi-source images in the intelligent mapping process, and adaptively adjust the feature fusion strategy accordingly to realize the collaborative use of heterogeneous information, thereby improving the consistency and accuracy of the fused geographic spatial data product, and generating high-precision mapping products.

[0005] To this end, a remote sensing image intelligent mapping method and system based on multi-source data fusion are proposed. SUMMARY

[0006] The present application aims to provide a remote sensing image intelligent mapping method and system based on multi-source data fusion to solve the problems raised in the background art.

[0007] To achieve the above purpose, the present application provides the following technical solution: a remote sensing image intelligent mapping method and system based on multi-source data fusion, comprising: Image data from multiple sensors in the same geological exploration area is acquired; semantic scene analysis is performed on the image data using a parallel neural network processing framework; a first deep neural network encoder is used to extract a geological scene representation map that represents the geological structures in the image data; and a second deep neural network encoder is used to extract an image quality assessment map that represents the image artifacts and statistical degradation features of the image data at different spatial locations. Based on the image quality assessment map, the reliability of the image data at each spatial location is quantified to generate a multi-channel confidence map. The geological scene representation map and confidence map corresponding to the image data are input into a gated semantic fusion model. The gated semantic fusion model uses the confidence map as a spatial gate signal to perform pixel-level adaptive weighting on the geological scene representation map, dynamically adjusting the contribution of the image data in the feature fusion process, and generating a single fused scene map. Pixel-level semantic segmentation is performed using a single fused scene map to generate mapping products with geological boundaries.

[0008] Preferably, the specific construction of the parallel neural network processing framework includes: A content processing stream, including the first deep neural network encoder, is configured to receive the image data and separate and encode deep spatial patterns representing geological structures therefrom, and output the geological scene representation map; And a quality processing stream that runs in parallel with the content processing stream, including the second deep neural network encoder, configured to synchronously receive the image data, focus on extracting and quantifying low-level image artifacts and statistical degradation features introduced by sensor noise and atmospheric distortion, and output the image quality assessment map.

[0009] Preferably, the specific generation process of the geological scene representation map includes: The first deep neural network encoder performs layered processing through cascaded convolutional layers and downsampling layers. At each layer, local spatial features are extracted using convolutional kernels, the feature representation capability is enhanced by nonlinear activation functions, and the spatial resolution is gradually reduced and the receptive field is expanded through downsampling operations. Global geological structure and spatial pattern information is abstracted at a higher dimension to generate the geological scene representation map. The geological scene representation map, as a high-dimensional tensor, encodes different semantic attributes related to geological structures through different channels.

[0010] Preferably, the specific process for generating the image quality assessment map includes: The second deep neural network encoder is pre-trained using a self-supervised contrastive learning method based on multimodal data pairs. During the pre-training process, positive sample pairs are formed by acquiring and spatially aligning images from different sensor modalities that were captured at similar times at the same geographical location. The second deep neural network encoder is trained to maximize the similarity of the feature representations of the images in the positive sample pairs, while distinguishing the feature representations of the positive sample pairs from the feature representations of images from other geographical locations. By learning multimodal data, artifacts of multimodal data are suppressed, and an image quality assessment map characterizing the degree of artifacts in the image data is generated.

[0011] Preferably, the specific process for generating the confidence map includes: The image quality assessment map is input to a conversion module, which is configured to map quality features onto a set of predefined quality dimensions, explicitly quantify the reliability of the image data, and generate the multi-channel confidence map. Each channel of the confidence map corresponds to one of the quality dimensions, and the value of each pixel is normalized to represent the reliability level of the corresponding quality dimension. The quality dimensions include edge sharpness for evaluating image clarity, local noise variance for measuring signal purity, and a comprehensive signal-to-noise ratio.

[0012] Preferably, the specific generation process of the single fused scene graph includes: The geological scene representation map and the confidence map are input together into a gated semantic fusion model. The confidence map is used as an explicit external spatial gating signal to control the fusion process. A preliminary pixel-level weighting operation is performed based on the external spatial gating signal. The value of the confidence map at each spatial location is used as a gating weight and directly multiplied by the feature vector of the corresponding geological scene representation map at that spatial location to generate intermediate fusion features. The intermediate fusion features are then used through a collaborative attention mechanism to obtain fusion weights, which are then used to weight and combine the geological scene representation maps to generate a single fused scene map.

[0013] Preferably, the specific process of the collaborative attention mechanism includes: The confidence map is used to calculate a quality attention map to quantify the reliability of data quality; the geological scene representation map is used to calculate a content attention map to quantify the relevance of geological content to the surveying task; and the fusion weight is generated by multiplying the quality attention map and the content attention map pixel by pixel.

[0014] The intelligent mapping system for remote sensing images based on multi-source data fusion includes: Parallel processing module: acquires image data from multiple sensors in the same geological exploration area; performs semantic scene analysis on the image data through a parallel neural network processing framework; uses a first deep neural network encoder to extract a geological scene representation map that represents the geological structures in the image data; and uses a second deep neural network encoder to extract an image quality assessment map that represents the image data at different spatial locations, representing image artifacts and statistical degradation features. Intelligent fusion module: Based on the image quality assessment map, it quantifies the reliability of the image data at each spatial location and generates a multi-channel confidence map; it inputs the geological scene representation map and confidence map corresponding to the image data into a gated semantic fusion model; through the gated semantic fusion model, it uses the confidence map as a spatial gate signal to perform pixel-level adaptive weighting on the geological scene representation map, dynamically adjusts the contribution of the image data in the feature fusion process, and generates a single fused scene map; Data output module: Utilizes a single fused scene map to perform pixel-level semantic segmentation and generate mapping products with geological boundaries.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: By establishing a quality processing stream that operates independently of the content processing stream and runs in parallel, specifically extracting image artifacts and statistical degradation features characterizing the image data at different spatial locations, the quality assessment task is separated from the geological interpretation task. This decoupled design allows the content processing stream to focus more on learning pure geological patterns, while the quality processing stream can more accurately identify and quantify data quality issues. This avoids confusion between "what it is" and "how clear it is," providing clear and unambiguous input for subsequent removal of low-quality data and retention of high-quality information, mitigating the problem of decreased fusion accuracy caused by data quality uncertainty.

[0016] By acquiring and spatially aligning images from different sensor modalities but captured at similar times at the same geographical location to form positive sample pairs, this method cleverly utilizes the prior knowledge that "the same ground feature should have inherent consistency under different sensors." When the model is trained to maximize the similarity of feature representations in the images within the positive sample pairs, it can learn to uncover common and stable ground feature structure information among multimodal data and reduce sensitivity to sensor-specific imaging differences. Therefore, the degree of deviation between the features learned by the model and the ideal state becomes an indicator of the degree of artifacts. This method does not rely on any artificial quality labels, yet it can generate an image quality assessment map characterizing the degree of artifacts in the image data, solving the problem of being unable to effectively and reliably quantify the quality of heterogeneous data in the absence of ground-value quality data.

[0017] By using the confidence map as a spatial gating signal to control the feature fusion process, an effective means is provided to solve the key problem of "how to dynamically and adaptively fuse data with uneven quality". The confidence map provides a clear quantitative basis for the reliability of each pixel on each data source. During fusion, this confidence map acts as an external control signal, enabling pixel-level adaptive weighting of the geological scene representation map. In this way, the contribution of image data in the feature fusion process can be dynamically adjusted, ensuring that the generated single fused scene map utilizes reliable data source information at every pixel, effectively suppressing the propagation of noise and artifacts, thereby improving the accuracy and reliability of surveying and mapping products. Attached Figure Description

[0018] Figure 1 This is a flowchart of the intelligent mapping method for remote sensing images based on multi-source data fusion proposed in an embodiment of this invention application; Figure 2 This is a structural diagram of the intelligent mapping system for remote sensing images based on multi-source data fusion proposed in an embodiment of this invention application; Figure 3 This is a schematic diagram of a surveying product with geological boundaries proposed in an embodiment of this invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figures 1-3 The intelligent mapping method and system for remote sensing images based on multi-source data fusion provided by this invention includes the following specific steps: Image data from multiple sensors in the same geological exploration area is acquired; semantic scene analysis is performed on the image data using a parallel neural network processing framework; a first deep neural network encoder is used to extract a geological scene representation map that represents the geological structures in the image data; and a second deep neural network encoder is used to extract an image quality assessment map that represents the image artifacts and statistical degradation features of the image data at different spatial locations. Based on the image quality assessment map, the reliability of the image data at each spatial location is quantified to generate a multi-channel confidence map. The geological scene representation map and confidence map corresponding to the image data are input into a gated semantic fusion model. The gated semantic fusion model uses the confidence map as a spatial gate signal to perform pixel-level adaptive weighting on the geological scene representation map, dynamically adjusting the contribution of the image data in the feature fusion process, and generating a single fused scene map. Pixel-level semantic segmentation is performed using a single fused scene map to generate mapping products with geological boundaries.

[0021] The technical solution of the present invention will be further described in detail below with reference to specific embodiments.

[0022] Example 1 This application discloses an intelligent mapping method for remote sensing images based on multi-source data fusion. (See attached document.) Figure 1 The specific steps proposed in this invention include: S1. Acquiring image data from multiple sensors in the same geological exploration area; performing semantic scene analysis on the image data using a parallel neural network processing framework, and extracting a geological scene representation map representing the geological structures in the image data using a first deep neural network encoder; S2. Extracting an image quality assessment map representing image artifacts and statistical degradation features of the image data at different spatial locations using a second deep neural network encoder; S3. Quantifying the reliability of the image data at each spatial location based on the image quality assessment map, and generating a multi-channel confidence map; S4. Inputting the geological scene representation map and confidence map corresponding to the image data into a gated semantic fusion model; the gated semantic fusion model uses the confidence map as a spatial gate signal to perform pixel-level adaptive weighting on the geological scene representation map, dynamically adjusting the contribution of the image data in the feature fusion process, and generating a single fused scene map; S5. Performing pixel-level semantic segmentation using the single fused scene map to generate a mapping product with geological boundaries.

[0023] Further, image data from multiple sensors in the same geological exploration area are acquired; semantic scene analysis is performed on the image data using a parallel neural network processing framework, and a geological scene representation map representing the geological structures in the image data is extracted using a first deep neural network encoder; corresponding to step S1 above; the specific implementation process includes: This method acquires multi-source remote sensing imagery covering the same area, specifically including: 1) ASTER L1T-level data products from the Terra satellite, selecting three bands of visible near-infrared (VNIR) and six bands of shortwave infrared (SWIR), with spatial resolution uniformly resampled to 10 meters; 2) GRD products from the C-band synthetic aperture radar (SAR) of the Sentinel-1 satellite, containing data with both VV and VH polarization modes, with an original resolution of 10 meters. After georegistration including radiometric calibration, atmospheric correction (for ASTER data), and orthorectification, these data are stacked into a multi-channel image data cube. For example, for a 512x512 pixel analysis area, the input data is a three-dimensional tensor of size 512x512x11.

[0024] Image data is fed into a parallel neural network processing framework for scene analysis. This framework comprises two parallel processing streams. To improve the framework's computational efficiency, a dynamic computational resource allocation mechanism is introduced within the parallel neural network processing framework. After data input, a lightweight preliminary evaluation network quickly analyzes the overall statistical characteristics of the image. Based on the evaluation results, the parallel neural network processing framework dynamically and asymmetrically adjusts the number of channels in some convolutional layers in the content processing stream and the quality processing stream. If the image is evaluated as low-quality, more resources are allocated to the quality processing stream for more refined artifact analysis. This adaptive resource allocation makes the entire framework more targeted and efficient in processing data of different quality levels. In the first deep neural network encoder in the content processing stream, this encoder adopts a ResNet-50 (Residual Network 50) architecture in this embodiment. To accommodate the 11-channel input data described in this invention, we modified the first convolutional layer of the standard ResNet-50 (usually named conv1), adjusting its input channel count (in_channels) from 3 to 11, while the kernel size (kernel_size) remains 7x7 and the stride (stride) is 2. Other network layer structures, such as residual modules (Conv2_x to Conv5_x), remain consistent with the standard ResNet-50 architecture.

[0025] The input 512x512x11 tensor is processed hierarchically within the ResNet-50 encoder through a series of cascaded convolutional layers and downsampling layers (max pooling layers). In the initial convolutional layer (e.g., Conv1), a 7x7 convolutional kernel is used to extract basic spatial features such as edges and textures. Subsequently, the data flows through multiple residual modules (e.g., Conv2_x, Conv3_x, Conv4_x), where more complex local spatial features are continuously extracted using 3x3 and 1x1 convolutional kernels, and the expressive power of the features is enhanced by the ReLU non-linear activation function. Between each residual module, a downsampling operation with a stride of 2 is used to progressively reduce the spatial resolution of the feature map from 512x512 to 256x256, then to 128x128, and finally to 16x16. The reduction in spatial resolution is accompanied by a systematic expansion of the receptive field, allowing deeper neurons in the network to integrate information from a wider range of original images. This enables the abstraction of global geological structures (such as rock mass outlines and fault strikes) and spatial patterns at a higher dimension. The encoder outputs a high-dimensional tensor as a geological scene representation map. For example, different channels of a 16x16x2048 feature map encode dissimilar semantic attributes related to geological structures (such as different types of altered rocks and dikes), providing rich geological content information for subsequent fusion and analysis.

[0026] By designing a unique parallel neural network processing framework, the confusion that may arise during the learning process of a single network—that is, the difficulty in distinguishing whether the observed signal is a genuine geological structural feature or an artifact introduced by the sensor—is avoided. The content processing stream can focus on in-depth mining and encoding of complex, multi-layered geological spatial patterns, reducing interference from image quality issues. This specialized division of labor enables both encoders to achieve higher performance, ensuring that the geological scene representation information required for subsequent fusion is purer, laying a solid foundation for achieving high-quality intelligent fusion.

[0027] Furthermore, using a second deep neural network encoder, image quality assessment maps representing image artifacts and statistical degradation features of the image data at different spatial locations are extracted; corresponding to step S2 above; the specific process includes: Image data (e.g., a 3D tensor of size 512x512x11) is synchronously input into a second deep neural network encoder in the quality processing stream. In this embodiment, the second deep neural network encoder employs a lightweight convolutional neural network architecture. Specifically, the encoder consists of five cascaded convolutional blocks, each containing a 3x3 convolutional layer, a batch normalization layer, and a ReLU activation function. A 2x2 max-pooling layer is appended after the second and fourth convolutional blocks for downsampling. The encoder's task is to extract and quantify low-level image artifacts and statistical degradation features introduced by sensor noise, atmospheric distortion, and other factors, outputting an image quality assessment map.

[0028] The specific configurations are as follows: Convolutional Block 1: 11 input channels (consistent with the input data), 64 output channels (stride 1, padding 1), followed by a batch normalization layer and a ReLU activation function; Convolutional Block 2: 64 input channels, 128 output channels, followed by a batch normalization layer, a ReLU activation function, and a 2x2 max pooling layer (stride 2); Convolutional Block 3: 128 input channels, 256 output channels, followed by a batch normalization layer and a ReLU activation function; Convolutional Block 4: 256 input channels, 512 output channels, followed by a batch normalization layer, a ReLU activation function, and a 2x2 max pooling layer; Convolutional Block 5: 512 input channels, 512 output channels, followed by a batch normalization layer and a ReLU activation function.

[0029] To enable the encoder to perceive image quality, this method employs a self-supervised contrastive learning method based on multimodal data pairs (e.g., the SimCLR framework) for pre-training. This pre-training process does not rely on any manually labeled quality tags, and is implemented as follows: From a broad geographic archive, images of the same geographic location captured within a similar timeframe (e.g., within a week) are acquired and spatially aligned. The key is that these images originate from different sensor modes with vastly different physical observation characteristics. For example, a pre-processed optical image (such as Landsat-8 or Sentinel-2) with spatial resolution uniformized to that of a SAR image (e.g., uniformized to 10 meters through downsampling or pixel aggregation) is paired with a Sentinel-1 C-band SAR image of the same region to form a positive sample pair. Although their imaging mechanisms are completely different—the former records surface reflectivity and is susceptible to cloud cover; the latter records backscattering coefficients, is sensitive to surface roughness and dielectric constant, and carries speckle noise—they describe the same objective surface entity.

[0030] During training, the second deep neural network encoder is trained to maximize the similarity of the feature representations of the two images in a positive sample pair. Specifically, a contrastive loss function (such as NT-Xent loss) is used to bring the feature vectors of the positive sample pair closer together in the feature space. Simultaneously, the feature representations of the positive sample pair are distinguished from the feature representations of images from other geographical locations (i.e., negative samples). In this embodiment, the contrastive loss function specifically adopts the NT-Xent loss function with a temperature coefficient τ, where τ is set to 0.5. The pre-training process uses the AdamW optimizer, with an initial learning rate of 0.001, a weight decay coefficient of 0.01, and a batch size of 64. The second deep neural network encoder is pre-trained for 50 rounds on a multimodal image pair dataset constructed from various publicly available remote sensing datasets (such as Million-AID) to enable it to have robust image quality perception capabilities. The construction process of the multimodal image pair dataset used for pre-training is as follows: 1) Data source: Select publicly available archives of Million-AID and GF-2 satellite images. 2) Positive Sample Pair Construction: The dataset is traversed, and GF-2 panchromatic images and Sentinel-1 C-band SAR images with a spatial overlap greater than 80% and an imaging time interval of less than 7 days are selected. Georegistration is performed using the image's built-in RPC model, and sub-pixel-level fine alignment is achieved using SIFT feature point matching to ensure spatial alignment error is within 1 pixel. The aligned image pairs are then cropped into 256x256 pixel tiles to form positive sample pairs. 3) Negative Sample Selection: For a given positive sample pair, all other image tiles within the training batch are considered negative samples.

[0031] By learning to align these multimodal data with their diverse physical observation characteristics, the learning process incentivizes the encoder to focus more on common ground feature information across modalities. Since artifacts unique to each modality (such as thin clouds in optical images and speckle noise in SAR images) lack feature consistency, the model tends to reduce the weight of these inconsistent features during the learning process. This helps to construct a high-dimensional feature space that is more robust to common and stable ground feature information.

[0032] After pre-training, the encoder is deployed in the quality processing stream for inference. When the image data cube to be mapped is input, the encoder maps the features of the image data pixel by pixel into the constructed feature space. For regions with good image quality, their content conforms to the common structural paradigm learned by the model during pre-training, so the output feature vectors exhibit stable and convergent activation patterns in the feature space. Conversely, for regions with artifacts or statistical degradation (e.g., cloud cover), their input features do not match all known common ground features, causing the encoder to produce anomalous or discrete feature activations at these locations. Therefore, the activation patterns of the feature vectors in the high-dimensional feature map output by the encoder can reflect the deviation between the input data and the ideal model. In this embodiment, the encoder outputs a high-dimensional feature map of size 128x128x512. To transform this high-dimensional abstract feature into a quantitative quality metric that directly guides subsequent steps, the feature map needs to be processed to generate an "image quality assessment map". The specific processing step is as follows: Calculate the L2 norm of the 512-dimensional feature vector at each spatial location (i, j) to form an initial 128x128 quality score map. This norm value can serve as an indicator of the degree of deviation of the feature activation pattern. An increase in its value usually means that the feature activation deviates from the common structural paradigm learned by the model during the pre-training stage, and can reflect the degree of artifacts or statistical degradation at the corresponding location to a certain extent. Its value is negatively correlated with data quality. This initial quality score map is the final image quality evaluation map generated in this step and used as input to the transformation module.

[0033] By constructing positive and negative sample pairs using massive, easily accessible, and manually annotated multimodal remote sensing images, the model can learn, in an unsupervised manner, general features that are less sensitive to artifacts inherent in different sensors (such as clouds, shadows, and speckle noise) and represent the essence of ground features. This gives the trained encoder powerful generalization ability, enabling it to identify various image quality defects. This not only reduces the cost and time required for model training but also improves the accuracy and robustness of quality assessment.

[0034] Furthermore, based on the image quality assessment map, the reliability of the image data at each spatial location is quantified to generate a multi-channel confidence map; corresponding to step S3 above; the specific implementation process includes: The quality processing stream has generated an image quality assessment map characterizing image artifacts and statistical degradation features. This image quality assessment map is input to a transformation module, which in this embodiment is a lightweight neural network. Its core function is to decouple and map the high-dimensional, abstract quality features extracted by the encoder to a low-dimensional, more interpretable quality representation space. The different dimensions of this space functionally correspond to key quality factors affecting the fusion effect. Specifically, the structure is as follows: a 3x3 convolutional layer with a stride of 2 reduces the feature map spatial resolution from 128x128 to 64x64; then another 3x3 convolutional layer with a stride of 2 reduces it to 32x32; finally, another 3x3 convolutional layer with a stride of 2 reduces it to 16x16. After these spatial reduction operations, two consecutive 1x1 convolutional layers are connected.

[0035] In this embodiment, the quality representation space is defined as containing three dimensions corresponding to key quality factors of the fusion effect. The training objective of the transformation module is to ensure that its three output channel values ​​can be regressed to the three pseudo-labels generated for the training data. The correspondence between these three channels and the quality dimensions is fixed: Channel 1 (Sharpness): The output value of this channel is trained to fit the "sharpness" pseudo-label calculated by the variance of the Laplacian operator. Its activation value is positively correlated with the sharpness of high-frequency details in the image.

[0036] Channel 2 (Noise Level): The output value of this channel is trained to fit the "noise" pseudo-label calculated through Gaussian filtering residuals. Its activation value is positively correlated with the local statistical inconsistency of the signal.

[0037] Channel 3 (Signal-to-Noise Ratio): The output value of this channel is trained to fit a pseudo-label of "signal-to-noise ratio" calculated by the ratio of the mean to the standard deviation within a local window, used to capture overall signal quality. After training, the three channels of the 16x16x3 tensor output by the conversion module correspond to the three quality dimensions of sharpness, noise level, and signal-to-noise ratio as clearly defined above.

[0038] The training process of the conversion module is as follows: 1) Training dataset: A quality assessment dataset containing 10,000 remote sensing image patches was constructed. For each patch, pseudo-labels were generated in the following ways: a) Sharpness label: A 5x5 Laplacian convolution kernel was used to convolve the patch, and the variance of the output result was calculated as the sharpness score; b) Noise label: A Gaussian filter with a kernel size of 3x3 and a standard deviation of 1.5 was applied to the patch, and the original patch was subtracted from the filtered result to obtain the residual image. The standard deviation of the pixel values ​​in the residual image was calculated as the noise level score; c) Signal-to-noise ratio label: Within a local 5x5 window of the image, the ratio of the signal strength of the center pixel (approximately the mean of pixels within the window) to the noise strength (approximately the standard deviation of pixels within the window) was calculated, and the average ratio calculated for all windows of the entire patch was used as the signal-to-noise ratio score of the patch. All scores were normalized to the [0, 1] interval. 2) Loss function: The mean squared error loss function is used to calculate the difference between the network output and the pseudo label. The Adam optimizer (learning rate of 0.001) is used to iteratively update the weight parameters of the two 1x1 convolutional layers in this transformation module through the backpropagation algorithm until the model converges.

[0039] In the specific data processing flow, the 128x128x1 image quality assessment map flows through the conversion module. Since the input is a single-channel image, the module's primary task is to extract and construct a multi-dimensional feature representation from this quality score map. Therefore, the module first performs feature extraction and channel expansion on the single-channel input through a set of convolutional layers (e.g., a 3x3 convolutional layer, expanding the number of channels from 1 to 64) to aggregate neighborhood information and generate a richer intermediate feature map. Subsequently, this intermediate feature map passes through two consecutive 1x1 convolutional layers: the first 1x1 convolutional layer reduces the number of channels from 64 to 16 and uses the ReLU activation function; the second 1x1 convolutional layer reduces the number of channels from 16 to 3 and does not use an activation function. Through this series of operations, the single-channel quality assessment map is finally mapped to a size and geological scene representation. Figure 1 A 16x16 intermediate quality tensor is generated, containing three channels with clearly defined physical meanings (sharpness, noise level, and signal-to-noise ratio). A global average pooling layer is connected after the second 1x1 convolutional layer to compress the 16x16x3 feature map into a vector of length 3. This vector is then passed through a fully connected layer (FC layer) and a softmax function to generate three weight values, representing the relative importance of the three quality dimensions for the current input image. These weights are broadcast and multiplied by the 16x16x3 intermediate quality tensor to dynamically weight the three channels, thereby generating a weighted quality metric that integrates the characteristics of the data itself, improving the accuracy and adaptability of the confidence map in varying scenarios.

[0040] The output of this operation is an intermediate quality tensor of size 16x16x3. To generate the confidence map, each value in this tensor is normalized. Specifically, a sigmoid activation function maps the value of each pixel in each channel to a closed interval [0, 1]. This normalized value represents the confidence level for the corresponding quality dimension.

[0041] Finally, a multi-channel confidence map of size 16x16x3 is generated, where each channel corresponds to a quality dimension. For example, a value vector [0.95, 0.21, 0.91] at pixel location (i, j) physically means that the edge sharpness of the original image at this spatial location has extremely high confidence (0.95), but may be significantly contaminated by noise (the confidence of the local noise variance is only 0.21), while the confidence of its overall signal-to-noise ratio is still relatively high (0.91). This map provides spatially varied and refined quality weighting guidance for subsequent feature fusion.

[0042] By introducing a transformation module, the abstract quality features output by the quality encoder are mapped to a set of surrogate dimensions associated with traditional image quality assessment operators, generating a multi-channel confidence map. This multi-channel structure provides richer and more refined control signals for subsequent fusion, enabling the fusion model to apply differentiated weights based on different quality dimensions. This heuristic-based, multi-dimensional confidence approximation improves the interpretability of the entire method and the granularity of the fusion control.

[0043] Further, the geological scene representation map and confidence map corresponding to the image data are input into a gated semantic fusion model; the gated semantic fusion model uses the confidence map as a spatial gating signal to perform pixel-level adaptive weighting on the geological scene representation map, dynamically adjusting the contribution of the image data in the feature fusion process, and generating a single fused scene map; corresponding to step S4 above; the specific implementation process includes: The geological scene representation map (e.g., a high-dimensional tensor with a size of 16x16x2048) and the multi-channel confidence map (a tensor with a size of 16x16x3) are jointly input into the gated semantic fusion model.

[0044] The confidence map is used here as an explicit external spatial gating signal to perform preliminary pixel-level filtering and weighting of geological content features. To achieve this, a spatial smoothing preprocessing is first performed, using a 3x3 Gaussian filter to slightly convolve and smooth each channel of the confidence map. In real remote sensing images, image quality degradation caused by factors such as clouds, fog, and sensor stripe noise usually has spatial continuity rather than isolated pixels. Smoothing can reduce noise or abrupt values ​​that may exist in the confidence map, making the generated spatial gating signal more consistent and stable in local areas, thereby avoiding the introduction of unnatural high-frequency artifacts in the subsequent weighting process. The model processes the 16x16x3 confidence map through two 1x1 convolutional layers. The role of this convolutional layer is to integrate the channel information representing the three quality dimensions (edge ​​sharpness, local noise variance, and signal-to-noise ratio) into a gating weight tensor that matches the number of channels in the geological scene representation map. Specifically, the first 1x1 convolutional layer expands the number of channels from 3 to an intermediate dimension (e.g., 256), followed by a ReLU activation function to introduce non-linearity and perform initial integration of quality features; the second 1x1 convolutional layer maps the number of channels from the intermediate dimension to 2048, followed by a Sigmoid activation function to ensure that the generated gating weight values ​​fall within the range of 0 to 1.

[0045] The 16x16x2048 gating tensor is multiplied element-wise with the 16x16x2048 geological scene representation map. In this operation, the value of the confidence map at each spatial location (i, j) is transformed and used as the gating weight at that location, which is then directly multiplied by the 2048-dimensional feature vector of the corresponding geological scene representation map at that location.

[0046] Suppose that a thin cloud cover exists in a certain pixel area of ​​the original image, resulting in a low confidence map value for the corresponding pixel in that area (e.g., [0.4, 0.5, 0.3]). After a 1x1 convolution transformation, the 2048 channel values ​​of the generated gate weights at that pixel location will also be correspondingly low. When multiplied with the original geological scene representation map, the overall amplitude of the geological feature vector at that pixel location will be significantly suppressed. Conversely, for a region with clear image quality and no noise, the corresponding gate weight value will be higher, allowing its geological features to be preserved or even enhanced.

[0047] The above yields an intermediate fusion feature (still 16x16x2048 in size), which serves as a geological content representation after initial "purification" of data quality. The feature contribution of low-quality areas has been effectively suppressed. Then, a more refined set of final fusion weights is generated using a collaborative attention mechanism, which considers both data quality and the correlation between geological content.

[0048] Quality Attention Map Calculation: The quality attention map is calculated using a confidence map (16x16x3 in size). This quality attention submodule consists of a 3x3 convolutional layer (3 to 16 channels, ReLU activation function, stride 1, padding 1) and a 1x1 convolutional layer (16 to 1 channels, Sigmoid activation function) concatenated. This structure first performs preliminary feature integration on multi-channel quality information, and then refines it into a single-channel quality attention map (16x16x1 in size). Each pixel value (range 0-1) in this map comprehensively quantifies the overall reliability of the multi-source data quality at the corresponding spatial location.

[0049] Content Attention Map Calculation: A content attention map is calculated using a geological scene representation map (16x16x2048 pixels). The structure of this content attention submodule is similar to that of the quality attention submodule, consisting of a 3x3 convolutional layer (channel count reduced from 2048 to 256, ReLU activation function, stride 1, padding 1) and a 1x1 convolutional layer (channel count reduced from 256 to 1, Sigmoid activation function) concatenated. This structure compresses and refines high-dimensional content features into a single-channel content attention map (16x16x1 pixels). Each pixel value (range 0-1) in this map is designed to quantify the saliency or information entropy of the geological features at that location, i.e., the complexity and discriminative power of the features.

[0050] The final fusion weight W_final is generated collaboratively from the quality attention map A_q (16x16x1) and the content attention map A_c (16x16x1). The specific calculation formula is: W_final = A_q ⊙ A_c, where ⊙ represents element-wise multiplication. Each pixel value in this weight map simultaneously contains the answers to the questions: "Is the data quality at this location reliable?" and "Is the geological content at this location important?"

[0051] This set of fusion weights (16x16x1) was applied to the geological scene representation map. Through a broadcast mechanism, the single-channel fusion weight map was element-wise multiplied with the 2048-channel geological scene representation map. This operation achieved adaptive weighting of geological features, dynamically and pixel-level adjusting the contribution of the original image data at each spatial location to the feature fusion process.

[0052] The output of this process is a single fused scene map (16x16x2048 pixels), which is highly optimized, effectively suppresses artifacts, preserves real features, and highlights the unified feature expression of key geological information, laying a solid foundation for the subsequent generation of high-precision mapping products.

[0053] This model not only utilizes confidence maps as external spatial gating signals for initial data quality screening, but also employs a collaborative attention mechanism to consider both data reliability and content relevance. The model dynamically assesses the reliability of data quality and the importance of geological content to the current mapping task at each pixel location. By combining these two factors to generate fusion weights, the model can make more intelligent decisions. This adaptive weighting mechanism, driven by both quality and content, ensures that the final fused scene map not only has fewer artifacts but also highlights more valuable information for geological interpretation, thus achieving better feature fusion results.

[0054] Furthermore, pixel-level semantic segmentation is performed using a single fused scene map to generate a mapping product with geological boundaries; this corresponds to step S5 above; the specific implementation process includes: The single fused scene map is input into a decoder network specifically designed for pixel-level decoding and segmentation. In this embodiment, the core task of this decoder network architecture is to gradually and hierarchically restore the spatial resolution (e.g., 512x512 pixels) of the original input image from the low spatial resolution, high semantic dimension feature map through a series of upsampling operations (such as transposed convolution or bilinear interpolation) and convolutional layers. This decoder network adopts a U-Net-like structure and contains four upsampling decoding blocks to progressively restore the spatial resolution.

[0055] Decoding Block 1: The input is a 16x16x2048 feature map. First, it passes through a 2x upsampling layer (implemented via transposed convolution, with a 4x4 kernel and a stride of 2) to obtain a 32x32x1024 feature map. Then, through skip connections, this feature map is concatenated with the feature map (32x32x1024) output from the Conv4_x residual block of the first deep neural network encoder (ResNet-50). The concatenated feature map then passes through two consecutive 3x3 convolutional layers (each followed by batch normalization and a ReLU activation function), finally outputting a 32x32x512 feature map.

[0056] Decoding blocks 2, 3, and 4: Similar to decoding block 1, the feature map resolutions are sequentially increased to 64x64, 128x128, and 256x256, and then skip connections are made with the corresponding resolutions of Conv3_x, Conv2_x in the ResNet-50 encoder and the feature maps output from the initial convolutional layer.

[0057] Final output layer: After the resolution is restored to 256x256 in the last decoding block, it is restored to 512x512 by a 2x upsampling.

[0058] To ensure the clarity and positioning accuracy of geological boundaries in the generated mapping products, a key mechanism of this decoder network is the use of skip connections. In each upsampling stage, the decoder not only processes features from the previous deep network but also, through skip connections, receives and stitches together high-resolution feature maps from the first deep neural network encoder (ResNet-50) in the content processing stream at the corresponding spatial resolution. For example, when the decoder upsamples the feature map to 128x128 resolution, it stitches together a shallow feature map from the ResNet-50 encoder, also at 128x128 resolution. This operation directly transmits information captured early in the encoder, rich in fine spatial details (such as the location of micro-fractures and lithological contact zones), to the decoder, effectively compensating for the spatial positioning information lost during continuous downsampling in the deep encoding process.

[0059] After multiple layers of upsampling and feature fusion, the final layer of the decoder network is typically a 1x1 convolutional layer followed by a Softmax activation function. This 1x1 convolutional layer transforms the high-dimensional feature channels (e.g., 64) into a number of channels equal to the predefined number of geological categories (e.g., if the goal is to identify four alteration zones and background rocks, this is transformed into five channels). The Softmax function then calculates the values ​​for each pixel across these five channels, outputting a probability distribution indicating the probability that the pixel belongs to each geological category.

[0060] By taking the category with the highest probability at each pixel location as its final label, the model outputs a raster-format semantic segmentation map. This map is the mapping product with geological boundaries generated by this method. Each pixel in the map is assigned a clear geological attribute (such as "sericitization alteration zone" or "propylitization alteration zone"). The boundaries between different geological units become clear, sharp, and spatially continuous due to the introduction of the jump connection mechanism, which highly conforms to the actual requirements of geological interpretation.

[0061] To achieve end-to-end unified optimization, the entire mapping method is constructed as a single, differentiable neural network model. In this model, the weights of the second deep neural network encoder and its subsequent transformation module are frozen after pre-training, serving only as a static quality assessment branch. The first deep neural network encoder for content processing, the gated semantic fusion model (containing two sub-modules with a collaborative attention mechanism), and the decoder network for the data output module together form a trainable backbone network. During training, multi-source remote sensing image data cubes and corresponding geological category ground truth map images are used as input pairs. The data flows through the backbone network, and the decoder ultimately outputs a predicted probability map with the same spatial size as the ground truth map. The predicted probability map and the ground truth map are input into the cross-entropy loss function to calculate the total loss. Subsequently, through backpropagation, the gradient of this loss is propagated back from the decoder, synchronously updating all trainable parameters of the decoder, the gated semantic fusion model (including quality and content attention calculations), and the first deep neural network encoder. Specifically, the end-to-end training process uses the AdamW optimizer with an initial learning rate of 1e-4 and a weight decay coefficient of 0.01. The batch size during training is set to 8. To ensure stable convergence, a cosine annealing learning rate scheduling strategy is adopted, dynamically adjusting the learning rate over a total of 100 training epochs.

[0062] This invention effectively solves the problems of poor fusion results and low accuracy of mapping products when processing multi-source remote sensing data containing noise, artifacts, or missing data by constructing an intelligent mapping method based on multi-source data fusion. Through parallel content and quality dual-processing streams, a second deep neural network encoder independently evaluates image quality at the pixel level and generates an explicit confidence map. This confidence map acts like an intelligent filter, dynamically and adaptively adjusting the contribution of data from different sources at various geospatial locations within the gated semantic fusion model. This ensures that only high-confidence feature information plays a dominant role in the mapping results, thereby suppressing noise and artifact interference and improving the purity and reliability of the single fused scene map. The generated mapping products thus have clearer and more accurate geological boundaries, significantly improving the reliability and accuracy of overall geological interpretation and providing higher-quality data support for geological exploration.

[0063] Example 2 This second embodiment demonstrates the specific implementation process of the present invention applied to a parallel processing module in geological mapping; see reference Figure 2 The specific implementation method is as follows: The system acquires and preprocesses multi-source remote sensing image data covering the target exploration area, generating an initial image data cube. This cube has a three-dimensional tensor structure with dimensions (512, 512, 11) and a 32-bit floating-point data type. Its 11 channels correspond to preprocessed data layers from different sensors (e.g., the optical bands of ASTER and the radar polarization channels of Sentinel-1). This image data cube is fed as initial input into a parallel processing module and copied to simultaneously enter both the content processing and quality processing streams. In the content processing stream, the cube flows through a first deep neural network encoder, outputting a three-dimensional tensor with a data structure of "geological scene representation map" (16, 16, 2048) and a 32-bit floating-point data type. Each channel represents a deeply abstract geological scene representation map. Simultaneously, in the quality processing stream, the cube flows through a second deep neural network encoder, outputting a two-dimensional tensor with a data structure of "image quality assessment map" (128, 128, 1).

[0064] Example 3 This third embodiment demonstrates the specific implementation process of the present invention applied to the intelligent fusion module and data output module in geological mapping; see reference Figure 2 and Figure 3 The specific implementation method is as follows: It receives two outputs from the previous stage (parallel processing module) as its input data. The data structures of these two inputs are: a geological scene representation map, a three-dimensional floating-point tensor with dimensions (16, 16, 2048); and an image quality assessment map, a floating-point tensor with dimensions (128, 128, 1).

[0065] Within the intelligent fusion module, data flow first involves processing the image quality assessment map. This tensor is fed into a transformation module and converted into a three-dimensional tensor with a "multi-channel confidence map" data structure, dimensions (16, 16, 3), and data type 32-bit floating-point. This confidence map, along with the geological scene representation map, is input into a gated semantic fusion model. Within this model, a series of gated and collaborative attention weighting operations based on the confidence map generate an optimized single feature representation. The output of the intelligent fusion module is a three-dimensional tensor with a "single fused scene map" data structure, dimensions (16, 16, 2048), and data type 32-bit floating-point.

[0066] The single fused scene map is passed to the data output module. Inside this module, the single fused scene map serves as input and flows through a decoder network. This network progressively restores the spatial dimension of the input tensor from (16, 16) to (512, 512) of the original image through a series of upsampling, convolutions, and skip connections with feature maps from the content processing stream encoder, while simultaneously reducing the channel dimension from 2048. At the end of the decoder network, a temporary probability map is generated through a 1x1 convolutional layer and a Softmax activation function. Its data structure is a three-dimensional tensor with dimensions (512, 512, N_classes), where N_classes is the predefined number of geological categories.

[0067] The module generates a mapping product by assigning the most probable category label to each pixel. The output data structure of the data output module is a "geological boundary mapping product," which is also a semantic segmentation map. Figure 3 The product's data structure is a two-dimensional matrix (raster image), with the same spatial dimensions as the initial input image data cube, i.e., (512, 512). Its data type is 8-bit unsigned integer, and each integer value in the matrix represents a specific geological category label.

[0068] By clearly dividing the complex methodology into three functional modules—parallel processing, intelligent fusion, and data output—this system can automatically complete the entire process from inputting multi-source raw image data to outputting high-precision geological mapping products. This reduces reliance on manual intervention and improves the efficiency and automation level of geological mapping. Compared to traditional mapping workflows, this system demonstrates better robustness and accuracy when processing complex, noisy multi-source data, providing rapid and reliable technical support for fields such as geological exploration, resource assessment, and environmental monitoring.

[0069] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent surveying and mapping of remote sensing images based on multi-source data fusion, characterized in that, The method comprises: acquiring image data from multiple sensors in the same geological exploration area; performing semantic scene analysis on the image data through a parallel neural network processing framework, using a first deep neural network encoder to extract a geological scene representation map representing geological structures in the image data, and using a second deep neural network encoder to extract an image quality evaluation map representing image artifacts and statistical degradation features at different spatial positions in the image data; quantifying the reliability of the image data at each spatial position based on the image quality evaluation map to generate a multi-channel confidence map; inputting the geological scene representation map and the confidence map corresponding to the image data into a gated semantic fusion model; and 2.The multi-source data fusion based remote sensing image intelligent mapping method according to claim 1, characterized in that, using the single fusion scene map to perform pixel-level semantic segmentation to generate a surveying and mapping product with geological boundaries. The specific structure of the parallel neural network processing framework comprises: a content processing stream containing the first deep neural network encoder, configured to receive the image data and separate and encode deep spatial patterns representing geological structures from the image data to output the geological scene representation map; 3.The multi-source data fusion based remote sensing image intelligent mapping method according to claim 1, characterized in that, and a quality processing stream running in parallel with the content processing stream and containing the second deep neural network encoder, configured to synchronously receive the image data and focus on extracting and quantifying low-level image artifacts and statistical degradation features introduced by sensor noise and atmospheric distortion to output the image quality evaluation map. The specific generation process of the geological scene representation map comprises: 4.The multi-source data fusion based remote sensing image intelligent mapping method according to claim 1, characterized in that, The first deep neural network encoder performs hierarchical processing through cascaded convolutional layers and down-sampling layers; at each level, local spatial features are extracted using convolutional kernels, feature expression capabilities are enhanced through nonlinear activation functions, and spatial resolution is gradually reduced and receptive fields are expanded through down-sampling operations to abstract global geological structures and spatial patterns in higher dimensions to generate the geological scene representation map; the geological scene representation map is a high-dimensional tensor, and different channels encode distinct semantic attributes related to geological structures. The specific generation process of the image quality evaluation map comprises: 5.The multi-source data fusion based remote sensing image intelligent mapping method according to claim 1, characterized in that, The second deep neural network encoder is pre-trained using a self-supervised contrastive learning method based on pairs of multi-modal data; in the pre-training process, image pairs from different sensor modalities but captured at the same geographical location at similar times are obtained and spatially aligned to form positive sample pairs; the second deep neural network encoder is trained to maximize the similarity of feature representations of the images in the positive sample pairs, while distinguishing the feature representations of the positive sample pairs from those of images from other geographical locations; by learning multi-modal data, multi-modal artifacts are suppressed to generate the image quality evaluation map representing the degree of artifacts in the image data. The specific generation process of the confidence map comprises: The image quality evaluation map is input into a conversion module configured to map quality features onto a set of predefined quality dimensions, explicitly quantify the reliability of the image data, and generate the multi-channel confidence map; each channel of the confidence map corresponds to a quality dimension, and the value of each pixel is normalized to represent the reliability level of the corresponding quality dimension; the quality dimensions include edge sharpness for evaluating image sharpness, local noise variance for measuring signal purity, and comprehensive signal-to-noise ratio. 6.The multi-source data fusion based remote sensing image intelligent mapping method according to claim 1, characterized in that, The specific generation process of the single fused scene map includes: The geological scene representation map and the confidence map are jointly input into a gated semantic fusion model; the confidence map is used as an explicit external space gating signal to control the fusion process, and a preliminary pixel-level weighting operation based on the external space gating signal is performed; the value of the confidence map at each spatial location is used as a gating weight to directly multiply the feature vector of the corresponding spatial location of the geological scene representation map to generate intermediate fusion features; the intermediate fusion features are subjected to a cooperative attention mechanism to obtain fusion weights to weight and combine the geological scene representation map to generate a single fused scene map. 7.The multi-source data fusion based remote sensing image intelligent mapping method according to claim 6, characterized in that, The specific process of the cooperative attention mechanism includes: The confidence map is used to calculate a quality attention map to quantify the reliability of data quality; the geological scene representation map is used to calculate a content attention map to quantify the relevance of geological content to the mapping task; the fusion weights are generated by pixel-by-pixel multiplication of the quality attention map and the content attention map.

8. The intelligent remote sensing image surveying and mapping system based on multi-source data fusion, characterized in that, It includes: Parallel processing module: obtaining image data from multiple sensors in the same geological exploration area; Performing semantic scene analysis on the image data through a parallel neural network processing framework, using a first deep neural network encoder to extract a geological scene representation map representing geological structures in the image data, and using a second deep neural network encoder to extract an image quality evaluation map representing image artifacts and statistical degradation features at different spatial locations of the image data; Intelligent fusion module: based on the image quality evaluation map, quantifying the reliability of the image data at each spatial location to generate a multi-channel confidence map; inputting the corresponding geological scene representation map and confidence map of the image data into a gated semantic fusion model; using the confidence map as a spatial gating signal by the gated semantic fusion model to perform pixel-level adaptive weighting on the geological scene representation map, dynamically adjusting the contribution of the image data in the feature fusion process to generate a single fused scene map; Data output module: using the single fused scene map to perform pixel-level semantic segmentation to generate a mapping product with geological boundaries.

Images