Multi-source land coverage data fusion method based on conflict index and self-supervised reconciliation

Abstract

The invention belongs to the technical field of remote sensing land coverage mapping, and particularly relates to a multi-source land coverage data fusion method based on a conflict index and self-supervised reconciliation. Comprising the following steps of 1, data preparation and data preprocessing; step 2, feature construction; 3, patch data construction and enhancement are carried out; 4, model construction and strategy training; and 5, reasoning the whole graph and outputting a fusion result. According to the method, the precision of land coverage data can be improved, the space consistency and the distinguishing and repairing capability of a high-conflict region can be enhanced, the region perception and self-adaptability of a fusion model are realized, the dependence on a large amount of labeled data is reduced, and on the basis of a fusion thought of deep learning, conflict perception and self-supervision optimization, the method is suitable for large-scale popularization and application. By constructing a teacher-student double-model architecture, the model is guided to seek balance between global consistency and local difference.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing land cover mapping, in particular to a multi-source land cover data fusion method based on conflict index and self-supervision reconciliation. Background Art

[0002] Land cover refers to the natural or artificially modified physical and biological cover types on the Earth's surface, such as forests, farmland, water bodies, and cities. Land cover data is one of the fundamental datasets in geographic information science, accurately reflecting the distribution of surface types and their temporal and spatial variations. It is widely used in numerous fields, including water resource evolution analysis, geological hazard monitoring, land degradation and desertification assessment, environmental quality evaluation, and ecological security pattern research.

[0003] With the advancement of remote sensing technology, an increasing number of land cover products have been released. Faced with the coexistence of land cover products from multiple sources, a single land cover data set often struggles to achieve both accuracy and versatility. Data fusion can be used to improve data accuracy. Existing methods for multi-source land cover data fusion primarily include rule-based decision-making, probabilistic modeling, and machine learning.

[0004] A common method is majority voting, which uses the category with the highest frequency of occurrence of the same pixel in multiple data points as the final result. However, this method is highly sensitive to weights and tends to amplify the influence of outliers in areas with large errors.

[0005] To address the classification inconsistencies and uncertainties between multi-source data, the Dempster-Shafer evidence theory method has been applied to land cover data fusion. However, when faced with severe classification conflicts between multiple land cover data sets, the traditional Dempster-Shafer evidence theory method is susceptible to unstable fusion results due to the influence of the trust function design. It also requires extensive prior knowledge to design rules and models, which can be difficult to implement when data is scarce or domain knowledge is insufficient.

[0006] While the multi-source land use and land cover data fusion method considers spatial correlation, it does not address conflicting areas of classification across multiple sources. Instead, it uses a modified Dempster-Shafer evidence theory to fuse areas of low consistency. By integrating the credibility of multiple land cover products, it determines land cover type, effectively addressing areas of uncertainty and improving classification accuracy. However, this consistency analysis focuses only on surface consistency, ignoring the consistency of spatial context between pixels and fuzzy boundaries.

[0007] Therefore, we propose a multi-source land cover data fusion method based on conflict index and self-supervised reconciliation to solve the above problems. Summary of the Invention

[0008] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of this application to avoid blurring the purpose of this section, the abstract and the title of the invention, and such simplifications or omissions should not be used to limit the scope of the present invention.

[0009] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions: A multi-source land cover data fusion method based on conflict index and self-supervised reconciliation includes the following steps: Step 1: Data preparation and data preprocessing: Download land cover data and multiple geographic environmental factors, and reclassify and process the source land cover data; Step 2: Feature construction: One-hot encoding is performed on each category of each land cover data, each geographical environmental factor is stacked into a channel, and the conflict index is constructed and treated as a separate channel; Step 3: Patch data construction and enhancement: Patches are extracted from the study area using a sliding window method. Each patch contains multiple channel input features and a corresponding label map. Step 4: Model construction and training strategy: Build a pair of deep learning semantic segmentation models with identical structure and complementary functions: teacher model and student model; The teacher model includes one-hot encoding of multiple land cover data categories, multiple normalized geographic environment factors, and a conflict index; Self-supervised uncertainty guidance module: Utilize the output of the teacher model and the conflict index to automatically generate high-confidence pseudo labels; The training data of the student model includes the original patches from the true labels and the high-confidence pseudo-label patches selected based on the teacher's softmax output and the conflict index; Step 5: Whole-image reasoning and fusion result output: The reasoning results of teachers and students are integrated and saved.

[0010] As a preferred solution of the multi-source land cover data fusion method based on conflict index and self-supervised reconciliation described in the present invention, in the step 2, the function formula of the conflict index is: ; in, Represents pixels The conflict index value reflects the inconsistency of the classification results in the multi-source layer. Represents pixels The mode frequency of Represents pixels The classification consistency with the pixels in its neighborhood, For pixels The classification information entropy of .

[0011] As a preferred solution of the multi-source land cover data fusion method based on conflict index and self-supervised reconciliation described in the present invention, The formula is: ; in, is the number of occurrences of the category that most frequently appears in all input layers for this pixel. is the total number of data. The higher the mode frequency, the more consistent the multi-source classification results at that location and the lower the degree of conflict.

[0012] As a preferred solution of the multi-source land cover data fusion method based on conflict index and self-supervised reconciliation described in the present invention, The formula is: ; for The neighborhood set of is the indicator function, if , then it is 1, otherwise it is 0. This item is used to measure the local smoothness of the pixel in space. If the classification consistency between the pixel and its neighborhood is low, it means that it may be in a boundary or conflict transition zone.

[0013] As a preferred solution of the multi-source land cover data fusion method based on conflict index and self-supervised reconciliation described in the present invention, The formula is: ; in, Indicates that the pixel is marked as k The probability of the class, kis the total number of categories, and the information entropy reflects the degree of distribution disorder of the classification results at that location. The higher the entropy, the more uncertain and unstable the classification of the pixel is.

[0014] As a preferred solution of the multi-source land cover data fusion method based on conflict index and self-supervised reconciliation described in the present invention, the teacher model training data all comes from the real labels of visual interpretation, and is optimized using a cross-entropy loss function with class weights. The loss function is defined as: ; in, Indicates the total number of categories; Indicates that the value is 1 when the label is category c, otherwise it is 0; Represents the model for category c The predicted probability of Representation category c The weight of Represents the overall loss function, which is weighted and summed according to the difference in the number of class samples. The class weight is obtained by normalizing the inverse of the frequency of each class pixel.

[0015] As a preferred solution of the multi-source land cover data fusion method based on conflict index and self-supervised reconciliation described in the present invention, the teacher model and the student model both use the U-Net network as the basic framework, consisting of a symmetrical encoder and decoder, constructing multi-channel input by introducing multi-source land cover data and geographic environmental factors, and performing differentiated design in training strategies and data sources.

[0016] As an optimal solution of the multi-source land cover data fusion method based on conflict index and self-supervised reconciliation described in the present invention, the teacher model relies on real labels to provide stable and reliable full-map classification results, which is suitable for processing areas with strong consistency; the student model, based on the introduction of pseudo labels, has stronger adaptability and repair capabilities, and can improve classification performance in areas with significant conflicts and blurred boundaries. Finally, through a decision-making mechanism guided by conflict perception, the advantages of the two are integrated to achieve a land cover fusion optimization strategy that emphasizes both stability and flexibility.

[0017] As a preferred solution of the multi-source land cover data fusion method based on conflict index and self-supervision reconciliation described in the present invention, the first step of the self-supervised uncertainty guidance module is Softmax entropy calculation. The information entropy is calculated pixel by pixel through the softmax probability map output by the teacher model to measure the prediction uncertainty of the model at each pixel point. The higher the entropy value, the lower the classification confidence of the model at that pixel; conversely, low entropy pixels can be regarded as high-confidence prediction results. The second step is to read the conflict index. The conflict index is calculated based on the inconsistency of the classification results of multi-source land cover products. It can reflect the degree of conflict between different data sources at a certain pixel. By reading the conflict index map and jointly analyzing it with the entropy value, high-conflict and high-uncertainty areas can be distinguished from ordinary areas, providing a basis for subsequent pseudo-label screening. The third step is to apply double thresholds, which sets two screening criteria: entropy threshold and conflict index threshold; The fourth step is to filter pseudo labels. For pixels that meet the conditions, the category with the highest probability is directly selected from the softmax output of the teacher model as the pseudo label. The last step is mixing with the true label. In order to maintain the diversity and stability of the training data, the module mixes the generated pseudo-label patch with the true label patch obtained by visual interpretation in a 1:1 ratio to construct the training set of the student model.

[0018] Compared with the existing technology, the beneficial effects of the present invention are: 1. Improving the accuracy of land cover data. By introducing the teacher-student model collaborative mechanism and the conflict index-driven partitioning decision strategy, the teacher model maintains high accuracy in stable areas with low conflict, while the student model focuses on repairing areas with high conflict and blurred boundaries, thereby improving the overall classification accuracy indicators such as mIoU, OA, and kappa coefficient.

[0019] ② Enhanced identification and repair capabilities for spatial consistency and high-conflict areas. This method constructs a conflict index to accurately identify high-conflict areas and guides the student model to focus on restorative learning in these areas. This network architecture strengthens the model's ability to identify and optimize areas of inconsistent classification across data sources, effectively improving classification confusion in conflicting areas and enhancing the consistency and semantic stability of the fused data.

[0020] ③ Realize regional perception and adaptability of the fusion model. Unlike the traditional method of unified classification decision for the entire image, the present invention can make adaptive decisions based on the conflict index of each pixel, realizing pixel-level fusion control. This fusion mechanism significantly improves the flexibility and generalization ability of the model.

[0021] ④ It reduces the dependence on large amounts of labeled data. By using the output of the teacher model and the dual-constraint screening mechanism to generate pseudo labels, the student model training set is constructed, which effectively expands the training sample size. Even when data annotation is scarce, it can still achieve good results. It has the advantages of self-supervised learning and saves human resources and time costs.

[0022] ⑤ The method is highly versatile and easily extensible. This method is applicable to the fusion of land cover products from a variety of spatial resolutions, classification systems, and data sources. Furthermore, the input of geographic environmental factors is highly scalable, allowing the model to incorporate a variety of geographic environmental factors as needed. The overall framework features a modular design and standardized processes, making it easy to migrate to fusion tasks in other regions or with other data. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort. Among them: Figure 1 This is a flow chart of a multi-source land cover data fusion method based on conflict index and self-supervised reconciliation according to the present invention; Figure 2 This is a model architecture diagram of a multi-source land cover data fusion method based on conflict index and self-supervised reconciliation in the present invention; Figure 3 A visualization diagram of the conflict index (CI) within the study area in Example 1 of the multi-source land cover data fusion method based on conflict index and self-supervised reconciliation of the present invention; Figure 4 A visualization diagram of the conflict index indicator, information entropy (Entropy), within the study area in Example 1 of a multi-source land cover data fusion method based on conflict index and self-supervised reconciliation of the present invention; Figure 5 This is a visualization graph of the conflict index indicator, Model Frequency, in the study area in Example 1 of the multi-source land cover data fusion method based on conflict index and self-supervised reconciliation of the present invention; Figure 6 A visualization diagram of the conflict index indicator, namely, neighborhood consistency (Local Consistency), in the study area in Example 1 of a multi-source land cover data fusion method based on conflict index and self-supervised reconciliation of the present invention; Figure 7 A comparison diagram of the real land features, the source land cover map, and the fused land cover map at the same location in Example 1 of a multi-source land cover data fusion method based on conflict index and self-supervised reconciliation of the present invention; Figure 8 The ESRI 2020 Land Cover map used in the fusion of Example 1 of a multi-source land cover data fusion method based on conflict index and self-supervised reconciliation of the present invention; Figure 9 The MODIS MCD12Q1 LCCS2 land cover map used in the fusion of Example 1 of the multi-source land cover data fusion method based on conflict index and self-supervised reconciliation of the present invention; Figure 10 The MODIS MCD12Q1 IGBP land cover map used in the fusion of Example 1 of the multi-source land cover data fusion method based on conflict index and self-supervised reconciliation of the present invention; Figure 11 The ESA CCI-LC land cover map used in the fusion of Example 1 of the multi-source land cover data fusion method based on conflict index and self-supervised reconciliation of the present invention; Figure 12 The GLC-FCS30 land cover map used in the fusion of Example 1 of the multi-source land cover data fusion method based on conflict index and self-supervised reconciliation of the present invention; Figure 13 The land cover map generated by fusion using the traditional U-Net architecture in Example 1 of the multi-source land cover data fusion method based on conflict index and self-supervised reconciliation of the present invention; Figure 14 This is a land cover map generated by using a self-repairing U-Net network fusion in Example 1 of a multi-source land cover data fusion method based on conflict index and self-supervised reconciliation of the present invention. DETAILED DESCRIPTION

[0024] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0025] Next, the present invention is described in detail with reference to schematic diagrams. For ease of illustration, cross-sectional views of device structures may be partially enlarged and not to scale when describing the embodiments of the present invention. Furthermore, the schematic diagrams are merely illustrative and should not limit the scope of protection of the present invention. Furthermore, in actual production, three-dimensional dimensions, including length, width, and depth, should be included.

[0026] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0027] This invention aims to resolve classification conflicts caused by inconsistent classification systems, varying spatial resolutions, and different classification methods in the fusion of multivariate land cover data, thereby constructing more accurate land cover data. The working principle is based on the fusion of deep learning, conflict perception, and self-supervised optimization. By constructing a teacher-student dual-model architecture, the model is guided to strike a balance between global consistency and local differences.

[0028] A multi-source land cover data fusion method based on conflict indices and self-supervised reconciliation addresses some of the challenges faced by existing methods in identifying and addressing areas of inconsistent classification across different data. The core idea is to identify inconsistent regions by constructing a conflict index. This, combined with uncertainty estimates from model predictions, guides the construction of self-supervised pseudo-labels, thereby achieving focused restoration of high-conflict areas and optimizing the overall classification results. This method enhances the model's ability to represent the fusion of multi-source data while effectively improving classification accuracy and spatial consistency in high-conflict regions.

[0029] In order to identify high-conflict areas, the present invention constructs a quantifiable indicator to express the inconsistency of the classification of multiple land cover data - the conflict index CI (Conflict Index), which comprehensively considers the category mode frequency, information entropy and local consistency, accurately measures the classification differences between multi-source data, and provides data support for the identification of high-uncertainty classification areas and the adjustment of fusion strategies. In order to ensure that the influence of the three is balanced when evaluating the degree of conflict in multi-source classification results, an equal weight distribution strategy is adopted, that is, the weight of each indicator is 1 / 3. This distribution method not only makes the feature contribution of each dimension equal, but also satisfies a basic constraint: the sum of the weights is equal to 1. This average weighting method helps to comprehensively reflect the classification conflict of pixels from the three dimensions of category consistency, category confusion and spatial continuity, thereby constructing a more objective and stable conflict index. The function formula of the conflict index is:

[0030] in, Represents pixels The conflict index value reflects the inconsistency of the classification results in the multi-source layer. The formula contains the following three sub-formulas:

[0031] in, Represents pixels The mode frequency of is the number of occurrences of the category that most frequently appears in all input layers for this pixel. is the total number of data. The higher the mode frequency, the more consistent the multi-source classification results at that location and the lower the degree of conflict.

[0032]

[0033] in, Represents pixels The classification consistency with the pixels in its neighborhood; for Neighborhood set of (3×3 neighborhood window), is the indicator function (if , then it is 1, otherwise it is 0). This item is used to measure the local smoothness of the pixel in space. If the classification consistency between the pixel and its neighborhood is low, it means that it may be in a boundary or conflict transition zone.

[0034]

[0035] in, For pixels The classification information entropy of Indicates that the pixel is marked as k The probability of the class, k is the total number of categories. Information entropy reflects the degree of distribution disorder of the classification results at that location. The higher the entropy, the more uncertain and unstable the classification of the pixel.

[0036] The conflict index (CI) is a metric used to measure the inconsistency of classification results from multi-source land cover data. It quantifies the degree of classification conflict for each pixel across different products. A higher CI value indicates greater discrepancies and uncertainty in the classification of that pixel across multiple data sets. The conflict index not only identifies problem areas but also guides subsequent classification optimization and model reconstruction tasks, making it the core scheduling mechanism in this paper.

[0037] This paper constructs a self-healing U-Net network, an uncertainty-aware fusion model based on a U-Net core architecture that incorporates self-supervised learning. This model utilizes a teacher-student collaborative mechanism and consists of three main modules: a teacher U-Net, a self-supervised uncertainty-guided module, and a student U-Net. These three modules work together to improve the model's classification capabilities in high-conflict areas and enhance overall spatial consistency and robustness.

[0038] The deep learning model constructed in this paper uses the classic U-Net encoder-decoder structure. The network architecture remains consistent in both the teacher and student models, but differs in the input source and training strategy. In the encoder, the model undergoes four downsampling stages, each consisting of two 3×3 convolutional layers (Conv2d + BatchNorm + ReLU) and a 2×2 max pooling operation (MaxPooling). The number of channels is expanded from 64, 128, 256, to 512 after each downsampling, and the feature map size is gradually reduced to one-quarter of the original input. The bottleneck layer uses two 1024-channel convolutional layers for deep feature extraction. In the decoder, each stage restores the feature map to a higher spatial resolution through 2×2 upsampling (transposed convolution). The feature map is then concatenated (skip connection) with the feature map from the corresponding encoding stage, followed by two convolutional layers for fusion. The number of channels in the decoding phase is reduced from 512, 256, and 128 to 64, and the output is ultimately mapped to a prediction map for the target class via a 1×1 convolution. The teacher model is trained using true labels and generates high-quality softmax classification probability maps. The student model is trained using a mixture of true label patches and pseudo-label patches selected based on the conflict index and softmax entropy. The training strategy (loss function, early stopping, learning rate schedule, etc.) is consistent with that of the teacher model. The output is still the same softmax classification map as the teacher model.

[0039] The self-supervised uncertainty guidance module, located between the teacher and student models, is primarily responsible for generating pseudo-labels and constructing a training set for the student model. This module first calculates information entropy based on the softmax probability map output by the teacher model to measure the model's classification uncertainty for each pixel. It then reads the conflict index map to comprehensively determine whether a pixel falls into an unreliable region. The module uses dual thresholds (entropy and conflict index) to select high-confidence pixels as pseudo-labels. The pseudo-label patches are then mixed with the true label patches in a 1:1 ratio to form the training dataset for the student model.

[0040] The model training phase uses a cross-entropy loss function for optimization. Considering the imbalanced class ratios in land cover data, a weighted cross-entropy loss function is used, assigning different weights to different classes. Class weights are calculated as the inverse of the pixel frequency of each class and normalized, with higher weights for less common classes to prevent the model from being overly biased towards the dominant class. Pixels outside the region or without a label are assigned class 0, and these regions are excluded from the loss function using ignore_index = 0 to avoid affecting training.

[0041] The Adam optimizer was selected to accelerate convergence. The initial learning rate was set to 0.001, and the weight decay parameter was set to 1e-5 to prevent overfitting. To improve training stability, the learning rate scheduling strategy StepLR was introduced, automatically decaying the learning rate by 0.1 times after every 10 epochs. During training, an EarlyStopping strategy was implemented, terminating training early if the validation set mIoU did not improve for 15 consecutive epochs to prevent the model from overfitting to the training set. The batch size was set to 8, and each training session lasted a maximum of 100 epochs. In most cases, training could be terminated early within 30 to 50 epochs.

[0042] Ultimately, the teacher model provides stable predictions across the entire map, while the student model focuses on repairing and optimizing areas of significant conflict. The fusion phase makes region-aware decisions based on the spatial distribution of the conflict index map: the student model output is used in high-conflict areas, while the teacher model output is retained in other areas. This results in a high-quality land cover classification map output that combines stability and flexibility. The model's overall parameter size is well-balanced, ensuring both strong expressive power and efficient operation within acceptable computing resources.

[0043] like Figure 1-2 As shown, the present invention proposes a multi-source land cover data fusion method based on conflict index and self-supervised reconciliation, which includes the following steps: Step 1: Data preparation and data preprocessing Download the land cover data and multiple geographic environment factors that need to be integrated.

[0044] Use ArcGIS Pro software to unify the spatial resolution of all data by resampling, and also unify the projection coordinate system of all data.

[0045] Select the required target category according to the research purpose.

[0046] A classification mapping table was constructed between the classification systems of multiple source land cover products and the target classification based on the semantic affinity scores. All source land cover data were reclassified using ArcGIS Pro software to achieve a unified classification system.

[0047] A certain number of random points are generated within the study area as verification point data, and each point is visually interpreted through high-definition satellite remote sensing images to obtain the land cover type of each point.

[0048] The research area data and the interpreted verification point data serve as the basis for effective pixel screening and accuracy evaluation. After they are rasterized, the spatial resolution and projection coordinate system must also be unified.

[0049] All geographical environmental factor data are normalized.

[0050] After all data processing is completed, convert it into Numpy format.

[0051] Step 2: Feature Construction One-hot encoding is performed on each category of each land cover data, and each category of each land cover data is regarded as a channel. The corresponding number of channels is the product of the number of land cover data and the number of target categories.

[0052] Each geographical environment factor is stacked into a channel.

[0053] Construct a conflict index as a separate channel.

[0054] Step 3: Patch data construction and enhancement Patches were extracted by sliding across the study area using a sliding window approach (patch size = 256, stride = 128). Each patch contained multiple channel input features and a corresponding label map. Labels were derived from visual interpretation verification points. Pixels outside the region or without labels were set to 0 and ignored during training. To improve model generalization, data augmentation was performed, including horizontal flipping, vertical flipping, and 90° rotation. Three augmented samples were generated for each original patch.

[0055] Step 4: Model construction and training strategy This paper constructs a pair of deep learning semantic segmentation models with identical structure and complementary functions: the Teacher model and the Student model. Both models utilize the U-Net network as their foundational framework, comprised of symmetrical encoders and decoders, and possess excellent feature extraction and spatial localization capabilities. By incorporating multi-source land cover data and geographic environmental factors into a multi-channel input, and employing differentiated training strategies and data sources, they achieve the identification and optimization of conflicting areas, enhancing the spatial consistency of classification results.

[0056] Teacher Model The teacher model is the core classifier in the first stage of the entire network architecture. It is responsible for learning the mapping from fused features to standard land cover classification results under supervised conditions. The model input is a multi-channel feature map consisting of one-hot encoded land cover data, multiple normalized geographic environmental factors, and a conflict index. The model output is multi-channel (the number of target classes plus 1) prediction data corresponding to all valid target classes and one invalid class (class 0 is set as the ignore index and does not participate in the loss calculation).

[0057] The teacher model training data all comes from the real labels of visual interpretation, and is optimized using the weighted cross entropy loss function (Weighted CrossEntropyLoss) with category weights. The loss function is defined as:

[0058] in, Indicates the total number of categories; If the label is category c, the value is 1, otherwise it is 0 (i.e. the first c Class component); Represents the model for category c The predicted probability of (softmax output); Representation category c The weight is usually calculated based on the inverse of the frequency of occurrence of the category (see the following formula); This is the overall loss function, weighted by the number of class samples. This formula is used to address class imbalance and prevent minor classes from being neglected.

[0059] The class weight is obtained by normalizing the inverse of the frequency of each class of pixels. The specific calculation method is as follows:

[0060] in, Indicates the frequency of category c in the training set (i.e., the proportion of pixels to the total number); Represents the weighting coefficient of category c; the denominator is the sum of the inverses of the frequencies of all categories, which is used to normalize the sum of the weights to 1. This formula ensures that rare classes have a large weight and common classes have a small weight, and is the basis of weighted cross entropy.

[0061] Self-supervised Uncertainty-Guided Module This module is the key link between the teacher and student in the architecture. It uses the output of the teacher model and the conflict index to automatically generate high-confidence pseudo-labels, thereby constructing supplementary training data and guiding the student model to better learn the classification features of high-conflict areas.

[0062] The first step of the module is Compute Softmax Entropy.

[0063] Using the softmax probability map output by the teacher model, we calculate information entropy pixel by pixel to measure the uncertainty of the model's prediction at each pixel. A higher entropy value indicates a lower confidence level in the model's classification at that pixel; conversely, a pixel with low entropy can be considered a high-confidence prediction. This process helps us identify areas of high uncertainty that require special attention.

[0064] The second step is Read Conflict Index.

[0065] The conflict index is calculated based on the inconsistency of the classification results of multi-source land cover products. It reflects the degree of conflict between different data sources at a specific pixel. By reading the conflict index map, high-conflict and high-uncertainty areas can be distinguished from normal areas, providing a basis for subsequent pseudo-label screening.

[0066] The third step is to Apply Dual Thresholds.

[0067] This step sets two screening criteria: the entropy threshold and the conflict index threshold. By setting these two thresholds, we can effectively remove low-confidence and high-conflict noise pixels, ensuring the reliability of the pseudo-labels.

[0068] The fourth step is Select Pseudo Labels.

[0069] For pixels that meet the requirements, the class with the highest probability is directly selected from the softmax output of the teacher model as the pseudo label. This can quickly obtain a set of more reliable label data, covering the shortcomings of the teacher model in complex areas.

[0070] The last step is Mix with Real Labels (1:1).

[0071] To maintain the diversity and stability of the training data, the module mixes the generated pseudo-label patches with the true-label patches obtained through visual interpretation in a 1:1 ratio to construct the student model's training set. This hybrid strategy leverages the high accuracy of true labels and the high coverage of pseudo-labels, improving the student model's discriminative capabilities in high-conflict areas.

[0072] Student Model The student model, serving as a supplementary classifier in the second stage, is structurally identical to the teacher model, also employing a U-Net network based on multi-channel input. However, its training data consists of two components: original patches with true labels and patches with high-confidence pseudo-labels selected based on the teacher's softmax output and the conflict index. Specifically, the softmax entropy of each pixel is calculated and combined with the conflict index to set masking conditions, thereby selecting credible regions and generating pseudo-labels to construct the self-supervised training set. During the validation phase, only true labeled data is used for evaluation to maintain consistency and reliability of the evaluation metrics. The training strategy (loss function, early stopping, and learning rate scheduling) remains the same as that of the teacher model.

[0073] While the teacher and student models share the same network structure, both based on the multi-channel input U-Net framework, they achieve a functional division of labor and collaboration. The teacher model relies on true labels to provide stable and reliable full-map classification results, suitable for processing areas with strong consistency. The student model, by introducing pseudo-labels, possesses greater adaptability and resilience, improving classification performance in areas with significant conflicts and ambiguous boundaries. Ultimately, through a conflict-aware decision-making mechanism, the advantages of both models are combined to achieve a land cover fusion optimization strategy that balances stability and flexibility.

[0074] Step 5: Whole-image inference and fusion result output After the teacher model training is completed, it is used for sliding window reasoning of the entire image, outputting a softmax probability map, and obtaining a preliminary classification map through the argmax operation, providing a probabilistic basis for subsequent self-supervised pseudo-label generation.

[0075] After the student model training is completed, the student model is used to perform whole-image reasoning, and a softmax probability map is also generated. The classification map based on the conflict mask is obtained through the argmax operation.

[0076] Output fusion optimization: The teacher and student reasoning results are fused. For high-conflict areas above the conflict index threshold, the student's classification results are used, and the teacher's output is retained for the rest of the areas. Pixels outside the area are set to 0, and the final fusion result is saved.

[0077] Example 1 See also Figure 3-14 ,in Figure 7 In the figure, a1 b2 represent real objects in two high-resolution remote sensing satellite images, a2b2 represents the land cover map fused using the self-repairing U-Net network, a3 b3 represents the land cover map fused using the traditional U-Net architecture, a4 b4 represents the land cover map of ESRI 2020 Land Cover, a5 b5 represents the land cover map of MODIS MCD12Q1LCCS2, a6 b6 represents the land cover map of MODIS MCD12Q1 IGBP, a7 b7 represents the land cover map of ESA CCI-LC, and a8 b8 represents the land cover map of GLC-FCS30.

[0078] The experimental areas were the Sanjiang Plain region of my country and its adjacent Primorsky Krai of Russia. Both regions are geographically adjacent and located in the mid- and high-latitude monsoon climate zone of East Asia. They share geographical and ecological similarities, yet exhibit differences in geomorphic structure and land use patterns, representing a typical ecological transition zone. These regions, characterized by complex land class boundaries, intertwined heterogeneous information, and significant classification conflicts, are ideal for testing the applicability and stability of multi-source land cover fusion methods.

[0079] This example selected five mainstream land cover data products from 2020: ESA CCI-LC, GLC-FCS30, MODIS MCD12Q1 IGBP, MODIS MCD12Q1 LCCS2, and ESRI 2020 Land Cover. The goal of this fusion experiment was to improve the classification accuracy and spatial consistency of cultivated land, forest land, and grassland in the study area, generating a 2020 cultivated land, forest land, and grassland cover map for the study area. Auxiliary input data included five geographic environmental factors: digital elevation model (DEM), slope, soil moisture, NDVI (normalized vegetation index), and land surface temperature (LST). The data was uniformly resampled to a 30-meter resolution and maintained a consistent projection coordinate system (WGS84). After semantic comparison and expert revision, all land cover products were reclassified into four categories: cultivated land, forest land, grassland, and others. All geographic environmental factors were normalized.

[0080] 3686 points were evenly distributed in the study area as verification point data. At the same time, Google high-resolution remote sensing images were used to visually interpret each point. After the interpretation was completed, the verification point data was converted into raster format, and the projection and resolution were unified.

[0081] All data were then converted to NumPy format. The five land cover data were one-hot encoded (4 channels per product, for a total of 20 channels), the geographic factors were stacked into 5 channels by number, and the conflict index was constructed as the 26th channel. The final input feature tensor constructed had 26 channels and dimensions (H, W, 26).

[0082] The conflict index is constructed based on three sub-indicators: majority class frequency, local consistency, and information entropy, which measure multi-source classification consistency, neighborhood structure consistency, and classification uncertainty, respectively. These three sub-indicators are given equal weight, and the final sum of the weights is 1. The constructed conflict index map has a value range of [0, 3.46875] and is divided into eight intervals according to the values: 0.0, (0.0, 0.04444], (0.04444, 0.08889], (0.08889, 0.13333], (0.13333, 0.17778], (0.17778, 0.6], (0.6, 1.0], >1.0. Regions with pixel values ​​>1 are defined as high-conflict areas.

[0083] Then, the patch is split and the sliding window strategy (patch size = 256, stride = 128) is used to slide the study area to extract the original patch. At the same time, data augmentation operations (90° rotation, horizontal flip, vertical flip) are performed on each patch, and each original patch is enhanced to 4 samples to improve the generalization ability of the model.

[0084] This study employed a teacher-student model architecture. Both are based on the standard U-Net network structure, featuring a symmetrical encoder-decoder structure, skip connections, and adaptability to multi-channel inputs. In the first phase, the teacher model undergoes supervised training using ground-truth visual interpretation labels. The optimization objective is a weighted cross-entropy loss function, with class weights normalized by the inverse of the sample frequency. Training utilizes EarlyStopping (monitoring validation set mIoU, patience = 15) and a StepLR learning rate decay strategy, with an initial learning rate of 0.001.

[0085] After inference, the teacher model outputs a softmax probability map, which is used to construct pseudo labels for the student model in the second stage. Pseudo-label screening uses a joint thresholding strategy: pixels with a softmax entropy value < 0.8 and a conflict index < 1 are considered high-confidence pseudo-labels and participate in student model training. Pseudo-label patches are mixed with true label patches in a 1:1 ratio to form the student model training set.

[0086] The student model structure is the same as the teacher model, but the training data contains pseudo-labeled areas, which has stronger boundary recognition and region repair capabilities. The training strategy is the same as the teacher model.

[0087] In the final fusion stage, a conflict-aware decision-making mechanism is used: the prediction results of the student model are used for areas with a conflict index greater than 1, and the output of the teacher model is retained for the remaining areas. The final fused land cover map is saved, and pixels outside the area are set to invalid values ​​(0).

[0088]

[0089] Although the present invention has been described above with reference to embodiments, various modifications may be made thereto and equivalent components may be substituted without departing from the scope of the present invention. In particular, as long as there are no structural conflicts, the various features of the embodiments disclosed herein may be combined with each other in any manner, and the omission of an exhaustive description of such combinations in this specification is solely for the sake of space and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A multi-source land cover data fusion method based on conflict index and self-supervised reconciliation, characterized by: The following steps are involved: Step 1: Data preparation and data preprocessing: Download land cover data and multiple geographic environmental factors, and reclassify and process the source land cover data; Step 2: Feature construction: One-hot encoding is performed on each category of each land cover data, each geographical environmental factor is stacked into a channel, and the conflict index is constructed and treated as a separate channel; Step 3: Patch data construction and enhancement: Patches are extracted from the study area using a sliding window method. Each patch contains multiple channel input features and a corresponding label map. Step 4: Model construction and training strategy: Build a pair of deep learning semantic segmentation models with identical structure and complementary functions: teacher model and student model; The teacher model includes one-hot encoding of multiple land cover data categories, multiple normalized geographic environment factors, and a conflict index; Self-supervised uncertainty guidance module: Utilize the output of the teacher model and the conflict index to automatically generate high-confidence pseudo labels; The training data of the student model includes the original patches from the true labels and the high-confidence pseudo-label patches selected based on the teacher's softmax output and the conflict index; Step 5: Whole-image reasoning and fusion result output: The reasoning results of teachers and students are integrated and saved.

2. The multi-source land cover data fusion method based on conflict index and self-supervised reconciliation according to claim 1 is characterized in that: In step 2, the function formula of the conflict index is: ； in, Represents pixels The conflict index value reflects the inconsistency of the classification results in the multi-source layer. Represents pixels The mode frequency of Represents pixels The classification consistency with the pixels in its neighborhood, For pixels The classification information entropy of .

3. The multi-source land cover data fusion method based on conflict index and self-supervised reconciliation according to claim 2 is characterized in that: described The formula is: ； in, is the number of occurrences of the category that most frequently appears in all input layers for this pixel. is the total number of data. The higher the mode frequency, the more consistent the multi-source classification results at that location and the lower the degree of conflict.

4. The multi-source land cover data fusion method based on conflict index and self-supervised reconciliation according to claim 2 is characterized in that: described The formula is: ； for The neighborhood set of is the indicator function, if , then it is 1, otherwise it is 0. This item is used to measure the local smoothness of the pixel in space. If the classification consistency between the pixel and its neighborhood is low, it means that it may be in a boundary or conflict transition zone.

5. The multi-source land cover data fusion method based on conflict index and self-supervised reconciliation according to claim 2 is characterized in that: described The formula is: ； in, Indicates that the pixel is marked as k The probability of the class, k is the total number of categories, and the information entropy reflects the degree of distribution disorder of the classification results at that location. The higher the entropy, the more uncertain and unstable the classification of the pixel is.

6. The multi-source land cover data fusion method based on conflict index and self-supervised reconciliation according to claim 1 is characterized in that: The teacher model training data all comes from the real labels of visual interpretation, and is optimized using the cross-entropy loss function with category weights. The loss function is defined as: ； in, Indicates the total number of categories; Indicates that the value is 1 when the label is category c, otherwise it is 0; Represents the model for category c The predicted probability of Representation category c The weight of Represents the overall loss function, which is weighted and summed according to the difference in the number of class samples. The class weight is obtained by normalizing the inverse of the frequency of each class pixel.

7. The multi-source land cover data fusion method based on conflict index and self-supervised reconciliation according to claim 1 is characterized in that: Both the teacher model and the student model use the U-Net network as the basic framework, which consists of a symmetrical encoder and decoder. They construct multi-channel input by introducing multi-source land cover data and geographic environmental factors, and perform differentiated design in training strategies and data sources.

8. The multi-source land cover data fusion method based on conflict index and self-supervised reconciliation according to claim 1 is characterized in that: The teacher model relies on real labels to provide stable and reliable full-map classification results, and is suitable for processing areas with strong consistency; the student model, based on the introduction of pseudo-labels, has stronger adaptability and repair capabilities, and can improve classification performance in areas with significant conflicts and blurred boundaries. Ultimately, through a decision-making mechanism guided by conflict perception, the advantages of the two are integrated to achieve a land cover fusion optimization strategy that emphasizes both stability and flexibility.

9. The multi-source land cover data fusion method based on conflict index and self-supervised reconciliation according to claim 1, characterized in that: The first step of the self-supervised uncertainty guidance module is to calculate the softmax entropy. The information entropy is calculated pixel by pixel using the softmax probability map output by the teacher model to measure the prediction uncertainty of the model at each pixel. The higher the entropy value, the lower the classification confidence of the model at that pixel. Conversely, low entropy pixels can be regarded as high-confidence prediction results. The second step is to read the conflict index. The conflict index is calculated based on the inconsistency of the classification results of multi-source land cover products. It can reflect the degree of conflict between different data sources at a certain pixel. By reading the conflict index map and jointly analyzing it with the entropy value, high-conflict and high-uncertainty areas can be distinguished from ordinary areas, providing a basis for subsequent pseudo-label screening. The third step is to apply double thresholds, which sets two screening criteria: entropy threshold and conflict index threshold; The fourth step is to filter pseudo labels. For pixels that meet the conditions, the category with the highest probability is directly selected from the softmax output of the teacher model as the pseudo label. The last step is mixing with the true label. In order to maintain the diversity and stability of the training data, the module mixes the generated pseudo-label patch with the true label patch obtained by visual interpretation in a 1:1 ratio to construct the training set of the student model.

Images