A deep learning-based topographic map unlabeled region segmentation method

Abstract

The present application provides a kind of topographic map unlabelled area segmentation method based on deep learning, including collecting and segmenting topographic map grid, obtain multi-source tensor topographic map;Design backbone network, output the feature representation of multi-source tensor topographic map;Build geographic semantic library, increase the geographic semantic understanding of topographic map, generate semantic weight;Design main segmentation network, carry out initial segmentation prediction;Quantify the prediction uncertainty of main segmentation network, output initial segmentation result by the way of multiple iterations;For initial segmentation result, pixel connected domain detection, terrain constraint and spatial consistency improvement are carried out, and coherent segmentation result is obtained;Again, coherent segmentation result is visualized as heat map and spatial explainability layer.The present application can handle complex landform and spatial heterogeneous situation, make topographic map unlabelled area segmentation more accurate, intelligent and efficient, restore unlabelled area real landform, significantly optimize boundary and spatial distribution, promote the intelligent upgrading of geographic information system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of topographic map segmentation, in particular to a topographic map unlabeled region segmentation method based on deep learning. BACKGROUND

[0002] In the work scene, there are blank unlabeled regions in the topographic map, that is, topographic map unlabeled regions. The main function of the topographic map is to comprehensively and accurately present geographical information. If there are unlabeled regions, it means that the map data is incomplete, which will affect the analysis and decision-making based on the map, so it is necessary to segment the range of these regions to supplement the complete data topographic map and improve the usability of the map. The unlabeled region segmentation refers to a technology of automatically identifying and segmenting different topographic regions in the map only according to the information of the topographic map itself, such as contour lines, landform texture, color, spatial structure, etc., without the aid of human-labeled information.

[0003] At present, the existing topographic map unlabeled region segmentation methods include traditional manual methods and automatic exploration methods. The traditional manual method relies on manual comparison of remote sensing images to segment the unlabeled region, which is low in efficiency and has obvious pain points, and is less applied in practice. Although the automatic exploration method improves the segmentation efficiency, it still has the following defects: 1) large dependence on feature expression: traditional threshold segmentation is sensitive to color and gray scale, but the style of the topographic map is complex, and the same region has different colors, which leads to inaccurate segmentation; simple clustering does not consider the spatial and morphological structure, and it is difficult to distinguish complex landforms. 2) lack of spatial continuity and landform consistency: many methods are only pixel clustering, and lack of learning and constraint on contour lines, land feature structure and spatial relationship. 3) poor generalization ability: poor adaptability to new maps / style, and risk of overfitting to sample maps. 4) data dependence and evaluation difficulty: it is difficult to determine the "correctness" of the segmentation result for the unlabeled method; lack of topographic structure knowledge constraint. SUMMARY

[0004] In order to overcome the deficiencies of the prior art, the purpose of the present application is to provide a topographic map unlabeled region segmentation method based on deep learning, which can handle complex landforms and spatial heterogeneity, make the topographic map unlabeled region segmentation more accurate, intelligent and efficient, restore the real landform of the unlabeled region, significantly optimize the boundary and spatial distribution, and promote the intelligent upgrading of the geographic information system.

[0005] To achieve the above purpose, the present application provides the following scheme: a topographic map unlabeled region segmentation method based on deep learning, comprising:

[0006] Collect a terrain map grid, obtain DEM, remote sensing image and OpenStreetMap vector data corresponding to the terrain map grid, perform multi-channel division and region segmentation on the terrain map grid, and obtain a multi-source tensor terrain map;

[0007] A backbone encoder is selected and designed to obtain a backbone network, and self-supervised task design and position encoding design are performed on the backbone network to output a feature representation of the multi-source tensor terrain map;

[0008] According to the constructed geographic semantic short sentence database, the multi-source tensor terrain map is semantically mapped to obtain a landform semantic description, and a semantic weight is generated;

[0009] Based on the backbone network, a segmentation network architecture and a loss function are designed to obtain a backbone segmentation network, and the backbone segmentation network is used to output a spatial segmentation probability and a boundary explicit mask of the terrain map grid, and an initial terrain segmentation prediction is completed;

[0010] According to the spatial segmentation probability and the boundary explicit mask, the prediction uncertainty of the backbone segmentation network is quantified to obtain an uncertainty result, and multi-round self-iterative backflow training of the backbone segmentation network is performed to obtain a region segmentation model and an initial segmentation result output by the region segmentation model;

[0011] The initial segmentation result is subjected to pixel connected domain detection, terrain constraint and spatial consistency improvement to obtain a coherent segmentation result;

[0012] The coherent segmentation result is visualized as a soft probability heat map, a Monte-Carlo Dropout uncertainty heat map, and an Attention-Rollout attention area heat map, and a target class is selected for the end convolution or Transformer feature of the backbone segmentation network to generate a Grad-CAM spatial explainability graph layer.

[0013] Optionally, the terrain map grid is collected, and DEM, remote sensing image and OpenStreetMap vector data corresponding to the terrain map grid are obtained, and the terrain map grid is subjected to multi-channel division and region segmentation to obtain a multi-source tensor terrain map, comprising:

[0014] For the collected terrain map, a DEM service and a remote sensing API are introduced to eliminate stitching errors, align the terrain map, and call an OSM API to download vector elements of a region corresponding to the terrain map, intelligently filter the terrain map, and obtain a terrain map grid;

[0015] The terrain raster is cut into a plurality of 2048x2048 standard slice blocks, geographical coordinate indexes and meta information are labeled for each standard slice block, and each standard slice block is divided into a terrain shadow, a DEM-slope enhanced channel, an OSM road rasterized channel, and a fused vector attention 4 auxiliary channels;

[0016] By using unsupervised clustering, the standard slice blocks are divided into geomorphic divisions and labeled with geomorphic types according to geographical and statistical features, the geomorphic divisions are stratified sampled, and a mixed dataset including a training set, a validation set and a test set is output, so that each standard slice block includes a multi-source tensor terrain map;

[0017] The mixed dataset is mixed into each geomorphic area in the standard slice block in the form of different resolutions to expand the spatial scale of the terrain raster.

[0018] Optionally, a backbone encoder is selected and designed to obtain a backbone network, and a self-supervised task and position coding are designed for the backbone network to output a feature representation of the multi-source tensor terrain map, including:

[0019] Based on the multi-source tensor terrain map, Swin-Transformer-Lite is selected as the backbone encoder, a terrain mutation response layer is designed in the backbone encoder, and a second derivative statistic of height is superimposed before each window self-attention in the backbone encoder to assist the window to focus on areas where the geomorphic change is obvious, and a window sliding strategy is designed to align the DEM and the contour edge of the main map to complete the design of the backbone encoder and the geographical adaptation to obtain an initial backbone network.

[0020] For the multi-source tensor terrain map, a fused hierarchical mask strategy is designed in combination with patch masking to perform geographical sensitive masking, BYOL unsupervised learning is introduced for unlabeled contrast training and multi-view multi-source fusion, the self-supervised task design of the initial backbone network is completed, and a first backbone network is obtained.

[0021] Based on two-dimensional position coding, standardized DEM information and high-order terrain attributes are fused on each patch to generate terrain perception type position coding, and elevation gated attention is introduced in the encoding layer of the first backbone network to adaptively adjust the feature extraction weight of the plain and high-difference blocks, the position coding is completed, and a second backbone network that outputs feature representation is obtained.

[0022] Optionally, the fused hierarchical mask strategy includes 40% patch masking and 60% random patch of the whole map, and 20% of the 40% patch masking is a geomorphic mutation zone and 20% is an isocline dense zone.

[0023] Optionally, according to the constructed geographical semantic short sentence database, the multi-source tensor topographic map is semantically mapped to obtain a landform semantic description and generate a semantic weight, including:

[0024] A plurality of topographic map samples of different landform types are re-collected, landform-related paragraphs are crawled in an open source paper library, keywords and context description sentences of the landform-related paragraphs are extracted, and the keywords and context description sentences are automatically clustered, induced and de-duplicated to obtain a geographical semantic short sentence database;

[0025] The fixed-length visual vector of the multi-source tensor topographic map is extracted by using the main encoder, the geographical semantic short sentence data is encoded into a text embedding vector by using BERT or Transformer, the fixed-length visual vector and the text embedding vector are semantically mapped to obtain a landform semantic description and generate a semantic weight;

[0026] The multi-source tensor topographic map and the corresponding landform semantic description are set as positive samples, and the multi-source tensor topographic map and the non-corresponding landform semantic description are set as negative samples, a contrastive loss of the positive samples and the negative samples is calculated by using cross-modal contrastive learning, the contrastive loss is introduced, cross-modal contrastive training of the second main network is completed, and a third main network is obtained.

[0027] Optionally, based on the main network, a segmentation network architecture and a loss function are designed to obtain a main segmentation network, and the main segmentation network is used to output spatial segmentation probability and boundary explicit mask of the topographic map grid to complete initial topographic segmentation prediction, including:

[0028] Based on the third main network, a semantic prompt injection module is inserted between an encoder high-level feature layer and a decoder, a multi-scale decoder fused with the auxiliary channel is designed, and an output class probability and a main branch and an auxiliary branch outputting a region boundary are designed to obtain a segmentation network architecture;

[0029] FocalTversky Loss and Lovász Loss are combined to obtain a main loss function, boundary IoU Loss and direction consistency are combined to obtain an auxiliary boundary loss function, and the segmentation network architecture, the main loss function and the auxiliary boundary loss function are combined to obtain a main segmentation network;

[0030] The main segmentation network is used to output spatial segmentation probability and boundary explicit mask of the topographic map grid to complete initial topographic segmentation prediction.

[0031] The semantic prompt injection module realizes fusion of the landform semantic description and the encoded features through a cross-attention mechanism.

[0032] Optionally, according to the spatial segmentation probability and the boundary explicit mask, the prediction uncertainty of the backbone segmentation network is quantified to obtain an uncertainty result, so as to perform multi-round self-iterative backflow training of the backbone segmentation network to obtain a region segmentation model and an initial segmentation result output by the region segmentation model, including:

[0033] The prediction uncertainty of the backbone segmentation network is quantified by using Monte-Carlo Dropout to obtain the probability distribution of each patch to be segmented, and the per-pixel uncertainty, patch confidence and patch ranking of each patch to be segmented are calculated to obtain an uncertainty result, so as to assign a labeling task; the calculation expression of the uncertainty result is:

[0034]

[0035] Q(P)=(1-C1)·α+β·D(P)

[0036] Wherein, U(x) is the uncertainty entropy of pixel x, is the average probability of the kth class appearing at pixel x, C is the average confidence of the patch, |P| is the number of pixels in the patch, Q(P) is the patch priority ranking score, C1 is the average maximum confidence of the patch, D(P) is the average distance of the patch to the labeled set in the feature space, and α and β are weight coefficients;

[0037] The patches with a patch confidence greater than 0.9 are set as high-confidence pseudo-labels, the artificial labels, the high-confidence pseudo-labels and consistency constraints are set as training targets, and multi-round self-iterative backflow training of the backbone segmentation network is performed to obtain a region segmentation model, and an initial segmentation result is output by using the region segmentation model.

[0038] Optionally, pixel connected domain detection, terrain constraint and spatial consistency improvement are performed on the initial segmentation result to obtain a coherent segmentation result, including:

[0039] By using connected domain analysis, same-class pixel blocks in the initial segmentation result are classified into a node, and an undirected edge is added between adjacent nodes, attribute values of the nodes and the undirected edges are assigned, and then a graph data structure is obtained according to the nodes and the undirected edges; wherein the attributes of the nodes include class probability distribution, average elevation, position centroid and area, and the attribute of the undirected edge is class distribution similarity or elevation gradient at the boundary pixel;

[0040] Based on the graph data structure, a terrain constraint target is defined, and the constraint is expressed by using an indicator function and a penalty loss, and then constraint correction is performed according to the total constraint loss to obtain an initial coherent result;

[0041] Based on the initial coherent result, small nodes with pixels less than 100px are merged into the spatially nearest, same-class or highest-probability adjacent regions, and nodes with discrete class probability distribution and non-landform are deleted to complete the spatial consistency processing and obtain a coherent segmentation result.

[0042] Optionally, based on the graph data structure, a terrain constraint target is defined, and the constraint is expressed by using an indicator function and a penalty loss; then, according to a total constraint loss, constraint correction is performed to obtain an initial coherent result, including:

[0043] Based on the graph data structure, a terrain constraint target is defined; the terrain constraint target includes a unary constraint with a rule that a flat area cannot be directly adjacent to a valley, a binary constraint with a rule that a ridge is connected to more than two slopes, and a ternary constraint with a rule that the elevation of all valley-to-ridge paths monotonically increases;

[0044] The terrain constraint target is expressed by using an indicator function and a penalty loss to obtain a total constraint loss; a calculation expression of the total constraint loss is:

[0045] L1=∑ (i,j)∈E (c i =pb,c j =sg)·w ij

[0046]

[0047] L3=∑ p∈lj ∑ (i,j)∈p max(0,h i -h j )

[0048] L=λ1L1+λ2L2+λ3L3

[0049] wherein L1 is a flat-valley adjacent penalty, c i , and c j are node classes, w ij is a distance weight of an edge, L2 is a ridge-slope connection constraint, N(i) is a neighbor of node i, pb is a flat, sg is a valley, sj is a ridge, sp is a slope, L3 is an elevation monotonicity loss, lj is all valley-ridge paths, p is a path, h is an elevation, L is a total constraint loss, and λ1, λ2, and λ3 are weights;

[0050] All nodes and paths are traversed to find out a default node, the class probability of the default node is calculated, and a new class of the default node is selected to perform constraint correction to obtain an initial coherent result; a calculation expression of the new class of the default node is:

[0051]

[0052] wherein, p is a new category of default nodes, k is the probability of the node being predicted as category k, is the constraint loss difference brought by assigning the node to category k, and is the penalty trade-off parameter.

[0053] Optionally, the continuous segmentation result is visualized as a soft probability heat map, a Monte-Carlo Dropout uncertainty heat map, and an Attention-Rollout attention area heat map, and for the end convolution or Transformer feature of the backbone segmentation network, a target class is selected to generate a Grad-CAM spatial explainability layer, including:

[0054] Based on the continuous segmentation result, the class distribution probability of each pixel is calculated, the maximum value of the class distribution probability is selected as the pixel soft confidence, and the pixel soft confidence is displayed by color mapping to obtain a soft probability heat map.

[0055] Based on the continuous segmentation result, the uncertainty entropy of each pixel is calculated to obtain an uncertainty area, and the uncertainty area is highlighted locally by a heat map to obtain a Monte-Carlo Dropout uncertainty heat map.

[0056] Based on the continuous segmentation result, the attention matrix of each Transformer layer in the backbone segmentation network is extracted to calculate the global attention score of the pixel from input to output, and the global attention score is aggregated by class to obtain an Attention-Rollout attention area heat map of the pixel.

[0057] For the end convolution or Transformer feature of the backbone segmentation network, a target class is selected, the average gradient weight of the feature mapping output for the target class is calculated, and according to the average gradient weight, a Grad-CAM spatial explainability layer is generated in combination with a ReLU function.

[0058] The present application provides a terrain map unlabeled region segmentation method based on deep learning, and discloses the following technical effects:

[0059] 1. High data base quality: 1) By integrating topographic maps, DEMs, remote sensing images and OSM vector data into multi-source tensor data, the comprehensiveness of information can be improved. 2) By API, stitching errors are eliminated, intelligent filtering areas are cut into standard slice blocks, and multi-channels such as terrain shadow, DEM-slope enhancement, OSM road rasterization, and fusion of vectors are established to increase spatial salient features and ensure topographic diversity and partition rationality. 3) Unsupervised clustering and mixed data hierarchical sampling are used to expand the data spatial scale and improve the richness of training samples.

[0060] 2. High-discrimination feature expression solves the coding problem of complex topography: 1) Design a deep coding structure containing a terrain mutation response layer, and superimpose the second-order derivative statistics of elevation to achieve stronger perception of topographic mutation areas, adaptively focus on confusing boundary areas, and improve feature discrimination ability. 2) Fusion of high-order topographic attributes and position information coding, and introduction of elevation gated attention, adaptive plain elevation feature weight, can fully explore the topographic geometry and physical properties, and improve the pre-processing feature expression ability. 3) Through grid masking and BYOL unsupervised training closed loop, without large-scale artificial labels, realize more extensive regional model generalization.

[0061] 3. Intelligent identification of real topography in unlabeled areas: Build a geographic semantic short sentence database, extract the topographic paragraph from the paper library, and generate a semantic library by clustering, realize image and text semantic alignment, form the ability to understand the picture, make the topographic identification closer to the human understanding method, and greatly enrich the depth of topographic understanding.

[0062] 4. Fine segmentation and boundary optimization: 1) Introduce a semantic prompt injection module in the encoding and decoding network to assist multi-scale decoding in the channel, which can greatly improve the spatial segmentation accuracy. 2) Distinguish the main branch and the boundary auxiliary branch, and apply loss multi-constraint strategy, which can significantly improve the model's response ability to fine-grained features and boundary changes.

[0063] 5. Strong automatic segmentation ability in unlabeled areas: Through quantifying pixel / patch uncertainty and pseudo-label driven self-iterative backflow training, active sampling and task prioritization can be realized, effectively reducing the pressure of artificial labeling, and improving the generalization robustness and robustness of the model.

[0064] 6. Spatial consistency: Through the construction of pixel block graph structure, multi-element topographic physical constraint, punishment of default nodes, and spatial consistency processing, discrete small areas can be effectively eliminated, the coherence of similar blocks can be improved, the spatial self-consistency and physical reasonableness of the segmentation result can be realized, and the spatial distortion problems such as excessive fragmentation or excessive connectivity of traditional algorithms can be significantly improved, making the segmentation close to the actual topographic distribution law.

[0065] The technical solutions of the present application will be further described in detail below through the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0067] Figure 1A method flowchart provided by the embodiment of the present application is shown in the figure.

[0068] Figure 2 A flowchart of initial segmentation prediction provided by the embodiment of the present application is shown in the figure.

[0069] Figure 3 A flowchart of consecutive segmentation results provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present application will be described clearly and completely in the figures of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0071] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail in the following with reference to the figures and specific embodiments.

[0072] As shown in the figure, Figure 1 The present application provides a terrain map unlabeled region segmentation method based on deep learning, which is characterized by comprising the following steps:

[0073] Step 1, collect terrain map grids, and obtain DEM, remote sensing image and OpenStreetMap vector data corresponding to the terrain map grids, then perform multi-channel division and region segmentation on the terrain map grids to obtain multi-source tensor terrain maps; specifically including:

[0074] 1.1 For the collected terrain map, introduce DEM service and remote sensing API to eliminate stitching errors, align the terrain map, and call OSM API to download vector elements of the region corresponding to the terrain map, perform intelligent filtering of the terrain map, and obtain terrain map grids.

[0075] 1.2 divide the terrain map grids into a plurality of 2048x2048 standard slice blocks, label geographic coordinate index and meta information for each standard slice block, and then divide each standard slice block into terrain shadow, DEM-slope enhancement channel, OSM road rasterization channel and fusion vector attention map 4 auxiliary channels.

[0076] Terrain shadow: automatically generated from DEM, combined with morning and evening sun height multi-angle synthesis, and enhanced to enhance the recognition of topography levels.

[0077] DEM-slope enhancement channel: obtain the slope field by Sobel or Horn operator, and then multiply it with DEM pixel by pixel to generate a “geomorphology sensitive layer” that integrates elevation relief and slope mutation and is easy to segment.

[0078] OSM road rasterization channel: rasterize the vector road according to the weight (main road / auxiliary road) and fuse the buffer width to form a feature map expressing the road network structure, which helps regional segmentation to fit the actual road form.

[0079] Fusion vector attention map: mask all rasterized regions with human annotation on the map, and preliminarily detect potential but unannotated regions (using remote sensing + topographic feature screening) by weak supervision or semantic segmentation to generate an attention map, which is not only used for model assisted segmentation, but also provides a basis for subsequent active learning to automatically select key samples.

[0080] 1.3 Use unsupervised clustering to divide the standard slice block according to geographical and statistical features, and mark the geomorphology type, and then stratified sampling is performed on the geomorphology division to output a mixed dataset including training set, validation set and test set, so that each standard slice block includes a multi-source tensor topographic map.

[0081] Geographical and statistical features: elevation variation, slope distribution, road density, etc.

[0082] Landform type: mountain, hill, plain, valley, urban fringe, etc.

[0083] 1.4 Mix the mixed dataset into each geomorphology area in the standard slice block in different resolutions to expand the spatial scale of the topographic map grid and ensure the robustness of the subsequent segmentation algorithm to different map accuracy and grid resampling.

[0084] Output multi-source tensor topographic map, each training sample is a tensor with shape = [C, 2048, 2048], C ≥ 6, containing main map + 4 innovative channels + attention mask, with complete geographic meta information and division label, forming an innovative geographic intelligent segmentation data basis, providing high quality support for subsequent deep learning process.

[0085] Step 2, select and design the main encoder to obtain the main network, and design the self-supervised task and position coding for the main network to output the feature representation of the multi-source tensor topographic map; specifically including:

[0086] 2.1 Based on the multi-source tensor topographic map, Swin-Transformer-Lite is selected as the backbone encoder, a topographic mutation response layer is designed in the backbone encoder, and a height second derivative statistic is superimposed before each window self-attention in the backbone encoder to assist the window in paying attention to the area where the topography changes obviously, and a window sliding strategy is designed to align the DEM and the contour edge of the main map, thus completing the design of the backbone encoder and the geographical adaptation, and obtaining an initial backbone network.

[0087] Swin-Transformer-Lite: Swin-Transformer-Lite with small parameter quantity and window-level self-attention is selected as the backbone encoder, which is suitable for processing large-size maps. The model input is the multi-channel block in the multi-source tensor (main map + DEM + slope + road + attention map, etc.), and the first layer automatically adapts the channels.

[0088] Topographic mutation response layer: Before each window self-attention, superimpose the height second derivative statistic to assist the window in paying attention to the area where the topography changes obviously. Design the window sliding strategy to align the DEM and the contour edge of the main map, and improve the "contour-shading-elevation" sensitivity.

[0089] 2.2 For the multi-source tensor topographic map, a fusion hierarchical mask strategy is designed in combination with patch masking to perform geographical sensitive masking, and BYOL unsupervised learning is introduced for unlabeled contrast training and multi-view multi-source fusion, thus completing the self-supervised task design of the initial backbone network and obtaining a first backbone network.

[0090] The fusion hierarchical mask strategy includes 40% patch masking and 60% random patch of the whole map. In the 40% patch masking, 20% is a topographic mutation zone and 20% is a contour dense zone. The fusion hierarchical mask strategy can realize geographical sensitive masking and guide the model to learn to reconstruct key information. The mask object is multi-channel / full-band, not only the main map but also including DEM and attention map, etc.

[0091] Multi-view multi-source fusion, for example:

[0092] View Figure 1 : original topographic map + DEM;

[0093] View Figure 2 : add random channel disturbance, such as hiding roads or adjusting the slope channel, affine transformation.

[0094] 2.3 Based on two-dimensional position coding, fuse standardized DEM information and high-order terrain attributes on each patch to generate terrain-aware position coding, and introduce elevation-gated attention in the encoding layer of the first backbone network to adaptively adjust the feature extraction weights of plain and high-difference blocks, complete the design of position coding, and obtain the second backbone network of output feature representation.

[0095] High-order terrain attributes: slope, convexity, curvature, etc.

[0096] Step 3, according to the constructed geographical semantic short sentence database, the semantic mapping of the multi-source tensor topographic map is carried out, the geomorphic semantic description is obtained, and the semantic weight is generated; specifically including:

[0097] 3.1 Re-collect multiple topographic map samples of different landform types, crawl paragraphs related to landforms in an open source paper library, extract keywords and context description sentences of landform-related paragraphs, and automatically cluster, induce and remove duplicates of keywords and context description sentences to obtain a geographical semantic short sentence database.

[0098] 3.2 Use the backbone encoder to extract fixed-length visual vectors of the multi-source tensor topographic map, use BERT or Transformer to encode the geographical semantic short sentence data into text embedding vectors, and perform semantic mapping on the fixed-length visual vectors and the text embedding vectors to obtain a geomorphic semantic description and generate a semantic weight.

[0099] 3.3 Set the multi-source tensor topographic map and the corresponding geomorphic semantic description as positive samples, set the multi-source tensor topographic map and the non-corresponding geomorphic semantic description as negative samples, use cross-modal contrastive learning to calculate the contrastive loss of the positive samples and the negative samples, introduce the contrastive loss, complete the cross-modal contrastive training of the second backbone network, and obtain a third backbone network.

[0100] Step 4, as shown in Figure 2 , based on the backbone network, design the segmentation network architecture and loss function to obtain the backbone segmentation network, use the backbone segmentation network to output the spatial segmentation probability and boundary explicit mask of the topographic map grid, and complete the initial segmentation prediction of the terrain; specifically including:

[0101] 4.1 Based on the third backbone network, insert a semantic prompt injection module between the high-level feature layer in the encoder and the decoder, design a multi-scale decoder fused with the auxiliary channel, and design an output class probability and a main branch and an auxiliary branch outputting the region boundary to obtain a segmentation network architecture.

[0102] Semantic prompt injection module: fuse the geomorphic semantic description and the encoded features through cross-attention mechanism.

[0103] Main branch: four-layer multi-scale decoding, each layer is decoded before the encoder multi-level feature is fused through the jump connection, and the depth separable convolution is used to efficiently improve the resolution and fine-grained segmentation ability. Channel spatial attention is added to guide the model to pay attention to the landform boundary area with spatial heterogeneity. The output is a class probability map.

[0104] Auxiliary branch: through the main decoding output feature + high resolution skip fusion, the region boundary is explicitly predicted, and the edge sharpness and landform fracture zone accuracy are improved.

[0105] 4.2, combined with FocalTversky Loss and Lovász Loss, the main loss function is obtained, combined with boundary IoU Loss and direction consistency, the auxiliary boundary loss function is obtained, combined with the segmentation network architecture, the main loss function and the auxiliary boundary loss function, the backbone segmentation network is obtained.

[0106] FocalTversky Loss: highlights the extreme class imbalance and main area missing problems, and is highly sensitive to minority classes and edge areas.

[0107] Lovász Loss: optimizes the direct ordering in the IoU space, better fits the segmentation evaluation standard, and improves the global consistency.

[0108] Boundary IoU Loss: specifically measures the IoU of the predicted boundary and the real or pseudo-real boundary, and directly optimizes the edge hit rate.

[0109] Direction consistency: by comparing the normal / tangential direction of the predicted boundary with the main direction of the geographical true value, the consistent expression of the natural landform direction structure is strengthened, and the quantitative capture ability of the landform fracture line and valley trend is greatly improved.

[0110] 4.3, using the backbone segmentation network, outputting the spatial segmentation probability and boundary explicit mask of the terrain raster, completing the initial terrain segmentation prediction.

[0111] Step 5, as shown in Figure 3 , according to the spatial segmentation probability and boundary explicit mask, quantifying the prediction uncertainty of the backbone segmentation network to obtain an uncertain result, and performing multi-round self-iterative backflow training of the backbone segmentation network to obtain a region segmentation model and an initial segmentation result output by the region segmentation model; specifically including:

[0112] 5.1 Quantify the prediction uncertainty of the backbone segmentation network by Monte-Carlo Dropout to obtain the probability distribution of each patch to be segmented, and calculate the per-pixel uncertainty, patch confidence and patch ranking of each patch to be segmented to obtain the uncertainty result to assign the labeling task; the calculation expression of the uncertainty result is:

[0113]

[0114] Q(P) = (1 - C1) · a + b · D(P)

[0115] wherein U(x) is the uncertainty entropy of pixel x, is the average probability of the k-th class appearing at pixel x, C is the average confidence of the patch, |P| is the number of pixels in the patch, Q(P) is the patch priority ranking score, C1 is the average maximum confidence of the patch, D(P) is the average distance of the patch from the labeled set in the feature space, a and b are weight coefficients.

[0116] 5.2 Set the patches with patch confidence greater than 0.9 as high-confidence pseudo labels, set the artificial labels, the high-confidence pseudo labels and consistency constraints as training targets, and perform multi-round self-iterative backflow training of the backbone segmentation network to obtain a region segmentation model, and output an initial segmentation result using the region segmentation model.

[0117] Step 6, as shown in Figure 3 , pixel connected domain detection, terrain constraint and spatial consistency enhancement are performed on the initial segmentation result to obtain a coherent segmentation result; comprising:

[0118] 6.1 Using connected domain analysis, same-class pixel blocks in the initial segmentation result are grouped into a node, and an undirected edge is added between adjacent nodes, attribute values are assigned to the nodes and the undirected edges, and then a graph data structure is obtained based on the nodes and the undirected edges.

[0119] Node attributes: class probability distribution, average elevation, location centroid and area.

[0120] Undirected edge attributes: class distribution similarity or elevation gradient at the boundary pixels.

[0121] 6.2 Based on the graph data structure, define a terrain constraint target, express the constraint using an indicator function and a penalty loss, and then perform constraint correction based on the total constraint loss to obtain an initial coherent result; specifically comprising:

[0122] 6.2.1 Based on the graph data structure, define a terrain constraint target:

[0123] Unary constraint: Plateau cannot be directly adjacent to valley;

[0124] Binary constraint: Ridge is connected to more than 2 slopes;

[0125] Ternary constraint: All valley-to-ridge paths have monotonically increasing elevation.

[0126] 6.2.2 Express the terrain constraint objectives using indicator functions and penalized loss, resulting in total constraint loss; the calculation expression of the total constraint loss is:

[0127] L1 =∑ (i,j)∈E (c i = pb,c j = sg)·w ij

[0128]

[0129] L3 =∑ p∈lj ∑ (i,j)∈p max(0, h i -h j )

[0130] L = λ1L1 + λ2L2 + λ3L3

[0131] Wherein, L1 is a plateau-valley adjacency penalty, c i , c j are node classes, w ij is the distance weight of the edge, L2 is a ridge-slope connection constraint, N(i) is the neighbor of node i, pb is a plateau, sg is a valley, sj is a ridge, sp is a slope, L3 is an elevation monotonicity loss, lj is all valley-ridge paths, p is a path, h is an elevation, L is a total constraint loss, λ1, λ2, λ3 are weights.

[0132] 6.2.3 Traverse all nodes and paths to find out violation nodes such as plateau-valley adjacency, ridge isolation, etc. Calculate the class probability of the violation nodes and select the new class of the violation nodes to perform constraint correction, resulting in an initial coherent result; the calculation expression of the new class of the violation nodes is:

[0133]

[0134] Wherein, is the new class of the violation node, p k is the probability of the node being predicted as class k, ΔL(k) is the constraint loss difference brought by the node being assigned as class k, μ is a penalty trade-off parameter.

[0135] 6.3 Based on the initial coherent result, small nodes with pixel size less than 100px are merged into the spatially nearest, same category or highest probability adjacent region, and nodes with discrete category probability distribution and non-geomorphic are deleted to complete the spatial consistency processing and obtain the coherent segmentation result.

[0136] Step 7, the coherent segmentation result is visualized as a soft probability heat map, a Monte-Carlo Dropout uncertainty heat map, an Attention-Rollout attention region heat map, and Grad-CAM spatial explainability layers are generated by selecting target categories for the end convolution or Transformer features of the backbone segmentation network; specifically comprising:

[0137] 7.1 Based on the coherent segmentation result, the category distribution probability of each pixel is calculated, the maximum value of the category distribution probability is selected as the pixel soft confidence, and the pixel soft confidence is displayed by using color mapping to obtain a soft probability heat map, which is simple and intuitive to identify the model forward decision area.

[0138] 7.2 Based on the coherent segmentation result, the uncertainty entropy of each pixel is calculated to obtain an uncertainty region, and the uncertainty region is highlighted locally by using a heat map to obtain a Monte-Carlo Dropout uncertainty heat map, which quantifies the segmentation uncertainty and is superior to single probability judgment, and can mine vulnerabilities that need to be highlighted or manually checked.

[0139] 7.3 Based on the coherent segmentation result, the attention matrix of each Transformer layer in the backbone segmentation network is extracted to calculate the global attention score of the pixel from input to output, and the global attention score is aggregated by category to obtain an Attention-Rollout attention region heat map of the pixel.

[0140] 7.4 For the end convolution or Transformer features of the backbone segmentation network, the target category is selected, the average gradient weight of the feature mapping output for the target category is calculated, and Grad-CAM spatial explainability layers are generated according to the average gradient weight combined with the ReLU function.

[0141] The three types of heat maps are combined with Grad-CAM spatial explainability layers to provide rich spatial decision-making basis for geographic experts and others, and quickly locate the bias source, weak confidence area, attention blind area and segmentation rationality of the model.

[0142] Therefore, the present application provides a topographic map unlabeled region segmentation method based on deep learning, which can handle complex topography and spatial heterogeneity, make topographic map unlabeled region segmentation more accurate, intelligent and efficient, restore the true topography of the unlabeled region, significantly optimize the boundary and spatial distribution, and promote the intelligent upgrading of geographic information systems.

[0143] The various embodiments described in this specification are presented for the purpose of illustrating the principles of the present application and its best mode of operation. Each of the embodiments described in this specification has been provided for the purpose of illustration and is not intended to limit the application.

[0144] The principles and operation of the present application have been explained so far with the aid of specific examples, but it is apparent for those skilled in the art that the present application is capable of significant modifications and applications and is limited only by the following claims.

Claims

1. A deep learning-based unannotated region segmentation method for a topographic map, characterized by, The method comprises the following steps: Collecting a terrain raster, obtaining DEM, remote sensing image and OpenStreetMap vector data corresponding to the terrain raster, and performing multi-channel division and region segmentation on the terrain raster to obtain a multi-source tensor terrain map; A backbone encoder is selected and designed to obtain a backbone network, and self-supervised task design and position encoding design are performed on the backbone network to output feature representation of the multi-source tensor terrain map; including: Based on the multi-source tensor terrain map, Swin-Transformer-Lite is selected as the backbone encoder, a terrain mutation response layer is designed in the backbone encoder, and the elevation second-order derivative statistics is superimposed before each window self-attention in the backbone encoder to assist the window to pay attention to the area with obvious landform change, and then a window sliding strategy is designed to align the DEM and the contour line edge of the main map, thereby completing the design of the backbone encoder and the geographical adaptation to obtain an initial backbone network; For the multi-source tensor terrain map, a fusion hierarchical mask strategy is designed in combination with patch masking to perform geographical sensitive masking, BYOL unsupervised learning is introduced for unlabeled contrast training and multi-view multi-source fusion, the self-supervised task design of the initial backbone network is completed, and a first backbone network is obtained; the fusion hierarchical mask strategy includes 40% patch masking and 60% random patch of the whole map, and in the 40% patch masking, 20% is a landform mutation zone and 20% is an isoline dense zone; Based on two-dimensional position encoding, standardized DEM information and high-order terrain attributes are fused on each patch to generate terrain-aware position encoding, and elevation-gated attention is introduced in the encoding layer of the first backbone network to adaptively adjust the feature extraction weight of the plain and high-difference blocks, thereby completing the design of the position encoding and obtaining a second backbone network for output feature representation; According to the constructed geographical semantic short sentence database, semantic mapping is performed on the multi-source tensor terrain map to obtain a landform semantic description and generate a semantic weight; Based on the backbone network, a segmentation network architecture and a loss function are designed to obtain a backbone segmentation network, and the backbone segmentation network is used to output a spatial segmentation probability and a boundary explicit mask of the terrain raster, thereby completing the initial segmentation prediction of the terrain; According to the spatial segmentation probability and the boundary explicit mask, the prediction uncertainty of the backbone segmentation network is quantified to obtain an uncertainty result, so that multi-round self-iterative backflow training of the backbone segmentation network is performed to obtain a region segmentation model and an initial segmentation result output by the region segmentation model; For the initial segmentation result, pixel connected domain detection, terrain constraint and spatial consistency improvement are performed to obtain a coherent segmentation result; The coherent segmentation result is visualized as a soft probability heat map, a Monte-Carlo Dropout uncertainty heat map and an Attention-Rollout attention area heat map, and for the end convolution or Transformer feature of the backbone segmentation network, a target class is selected to generate a Grad-CAM spatial explainability graph layer.

2. The method of claim 1, wherein the method is based on deep learning. Collect terrain map grids, obtain DEM, remote sensing image and OpenStreetMap vector data corresponding to the terrain map grids, and perform multi-channel division and regional segmentation on the terrain map grids to obtain a multi-source tensor terrain map, including: For the collected terrain map, introduce DEM service and remote sensing API to eliminate stitching errors, align the terrain map, call OSM API to download vector elements of the region corresponding to the terrain map, intelligently filter the terrain map, and obtain terrain map grids; The terrain map grids are divided into a plurality of standard slice blocks of 2048x2048, and each standard slice block is labeled with geographic coordinate indexes and meta information. Each standard slice block is divided into a terrain shadow, a DEM-slope enhanced channel, an OSM road rasterization channel, and a fusion type vector attention map 4 auxiliary channels; Using unsupervised clustering, the standard slice blocks are divided into landforms and labeled according to geographical and statistical characteristics. The landform divisions are sampled to output a mixed dataset including a training set, a validation set and a test set, so that each standard slice block includes a multi-source tensor terrain map; The mixed dataset is mixed into each landform region in the standard slice block in different resolutions to expand the spatial scale of the terrain map grids.

3. The method of claim 2, wherein the method is based on deep learning. According to the constructed geographic semantic short sentence database, the multi-source tensor terrain map is semantically mapped to obtain a landform semantic description and generate a semantic weight, including: Recollecting a plurality of terrain map samples of different landform types, crawling paragraphs related to landforms in an open source paper library, extracting keywords and context description sentences of the paragraphs related to landforms, and automatically clustering, inducing and deduplicating the keywords and context description sentences to obtain a geographic semantic short sentence database; Using the main encoder to extract fixed-length visual vectors of the multi-source tensor terrain map, using BERT or Transformer to encode the geographic semantic short sentence data into text embedding vectors, and performing semantic mapping on the fixed-length visual vectors and the text embedding vectors to obtain a landform semantic description and generate a semantic weight; The multi-source tensor terrain map and the corresponding landform semantic description are set as positive samples, and the multi-source tensor terrain map and the non-corresponding landform semantic description are set as negative samples. Cross-modal contrast learning is used to train the contrast loss of the positive samples and the negative samples. The contrast loss is introduced to complete the cross-modal contrast training of the second main network to obtain a third main network.

4. The method of claim 3, wherein the method further comprises: Based on the main network, a segmentation network architecture and a loss function are designed to obtain a main segmentation network. The main segmentation network is used to output the spatial segmentation probability and boundary explicit mask of the terrain map grid to complete the initial terrain segmentation prediction, including: Based on the third main network, a semantic prompt injection module is inserted between the high-level feature layer in the encoder and the decoder, a multi-scale decoder is designed to fuse with the auxiliary channel, and an output class probability and a main branch and an auxiliary branch outputting a region boundary are designed to obtain a segmentation network architecture. The main loss function is obtained by combining the Focal Tversky Loss and the Lovász Loss, the auxiliary boundary loss function is obtained by combining the boundary IoU Loss and the direction consistency, and the backbone segmentation network is obtained by combining the segmentation network architecture, the main loss function, and the auxiliary boundary loss function. The spatial segmentation probability and the boundary explicit mask of the terrain raster are output by using the backbone segmentation network, and initial terrain segmentation prediction is completed. The semantic prompt injection module fuses the geomorphological semantic description and the encoded features by using a cross-attention mechanism.

5. The method of claim 4, wherein the method is based on deep learning. The prediction uncertainty of the backbone segmentation network is quantified according to the spatial segmentation probability and the boundary explicit mask, and an uncertainty result is obtained to perform multi-round self-iterative backflow training of the backbone segmentation network, so as to obtain a region segmentation model and an initial segmentation result output by the region segmentation model, including: The prediction uncertainty of the backbone segmentation network is quantified by using Monte-Carlo Dropout, and the probability distribution of each patch to be segmented is obtained, and the per-pixel uncertainty, patch confidence, and patch ranking of each patch to be segmented are calculated to obtain an uncertainty result, so as to allocate a labeling task; the calculation expression of the uncertainty result is: wherein, is the uncertainty entropy of the pixel , is the average probability of the class appearing on the pixel , is the average confidence of the patch, is the number of pixels within the patch, is the patch priority ranking score, is the average maximum confidence of the patch, is the average distance of the patch to the labeled set in the feature space, are all weight coefficients. The patches with a patch confidence greater than 0.9 are set as high-confidence pseudo-labels, the artificial labels, the high-confidence pseudo-labels, and consistency constraints are set as training targets, and multi-round self-iterative backflow training of the backbone segmentation network is performed to obtain a region segmentation model, and an initial segmentation result is output by using the region segmentation model.

6. The method of claim 5, wherein the method further comprises: Pixel connected domain detection, terrain constraint, and spatial consistency improvement are performed on the initial segmentation result to obtain a coherent segmentation result, including: Connected domain analysis is used to classify the same-class pixel blocks in the initial segmentation result into a node, and an undirected edge is added between adjacent nodes, attribute values of the nodes and the undirected edges are assigned, and a graph data structure is obtained according to the nodes and the undirected edges; the attributes of the nodes include class probability distribution, average elevation, position centroid, and area, and the attribute of the undirected edge is the class distribution similarity or the elevation gradient at the boundary pixels; Based on the graph data structure, a terrain constraint target is defined, and the constraint is expressed by using an indicator function and a penalty loss, and constraint correction is performed according to a total constraint loss to obtain an initial coherent result. Based on the initial coherent result, small nodes with a pixel size less than 100px are merged into adjacent regions that are spatially closest, have the same class, or have the highest probability, and nodes with discrete class probability distribution and non-geomorphological nodes are deleted, spatial consistency is processed, and a coherent segmentation result is obtained.

7. The method of claim 6, wherein the method is based on deep learning. Based on the graph data structure, a terrain constraint target is defined, and the constraint is expressed by using an indicator function and a penalty loss, and constraint correction is performed according to a total constraint loss to obtain an initial coherent result, including: Defining a terrain constraint target based on the graph data structure; the terrain constraint target includes a unary constraint with a rule that flat areas cannot be directly adjacent to valleys, a binary constraint with a rule that ridges are connected to more than two slopes, and a ternary constraint with a rule that the elevation of all valley-to-ridge paths monotonically increases; Expressing the terrain constraint target by using an indicator function and a penalty loss to obtain a total constraint loss; the calculation expression of the total constraint loss is: wherein, is a flat-valley adjacency penalty, are node classes, is a distance weight for edges, is a ridge-slope connection constraint, is a node neighbor, is a flat, is a valley, is a ridge, is a slope, is an elevation monotonicity loss, is all valley-ridge paths, is a path, is an elevation, is a total constraint loss, are weights; Traversing all nodes and paths to find out a violation node, calculating the category probability of the violation node and selecting a new category of the violation node to perform constraint correction to obtain an initial coherent result; the calculation expression of the new category of the violation node is: where, is a new class for breach nodes, is the probability that a node is predicted to be class , is the constraint loss difference incurred if a node is assigned to class , is a penalty trade-off parameter.

8. The method of claim 7, wherein the method is based on deep learning. Visualizing the coherent segmentation result as a soft probability heat map, a Monte-Carlo Dropout uncertainty heat map, and an Attention-Rollout attention area heat map, and selecting a target category for the end convolution or Transformer feature of the backbone segmentation network to generate a Grad-CAM spatial explainability layer, including: Based on the coherent segmentation result, calculating the category distribution probability of each pixel, selecting the maximum value of the category distribution probability as the pixel soft confidence, and displaying the pixel soft confidence by using color mapping to obtain a soft probability heat map; Based on the coherent segmentation result, calculating the uncertainty entropy of each pixel to obtain an uncertainty area, and highlighting the uncertainty area by using a heat map to obtain a Monte-Carlo Dropout uncertainty heat map; Based on the coherent segmentation result, extracting the attention matrix of each Transformer layer in the backbone segmentation network to calculate the global attention score of a pixel from input to output, aggregating the global attention score by category to obtain an Attention-Rollout attention area heat map of the pixel; For the end convolution or Transformer feature of the backbone segmentation network, selecting a target category, calculating the average gradient weight of the feature mapping output for the target category, and generating a Grad-CAM spatial explainability layer according to the average gradient weight combined with a ReLU function.

Images