Remote sensing regularization segmentation method and system fusing cross-modal calibration and topological constraint

Abstract

The application provides a remote sensing regularization segmentation method and system fusing cross-modal calibration and topological constraint, and relates to the technical fields of remote sensing image processing and computer vision. The method comprises the following steps: acquiring image data and elevation data of a target area; using a trained neural network model to process the image data and the elevation data to obtain a break point graph and a mask graph; the neural network model comprises a backbone network, a multi-scale cross-modal calibration module and a parallel multi-task learning architecture, and the parallel multi-task learning architecture comprises a semantic segmentation branch and a geometric break point detection branch; the vectorization result output by the application has high physical regularity and topological consistency, and significantly improves the mapping accuracy of remote sensing feature extraction.

Description

Technical Field

[0001] This invention relates to the fields of remote sensing image processing and computer vision technology, and in particular to a remote sensing regularization segmentation method and system that integrates cross-modal calibration and topological constraints. Background Technology

[0002] With the rapid popularization of low-altitude unmanned aerial vehicle (UAV) remote sensing technology, acquiring high-resolution imagery and high-precision elevation data has become increasingly convenient, demonstrating enormous application potential in areas such as urban planning, natural resource monitoring, and updating basic geographic information databases. Against this backdrop, achieving automated and standardized extraction of ground features has become a core task in remote sensing applications.

[0003] In recent years, large-scale pre-trained basic models, represented by SAM3 (SegmentAnythingModel3), have made groundbreaking progress in the field of computer vision. SAM3 possesses powerful zero-shot transfer capabilities and extremely high semantic perception accuracy, enabling precise mask segmentation of various target objects in complex scenes through pre-training on massive amounts of data. In remote sensing image processing, SAM3 can effectively handle diverse ground textures and complex environmental backgrounds, providing a solid feature representation foundation for ground feature extraction. However, despite its excellent performance in general segmentation tasks, SAM3 still faces the following bottlenecks that urgently need to be addressed when dealing with specific low-altitude remote sensing scenarios requiring high-precision surveying and regularized mapping: First, remote sensing features have stringent requirements for geometric regularity. Traditional large-scale models and their derivative technologies often focus on pixel-level classification accuracy, neglecting the inherent geometric properties of man-made features such as buildings and roads. This results in irregular, jagged, or fragmented edges in the model output at the pixel level, lacking the right-angled features and long, straight boundaries that buildings should have. Due to the lack of explicit geometric topological constraints, the masks generated by the model cannot be directly converted into vector data according to surveying standards, requiring significant manual interactive re-surveying, which severely limits operational efficiency.

[0004] Secondly, the fusion depth of cross-modal information is insufficient. Although optical images contain rich colors and textures, in complex situations such as light and shadow interference, vegetation occlusion, or extremely similar spectral characteristics between the target and the background, relying solely on visual information is prone to missed detections or false detections. Although Digital Surface Models (DSMs) provide crucial elevation geometric priors, existing fusion strategies often simply overlay elevation information as an auxiliary channel, failing to achieve deep alignment and dynamic calibration of elevation gradients and visual features in the spatial dimension. Consequently, the guiding role of elevation information in boundary determination is not fully utilized.

[0005] Furthermore, maintaining topological consistency for contiguous targets is challenging. In densely built-up urban areas, different types of features are often spatially closely connected. When conventional segmentation models process such contiguous targets, features at category boundaries are prone to sticking or becoming confused, leading to the loss of geometric vertices of shared boundaries. Accurately extracting and preserving these complex topological connections is crucial for achieving surveying-grade vectorization results.

[0006] Finally, the extraction of key geometric signals faces the challenge of training imbalance. Introducing inflection point detection into segmentation tasks to enhance regularity must address the objective reality of extremely sparse inflection point pixels in remote sensing images. In images with millions of pixels, key inflections typically account for less than one ten-thousandth. This extreme sample imbalance can cause gradient drowning in traditional loss functions during training, making it difficult for the model to capture weak geometric constraint signals, resulting in unstable training or poor model convergence performance.

[0007] In summary, although basic models such as SAM3 provide excellent semantic perception capabilities, there is still an urgent need for a regularized segmentation scheme that can deeply integrate cross-modal features, strengthen hierarchical topological constraints, and have stable learning capabilities under extremely sparse signals for remote sensing ground object extraction. Summary of the Invention

[0008] This invention provides a remote sensing regularization segmentation method and system that integrates cross-modal calibration and topological constraints, which can significantly improve the mapping accuracy of remote sensing ground feature extraction.

[0009] In a first aspect, the present invention provides a remote sensing regularization segmentation method that integrates cross-modal calibration and topological constraints, comprising: Acquire image and elevation data of the target area; The image data and elevation data are processed using a trained neural network model to obtain a vertex map and a mask map. The neural network model includes a backbone network, a multi-scale cross-modal calibration module, and a parallel multi-task learning architecture. The parallel multi-task learning architecture includes a semantic segmentation branch and a geometric vertex detection branch. The processing of the image data and elevation data using the trained neural network model includes: The image data is used to extract features using a backbone network to obtain the visual features of the image data; based on the elevation data, the visual features are calibrated using a multi-scale cross-modal calibration module to obtain the calibrated visual features; The calibrated visual features are processed using a semantic segmentation branch to obtain a mask map representing the semantic segmentation result of the visual features; the calibrated visual features are processed using a geometric inflection detection branch to obtain an inflection map representing the geometric structure information of the visual features. The mask image is corrected using the inflection point image to obtain the vector results of the ground feature elements.

[0010] Secondly, the present invention also provides a remote sensing regularization segmentation system that integrates cross-modal calibration and topological constraints, comprising: The data acquisition module is used to acquire image data and elevation data of the target area; A data processing module is used to process the image data and elevation data using a trained neural network model to obtain a vertex map and a mask map. The neural network model includes a backbone network, a multi-scale cross-modal calibration module, and a parallel multi-task learning architecture. The parallel multi-task learning architecture includes a semantic segmentation branch and a geometric vertex detection branch. The processing of the image data and elevation data using the trained neural network model includes: The image data is used to extract features using a backbone network to obtain the visual features of the image data; based on the elevation data, the visual features are calibrated using a multi-scale cross-modal calibration module to obtain the calibrated visual features; The calibrated visual features are processed using a semantic segmentation branch to obtain a mask map representing the semantic segmentation result of the visual features; the calibrated visual features are processed using a geometric inflection detection branch to obtain an inflection map representing the geometric structure information of the visual features. The correction module is used to correct the mask map using the vertex map to obtain the vector results of the ground feature elements.

[0011] Thirdly, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the remote sensing regularization segmentation method integrating cross-modal calibration and topological constraints as described above.

[0012] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the remote sensing regularization segmentation method integrating cross-modal calibration and topological constraints as described above.

[0013] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the remote sensing regularization segmentation method that integrates cross-modal calibration and topological constraints as described above.

[0014] According to the specific embodiments provided in this application, the following technical effects are disclosed: The remote sensing regularization segmentation method and system provided by this invention, which integrates cross-modal calibration and topological constraints, first extracts visual features from the acquired image data through a backbone network. Then, using elevation data as a guiding signal, a multi-scale cross-modal calibration module dynamically adjusts the response intensity of visual features at object edges, achieving depth alignment and edge enhancement of cross-modal features. Subsequently, the features are input into the semantic segmentation branch and the geometric inflection point detection branch to achieve joint decoding of semantic information and geometric structure, outputting a mask map and an inflection point map. Finally, using the geometric inflection points in the inflection point map as physical anchor points, a coordinate snapping algorithm is used to reshape the mask map, eliminating pixel-level irregular fluctuations. Combined with vector orthogonalization correction for building elements, this ensures that building edges have strict straightness and vertical physical characteristics. This results in a qualitative change in the generated vectorized output from pixel probability distribution to regularized polygons. The generated vectorized output has high physical regularity and topological consistency, significantly improving the mapping accuracy of remote sensing ground feature extraction. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating the remote sensing regularization segmentation method that integrates cross-modal calibration and topological constraints provided by the present invention. Figure 2 This is a schematic diagram of the neural network model in the remote sensing regularization segmentation method that integrates cross-modal calibration and topological constraints provided by the present invention; Figure 3 This is a schematic diagram of the structure of the multi-scale cross-modal calibration module in the remote sensing regularization segmentation method that integrates cross-modal calibration and topological constraints provided by the present invention; Figure 4 This is a schematic diagram of the structure of the depth-separable convolutional block in the remote sensing regularization segmentation method that integrates cross-modal calibration and topological constraints provided by the present invention; Figure 5 This is a schematic diagram of the geometric inflection point detection branch in the remote sensing regularization segmentation method that integrates cross-modal calibration and topological constraints provided by the present invention; Figure 6 This is a schematic diagram of the hierarchical topology vertices extraction and encoding results provided by the present invention. Figure 6 (a) in the diagram is a truth mask for multi-class classification; Figure 6 (b) in the figure represents the result of closed vector contour extraction; Figure 6 (c) in the diagram is a schematic diagram of the key topological inflection point; Figure 6 (d) in the text is the Gaussian thermal field label; Figure 7 This diagram illustrates a performance comparison between the remote sensing regularization segmentation method integrating cross-modal calibration and topological constraints provided by this invention and the multi-class regularization segmentation performance of U-Net, DeepLabV3+, Segformer, CMX, and the original SAM3 model. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0018] like Figure 1 As shown, this application provides a remote sensing regularization segmentation method that integrates cross-modal calibration and topological constraints. This method is executed by a computer device, specifically by a terminal or server alone, or by both. In this embodiment, the remote sensing regularization segmentation method integrating cross-modal calibration and topological constraints includes the following steps S101 to S103. Wherein: Step S101: Obtain image data and elevation data of the target area; the image data is low-altitude image data, specifically remote sensing data, and the elevation data is DSM data. After obtaining the above image data and elevation data, preprocessing is required, such as linear normalization of the elevation data.

[0019] Step S102: Process the image data and elevation data using the trained neural network model to obtain a vertex map and a mask map; the neural network model includes a backbone network, a multi-scale cross-modal calibration module, and a parallel multi-task learning architecture, the parallel multi-task learning architecture including a semantic segmentation branch and a geometric vertex detection branch; wherein, processing the image data and elevation data using the trained neural network model includes the following steps S201 to S202. Wherein: Step S201: Use the backbone network to extract features from the image data to obtain the visual features of the image data; calibrate the visual features using a multi-scale cross-modal calibration module based on the elevation data to obtain the calibrated visual features; Step S202: Process the calibrated visual features using the semantic segmentation branch to obtain a mask map representing the semantic segmentation result of the visual features; process the calibrated visual features using the geometric inflection detection branch to obtain an inflection map representing the geometric structure information of the visual features; Step S103: Use the inflection point map to correct the mask map to obtain the vector results of the ground feature elements.

[0020] By implementing steps S101 to S103, this application first extracts visual features from the acquired image data through a backbone network. Then, using elevation data as a guiding signal, a multi-scale cross-modal calibration module is used to dynamically adjust the response intensity of visual features at the object edges, achieving depth alignment and edge enhancement of cross-modal features. Subsequently, the features are input into the semantic segmentation branch and the geometric inflection point detection branch to achieve joint decoding of semantic information and geometric structure, outputting a mask map and an inflection point map. Finally, using the geometric inflection points in the inflection point map as physical anchor points, a coordinate snapping algorithm is used to reshape the mask map, eliminating pixel-level irregular fluctuations. Combined with vector orthogonalization correction for building elements, this ensures that the building edges have strict straightness and vertical physical features, resulting in a qualitative change from pixel probability distribution to regular polygons. The generated vectorized results have high physical regularity and topological consistency, significantly improving the mapping accuracy of remote sensing ground feature extraction.

[0021] As one embodiment of this application, step S201, which involves using a backbone network to extract features from the image data to obtain the visual features of the image data, includes steps S301 to S302. Wherein: Step S301: Perform multi-stage feature extraction operations on the image data using the backbone network; Step S302: Extract the output of the feature extraction operation at each execution stage to obtain a multi-scale feature map; the multi-scale feature map is the visual feature of the image data.

[0022] In this embodiment, the backbone network is the image editor in the SAM3 visual large model. This image editor is based on the ViT-Large (VisionTransformer) network architecture and is used to process the input image data. Specifically, a segmented extraction mechanism is used to obtain the feature responses of image data at different depth stages, and three visual feature maps of different resolutions (scales) are extracted in parallel. .in The resolution is the same as the original image. .like Figure 2As shown, Stage 1 in the backbone network receives the input image data. After the image data undergoes feature extraction operations at three different depth stages, Stage 2, Stage 3, and Stage 4, three visual features F2, F3, and F4 are obtained.

[0023] As one embodiment of this application, step S201, which involves calibrating the visual features using a multi-scale cross-modal calibration module based on the elevation data, includes steps S401 to S402. Wherein: Step S401: Use the multi-scale cross-modal calibration module to perform feature extraction on the elevation data to obtain the elevation geometric features of the elevation data.

[0024] In this embodiment of the application, the multi-scale cross-modal calibration module is an MS-CMC module, such as... Figure 2 As shown, multiple MS-CMC modules are configured, with each MS-CMC module deployed at each feature scale cascade, i.e., connected to the feature extraction operation at the corresponding depth, responsible for performing cross-modal feature alignment and enhancement. Specifically, as... Figure 3 As shown, the MS-CMC module contains four consecutive depthwise and pointwise conv blocks, which are used to process the normalized elevation data (DSM data). Process it. For example... Figure 4 As shown, the depthwise separable convolutional block includes a 3×3 depthwise convolutional layer (DepthwiseConv), a 1×1 pointwise convolutional layer (PointwiseConv), layer normalization, activation functions, and a projection layer. The depthwise convolutional layer extracts local geometric gradients and spatial abrupt changes from the elevation data, obtaining its spatial features. The pointwise convolutional layer projects the data along the channel dimension, transforming spatial features into channel features. Layer normalization and activation functions perform non-linear processing on the channel features. The projection layer maps the channel features to a dimension consistent with the visual features, generating corresponding elevation geometric features. .

[0025] Step S402: Generate spatial scaling parameters and spatial offset parameters based on the elevation geometric features, and use the spatial scaling parameters and spatial offset parameters to calibrate the visual features to obtain multiple calibrated visual features.

[0026] For the elevation geometry features generated in step S401 This application uses two sets of parallel 1×1 convolutional layers to map and generate spatial scaling parameters respectively. and spatial offset parameters Finally, the generated spatial scaling parameters are used. and spatial offset parameters The calibration formula for pixel-level reshaping of visual features is as follows: ; in, Represents the original visual features at the i-th scale. This represents the calibrated visual features. At the edges of objects where the elevation gradient changes drastically, the spatial scaling parameter... This is used to force enhanced feature contrast in boundary regions with abrupt changes in physical height, suppress texture noise in regions with gentle height transitions, and force visual features to align with physical boundaries; in flat regions, spatial offset parameters... Used to compensate for noise interference.

[0027] The aforementioned multi-scale cross-modal calibration module addresses the spatial mismatch between visual features and elevation geometric features by designing a calibration mechanism based on affine transformation. It utilizes the elevation gradient extracted from the DSM as a guiding signal to dynamically adjust the response intensity of visual features at object edges, achieving depth alignment and edge enhancement of cross-modal features. By implementing steps S401 to S402, this application uses the elevation geometric information of the elevation data to guide the perceptual distribution of visual features and, through calculating adaptive affine transformation parameters, performs spatial reshaping and calibration of the visual feature map in the feature mapping space. This mechanism enables the model to achieve accurate boundary definition even in complex scenes with extremely similar spectral features, relying on physical height jump features, significantly enhancing the model's ability to perceive three-dimensional spatial information.

[0028] As one embodiment of this application, after step S201, the method further includes using bilinear interpolation to uniformly map each calibrated visual feature to a high-resolution feature space and perform feature fusion; the fused depth features are used to inject parallel semantic segmentation branches and inflection detection branches to achieve joint decoding of semantic information and geometric structure in step S202, and output mask map and inflection map.

[0029] In step S202 of this embodiment, the semantic segmentation branch consists of multi-layered stacked bidirectional cross-attention modules. The semantic segmentation branch performs global attention addressing on the feature map using a preset set of learnable category query vectors. Each category query vector is responsible for capturing the spectral and textural features of a specific land feature, thereby achieving semantic-level decoupling. The geometric inflection point detection branch learns the topological corner information displayed in the feature map and outputs Gaussian thermodynamic field labels for key topological inflections, providing geometric anchors for subsequent regularization corrections; such as... Figure 5As shown, the geometric inflection detection branch adopts a lightweight fully convolutional structure, which includes three consecutive residual feature refinement blocks. Each residual feature refinement block consists of a 3×3 convolution, group normalization, and ReLU activation function. This structure can enhance the model's feature response capability to local geometric abrupt changes while maintaining computational efficiency.

[0030] In one embodiment of this application, step S103, which involves correcting the mask image using the inflection point image, includes steps S501 to S504. Wherein: Step S501: Extract the extreme points of the inflection point graph as predicted inflection points, and construct a global predicted inflection point set based on the predicted inflection points. Specifically, this application first uses 3×3 non-maximum suppression to locate extreme points in the inflection point graph, and then constructs a global predicted inflection point set based on the extreme points.

[0031] Step S502: Simplify the mask edges of the mask image using a polygon approximation algorithm to generate a preliminary vector contour; specifically, based on the above-mentioned global prediction vertex set, this application uses a polygon approximation algorithm to simplify the mask edges of the vertex image, according to the tolerance... Redundant edge points lacking geometric significance are removed to generate a preliminary vector profile; the tolerance... The formula for calculating tolerance is the same as the formula for calculating tolerance in the neural network model training process described below.

[0032] Step S503: Traverse the mask edge points of the preliminary vector contour, retrieve the corresponding points in the global predicted inflection point set within the preset radius, and perform coordinate snapping to force the mask edge points to align with the predicted inflection points, thereby eliminating pixel-level jagged edges and obtaining the snapped contour.

[0033] Step S504: Perform geometric constraint correction based on vector orthogonality on the building elements in the adsorbed contour to obtain the vector result of the land feature elements.

[0034] Furthermore, in step S504 of this embodiment, the geometric constraint correction based on vector orthogonality is performed on the architectural elements in the adsorbed contour, including steps S601 to S602. Wherein: Step S601: Perform vector analysis on the building elements in the adsorbed contour and calculate the cosine similarity of the adjacent edge vectors of the building elements.

[0035] Step S602: Determine whether the cosine similarity is less than a preset threshold. If the determination result is yes, use the least squares method to perform straight line fitting on the adjacent edges of the building elements and force their normal vectors to be perpendicular to each other to achieve strict regularization of the building edges.

[0036] By implementing steps S501 to S504 and steps S601 to S602, this application addresses the jaggedness and fragmentation problems commonly found in remote sensing segmentation. It treats inflection point detection as a strongly constrained auxiliary task, uses predicted geometric inflection points as physical anchor points, and reshapes the segmentation mask using a coordinate snapping algorithm to eliminate pixel-level irregular fluctuations. Furthermore, by combining vector orthogonalization correction for building elements, it ensures that building edges possess strict straightness and vertical physical characteristics, resulting in a qualitative leap from pixel probability distribution to regular polygons in the generated vectorized output.

[0037] This application also provides a method for training the neural network model in step S102, wherein the method for training the neural network model includes the following steps S701 to S704. Wherein: Step S701: Acquire image data and elevation data for different regions. The image data is labeled with ground truth labels, which correspond to the categories of ground features in the image data. The ground truth labels are manually interpreted ground truth labels for the ground features. After acquiring the image data and elevation data for the different regions, a semantic segmentation dataset containing training, validation, and test sets is constructed to be suitable for the training, validation, and testing phases of the neural network model. In this embodiment, the categories of ground features include "buildings," "water bodies," "roads," "cultivated land," "forest land," and "low vegetation."

[0038] Step S702: Based on the image data and the truth label (i.e., ... Figure 2 Generate Gaussian thermal field labels using the truth mask in the data; Step S703: Input the image data and elevation data of the different regions into the neural network model, and output the predicted inflection map and the predicted mask map; Step S704: Calculate the training loss of the neural network model based on the predicted inflection point map, the predicted mask map, and the Gaussian thermal field label. The training loss includes... Figure 2 The semantic loss and inflection point loss are used in the training process; the neural network model is optimized based on the training loss to obtain a trained neural network model.

[0039] Please see Figure 6 , Figure 6 (a) in the image is a multi-class ground truth mask, representing the ground truth labels of various features in the image data; Figure 6 (b) in the figure represents the closed vector contour extraction result, indicating the closed vector contour of each feature. Figure 6 (c) in the figure is a schematic diagram of key topological inflections. The circles in the figure represent the key topological inflections of each feature. Figure 6(d) in the text is the Gaussian thermodynamic field label of the inflection point obtained from the key topological inflection point encoding.

[0040] Based on the above Figure 6 In step S702 of this application, the generation of Gaussian thermal field labels based on the image data and the ground truth labels includes the following steps S801 to S803. Wherein: Step S801: The image data is split into multiple binary mask planes according to the ground truth labels. In step S801, this application first performs hierarchical topological vertex extraction on the image data according to the ground truth labels. Specifically, the foreground of the image data is traversed by hierarchical scanning, and the classification map containing N categories (each ground truth label corresponds to one category) is split into N independent binary mask planes according to the ground truth labels.

[0041] Step S802: The closed vector contours of each feature element in the binary mask plane are extracted using a topological contour tracing algorithm. In step S802, for each binary mask plane, the topological contour tracing algorithm is independently called to extract the polygon point set, thereby obtaining the closed vector contour of the feature element. It can be understood that by implementing the above steps S801 to S802, this application eliminates the contour fusion phenomenon caused by different types of features being closely adjacent in spatial location through this layering mechanism, ensuring that common vertices on the boundary line are completely preserved in the geometric description of each category.

[0042] It should be noted that the specific implementation method for extracting polygon point sets by independently calling the topological contour tracing algorithm to obtain the closed vector contour of the ground feature is as follows: using a boundary-following-based topological tracing operator, the outer edge pixel coordinate sequence is extracted by scanning the connected regions with a pixel value of 1 in the binary mask plane. , This represents the coordinates of the m-th edge pixel on the periphery of the connected region. This mechanism, through physical decoupling, eliminates the contour fusion phenomenon that occurs when different features are spatially aligned, ensuring that common vertices on the boundary line are fully preserved in the geometric descriptions of all categories.

[0043] Step S803: The closed vector contour is simplified using an approximation algorithm with threshold capping constraints. The polygon vertices of the simplified ground features are retained as key topological vertices, and the key topological vertices are encoded as Gaussian thermal field labels.

[0044] Because the original closed vector contour extracted in step S802 contains a large number of redundant points and jagged noise, this invention uses an improved Douglas-Peucker algorithm (a polygon approximation algorithm with threshold capping constraints) to adaptively simplify the original closed vector contour. The tolerance calculation formula in the approximation algorithm with threshold capping constraints is as follows: ; In the formula, L represents the cumulative perimeter of the closed vector profile of the ground feature; k is the proportional sensitivity factor, with a value range of [value missing]. T represents the rigid capping threshold in the pixel dimension. It should be noted that the tolerance is the threshold of the maximum allowable deviation between the original closed vector contour and the simplified closed vector contour. This application uses tolerance as a constraint to preserve subtle textures of small objects and effectively suppresses oversimplification of corners caused by linear scaling failure in ultra-large perimeter objects, maintaining global topological consistency. In summary, this application introduces a dual constraint mechanism based on perimeter scaling and a rigid threshold to remove pixel-level noise while preserving key geometric features.

[0045] After simplification, the vertices of the polygon are retained as key topological vertices. The sequence of key topological vertices can be represented as follows: , This represents the k-th critical topological inflection point in the inflection point sequence. To enable end-to-end alignment of the discrete coordinates with the multi-scale feature map output by the backbone network, this invention encodes the inflection point sequence of critical topological inflections as spatially continuous Gaussian thermodynamic field labels. For each pixel in the image... Its response value in the thermal field is composed of the superposition of Gaussian distributions of all key topological inflection points, and the calculation formula is: ; in, Represents pixels The response value in the thermal field; Indicates the first Spatial coordinates of key topological inflection points; This represents the i-th key topological vertex in the vertex sequence; The standard deviation (scale factor) is used to control the propagation range of the inflection point signal in the thermal field. Through this process, discrete vector inflections are transformed into a heatmap with a spatial probability distribution, enabling the network to learn the local geometric features of ground feature corners through convolution operations and guide the mask edges to accurately adhere to the predicted geometric center during the inference stage.

[0046] By implementing steps S801 to S803, this application provides a hierarchical topological inflection point extraction technology. Addressing the topological fusion problem caused by the contiguous distribution of remotely sensed ground features, it proposes a category-by-category decoupled scanning logic. By independently extracting the contours of each category and adopting an adaptive tolerance formula constrained by both perimeter and capping threshold, it achieves accurate extraction of shared boundary inflection points in complex scenes, ensuring that the topological structure is not lost.

[0047] In step S704 of this application embodiment, the training loss of the neural network model includes semantic segmentation loss and inflection point detection loss. The formula for calculating the training loss of the neural network model is as follows: ; in, This represents the total loss, i.e., the training loss of the neural network model; These represent the weight coefficients of the semantic segmentation loss and the inflection point detection loss, respectively, used to balance the gradient contributions between different tasks. The semantic segmentation loss is used to constrain the accuracy of pixel classification and the integrity of the region, and its calculation formula is: ; In the formula, This represents the multi-class cross-entropy loss, used to constrain pixel-level classification accuracy; This represents the Dice loss, used to optimize the overlap between the predicted and real regions. This represents the weight adjustment coefficient, used to balance the contributions of cross-entropy loss and Dice loss to the model optimization process. Where: Multi-class cross-entropy loss The calculation formula is: ; Where M is the total number of pixels in the image, and N is the total number of land cover categories. For pixels Truth labels belonging to category c Indicates the model predicts pixels The probability of belonging to category c, this probability It is obtained by processing the mask image of the semantic branch output by the softmax function.

[0048] Multiclass Dice loss The calculation formula is: ; in, This represents a smoothing factor used to prevent the denominator from being zero. This loss term effectively mitigates the boundary fragmentation problem caused by class imbalance in remote sensing images.

[0049] To address the gradient flooding problem caused by the extreme imbalance in the proportion of positive and negative sample pixels in the inflection point heatmap, this application employs an improved numerically balanced FocalLoss as the numerically balanced inflection point loss. Numerical equilibrium inflection point loss The calculation formula is: ; in, The number of positive sample pixels with a probability value greater than a preset threshold in the Gaussian thermal field is calculated. The probability value in the Gaussian thermal field is the response value of the pixel in the thermal field, which is calculated by the above calculation formula of superimposed Gaussian distribution. Count the negative sample pixels in the background region; , For the first The positive sample pixel or the first FocalLoss basic loss for each negative sample pixel; This is a numerical smoothing factor to prevent the denominator from being zero and to ensure the stability of numerical calculations. This numerical balances the inflection point loss. By independently calculating the mean and then summing the results, the contribution weights of sparse keypoint features and the overall background to the model gradient are forcibly balanced.

[0050] In summary, this application designs a numerical balance loss function for training neural networks. To address the gradient flooding problem caused by extremely sparse inflection point signals (usually accounting for less than one ten-thousandth), it constructs an independent mean-normalized inflection point loss term. By forcibly balancing the average gradient contribution of positive and negative sample regions, it solves the technical problem of unstable training and difficulty in convergence of the model in large-scale remote sensing images.

[0051] As an optional implementation of this application, the training method of the neural network further includes a virtual batch gradient management step, which includes: Step 1: Under the condition of limited single-pass memory processing units (the memory processing units cannot support large batch training), set the gradient accumulation step number to M; Step 2: Perform M consecutive forward propagations on the neural network model and calculate the loss, but do not update the parameters of the neural network model until the accumulated gradient reaches the equivalent large batch size. Then, execute the optimizer stepping to ensure the statistical directional stability of the extremely sparse signal. For example, if a training batch contains 32 images, and the gradient accumulation step count is set to M=8, with 4 images processed in each forward propagation, after 8 forward propagations, the accumulated gradient reaches the equivalent large batch size (i.e., 32), and then the optimizer stepping is executed to optimize the parameters of the neural network.

[0052] In summary, to address the gradient submersion problem caused by the extremely low proportion of inflection pixels in large-scale remote sensing imagery, this invention constructs an independent mean-normalized numerical balance loss function. This mechanism forcibly balances the contribution weights of extremely sparse geometric signal regions and massive background regions to the gradient, resolving the numerical oscillations or slow convergence issues that traditional FocalLoss is prone to in such tasks. Furthermore, by setting up virtual batch gradient management, this application enables the model to capture weak geometric constraint signals in a large statistical space even with limited GPU memory resources, greatly improving the robustness of model training.

[0053] Based on the same inventive concept, this application provides a remote sensing regularized segmentation system that integrates cross-modal calibration and topological constraints. The remote sensing regularized segmentation system that integrates cross-modal calibration and topological constraints provided by this invention will be described below. The remote sensing regularized segmentation system that integrates cross-modal calibration and topological constraints described below can be referred to in correspondence with the remote sensing regularized segmentation method that integrates cross-modal calibration and topological constraints described above.

[0054] A remote sensing regularization segmentation system integrating cross-modal calibration and topological constraints includes a data acquisition module, a data processing module, and a correction module. Wherein: The data acquisition module is used to acquire image data and elevation data of the target area; A data processing module is used to process the image data and elevation data using a trained neural network model to obtain a vertex map and a mask map. The neural network model includes a backbone network, a multi-scale cross-modal calibration module, and a parallel multi-task learning architecture. The parallel multi-task learning architecture includes a semantic segmentation branch and a geometric vertex detection branch. The processing of the image data and elevation data using the trained neural network model includes: The image data is used to extract features using a backbone network to obtain the visual features of the image data; based on the elevation data, the visual features are calibrated using a multi-scale cross-modal calibration module to obtain the calibrated visual features; The calibrated visual features are processed using a semantic segmentation branch to obtain a mask map representing the semantic segmentation result of the visual features; the calibrated visual features are processed using a geometric inflection detection branch to obtain an inflection map representing the geometric structure information of the visual features. The correction module is used to correct the mask map using the vertex map to obtain the vector results of the ground feature elements.

[0055] Please see Figure 7 , Figure 7This paper demonstrates the performance of the proposed method in multi-class regularized segmentation when processing low-altitude remote sensing imagery. A qualitative comparison is made between the proposed method and mainstream semantic segmentation networks (U-Net, DeepLabV3+, Segformer, CMX) and the original SAM3 model without elevation prior fusion. The results show that the proposed method has significant advantages in utilizing DSM spatial geometric information, maintaining mask regularity, and ensuring topological consistency. Figure 7 It is known that masks generated by traditional models generally suffer from technical bottlenecks such as severe jagged edges, fragmented internal structures, and loss of key topological corners. In contrast, the extraction results generated by this invention exhibit high edge straightness, coherent internal regions, and no salt-and-pepper noise, with their geometric contours most closely resembling the true physical boundaries of ground features. This fully verifies the superior effectiveness of the cross-modal calibration module and topological constraint logic in improving the refinement and regularization of remote sensing ground feature extraction.

[0056] In one exemplary embodiment, this application also provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0057] In one exemplary embodiment, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0058] In one exemplary embodiment, this application provides a computer program product including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0059] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0060] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0061] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0062] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0063] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A remote sensing regularization segmentation method integrating cross-modal calibration and topological constraints, characterized in that, include: Acquire image and elevation data of the target area; The image data and elevation data are processed using a trained neural network model to obtain a vertex map and a mask map. The neural network model includes a backbone network, a multi-scale cross-modal calibration module, and a parallel multi-task learning architecture. The parallel multi-task learning architecture includes a semantic segmentation branch and a geometric vertex detection branch. The processing of the image data and elevation data using the trained neural network model includes: The image data is used to extract features using a backbone network to obtain the visual features of the image data; based on the elevation data, the visual features are calibrated using a multi-scale cross-modal calibration module to obtain the calibrated visual features; The calibrated visual features are processed using a semantic segmentation branch to obtain a mask map representing the semantic segmentation result of the visual features; the calibrated visual features are processed using a geometric inflection detection branch to obtain an inflection map representing the geometric structure information of the visual features. The mask image is corrected using the inflection point image to obtain the vector result of the ground feature elements; The step of modifying the mask image using the inflection point image includes: Extract the extreme points of the inflection plot as predicted inflection points, and construct a global predicted inflection point set based on the predicted inflection points; A polygon approximation algorithm is used to simplify the mask edges of the mask image to generate a preliminary vector contour; Traverse the mask edge points of the preliminary vector contour, retrieve the corresponding points in the global predicted inflection point set within the preset radius and perform coordinate snapping to force the mask edge points to align with the predicted inflection points, thus obtaining the snapped contour. Perform geometric constraint correction based on vector orthogonality on the building elements in the adsorbed contour to obtain the vector result of the ground feature elements; The step of performing geometric constraint correction based on vector orthogonality on the architectural features in the adsorbed contour includes: Vector analysis is performed on the building elements in the adsorbed contour to calculate the cosine similarity of the adjacent edge vectors of the building elements. Determine whether the cosine similarity is less than a preset threshold. If the determination result is yes, use the least squares method to perform straight line fitting on the adjacent edges of the building elements and force their normal vectors to be perpendicular to each other. The training method for the neural network model includes the following steps: Acquire image data and elevation data for different regions. The image data is labeled with ground value tags, and the ground value tags correspond to the ground features in the image data. Gaussian thermal field labels are generated based on the image data and the ground truth labels; The image data and elevation data of the different regions are input into the neural network model, which outputs a predicted inflection map and a predicted mask map. The training loss of the neural network model is calculated based on the predicted inflection point map, the predicted mask map, and the Gaussian thermal field label; the neural network model is optimized based on the training loss to obtain a trained neural network model. The step of generating Gaussian thermal field labels based on the image data and the ground truth labels includes: The image data is split into multiple binary mask planes based on the truth labels; The closed vector contours of each feature in the binary mask plane are extracted using a topological contour tracing algorithm. An approximation algorithm with threshold capping constraints is used to simplify the closed vector contour. The polygon vertices of the simplified ground features are retained as key topological vertices, and the key topological vertices are encoded as Gaussian thermal field labels.

2. The remote sensing regularization segmentation method integrating cross-modal calibration and topological constraints according to claim 1, characterized in that, The step of using a backbone network to extract features from the image data to obtain the visual features of the image data includes: The image data is subjected to multi-stage feature extraction operations using a backbone network. The outputs of the feature extraction operation at each execution stage are extracted to obtain a multi-scale feature map; the multi-scale feature map is the visual feature of the image data.

3. The remote sensing regularization segmentation method integrating cross-modal calibration and topological constraints according to claim 1, characterized in that, The step of calibrating the visual features using a multi-scale cross-modal calibration module based on the elevation data includes: The elevation data is subjected to feature extraction using a multi-scale cross-modal calibration module to obtain the elevation geometric features of the elevation data. Spatial scaling parameters and spatial offset parameters are generated based on the elevation geometric features. The visual features are then calibrated using the spatial scaling parameters and spatial offset parameters to obtain multiple calibrated visual features.

4. A remote sensing regularization segmentation system integrating cross-modal calibration and topological constraints, characterized in that, include: The data acquisition module is used to acquire image data and elevation data of the target area; A data processing module is used to process the image data and elevation data using a trained neural network model to obtain a vertex map and a mask map. The neural network model includes a backbone network, a multi-scale cross-modal calibration module, and a parallel multi-task learning architecture. The parallel multi-task learning architecture includes a semantic segmentation branch and a geometric vertex detection branch. The processing of the image data and elevation data using the trained neural network model includes: The image data is used to extract features using a backbone network to obtain the visual features of the image data; based on the elevation data, the visual features are calibrated using a multi-scale cross-modal calibration module to obtain the calibrated visual features; The calibrated visual features are processed using a semantic segmentation branch to obtain a mask map representing the semantic segmentation result of the visual features; the calibrated visual features are processed using a geometric inflection detection branch to obtain an inflection map representing the geometric structure information of the visual features. The correction module is used to correct the mask map using the vertex map to obtain the vector result of the ground feature elements; The step of modifying the mask image using the inflection point image includes: Extract the extreme points of the inflection plot as predicted inflection points, and construct a global predicted inflection point set based on the predicted inflection points; A polygon approximation algorithm is used to simplify the mask edges of the mask image to generate a preliminary vector contour; Traverse the mask edge points of the preliminary vector contour, retrieve the corresponding points in the global predicted inflection point set within the preset radius and perform coordinate snapping to force the mask edge points to align with the predicted inflection points, thus obtaining the snapped contour. Perform geometric constraint correction based on vector orthogonality on the building elements in the adsorbed contour to obtain the vector result of the ground feature elements; The step of performing geometric constraint correction based on vector orthogonality on the architectural features in the adsorbed contour includes: Vector analysis is performed on the building elements in the adsorbed contour to calculate the cosine similarity of the adjacent edge vectors of the building elements. Determine whether the cosine similarity is less than a preset threshold. If the determination result is yes, use the least squares method to perform straight line fitting on the adjacent edges of the building elements and force their normal vectors to be perpendicular to each other. The training method for the neural network model includes the following steps: Acquire image data and elevation data for different regions. The image data is labeled with ground value tags, and the ground value tags correspond to the ground features in the image data. Gaussian thermal field labels are generated based on the image data and the ground truth labels; The image data and elevation data of the different regions are input into the neural network model, which outputs a predicted inflection map and a predicted mask map. The training loss of the neural network model is calculated based on the predicted inflection point map, the predicted mask map, and the Gaussian thermal field label; the neural network model is optimized based on the training loss to obtain a trained neural network model. The step of generating Gaussian thermal field labels based on the image data and the ground truth labels includes: The image data is split into multiple binary mask planes based on the truth labels; The closed vector contours of each feature in the binary mask plane are extracted using a topological contour tracing algorithm. An approximation algorithm with threshold capping constraints is used to simplify the closed vector contour. The polygon vertices of the simplified ground features are retained as key topological vertices, and the key topological vertices are encoded as Gaussian thermal field labels.

5. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the remote sensing regularization segmentation method that integrates cross-modal calibration and topological constraints as described in any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the remote sensing regularization segmentation method that integrates cross-modal calibration and topological constraints as described in any one of claims 1 to 3.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the remote sensing regularization segmentation method that integrates cross-modal calibration and topological constraints as described in any one of claims 1 to 3.

Images