Topographic data extraction and processing methods, apparatus, computer equipment and storage media

Abstract

This invention provides a method, apparatus, computer device, and readable storage medium for terrain data extraction and processing, relating to the field of surveying and mapping data processing technology. The method includes: identifying ground features based on three-dimensional scene data of the area to be processed, obtaining an initial identification result including the identified ground features and initial terrain data; determining the target location in response to a target selection operation, and inputting it into a first segmentation model to obtain an initial region of the target ground feature; inputting the initial region as a first segmentation prior into a second segmentation model to obtain an optimized region; determining the correction location and correction information in response to a correction operation, and inputting the optimized region as a second segmentation prior into a second segmentation model to obtain a corrected region; updating the initial terrain data using the restored region including the corrected region to obtain terrain data. This invention improves the accuracy and processing efficiency of terrain extraction.

Description

Technical Field

[0001] This invention relates to the field of surveying and mapping data processing technology, and in particular to a method, apparatus, computer equipment, and readable storage medium for extracting and processing terrain data. Background Technology

[0002] With the development of UAV aerial surveying, oblique photogrammetry, and 3D reconstruction technologies, the acquisition of 3D spatial information of construction sites, engineering sites, mining areas, or land reclamation areas based on oblique photogrammetry data has been gradually applied to scenarios such as building construction, engineering surveying, land mapping, and earthwork calculation. Oblique photogrammetry typically uses UAVs equipped with multi-lens cameras or single-lens multi-angle shooting equipment to collect images of ground features from different directions. Combined with GPS information, POS pose information, camera parameters, and other data, 3D reconstruction of the survey area is performed to generate point cloud data, mesh models, or realistic 3D models that reflect the spatial morphology of the site.

[0003] In the surveying and mapping process at infrastructure construction sites, the calculation of earthwork volume usually requires surface topographic data as a basis. For data collected at different stages before and after construction, the change in earthwork volume can be calculated through methods such as cut-and-fill analysis, comparison of topography between two periods, and cross-sectional measurement. Therefore, the ability to accurately extract the real topography from oblique photogrammetry data or 3D reality models has a significant impact on the accuracy of subsequent earthwork volume calculations.

[0004] However, in actual construction sites, the 3D models reconstructed from oblique photogrammetry data often include not only ground topography but also a large number of above-ground features, such as buildings, temporary facilities, vehicles, vegetation, material stockpiles, and machinery. These features are not part of the actual surface morphology used for earthwork calculation. If earthwork calculations are performed directly based on a 3D model containing these features, it can easily lead to distortion of the terrain elevation, thus affecting the accuracy of excavation and fill volumes, as well as the accuracy of comparisons between different periods. Therefore, before performing earthwork calculations, it is usually necessary to identify, remove, and flatten the above-ground features in the 3D model to obtain terrain data that more closely approximates the actual ground surface.

[0005] In existing technologies, one approach involves using commercial oblique photogrammetry modeling or mapping processing platforms to perform 3D reconstruction of UAV aerial survey images and provide basic terrain processing functions. While this approach can effectively complete image modeling and generate 3D results, its ability to automatically identify and remove ground features such as buildings, vegetation, vehicles, and material stockpiles is limited, especially in complex construction scenarios. This is particularly problematic when ground features have complex shapes, severe occlusion, similar material textures, or cluttered construction site environments. Issues such as missed or incorrect removal of ground features, or inaccurate removal boundaries, can easily arise, making it difficult to directly meet the requirements for high-precision earthwork quantity calculation.

[0006] Another approach involves terrain editing through manual interaction. Operators can manually select areas of ground features in the 3D scene or point cloud results and then delete, flatten, or repair them. This approach can achieve high terrain processing accuracy with the participation of experienced operators, but the manual operation process is usually quite cumbersome, especially in large construction sites or when there are many ground features. It requires a large number of repetitive selection, editing, flattening, and checking operations, resulting in low processing efficiency. Users often find it difficult to quickly determine whether the processing results meet the calculation requirements, leading to a poor overall user experience.

[0007] Another type of solution relies on artificial intelligence algorithms to automatically identify and remove ground features. This approach can improve processing efficiency and reduce manual editing workload to some extent. However, in non-standardized scenarios such as construction sites, ground features are diverse, spatially complex, and vary significantly in scale. Furthermore, different ground features may obstruct, overlap, or have blurred textures. Relying solely on automatic identification algorithms can easily lead to unstable identification results. Especially in engineering surveying and earthwork quantity calculation scenarios, the boundary accuracy of ground feature removal and the quality of elevation restoration directly affect the engineering quantity calculation results. Existing purely automated solutions often struggle to balance processing efficiency and engineering accuracy.

[0008] In summary, existing terrain extraction and earthwork calculation schemes based on oblique photogrammetry data have at least the following problems: First, manual interactive terrain editing schemes have high accuracy, but are complex to operate and have low processing efficiency, making them difficult to adapt to the needs of large-scale data processing; Second, purely automated ground object identification and removal schemes have high processing efficiency, but their accuracy is insufficient in complex construction scenarios, making it difficult to meet the engineering quantity calculation requirements.

[0009] Therefore, how to simultaneously improve the accuracy and processing efficiency of terrain extraction in complex construction scenarios has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0010] The purpose of this invention is to provide a method, apparatus, computer device, and readable storage medium for extracting and processing terrain data, in order to solve the aforementioned technical problems in the prior art.

[0011] On the one hand, in order to achieve the above objectives, the present invention provides a method for extracting and processing terrain data.

[0012] The terrain data extraction and processing method includes: identifying ground objects in the area to be processed based on the 3D scene data of the area to be processed to obtain an initial identification result, wherein the 3D scene data is used to characterize the terrain and ground objects in the area to be processed, and the initial identification result includes the identified ground objects and the initial terrain data obtained by removing the identified ground objects; in response to a target selection operation, determining the target location corresponding to the target selection operation, wherein the target selection operation is used to select the target ground object on the initial image corresponding to the initial identification result; inputting the target location and the initial image into a first segmentation model to obtain the initial region of the target ground object; using the initial region as a first segmentation prior, inputting the initial image, target location, and first segmentation prior into a second segmentation model to obtain the optimized region of the target ground object; in response to a correction operation, determining the correction location and correction information operated on by the correction operation, wherein the correction information is used to indicate whether the correction location belongs to the target ground object; using the optimized region as a second segmentation prior, inputting the initial image, correction location, correction information, and second segmentation prior into the second segmentation model to obtain the corrected region of the target ground object, wherein the restored region includes the corrected region; and updating the initial terrain data using the restored region to obtain the terrain data of the area to be processed.

[0013] Furthermore, the step of inputting the target location and initial image into the first segmentation model to obtain the initial region of the target ground object includes: extracting features from the initial image to obtain image features; encoding the target location to obtain click prompt features; generating an initial segmentation probability map based on the image features and click prompt features; and determining the initial region of the target ground object based on the initial segmentation probability map.

[0014] Furthermore, the step of inputting the initial image, target location, and first segmentation prior into the second segmentation model to obtain the optimized region of the target ground object includes: generating a click response heatmap based on the target location; concatenating the initial image, click response heatmap, and first segmentation prior through channel stitching to obtain a first joint input feature; extracting multi-scale features from the first joint input feature to obtain a first multi-scale feature; generating an optimized segmentation probability map of the target ground object based on the first multi-scale feature; and determining the optimized region of the target ground object based on the optimized segmentation probability map.

[0015] Furthermore, the optimized region is used as the second segmentation prior. The steps of inputting the initial image, the corrected position, the corrected information, and the second segmentation prior into the second segmentation model to obtain the corrected region of the target ground object include: generating a corrected click response heatmap based on the corrected position and the corrected information; concatenating the initial image, the corrected click response heatmap, and the second segmentation prior through channel stitching to obtain the second joint input feature; extracting multi-scale features from the second joint input feature to obtain the second multi-scale feature; generating a corrected segmentation probability map of the target ground object based on the second multi-scale feature; and determining the corrected region of the target ground object based on the corrected segmentation probability map.

[0016] Furthermore, in the presence of multiple correction operations, the correction region obtained in the previous iteration is used as the segmentation prior for the next iteration. Combined with the correction position and correction information corresponding to the current correction operation, the second segmentation model continues to iterate and update until a preset stopping condition is met. The preset stopping condition includes the number of times the user confirms the correction region or the number of correction operations reaches a preset number.

[0017] Furthermore, after obtaining the corrected area of ​​the target ground features and before updating the initial terrain data using the restored area to obtain the terrain data of the area to be processed, the method further includes: extracting candidate ground features from the initial image to obtain a set of candidate ground feature areas; extracting visual features from the candidate ground feature areas in the set to obtain candidate features; extracting visual features from the corrected area to obtain reference features; calculating the similarity between the candidate features and the reference features; and determining the candidate ground feature areas whose similarity meets the preset similarity conditions as similar ground feature areas, wherein the restored area also includes similar ground feature areas.

[0018] Furthermore, the steps of updating the initial terrain data using the restored area to obtain the terrain data of the area to be processed include: determining the boundary of the restored area to obtain the target outline; obtaining the boundary elevation information of the restored area based on the target outline; generating a restored terrain surface to replace the restored area based on the boundary elevation information; and merging the restored terrain surface into the initial terrain data to obtain the terrain data of the area to be processed.

[0019] On the other hand, in order to achieve the above objectives, the present invention provides a terrain data extraction and processing apparatus.

[0020] The terrain data extraction and processing device includes: a recognition module, used to recognize ground objects in the area to be processed based on the three-dimensional scene data of the area to be processed, and obtain an initial recognition result, wherein the three-dimensional scene data is used to characterize the terrain and ground objects in the area to be processed, and the initial recognition result includes the recognized ground objects and the initial terrain data obtained by removing the recognized ground objects; and a first response module, used to respond to a target selection operation, determine the target location corresponding to the target selection operation, wherein the target selection operation is used to select target ground objects on the initial image corresponding to the initial recognition result, input the target location and the initial image into a first segmentation model to obtain the initial region of the target ground objects, and then input the initial region into the first segmentation model. The region is used as the first segmentation prior. The initial image, target location, and the first segmentation prior are input into the second segmentation model to obtain the optimized region of the target ground object. The second response module is used to respond to the correction operation, determine the correction location and correction information operated on by the correction operation, wherein the correction information is used to indicate whether the correction location belongs to the target ground object. The optimized region is used as the second segmentation prior. The initial image, correction location, correction information, and the second segmentation prior are input into the second segmentation model to obtain the corrected region of the target ground object, wherein the restored region includes the corrected region. The restoration module is used to update the initial terrain data using the restored region to obtain the terrain data of the area to be processed.

[0021] On the other hand, to achieve the above objectives, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above method.

[0022] On the other hand, to achieve the above objectives, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above method.

[0023] The terrain data extraction and processing method, apparatus, computer equipment, and readable storage medium provided by this invention first identify ground objects based on the three-dimensional scene data of the area to be processed, and generate an initial identification result including the identified ground objects and initial terrain data; then, based on the user's target selection operation on the initial image, determine the target location, and use a first segmentation model to obtain the initial region of the target ground object; then, use the initial region as the first segmentation prior, and input it together with the initial image and the target location into a second segmentation model to obtain an optimized region; subsequently, based on the user's correction operation on the optimized region, determine the correction location and correction information, and use the optimized region as the second segmentation prior, and input it together with the initial image, correction location, and correction information into the second segmentation model to obtain a corrected region; finally, use the restored region including the corrected region to update the initial terrain data to obtain the terrain data of the area to be processed. This invention, after automatically identifying ground features and obtaining initial terrain data, does not rely entirely on the automatic identification results. Instead, it introduces target selection and correction operations, allowing users to interactively correct target ground features missed by the automatic identification or with inaccurate boundaries. Simultaneously, an initial region is obtained through a first segmentation model, and an optimized region is generated using the prior information from the first segmentation model through a second segmentation model. After the correction operation, the corrected region is generated again using the prior information from the second segmentation model, allowing the target ground feature region to gradually converge under the combined effect of the automatic segmentation results and user correction information. This reduces repetitive manual selection and editing, while avoiding the problems of inaccurate boundaries or missed identifications in complex construction scenarios where purely automatic identification is used. Furthermore, updating the initial terrain data using the restored region can restore the terrain data corresponding to the target ground features to more realistic surface morphology, thereby simultaneously improving the accuracy and processing efficiency of terrain extraction. Attached Figure Description

[0024] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart of the terrain data extraction and processing method provided in Embodiment 1 of the present invention; Figure 2 This is a block diagram of the terrain data extraction and processing device provided in Embodiment 2 of the present invention; Figure 3 This is a hardware structure diagram of a computer device provided in Embodiment 3 of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0026] Example 1 This invention provides a terrain data extraction and processing method. This method enables the identification, interactive correction, and terrain reconstruction of above-ground objects in complex construction scenarios, resulting in terrain data that more closely approximates the actual landform. Specifically, Figure 1 The flowchart of the terrain data extraction and processing method provided in Embodiment 1 of the present invention is as follows: Figure 1 As shown, the terrain data extraction and processing method provided in this embodiment includes the following steps S101 to S107.

[0027] Step S101: Identify the ground objects in the area to be processed based on the 3D scene data of the area to be processed, and obtain the initial identification results.

[0028] Among them, the three-dimensional scene data is used to characterize the terrain and ground objects of the area to be processed, and the initial identification results include the identified ground objects and the initial terrain data obtained by removing the identified ground objects.

[0029] The area to be processed can be a construction site, engineering site, mining area, land reclamation area, road construction area, storage yard area, foundation pit area, or other areas requiring terrain extraction processing. The 3D scene data can be a realistic 3D model reconstructed from UAV oblique photogrammetry data, or data generated by laser scanning, photogrammetry, existing surveying results, or a 3D modeling platform. The 3D scene data can represent the spatial morphology of the area to be processed, including both actual surface undulations and above-ground features such as buildings, temporary facilities, vehicles, vegetation, stockpiles, and machinery.

[0030] In this embodiment, 3D scene data can be input into an AI-automated terrain extraction module, which automatically identifies ground objects in the 3D scene data. Specifically, various ground objects can be identified from the 3D scene data based on image semantic segmentation, point cloud classification, 3D mesh classification, deep learning object detection, or multi-source feature fusion. After identification, the data corresponding to the identified ground objects can be removed, hidden, marked, or flattened to obtain initial terrain data.

[0031] It should be noted that the initial terrain data does not require the complete removal of all ground features, but rather represents the preliminary terrain processing results obtained during the automatic identification phase. Due to the complexity and significant differences in the types and shapes of ground features at construction sites, and the potential for occlusion, adhesion, similar textures, or unclear boundaries, the AI-automated terrain extraction module may miss, misidentify, or inaccurately identify boundaries. Therefore, the initial identification results, in addition to including identified ground features and the initial terrain data, also serve as the basis for subsequent manual interactive corrections.

[0032] Step S102: In response to the target selection operation, determine the target location corresponding to the target selection operation.

[0033] The target selection operation is used to select target ground objects on the initial image corresponding to the initial recognition result.

[0034] The initial image can be a two-dimensional image rendered based on the initial recognition results, or it can be a screenshot, orthophoto, partial view, or other image of a three-dimensional scene from the current perspective that allows the user to view and select target objects on the ground. The target selection operation can be a click operation performed by the user on the initial image in the front-end interface, or it can be a point selection, box selection, circle selection, touch selection, or other interactive operation that can indicate the location of the target object on the ground.

[0035] The target location is the image location determined based on the target selection operation. It can be pixel coordinates, normalized image coordinates, screen coordinates, or 3D scene coordinates obtained after coordinate mapping. The target ground objects are the ground objects that the user wants to process further. These are typically ground objects not identified during the AI's automatic recognition process, ground objects with inaccurate recognition boundaries, or ground objects that the user believes require re-terrain reconstruction. For example, if a vehicle, a stockpile of materials, or a patch of vegetation is not completely removed in the initial recognition results, the user can click on the location of the vehicle, stockpile, or vegetation on the initial image to trigger subsequent interactive segmentation processing.

[0036] Step S103: Input the target location and initial image into the first segmentation model to obtain the initial region of the target ground object.

[0037] The first segmentation model is used to perform initial segmentation of target objects on the ground in the initial image based on the target location selected by the user. The target location can be used as a click prompt information input to the first segmentation model, and the initial image is used as the image input data input to the first segmentation model. The first segmentation model can output the initial region where the target object is located based on the image features of the initial image and the prompt features corresponding to the target location.

[0038] The initial region represents the area of ​​the target ground object initially determined by the first segmentation model based on the target location. The initial region can be represented as a segmentation mask, a segmentation probability map, the target region boundary, or region data obtained through further transformation of the above data. For example, when a user clicks on the location of a vehicle, the first segmentation model can generate an initial region roughly covering the vehicle based on the click location and the texture, edges, color, and spatial structure features in the initial image. This initial region can be used to represent the general extent of the target ground object and serves as the basis for subsequent refinement processing by the second segmentation model.

[0039] Step S104: Using the initial region as the first segmentation prior, input the initial image, target location, and first segmentation prior into the second segmentation model to obtain the optimized region of the target ground object.

[0040] The first segmentation prior is used to characterize the target ground object region information already identified by the first segmentation model. The first segmentation prior can be obtained directly from the initial region, or it can be obtained after smoothing, normalizing, resizing, or format conversion of the initial region. The second segmentation model is used to further optimize the regional boundaries of the target ground objects under the joint constraints of the first segmentation prior and the target location.

[0041] In this embodiment, the second segmentation model receives an initial image, a target location, and a first segmentation prior. The initial image provides visual information about the target object and its surrounding environment, the target location provides interactive constraints for the user to select the target, and the first segmentation prior provides the approximate area of ​​the target object. Based on the above information, the second segmentation model can optimize the initial region to address issues such as boundary divergence, misselected regions, missing regions, or edge misalignment, thereby obtaining an optimized region.

[0042] The optimized region is a more refined area of ​​the target ground features compared to the initial region. Compared to the initial region, the optimized region better matches the actual boundaries of the target ground features, allowing subsequent user correction operations to be performed based on a higher-quality region result, thereby reducing the number of repeated user operations.

[0043] Step S105: In response to the correction operation, determine the correction location and correction information operated by the correction operation.

[0044] The correction information is used to indicate whether the correction location belongs to the target aboveground feature.

[0045] Correction operations can be user-initiated clicks, selections, bounding boxes, selections, or other interactive actions on the optimized area. The correction location is the position corresponding to the correction operation, and the correction information is used to express the relationship between that location and the target ground object. For example, when the optimized area omits part of the target ground object, the user can perform a correction operation within the omitted area, and the correction information will indicate that the corrected location belongs to the target ground object; when the optimized area incorrectly includes surrounding ground, shadows, or other non-target objects, the user can perform a correction operation within the incorrectly selected area, and the correction information will indicate that the corrected location does not belong to the target ground object.

[0046] In one specific implementation, the correction information can include foreground markers and background markers. Foreground markers indicate that the corrected location belongs to a target surface feature and needs to be included in the target region; background markers indicate that the corrected location does not belong to a target surface feature and needs to be excluded from the target region. By using the corrected location and correction information, the second segmentation model can obtain explicit feedback from the user on the current segmentation result, thereby enabling targeted corrections to the optimized region.

[0047] Step S106: Using the optimized region as the second segmentation prior, input the initial image, the corrected position, the corrected information, and the second segmentation prior into the second segmentation model to obtain the corrected region of the target ground object.

[0048] The restored region includes the correction region.

[0049] The second segmentation prior is used to characterize the optimized region obtained from the previous segmentation. After receiving the initial image, correction position, correction information, and second segmentation prior, the second segmentation model can combine the previous optimized region and the current user correction feedback to update the target ground object region again. In other words, the second segmentation model does not segment the target ground object from scratch, but rather, based on the existing optimized region, it makes local adjustments or updates the target boundary locally or globally according to the correction position and correction information.

[0050] The corrected area represents the target ground feature area obtained after user correction and reprocessing by the second segmentation model. This corrected area can be considered as the area requiring terrain restoration processing from the initial terrain data; therefore, in this embodiment, the restored area includes the corrected area. If there is no further batch processing of similar ground features, the restored area can directly be the corrected area; if similar ground feature areas are subsequently identified, the restored area can also include these similar ground feature areas.

[0051] Step S107: Update the initial terrain data using the restored area to obtain the terrain data of the area to be processed.

[0052] The restoration area refers to the area of ​​above-ground features that requires terrain restoration processing. Updating the initial terrain data using the restoration area may include: determining the extent of above-ground features corresponding to the restoration area; removing or replacing above-ground feature data within that extent from the initial terrain data; generating a terrain surface to replace the above-ground features based on elevation information, terrain continuity information, or existing surface data surrounding the restoration area; and integrating this terrain surface into the initial terrain data. Through the above processing, the elevation or model data originally belonging to above-ground features in the restoration area can be replaced with terrain data that more closely approximates the actual surface.

[0053] The terrain data of the area to be processed is obtained after automatic identification, interactive correction and terrain restoration. It can be used for subsequent terrain display, engineering verification, terrain change analysis or earthwork volume calculation and other scenarios.

[0054] In the terrain data extraction and processing method provided in this embodiment, ground objects are first identified based on the three-dimensional scene data of the area to be processed, and an initial identification result including the identified ground objects and initial terrain data is generated. Then, the target position is determined based on the user's target selection operation on the initial image, and the initial region of the target ground object is obtained using a first segmentation model. The initial region is then used as the first segmentation prior, and together with the initial image and the target position, it is input into the second segmentation model to obtain the optimized region. Subsequently, the correction position and correction information are determined based on the user's correction operation for the optimized region, and the optimized region is used as the second segmentation prior, and together with the initial image, correction position, and correction information, it is input into the second segmentation model to obtain the corrected region. Finally, the initial terrain data is updated using the restored region including the corrected region to obtain the terrain data of the area to be processed.

[0055] The terrain data extraction and processing method provided in this embodiment, after automatically identifying ground features and obtaining initial terrain data, does not rely entirely on the automatic identification results. Instead, it introduces target selection and correction operations, allowing users to interactively correct target ground features that were missed by automatic identification or whose boundaries are inaccurate. Simultaneously, an initial region is obtained through a first segmentation model, and an optimized region is generated using the first segmentation prior through a second segmentation model. After the correction operation, the second segmentation prior is used to generate a corrected region, allowing the target ground feature region to gradually converge under the combined effect of automatic segmentation results and user correction information. This reduces repetitive manual selection and editing, while avoiding the problems of inaccurate boundaries or missed identifications in complex construction scenarios where purely automatic identification is used. Furthermore, updating the initial terrain data using the restored region restores the corresponding region of the target ground feature to terrain data that more closely approximates the actual surface morphology, thereby simultaneously improving the accuracy and processing efficiency of terrain extraction.

[0056] Optionally, in one embodiment, the step of inputting the target location and an initial image into a first segmentation model to obtain the initial region of the target ground object includes: extracting features from the initial image to obtain image features; encoding the target location to obtain click prompt features; generating an initial segmentation probability map based on the image features and the click prompt features; and determining the initial region of the target ground object based on the initial segmentation probability map.

[0057] In this embodiment, the first segmentation model may include an image feature extraction part, a cue encoding part, and a segmentation prediction part. The image feature extraction part extracts features from the initial image to obtain image features representing the edges, textures, colors, shapes, and relationships with the surrounding background of the target object. The cue encoding part encodes the target location to obtain click cue features, which represent the target location the user wishes to select. The segmentation prediction part generates an initial segmentation probability map based on the image features and the click cue features.

[0058] The initial segmentation probability map represents the probability that each pixel in the initial image belongs to a target ground object. Typically, the main body of the target ground object has a higher probability, the probability near the object's boundary may be intermediate, and the probability of surrounding ground, background, or other objects is lower. When determining the initial region based on the initial segmentation probability map, pixel regions that meet the probability criteria are identified as the initial regions for the target ground object.

[0059] The terrain data extraction and processing method provided in this embodiment generates an initial segmentation probability map by combining image features and click-hint features after the user selects a target object. This allows the first segmentation model to utilize both the visual features of the initial image itself and the spatial hindrances provided by the user's target selection operation, thereby more accurately determining the approximate range of the target object. This initial region is further used as a priori for subsequent first segmentation in the second segmentation model, which helps improve the generation quality of the subsequent optimized region and reduces the computational and recognition burden on the second segmentation model when performing target localization without prior knowledge.

[0060] Optionally, in one embodiment, the step of inputting the initial image, target location, and first segmentation prior into the second segmentation model to obtain the optimized region of the target ground object includes: generating a click response heatmap based on the target location; performing channel concatenation on the initial image, click response heatmap, and first segmentation prior to obtain a first joint input feature; performing multi-scale feature extraction on the first joint input feature to obtain a first multi-scale feature; generating an optimized segmentation probability map of the target ground object based on the first multi-scale feature; and determining the optimized region of the target ground object based on the optimized segmentation probability map.

[0061] In this embodiment, the click response heatmap is used to convert the user's target location into an image-based constraint that can be processed by the second segmentation model. Optionally, a Gaussian response heatmap can be generated centered on the target location, so that pixels near the target location have higher response values ​​and pixels farther away from the target location have lower response values. This click response heatmap can express the spatial constraints of the user's target selection operation in the image plane.

[0062] The first joint input feature is input data obtained by concatenating the initial image, click response heatmap, and first segmentation prior along the channel dimension. The initial image provides visual content, the click response heatmap provides user interaction constraints, and the first segmentation prior provides the target region prior obtained by the first segmentation model. The second segmentation model performs multi-scale feature extraction on the first joint input feature, simultaneously obtaining the overall structural information and local boundary information of the target ground objects. After generating an optimized segmentation probability map based on the first multi-scale feature, the optimized region can be further determined based on probability thresholds, connected component filtering, or boundary smoothing.

[0063] The terrain data extraction and processing method provided in this embodiment uses channel stitching of the initial image, click response heatmap, and first segmentation prior. This allows the second segmentation model to simultaneously utilize image content, user target location, and region prior output by the first segmentation model within the same input space. Compared to re-segmenting based solely on click location, this method can reduce the target search range by utilizing existing initial regions; compared to boundary correction based solely on initial regions, it can also preserve the user's interactive intent regarding the selected target through the click response heatmap. Therefore, it can improve the boundary fit and region integrity of the optimized region.

[0064] Optionally, in one embodiment, the step of using the optimized region as a second segmentation prior and inputting the initial image, correction position, correction information, and second segmentation prior into the second segmentation model to obtain the correction region of the target ground object includes: generating a correction click response heatmap based on the correction position and correction information; performing channel concatenation on the initial image, the correction click response heatmap, and the second segmentation prior to obtain a second joint input feature; performing multi-scale feature extraction on the second joint input feature to obtain a second multi-scale feature; generating a correction segmentation probability map of the target ground object based on the second multi-scale feature; and determining the correction region of the target ground object based on the correction segmentation probability map.

[0065] In this embodiment, the modified click response heatmap not only expresses the modified location but also the regional attributes indicated by the modified information. When the modified information indicates that the modified location belongs to the target ground object, the modified click response heatmap can generate a response near that modified location to enhance the target region; when the modified information indicates that the modified location does not belong to the target ground object, the modified click response heatmap can generate a response near that modified location to suppress misselected regions. In this way, both the user's positive and negative modification intentions can be converted into spatial constraint information that the second segmentation model can recognize.

[0066] The second joint input feature is obtained by concatenating the initial image, the corrected click response heatmap, and the second segmentation prior. The second segmentation prior is derived from the optimization region, representing the segmentation result obtained in the previous round; the corrected click response heatmap represents the current user's local feedback to the previous round's result; the initial image still provides the original visual content. After performing multi-scale feature extraction on the second joint input feature, the second segmentation model generates a corrected segmentation probability map and determines the corrected region accordingly.

[0067] The terrain data extraction and processing method provided in this embodiment can further optimize the target ground object area after obtaining the optimized area, based on the user's correction operations. Since the corrected segmentation is not recalculated from the previous result, but rather processed based on the prior information of the second segmentation combined with the correction location and correction information, it can make targeted adjustments for misselected or missed areas, improve correction efficiency, and make the final corrected area more consistent with the user's judgment of the target ground object boundary.

[0068] Optionally, in one embodiment, when there are multiple correction operations, the correction region obtained in the previous iteration is used as the segmentation prior for the next iteration, and combined with the correction position and correction information corresponding to the current correction operation, the second segmentation model continues to iterate and update until a preset stopping condition is met. The preset stopping condition includes the number of times the user confirms the correction region or the number of correction operations reaches a preset number.

[0069] In this embodiment, the user can perform multiple correction operations on the same target ground object. For example, the first correction operation removes misselected ground areas, the second correction operation supplements missed vehicle rear areas, and the third correction operation further narrows the target boundary. After each correction, the corrected area output by the second segmentation model can be used as the segmentation prior for the next correction operation. The correction position and correction information corresponding to the current correction operation are used to express the user feedback in this round, and the correction area in the previous round is used to express the segmentation state that has been obtained. The second segmentation model continues to iterate and update accordingly.

[0070] Preset stopping conditions may include the user clicking the confirmation button on the front-end interface, the user ceasing to enter new correction operations, the number of correction operations reaching a preset number, or the change between two consecutive correction areas being less than a preset change threshold. After the preset stopping conditions are met, the correction area obtained from the last iteration can be used as the final area for restoration processing of the target aboveground features.

[0071] The terrain data extraction and processing method provided in this embodiment combines multiple rounds of user correction operations with the segmentation prior update mechanism of the second segmentation model, enabling the target ground object area to gradually converge to the range expected by the user. Compared with one-time manual selection, this method can continuously correct local errors with fewer interactive actions; compared with single automatic segmentation, this method can continuously utilize user feedback to improve the accuracy of the region in scenarios with complex ground object boundaries, occlusion, or adhesion, thereby improving the reliability of the reconstructed region determination.

[0072] Optionally, in one embodiment, after obtaining the corrected area of ​​the target ground object and before updating the initial terrain data using the restored area to obtain the terrain data of the area to be processed, the method further includes: extracting candidate ground objects from the initial image to obtain a set of candidate ground object regions; extracting visual features of the candidate ground object regions in the set to obtain candidate features; extracting visual features of the corrected area to obtain reference features; calculating the similarity between the candidate features and the reference features; and determining the candidate ground object regions whose similarity meets a preset similarity condition as similar ground object regions, wherein the restored area also includes similar ground object regions.

[0073] In this embodiment, after the user has determined the correction area of ​​a target ground object through target selection and correction operations, the correction area can be used as a reference area to search for other ground objects with similar visual features to the target ground object in the initial image. For example, if the user has accurately corrected a construction vehicle, the system can search for other similar vehicles in the initial image based on the visual features of the correction area; if the user has corrected a pile of materials, the system can search for other similar material pile areas.

[0074] The set of candidate ground object regions can be obtained by performing instance-level mask extraction on the initial image. Instance-level mask extraction means generating a corresponding mask for each independent candidate ground object in the initial image, rather than generating a whole region simply according to category. For example, if there are multiple vehicles in the initial image, candidate ground object regions corresponding to vehicle 1, vehicle 2, and vehicle 3 can be generated separately. After obtaining the candidate ground object regions, the visual features of the candidate ground object regions are extracted to obtain candidate features. The visual features of the modified regions are then extracted to obtain reference features.

[0075] Similarity can be cosine similarity, Euclidean distance transformed similarity, normalized dot product similarity, or other similarity metrics that measure the closeness of visual features. Preset similarity criteria can be that the similarity is greater than or equal to a preset similarity threshold, or that the top few candidate ground object regions are selected after sorting by similarity. When the similarity between a candidate feature and a reference feature meets the preset similarity criteria, the candidate ground object region is considered to belong to the same class of ground objects as the target ground object in the correction region, and is added to the restored region.

[0076] The terrain data extraction and processing method provided in this embodiment, after obtaining a relatively accurate correction area through user interaction, can further utilize the visual features of the correction area to automatically discover areas with similar ground objects. This ensures that the restored area includes not only the target ground object directly corrected by the user, but also other candidate ground objects similar to the target ground object. This reduces the workload for users to click and correct multiple similar ground objects one by one, improving batch processing efficiency while maintaining interactive accuracy. It is particularly suitable for scenarios with a large number of similar targets such as vehicles, temporary facilities, material stockpiles, or vegetation at construction sites.

[0077] Optionally, in one embodiment, the step of updating the initial terrain data using the restored region to obtain the terrain data of the area to be processed includes: determining the boundary of the restored region to obtain the target contour; obtaining the boundary elevation information of the restored region based on the target contour; generating a restored terrain surface to replace the restored region based on the boundary elevation information; and merging the restored terrain surface into the initial terrain data to obtain the terrain data of the area to be processed.

[0078] In this embodiment, the restored region represents the area that needs to undergo terrain restoration processing from the initial terrain data. The boundary of the restored region is determined to obtain the target contour. The target contour can be a two-dimensional image contour or can be converted into a spatial contour in a three-dimensional scene through coordinate mapping. Boundary elevation information can be obtained from terrain points, grid vertices, or elevation gratings located near the target contour in the initial terrain data, and is used to represent the elevation changes of the real terrain surrounding the restored region.

[0079] When generating a restored terrain surface based on boundary elevation information, interpolation, fitting, or mesh construction can be performed using boundary points on the target contour and their corresponding elevation values. For example, several boundary sampling points can be obtained along the target contour, the elevation values ​​of these boundary sampling points can be read, and a triangular or regular mesh can be constructed within the restored area. The elevations of the internal points can then be interpolated using the boundary elevation values ​​to generate a restored terrain surface used to replace the surface features. The restored terrain surface is used to simulate the surface morphology that the restored area should present after the surface features are removed.

[0080] Optionally, a 2.5D Delaunay triangulation method can be used to construct the reconstructed terrain surface. Specifically, the boundary points corresponding to the target contour are projected onto a horizontal plane, and Delaunay triangulation is performed within this horizontal plane. The elevation values ​​corresponding to the boundary points are used as elevation constraints for the vertices of the triangular mesh, and the elevations of the internal mesh points are determined by interpolation. Furthermore, the generated triangular mesh can be subjected to vertical surface detection, burr detection, or removal of abnormal elevation points to avoid abrupt changes in the reconstructed terrain surface due to ground object residue, abnormal boundary points, or local occlusion.

[0081] When integrating the restored terrain surface into the initial terrain data, the restored terrain surface can be used to directly replace the ground features within the restored area, or an elevation smoothing fusion can be performed between the restored terrain surface and the surrounding initial terrain data. Optionally, a connecting region can be set near the target contour, the distance from the point to be fused within the connecting region to the target contour can be calculated, and the elevation fusion weight can be determined based on the distance; then, based on the elevation fusion weight, the restored elevation value and the original elevation value of the point to be fused can be weighted and calculated. The closer the location is to the target contour, the smoother the transition between the original elevation and the restored elevation can be achieved; for locations farther from the target contour and located within the restored area, the restored elevation value can be used more. This method can reduce steps, breaks, or abrupt changes between the restored terrain surface and the surrounding terrain.

[0082] The terrain data extraction and processing method provided in this embodiment can further convert the segmented restored area into a restored terrain surface that can be used to update the initial terrain data. By obtaining boundary elevation information based on the target contour and generating the restored terrain surface using the boundary elevation information, the area where the ground features have been replaced can maintain elevation continuity with the surrounding terrain. Furthermore, by fusing the restored terrain surface with the initial terrain data, terrain data that more closely resembles the actual surface morphology can be obtained, thereby improving the accuracy of subsequent terrain analysis or engineering quantity calculations.

[0083] Optionally, in one embodiment, after obtaining the terrain data of the area to be processed, the method further includes: in response to the repair operation for the over-processed area, determining the repair area corresponding to the repair operation; obtaining the original three-dimensional scene data corresponding to the repair area; fusing the original three-dimensional scene data with the terrain data of the area to be processed to obtain the repaired terrain data; and performing continuity optimization on the boundary of the repair area.

[0084] In this embodiment, the over-processed area can be an area that the user believes has been mistakenly flattened, deleted, or over-restored. For example, a real terrain protrusion may be incorrectly identified as material piles and restored, or the boundary of the restored area may exceed the actual ground features. The user can initiate a repair operation for this area on the front-end interface. The system determines the repair area based on the repair operation and reads the original point cloud, mesh, or elevation data corresponding to the repair area from the original 3D scene data. Subsequently, the original 3D scene data is fused with the current terrain data to restore the true or original spatial form of the repair area. To avoid discontinuities between the repair area and the surrounding area, the boundary of the repair area can be smoothed, interpolated, or optimized for elevation transition.

[0085] The terrain data extraction and processing method provided in this embodiment can provide reverse repair capability after automatic identification, interactive correction and terrain restoration, so that users can restore and optimize the boundaries of over-processed areas, thereby reducing the impact of misprocessing on the final terrain data and improving the fault tolerance and controllability of the terrain processing process.

[0086] Optionally, in one embodiment, after at least one of the following operations—target selection, correction, and updating initial terrain data using the restored area—is completed, the current operation state is stored in association with the user identifier and the project identifier. The current operation state includes at least one of the following: selected surface feature area, correction area, similar surface feature area, restored area, terrain data update area, and operation time. When re-entering the terrain data extraction processing flow corresponding to the area to be processed, the historical operation states associated with the user identifier and the project identifier are queried, and at least one of the following—selected surface feature area, correction area, similar surface feature area, restored area, and terrain data update area—is restored based on the historical operation states.

[0087] In this embodiment, the current operation state can be saved after each time the user completes target selection, correction, confirmation of similar areas, or terrain data update. Saving can be done by writing to a database, project file, cache file, or cloud project storage space. User identifiers are used to distinguish different users, and project identifiers are used to distinguish different areas to be processed or different engineering projects. By associating the operation state with user identifiers and project identifiers, processing progress can be resumed after the user closes the page, switches projects, processes large areas, or interrupts the operation.

[0088] The terrain data extraction and processing method provided in this embodiment can save user interaction and terrain update status during the processing of large-scale three-dimensional scene data, allowing users to complete terrain processing tasks in stages, avoiding the loss of completed target selection, correction and restoration results due to interruption of operation, and improving the continuity and operation efficiency of large-scale terrain data processing.

[0089] Optionally, in one embodiment, the 3D scene data includes at least one of oblique photogrammetry data, 3D model data generated based on oblique photogrammetry data, point cloud data, mesh model data, OSGB model data, 3DTiles model data, and LAS point cloud data.

[0090] In this embodiment, when the input data is oblique photographic image data, 3D reconstruction can be performed based on image metadata, camera parameters, POS pose information, and control point information to obtain point cloud data, mesh model data, or a real-world 3D model. When the input data is existing 3D model data, format recognition, coordinate unification, and data parsing can be performed to enable it to enter subsequent processes such as ground object recognition, initial image generation, interactive segmentation, and terrain reconstruction.

[0091] The terrain data extraction and processing method provided in this embodiment is compatible with a variety of surveying and mapping and 3D modeling data sources, and is not limited to a single oblique photogrammetry input. This allows the method to be used for terrain extraction and processing starting from the original image, as well as for reprocessing existing 3D models, point clouds, or mesh results, thereby improving the applicability of the method in different engineering surveying scenarios.

[0092] Optionally, in one embodiment, after obtaining the terrain data of the area to be processed, the method further includes: performing terrain application processing based on the terrain data of the area to be processed, wherein the terrain application processing includes at least one of earthwork volume calculation, terrain change monitoring, project acceptance verification, updating of the three-dimensional base map of the construction site, and project auxiliary analysis. When the terrain application processing includes earthwork volume calculation, at least one of cut and fill analysis, two-phase terrain comparison, and cross-sectional acceptance can be performed based on the terrain data of the area to be processed.

[0093] In this embodiment, the obtained topographic data can serve as the basis for subsequent engineering surveying and construction management. When used for earthwork volume calculation, the current topographic data can be compared with the design topographic surface, historical topographic data, or benchmark topographic surface to calculate the excavation volume, fill volume, or net volume; alternatively, two-period comparisons can be performed based on multi-period topographic data to determine changes in earthwork before and after construction; and cross-sectional results can be generated according to set cross-sectional lines for cross-sectional measurement. When used for topographic change monitoring, project acceptance verification, or updating of the 3D base map of the construction site, the topographic data can be overlaid and analyzed with the design model, BIM model, GIS data, or project progress data.

[0094] The terrain data extraction and processing method provided in this embodiment can be used to further apply the terrain data after the ground objects have been restored to engineering application scenarios. This means that subsequent earthwork calculations, change monitoring, or acceptance verification no longer directly rely on three-dimensional scene data containing ground objects, thereby reducing the interference of ground objects on elevation and volume calculation results and improving the accuracy and usability of engineering survey results.

[0095] Example 2 This invention provides a terrain data extraction and processing method, which is implemented through an intelligent interactive system. The complete process of terrain data extraction and processing by a user using the intelligent interactive system is described below. The intelligent interactive system includes a front-end for user operation and a back-end for interaction with the front-end.

[0096] Before entering the intelligent interaction process, users can upload data of the area to be processed via the web or desktop client. The uploaded data can be oblique photogrammetry data or existing 3D model data. Oblique photogrammetry data can include metadata such as GPS information, POS pose information, shooting height, and camera attitude angle; existing 3D model data can include OSGB model data, 3DTiles model data, LAS point cloud data, point cloud data, or mesh model data. Upon receiving the uploaded data, the system automatically parses the EXIF ​​information of the oblique photogrammetry data, extracts metadata such as GPS coordinates, shooting height, and camera attitude angle, and performs 3D reconstruction based on this metadata, generating point clouds and mesh models. For existing 3D model data, the system performs format recognition and coordinate system unification processing to ensure that subsequent ground object recognition, intelligent interaction, real-time flattening, and earthwork calculations are all performed under a unified coordinate system and a unified data model. Through this pre-processing, subsequent modules can share the same 3D scene data, eliminating the need for repeated import and export between multiple platforms and avoiding accuracy loss due to format and coordinate conversions.

[0097] Step 1: Access Smart Interaction Functions and History Recovery [User Front-End Operation] In the 3D scene on the Web, users can enter the intelligent interactive function module for terrain restoration by clicking the "Terrain Restoration" button in the toolbar.

[0098] [Backend Processing] The frontend sends a session initialization request to the backend. Upon receiving the request, the backend queries the current user's historical operation records within the project, including a list of picked objects, information on flattened areas, operation timestamps, confirmed areas of similar objects, flattening parameters, repair records, etc. The historical operation records are then serialized and returned to the frontend. Optionally, historical operation records are stored in association with user identifiers, project identifiers, and data version identifiers to facilitate state recovery between different processing stages within the same project.

[0099] [Front-end Feedback] After receiving the historical operation records returned by the backend, if a history exists, a prompt box will pop up: "An incomplete operation record was detected last time. Do you want to restore it?" The user can choose "Restore" or "Restart". If the user chooses to restore, the front-end will reconstruct the processed ground object layer and flattened area based on the history and restore its display in the 3D scene. If the user chooses to restart, the history will be cleared, and the system will enter a blank interactive state. If no history exists, the system will directly enter an interactive waiting state. In this way, the processing progress can be resumed after the browser is closed, the system is restarted, the page exits abnormally, or the user actively pauses the process, achieving breakpoint resumption.

[0100] Step 2: Select the target area and have it automatically recognized by AI. [User Front-End Operation] Users drag and drop the mouse to draw a rectangular selection area in the 3D scene, selecting the target area for terrain reconstruction. After selecting, the user releases the mouse to confirm the area selection.

[0101] [Backend Processing] The frontend sends the boundary coordinates of the selected area, such as latitude and longitude range, projected coordinate range, or local coordinate range of the model, as well as the camera parameters of the current scene, to the backend. After receiving the request, the backend calls the AI ​​automated terrain extraction module to automatically identify ground objects in the 3D model or point cloud data within the selected area, identifying ground object categories such as buildings, vegetation, vehicles, material stockpiles, and machinery, performing intelligent segmentation and automatic removal, and generating preliminary terrain processing results.

[0102] In this embodiment, the preliminary terrain processing result can be an OBJ model or an initial terrain model in other formats after removing ground features such as buildings, vegetation, vehicles, and material stockpiles. This initial terrain model has already removed some ground features compared to the original 3D scene data, but compared to the final terrain data, there may still be cases where ground features are not completely removed, ground feature boundaries remain, or there is localized over-removal. Therefore, the preliminary terrain processing result serves as the basis for subsequent AI-driven interactive refinement, rather than being directly output as the final result.

[0103] After processing, the backend returns a list of identified ground features to the frontend. The list of ground features may include information such as the category, location, outline, mask data, confidence level, elevation range, and corresponding model segment identifier for each ground feature.

[0104] [Front-end Feedback] After receiving the AI ​​recognition results from the back-end, the front-end displays the recognized ground objects in the 3D scene with highlighted outlines. Different categories of ground objects are distinguished by different colors, such as red for buildings, green for vegetation, yellow for vehicles, and orange for material piles. Simultaneously, a list of recognized ground objects is displayed in the scene sidebar for users to view and select for subsequent refinement. Users can check the highlighted outlines in the 3D scene to determine if the AI ​​automatic recognition results have any omissions, misidentifications, or inaccurate boundaries.

[0105] Step 3: Single-target intelligent picking [User Front-End Operation] For ground objects that are missed or have inaccurate boundaries automatically identified by AI, the user can directly click on the target ground object's location in the 3D scene to initiate a single-target intelligent pickup request. This click operation can be a single click or multiple clicks for subsequent correction. The click location indicates the target ground object the user wishes to pick.

[0106] [Backend Processing] The frontend sends the user's click coordinates, current camera view parameters, and scene model data to the backend. The click coordinates can be converted from screen coordinates to image coordinates or 3D spatial coordinates. Upon receiving the request, the backend calls a combined SAM and SimpleClick algorithm to perform single-target intelligent picking.

[0107] Specifically, for scenarios where users quickly locate individual features, existing large-scale instance segmentation models such as SAM perform well on regular textured targets, but tend to exhibit segmentation divergence or boundary expansion issues for complex targets, occluded targets, or targets with chaotic textures in construction sites. While the SimpleClick algorithm for interactive segmentation has good convergence, its generalization ability is relatively insufficient on the first click. Therefore, this implementation adopts a cascaded interactive image segmentation method based on SAM and SimpleClick to improve the segmentation initialization quality and multi-round interactive convergence efficiency under conditions of a small number of user clicks.

[0108] This method uses the original RGB image as input data, denoted as I∈RH×W×3, and simultaneously receives a set of user interaction click information C={(xi,yi,li)}, where xi and yi represent pixel coordinates, and li represents foreground or background markers. First, the original RGB image and the initial click input SAM model are used for coarse segmentation inference. SAM extracts high-dimensional feature representations from the input image through an image coding network and combines this with click prompt encoding to generate semantic guidance information. A low-resolution segmentation logits map is then output via a mask decoder. This logits map is mapped using a Sigmoid function to obtain a pixel-level probability distribution, which is then upsampled to the original image size to form an initial segmentation probability map Psam, used to characterize the global structural prior of the target region.

[0109] Furthermore, the probability map output by SAM is smoothed and normalized, and then introduced into the SimpleClick model as prior information. Specifically, the RGB image, the Gaussian response heatmap generated by user clicks, and the SAM probability prior map are concatenated along the channel dimension to construct a joint input feature tensor. Based on this joint input feature tensor, SimpleClick performs multi-scale feature encoding through a convolutional feature extraction network, and generates a refined segmentation probability map Psimple under the joint guidance of click constraints and prior probabilities. Finally, the precise contour lines and mask data of the target features are output and returned to the front end. Optionally, the above algorithm is encapsulated through an inference service framework to support real-time and efficient collaboration between the front end and the back end, and to provide continuous intelligent interactive capabilities for terrain reconstruction to different users simultaneously. The entire processing can complete the return of target picking results in a short time.

[0110] [Front-end Feedback] After receiving the picking results from the backend, the frontend displays the target features in the 3D scene as highlighted outlines in real time, filling them with a semi-transparent color to indicate the picking area. Users can visually see the picking effect. If satisfied with the result, they confirm the picking; if the picking area is inaccurate, users can add clicks. Positive clicks add areas, while negative clicks remove areas. The frontend sends the added click information back to the backend, which performs incremental segmentation optimization based on SimpleClick and returns an updated mask. The frontend updates the display in real time until the user is satisfied.

[0111] In the multi-round interaction, the output of SimpleClick in each round can serve as the segmentation prior for the next round, and is iteratively updated by incorporating newly added click information, thus forming a closed-loop interaction mechanism that combines coarse segmentation guidance with fine segmentation optimization. Through this approach, the global structural prior provided by SAM improves the initial segmentation quality, while SimpleClick further optimizes boundary details under local click constraints, enabling the overall segmentation result to achieve stable convergence with fewer interactions. The final output is a pixel-level binary segmentation result and its corresponding probability map, achieving high-efficiency and high-precision interactive target segmentation.

[0112] Step 4: Batch picking of similar targets [User Front-End Operation] When there are multiple similar objects on the ground in the scene, such as multiple similar buildings, multiple trees, multiple vehicles, multiple piles of materials, etc., after the user has picked up a target, click the "Pick Similar Objects" button in the toolbar to initiate a batch recognition request for similar objects.

[0113] [Backend Processing] The frontend sends the mask data of the picked targets and the current scene range to the backend. After receiving the request, the backend calls the DINOv and DINOv2 enhanced algorithms to perform batch picking of similar targets. This process is used to avoid users repeatedly interacting with similar targets and improve the efficiency of batch positioning and selection of similar ground objects.

[0114] Specifically, the DINOv algorithm can be used to retrieve candidate instances within a scene and generate a candidate instance set. However, DINOv typically uses the COCO dataset as semantic training data, which may lead to semantic domain shift issues in construction site scenarios. Its candidate masks only possess coarse-grained target localization capabilities, and in complex ground object scenarios such as vehicles, material stockpiles, temporary facilities, vegetation, and machinery, it is prone to insufficient candidate target recall or inaccurate similar target selection. To address this issue, this implementation further introduces DINOv2 as a high-dimensional visual feature extraction network to construct a mask indexing and similar instance retrieval method based on DINOv feature enhancement.

[0115] Specifically, using the semantic segmentation results obtained from the pre-trained DINOv model as the initial candidate input, the input image I∈RH×W×3 is first subjected to instance-level mask extraction to obtain a candidate target set M={mi}, where each Mask mi represents an independent candidate target region index. Further, DINOv2 is used to encode the features of the original image, obtaining pixel-level feature representations F∈Rh×w×D. Based on this, feature aggregation calculation is performed on each candidate Mask region, i.e., average pooling is performed on the feature vectors corresponding to all pixels within the Mask to obtain instance-level feature vectors fi. Subsequently, the feature similarity between candidate instances and user-picked targets is calculated, and a similarity matrix can be constructed using cosine similarity or dot product normalization. A similarity distribution function is constructed based on the similarity matrix, and a threshold τ is set to filter candidate instances, retaining the set of instances with high similarity to the initial reference target, thereby achieving automatic clustering and index retrieval of similar targets.

[0116] Finally, the backend returns the outlines and mask data of all eligible similar ground features to the frontend in batches. The output may include a set of candidate target mask indices, similar instance grouping results, similarity values ​​for each candidate instance, and instance outlines.

[0117] [Front-end Feedback] After receiving the batch pickup results of similar targets from the backend, the frontend displays all identified similar ground objects in the 3D scene with highlighted outlines and shows the batch pickup results in a sidebar list. Users can view each pickup result one by one, deselect incorrectly picked targets, and manually add missed targets. After confirming that everything is correct, the user clicks the "Confirm Batch Picking" button, and the frontend submits the final confirmed target list to the backend for recording.

[0118] In this way, users only need one or a few interactions to complete the batch location and selection of similar targets. DINOv2's high-dimensional semantic feature enhancement mechanism can compensate for DINOv's insufficient generalization in the construction domain, improving the accuracy of similar target screening while ensuring the recall rate of candidate targets, and achieving a balanced optimization of accuracy and recall.

[0119] Step 5: Real-time flattening operation [User Front-End Operation] After confirming the target to be picked up, the user can click the "Flatten" button in the toolbar to flatten the picked-up ground object.

[0120] [Backend Processing] The frontend sends the confirmed ground object outline data to the backend. Upon receiving the request, the backend executes a real-time flattening algorithm. This real-time flattening algorithm must ensure that the flattened area generated by the algorithm is consistent with the flattened area displayed by the frontend Cesium shader, so that the user can obtain a WYSIWYG flattening effect.

[0121] Specifically, the backend quickly constructs a ground object layer based on the user-input ground object outlines. For the outlines of the picked targets, the backend uses a 2.5D Delaunay triangulation algorithm to construct a triangular mesh for the ground object layer. During the mesh construction process, vertical facets and burrs are avoided, and side faces are constructed to fill holes, thus ensuring the integrity of the ground object layer. For scenarios with few ground objects where the user wants to directly flatten them, the interaction algorithm is compatible with inputting realistic models and optimizes the process of generating the ground object layer from the outlines, enabling the flattened layer to adapt to ground object interactions in complex scenes.

[0122] Subsequently, the backend converts the generated ground feature layer file into GLB format to adapt to the Cesium visualization framework and ensure that the frontend can load and display it correctly. The backend also calculates the flattened elevation values ​​and uses a Gaussian weighted fusion method to perform elevation transition processing between the flattened area and the surrounding original terrain, so that the flattened area blends smoothly with the surrounding terrain. After processing, the backend returns the GLB file and flattening parameters to the frontend.

[0123] [Front-end Feedback] After receiving the GLB file and flattening parameters from the backend, the frontend loads the GLB file based on the Cesium shader principle and performs real-time rendering, instantly displaying the flattening effect in the 3D scene. The flattened terrain transitions smoothly with the surrounding original terrain, and users can view the flattening effect in real time. If there are multiple flattened areas, the frontend supports simultaneous flattening and display of multiple areas without interference between them. The frontend also automatically saves the flattening operation record, including area coordinates, flattening parameters, GLB file identifier, timestamp, etc., to the backend as an operation history.

[0124] Step Six: Repairing Overly Compacted Areas [User Front-End Operation] After viewing the flattening effect, if the user finds that some areas have been over-flattened, such as ground structures that should have been preserved being incorrectly flattened, or real terrain protrusions being mistakenly treated as ground features, the user can select the area to be repaired by drawing a box or clicking on it, and then click the "Repair" button to initiate a repair request.

[0125] [Backend Processing] The frontend sends the coordinates of the user-selected repair area to the backend. Upon receiving the request, the backend obtains the original real-world model data for that area, merges the original model with the currently flattened terrain, and simultaneously optimizes the continuity of the boundary between the repair area and the surrounding flattened areas to eliminate issues such as boundary discontinuities, cracks, and abrupt elevation changes. Optionally, the backend can smooth the transition between the original model and the current terrain based on the elevation and grid continuity near the repair area boundary to ensure the repair effect on above-ground structures.

[0126] [Front-end Feedback] After receiving the repair results from the back-end, the front-end updates and displays the real-world model of the repaired area in the 3D scene. The repaired area smoothly transitions with the surrounding flattened areas, with no visible cracks or abrupt changes in elevation. Users can repeat this operation on other areas that need repair. This repair process allows for local restoration of over-flattened areas after terrain reconstruction, improving the fault tolerance of interactive processing.

[0127] Step 7: Area Management (Show / Hide and Delete) [User Front-End Operations] In the processed area list in the scene sidebar, users can perform "hide," "show," or "delete" operations on one or more flattened areas. Simply click the corresponding operation button to execute.

[0128] [Backend Processing] These operations are primarily local frontend operations, requiring no real-time backend intervention. The frontend, through an architecture decoupled from the UI framework and rendering engine, directly controls the visibility of the corresponding layer within the Cesium scene. When a "delete" operation is performed, the frontend automatically restores the terrain of that area to its original state before flattening. Optionally, to maintain consistency between the backend records and the frontend display, the frontend synchronously updates the backend operation records with the deletion result after performing the deletion operation.

[0129] [Front-end Feedback] The front-end updates the 3D scene display in real time: After executing "Hide," the flattening effect of the corresponding area is temporarily hidden, and the original terrain is displayed; after executing "Show," the flattening effect is restored; after executing "Delete," the area is removed from the processed list, the terrain is permanently restored to its original state, and the back-end operation record is updated synchronously. Through the area management function, users can compare, check, and selectively retain multiple flattened areas, which facilitates the controllability of the final terrain processing result.

[0130] Step 8: Automatic saving and resume interrupted downloads [User Front-End Operation] Users do not need to manually save. The system automatically triggers saving at each key operation node, such as pick confirmation, batch pick confirmation, flattening completion, repair completion, area deletion completion, and interaction completion. Users can close the browser or pause the operation at any time.

[0131] [Backend Processing] The frontend automatically sends the current operation status to the backend asynchronously at each key operation node. The current operation status may include the list of picked targets, information on flattened areas, confirmation results for similar targets, operation step records, flattening parameters, repaired areas, area visibility status, timestamps, user identifiers, project identifiers, etc. The backend persists the operation records and associates them with user sessions and projects. Optionally, the system can also automatically save snapshots at key nodes for version management and historical state rollback.

[0132] [Front-end Feedback] The front-end displays a "Automatically saved" message in the interface status bar. When the user re-enters the intelligent interaction function, the system automatically queries the history and prompts for restoration. See step one for restoration instructions. Through automatic saving and resume functionality, the system effectively avoids the loss of historical operation records caused by browser crashes, system restarts, network interruptions, or user-initiated pauses, adapting to the realities of large-scale construction site 3D data processing being time-consuming.

[0133] Step Nine: Complete the interaction and data flow [User Front-End Operation] After the user confirms that all above-ground objects have been processed, click the "Complete" button to end the intelligent interaction process and enter the next stage, namely earthwork calculation.

[0134] [Backend Processing] The frontend submits the final high-precision terrain data to the backend, which then incorporates the processing results into the shared data model of the integrated system. This shared data model maintains a unified coordinate system, spatial index, and data structure across the modeling, terrain extraction, terrain reconstruction, and earthwork calculation modules. Upon completion of the previous stage, the system automatically triggers the next processing stage. Data is directly transferred from memory to the earthwork calculation module without the need for import / export or format conversion across multiple software platforms, thus avoiding coordinate transformation errors and data accuracy loss.

[0135] [Front-end Feedback] The front-end switches to the earthwork calculation module interface, performing earthwork calculations such as cut and fill analysis, two-phase comparison, and cross-section measurement based on high-precision terrain data. The entire process, from data input, 3D reconstruction, AI automatic recognition, interactive refinement, real-time flattening, local repair to calculation output, is completed in a unified system, eliminating the need for multi-platform workflows. The processing results at each stage can be previewed in real time, key nodes are automatically saved as snapshots, and any stage can be paused and resumed at a later time.

[0136] Through the complete process described above, this implementation method achieves integrated processing from data input, 3D reconstruction, AI automatic processing, intelligent interactive refinement, real-time flattening, over-flattening correction, to earthwork calculation. Compared to methods relying solely on manual editing, this process reduces repetitive operations through AI automatic recognition, intelligent single-target picking, and batch picking of similar targets. Compared to purely automatic recognition methods, this process improves processing accuracy in complex construction scenarios through user clicks, positive and negative corrections, real-time flattening previews, and over-flattening correction. Compared to multi-platform collaboration methods, this process reduces accuracy loss caused by format conversion and data migration through shared data structures, unified coordinate systems, and in-memory data flow, thereby improving the overall processing efficiency from terrain extraction to earthwork calculation.

[0137] Example 3 Corresponding to Embodiment 1 above, Embodiment 3 of the present invention provides a terrain data extraction and processing device. The technical features and corresponding technical effects can be referred to Embodiment 1 above, and will not be repeated in this embodiment. Figure 2 This is a block diagram of the terrain data extraction and processing device provided in Embodiment 3 of the present invention, as shown below. Figure 2 As shown, the device includes: an identification module 201, a first response module 202, a second response module 203, and a restoration module 204.

[0138] The recognition module 201 is used to recognize ground objects in the area to be processed based on the three-dimensional scene data of the area to be processed, and obtain an initial recognition result. The three-dimensional scene data is used to characterize the terrain and ground objects of the area to be processed. The initial recognition result includes the identified ground objects and initial terrain data obtained by removing the identified ground objects. The first response module 202 is used to respond to a target selection operation, determine the target location corresponding to the target selection operation, wherein the target selection operation is used to select a target ground object on the initial image corresponding to the initial recognition result. The target location and the initial image are input into a first segmentation model to obtain an initial region of the target ground object. The initial region is used as a first segmentation prior. The initial image, the target location, and the first segmentation prior are input into the second segmentation model to obtain the optimized region of the target ground object; the second response module 203 is used to respond to the correction operation, determine the correction location and correction information operated by the correction operation, wherein the correction information is used to indicate whether the correction location belongs to the target ground object, use the optimized region as the second segmentation prior, input the initial image, the correction location, the correction information, and the second segmentation prior into the second segmentation model to obtain the corrected region of the target ground object, wherein the restored region includes the corrected region; the restoration module 204 is used to update the initial terrain data using the restored region to obtain the terrain data of the area to be processed.

[0139] Optionally, in one embodiment, the first response module includes: an image feature extraction unit, configured to extract features from the initial image to obtain image features; a click prompt feature encoding unit, configured to encode the target location to obtain click prompt features; an initial segmentation probability map generation unit, configured to generate an initial segmentation probability map based on the image features and the click prompt features; and an initial region determination unit, configured to determine the initial region of the target ground object based on the initial segmentation probability map.

[0140] Optionally, in one embodiment, the first response module includes: a click response heatmap generation unit, configured to generate a click response heatmap based on the target location; a first feature stitching unit, configured to stitch the initial image, the click response heatmap, and the first segmentation prior through channels to obtain a first joint input feature; a first multi-scale feature extraction unit, configured to extract multi-scale features from the first joint input feature to obtain a first multi-scale feature; an optimized segmentation probability map generation unit, configured to generate an optimized segmentation probability map of the target ground object based on the first multi-scale feature; and an optimized region determination unit, configured to determine an optimized region of the target ground object based on the optimized segmentation probability map.

[0141] Optionally, in one embodiment, the second response module includes: a modified click response heatmap generation unit, configured to generate a modified click response heatmap based on the modified location and the modified information; a second feature stitching unit, configured to perform channel stitching of the initial image, the modified click response heatmap, and the second segmentation prior to obtain a second joint input feature; a second multi-scale feature extraction unit, configured to perform multi-scale feature extraction on the second joint input feature to obtain a second multi-scale feature; a modified segmentation probability map generation unit, configured to generate a modified segmentation probability map of the target ground object based on the second multi-scale feature; and a modified region determination unit, configured to determine the modified region of the target ground object based on the modified segmentation probability map.

[0142] Optionally, in one embodiment, the second response module is further configured to, in the presence of multiple correction operations, use the correction region obtained in the previous iteration as the segmentation prior for the next iteration, and combine it with the correction position and correction information corresponding to the current correction operation to continue iteratively updating through the second segmentation model until a preset stopping condition is met, wherein the preset stopping condition includes the number of times the user confirms the correction region or the number of times the correction operation is performed reaching a preset number.

[0143] Optionally, in one embodiment, the apparatus further includes: a similar ground object processing module, configured to extract candidate ground objects from the initial image to obtain a set of candidate ground object regions, extract visual features of the candidate ground object regions in the set to obtain candidate features, extract visual features of the modified region to obtain reference features, calculate the similarity between the candidate features and the reference features, and determine the candidate ground object regions whose similarity satisfies a preset similarity condition as similar ground object regions, wherein the restored region further includes the similar ground object regions.

[0144] Optionally, in one embodiment, the restoration module includes: a target contour determination unit, used to determine the boundary of the restoration area to obtain a target contour; a boundary elevation information acquisition unit, used to acquire the boundary elevation information of the restoration area based on the target contour; a restored terrain surface generation unit, used to generate a restored terrain surface to replace the restoration area based on the boundary elevation information; and a data fusion unit, used to fuse the restored terrain surface into the initial terrain data to obtain the terrain data of the area to be processed.

[0145] Example 4 This embodiment also provides a computer device, such as a smartphone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including a standalone server or a server cluster composed of multiple servers), etc., capable of executing programs. Figure 3 As shown, the computer device 01 in this embodiment includes, but is not limited to, a memory 012 and a processor 011 that are communicatively connected to each other via a system bus. It should be noted that... Figure 3 Only a computer device 01 with component memory 012 and processor 011 is shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0146] In this embodiment, the memory 012 (i.e., the readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 012 may be an internal storage unit of the computer device 01, such as the hard disk or memory of the computer device 01. In other embodiments, the memory 012 may also be an external storage device of the computer device 01, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 01. Of course, the memory 012 may include both the internal storage unit and its external storage device of the computer device 01. In this embodiment, the memory 012 is typically used to store the operating system and various application software installed on the computer device 01, such as the program code of the terrain data extraction and processing device in Embodiment 3. In addition, memory 012 can also be used to temporarily store various types of data that have been output or will be output.

[0147] In some embodiments, processor 011 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 011 is typically used to control the overall operation of computer device 01. In this embodiment, processor 011 is used to run program code stored in memory 012 or process data, such as terrain data extraction and processing methods.

[0148] Example 5 This embodiment also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App application store, etc., which stores a computer program. When the program is executed by a processor, it implements the corresponding function. In this embodiment, the computer-readable storage medium is used to store a terrain data extraction and processing device, which, when executed by a processor, implements the terrain data extraction and processing method of Embodiment 1.

[0149] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0150] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0151] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0152] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for extracting and processing terrain data, characterized in that, include: Based on the 3D scene data of the area to be processed, the ground objects in the area to be processed are identified to obtain an initial identification result. The 3D scene data is used to characterize the terrain and the ground objects in the area to be processed. The initial identification result includes the identified ground objects and the initial terrain data obtained by removing the identified ground objects. In response to a target selection operation, the target location corresponding to the target selection operation is determined, wherein the target selection operation is used to select a target ground object on the initial image corresponding to the initial recognition result; The target location and the initial image are input into the first segmentation model to obtain the initial region of the target ground object; Using the initial region as the first segmentation prior, the initial image, the target location, and the first segmentation prior are input into the second segmentation model to obtain the optimized region of the target ground object. In response to a correction operation, the correction location and correction information operated by the correction operation are determined, wherein the correction information is used to indicate whether the correction location belongs to the target ground feature; Using the optimized region as the second segmentation prior, the initial image, the corrected position, the corrected information, and the second segmentation prior are input into the second segmentation model to obtain the corrected region of the target ground object, wherein the restored region includes the corrected region; The initial terrain data is updated using the restored region to obtain the terrain data of the area to be processed.

2. The terrain data extraction and processing method according to claim 1, characterized in that, The step of inputting the target location and the initial image into the first segmentation model to obtain the initial region of the target ground object includes: Image features are obtained by performing feature extraction on the initial image; The target location is encoded to obtain a click prompt feature; An initial segmentation probability map is generated based on the image features and the click prompt features; and The initial region of the target ground object is determined based on the initial segmentation probability map.

3. The terrain data extraction and processing method according to claim 1, characterized in that, The step of inputting the initial image, the target location, and the first segmentation prior into the second segmentation model to obtain the optimized region of the target ground object includes: Generate a click response heatmap based on the target location; The initial image, the click response heatmap, and the first segmentation prior are concatenated by channels to obtain the first joint input feature; Multi-scale feature extraction is performed on the first joint input features to obtain the first multi-scale features; Generate an optimized segmentation probability map of the target ground features based on the first multi-scale features; and The optimized region of the target ground features is determined based on the optimized segmentation probability map.

4. The terrain data extraction and processing method according to claim 3, characterized in that, The steps of using the optimized region as a second segmentation prior, and inputting the initial image, the corrected location, the corrected information, and the second segmentation prior into the second segmentation model to obtain the corrected region of the target ground object include: A heatmap of corrected click responses is generated based on the corrected location and the corrected information; The initial image, the modified click response heatmap, and the second segmentation prior are concatenated by channels to obtain the second joint input feature; Multi-scale feature extraction is performed on the second joint input features to obtain the second multi-scale features; Generate a corrected segmentation probability map of the target aboveground features based on the second multi-scale features; and The corrected region of the target ground object is determined based on the corrected segmentation probability map.

5. The terrain data extraction and processing method according to any one of claims 1 to 4, characterized in that, In the presence of multiple correction operations, the correction region obtained in the previous iteration is used as the segmentation prior for the next iteration. Combined with the correction position and correction information corresponding to the current correction operation, the second segmentation model continues to iterate and update until a preset stopping condition is met. The preset stopping condition includes the number of times the user confirms the correction region or the number of times the correction operation is performed reaching a preset number.

6. The terrain data extraction and processing method according to claim 1, characterized in that, After obtaining the corrected area of ​​the target ground features, and before the step of updating the initial terrain data using the restored area to obtain the terrain data of the area to be processed, the method further includes: Extract candidate ground objects from the initial image to obtain a set of candidate ground object regions; Visual features of candidate aboveground object regions in the set are extracted to obtain candidate features; Extract the visual features of the corrected region to obtain reference features; Calculate the similarity between the candidate feature and the reference feature; Candidate aboveground object regions whose similarity meets preset similarity conditions are identified as similar aboveground object regions, wherein the restored region also includes the similar aboveground object regions.

7. The terrain data extraction and processing method according to claim 1, characterized in that, The steps of updating the initial terrain data using the restored region to obtain the terrain data of the area to be processed include: The boundaries of the restored region are determined to obtain the target contour; Obtain the boundary elevation information of the restored region based on the target contour; Generate a restored terrain surface to replace the restored area based on the boundary elevation information; The restored terrain surface is fused into the initial terrain data to obtain the terrain data of the area to be processed.

8. A terrain data extraction and processing device, characterized in that, include: The recognition module is used to recognize the ground objects in the area to be processed based on the three-dimensional scene data of the area to be processed, and to obtain an initial recognition result. The three-dimensional scene data is used to characterize the terrain and the ground objects in the area to be processed. The initial recognition result includes the recognized ground objects and the initial terrain data obtained by removing the recognized ground objects. A first response module is used to respond to a target selection operation and determine the target location corresponding to the target selection operation. The target selection operation is used to select a target ground object on the initial image corresponding to the initial recognition result. The target location and the initial image are input into a first segmentation model to obtain an initial region of the target ground object. The initial region is used as a first segmentation prior. The initial image, the target location, and the first segmentation prior are input into a second segmentation model to obtain an optimized region of the target ground object. The second response module is used to respond to the correction operation, determine the correction position and correction information operated by the correction operation, wherein the correction information is used to indicate whether the correction position belongs to the target ground object, take the optimized region as the second segmentation prior, input the initial image, the correction position, the correction information and the second segmentation prior into the second segmentation model to obtain the correction region of the target ground object, wherein the restored region includes the correction region; The restoration module is used to update the initial terrain data using the restored area to obtain the terrain data of the area to be processed.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images