An automatic editing method of real scene three-dimensional mesh model based on texture completion and generative backfilling
By using an automatic editing method that combines texture completion and generative backfilling, the problem of excessive manual intervention in existing technologies is solved, enabling efficient and seamless editing of real-world 3D mesh models, which is applicable to the updating of real-world 3D models in multiple fields.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU MOXI TECHNOLOGY CO LTD
- Filing Date
- 2026-03-28
- Publication Date
- 2026-07-14
AI Technical Summary
Existing real-world 3D mesh model editing technology requires a lot of manual intervention, making it difficult to achieve efficient and high-quality model editing. Furthermore, the generated new building models are difficult to automatically match with the scene, resulting in poor visual continuity.
An automatic editing method based on texture completion and generative backfilling is adopted. Through automatic registration and fusion technology, manual operation is reduced, a new building model with the same style as the surrounding area is generated, and automatic registration and fusion processing is performed.
It enables efficient and seamless editing of real-world 3D mesh models, improving editing efficiency, eliminating visual patchwork, and is suitable for editing single buildings and multi-objective projects at the park level, meeting the needs of fields such as digital twins and urban planning.
Smart Images
Figure CN122391468A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of real-scene 3D modeling and model editing technology, specifically involving an automatic editing method for real-scene 3D mesh models based on texture completion and generative backfilling. Background Technology
[0002] As a core data carrier in fields such as digital twins, urban planning, park management, and emergency simulation, the real-world 3D mesh model is composed of a set of vertices, a set of triangular faces, and texture maps. Through UV mapping, the texture is attached to the mesh surface, which can realistically restore the geometric shape and visual features of the real scene. In practical applications, it is often necessary to edit the existing real-world 3D mesh model, such as replacing buildings in the scene, repairing scene gaps after removing targets, and adding buildings that match the scene style, in order to adapt to business needs such as planning adjustments, scene updates, and scheme comparison.
[0003] Currently, editing of real-world 3D mesh models mainly involves manual editing workflows based on DCC software (such as Blender, 3ds Max, and Maya) or replacement workflows based on single generative 3D models. Among them, the manual editing workflow based on DCC software relies on operators to complete all editing work in professional 3D editing software. Specifically, it involves: manually deleting or masking target buildings in the scene, manually building new building models, manually applying textures to the new buildings and repeatedly adjusting material parameters, then manually moving, rotating, and scaling the new buildings to align them with the original scene coordinate system, and finally baking and exporting.
[0004] The replacement process based on a single generative 3D model reduces human intervention through generative technology. Specifically, it directly generates the target building in the scene void area after the target is removed using a generative model of text-based or image-based 3D. Then, the generated new building is placed back into the original scene manually or by simple rules.
[0005] However, both manual editing workflows based on DCC software (such as Blender, 3ds Max, and Maya) and replacement workflows based on single generative 3D models require a large amount of manual intervention. For example, in manual editing workflows, manual modeling and texturing cannot guarantee the consistency of the new building with the original real scene in terms of texture style, lighting effects, and detail granularity. The edited model is prone to obvious "collage" feel, which destroys the visual continuity of the scene. In the replacement workflow of 3D models, it is difficult to automatically match the scale and posture of the generated new building model with the empty areas of the scene. A lot of manual adjustment is still required to complete the placement, and the improvement in editing efficiency is limited. Therefore, it is difficult to meet the needs of efficient, high-quality, and low-manual-intervention model editing. Summary of the Invention
[0006] The purpose of this invention is to provide an automatic editing method for real-world 3D mesh models based on texture completion and generative backfilling. This method can reduce the amount of manual alignment work through automatic registration and fusion, and meet the needs of efficient, high-quality, and low-manual-intervention model editing, thereby solving the problems mentioned in the background art.
[0007] To achieve the above objectives, the present invention adopts the following technical solution:
[0008] An automatic editing method for real-world 3D mesh models based on texture completion and generative backfilling includes the following steps:
[0009] S1. Acquire the original image data and point cloud data of the target scene, and construct a textured 3D mesh model through 3D reconstruction, while recording its coordinate system, scale and texture map set.
[0010] S2. Perform target segmentation on the 3D mesh model to obtain the target building mesh and the model base mesh, and determine the empty space region by the difference between the target building mesh and the model base mesh;
[0011] S3. For the missing texture parts corresponding to the empty space area in the model base mesh, complete the missing texture by generating a model under conditions to obtain the updated base.
[0012] S4. Using the empty space area as the center, perform multi-channel rendering of the updated base from multiple perspectives to generate a set of rendering results.
[0013] S5. Combine the rendering result set with the editing constraint input prompt word generation module to automatically output structured building specifications and prompt words for generating the model;
[0014] S6. Use prompt words to generate new building exterior images that are consistent with the surrounding style in the generative image model, and select the best building exterior image.
[0015] S7. Input the optimal building exterior image into the graphical 3D model, generate a new textured 3D building mesh, and output its material, texture, and geometry.
[0016] S8. Based on the similarity transformation of the empty space area and the rendering camera parameters, the scene coordinates are solved, and the new building mesh is placed and registered in the scene to obtain the scene model after backfilling.
[0017] S9. Perform fusion processing on the backfilled scene model to output the final deliverable 3D model.
[0018] Preferably, the original image data is obtained through drone aerial photography or ground multi-view panoramic photography, and the point cloud data is obtained through LiDAR scanning.
[0019] Preferably, when constructing the 3D mesh model, camera pose estimation and point cloud reconstruction are performed first, then a mesh surface is generated from the dense point cloud, and finally, multi-view images are projected onto the mesh and stitched together to generate texture maps, thereby obtaining a textured 3D mesh model; the coordinate system and scale are determined and saved by GNSS / RTK control points, LiDAR absolute scale or known scale calibration objects.
[0020] Preferably, when determining the vacant space region, the target building mesh and the model base mesh are first geometrically aligned and topologically cleaned to ensure that they do not self-intersect or break surfaces in the same coordinate system; then the target building mesh is peeled off from the three-dimensional mesh model to obtain the contact boundary between the building and the base; then, using the contact boundary as a constraint, the corresponding closed boundary loop is extracted on the base surface and a polygonal region is constructed; finally, the polygonal region is extruded to a certain thickness in the direction of the base normal to obtain the vacant space region.
[0021] Preferably, when generating the rendering result set, the rendering viewpoint is first determined, and the camera intrinsics, resolution, and near and far clipping planes are set; then, the updated base is rendered under each viewpoint to output multiple buffer channels; finally, the multiple buffer channels of each viewpoint are packaged according to a unified naming and alignment method to form a rendering result set, wherein the buffer channels include RGB channels, empty mask channels, depth channels, and normal and boundary line channels.
[0022] Preferably, the prompt word generation module includes a visual parser, a constraint parser, a specification assembler, and a prompt word templater; the visual parser is used to extract key scene information from the rendering result set; the constraint parser is used to normalize editing constraints into computable fields; the specification assembler is used to fuse key scene information and fields to form structured building specifications; the prompt word templater is used to select prompt word templates based on structured building specifications and fill in keywords, generate positive and negative prompt words, and output them as prompt words.
[0023] Preferably, during the generation of the new building exterior image, when the geometric constraints of the empty space area need to be strictly met or the requirements for style consistency with the surrounding area are strong, the empty mask, depth, normal and boundary line are used as condition inputs to apply structural and spatial constraints to the generation process; when it is difficult to reliably meet the dual objectives of style consistency and constraint satisfaction by generating only once, or when it is necessary to select the best among multiple feasible appearance schemes, multiple candidate images are generated under the same conditions and the consistency index is calculated, and the optimal image is selected as the optimal building exterior image.
[0024] Preferably, the generative image model is used to produce a building appearance consistent with the scene at the two-dimensional level to provide texture or facade appearance priors for the new building's three-dimensional mesh; the image-generated three-dimensional model is used to upgrade the two-dimensional appearance into three-dimensional geometry and textured mesh to directly produce the new building's three-dimensional mesh; the original scene's three-dimensional mesh model and the updated base provide coordinate system, scale, and placement constraints so that the new building's three-dimensional mesh can be aligned and inserted into the empty space area under the same coordinates and scale.
[0025] Preferably, in step S8, the similarity transformation is solved by the least squares method to obtain the similarity transformation parameters, which are used to simultaneously align the new building's three-dimensional mesh with the boundary of the empty space region at three levels: scale, orientation, and position.
[0026] Preferably, in step S9, the fusion process includes boundary stitching, normal and texture transition, collision detection, terrain fitting, LOD generation and format export.
[0027] The present invention proposes an automatic editing method for real-world 3D mesh models based on texture completion and generative backfilling, which has the following advantages compared with existing technologies:
[0028] 1. This invention forms a closed-loop automated editing process for real-scene 3D mesh models through steps such as 3D mesh model construction, target segmentation and gap extraction, base texture completion, multi-channel rendering, prompt word generation, building appearance generation, image-generated 3D modeling, automatic registration and placement, and fusion export. From target segmentation and texture completion to generation modeling and registration fusion, each step requires no manual operation, significantly reducing the editing time for a single scene. At the same time, it supports generating multiple candidate building appearances under the same constraints and selecting the best one, which can quickly produce a variety of editing schemes to meet the needs of planning comparison, scheme demonstration and other business for rapid modeling, and significantly improve the editing efficiency of real-scene 3D mesh models.
[0029] 2. This invention performs conditional model completion operations on the missing texture parts in the model base mesh, filling the texture gaps in areas such as the ground and roof after the target building is removed, so that the model base remains visually continuous and complete, laying a seamless scene background foundation for subsequent new building backfilling, and eliminating the visual patchwork feel of the edited scene from the root.
[0030] 3. Based on the 3D scene context of multi-channel rendering, this invention combines editing constraints to automatically generate structured prompts, providing clear style, scale, and structural constraints for generative models. At the same time, in the building appearance generation stage, it can introduce geometric conditions such as gap masks, depth, and normals, so that the generated new building appearance is highly consistent with the surrounding scene in terms of style, material, and color tone at the 2D level. Then, through graph-generated 3D modeling, the 2D appearance is transformed into a 3D style that conforms to the scene constraints, improving the controllability and consistency of the new building generation.
[0031] 4. This invention is not only applicable to the replacement and addition of individual buildings, but can also be seamlessly adapted to multi-object batch editing at the park and street level. It can meet the needs of updating real-scene 3D models and reconstructing scenes in multiple fields such as digital twins, urban planning, park management, and emergency simulation. At the same time, the input and output of each step in the process have standardized features and can be compatible with existing 3D reconstruction software and generative model platforms. It does not require major modifications to the existing technology system and has strong practical application value and promotion potential. Attached Figure Description
[0032] Figure 1 This is a schematic diagram of the process of the present invention;
[0033] Figure 2 This is a system block diagram of the automatic editing platform of the present invention. Detailed Implementation
[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0035] This invention provides, for example Figure 1 The method for automatically editing a real-world 3D mesh model based on texture completion and generative backfilling, as shown, includes the following steps:
[0036] S1. Acquire the original image data and point cloud data of the target scene, and construct a textured 3D mesh model through 3D reconstruction, while recording its coordinate system, scale and texture map set.
[0037] Specifically, the original image data is obtained through drone aerial photography or ground multi-view panoramic photography to ensure sufficient overlap and coverage, and the point cloud data is obtained through LiDAR scanning, such as airborne, vehicle-mounted, and ground-based LiDAR.
[0038] When constructing the 3D mesh model, camera pose estimation and point cloud reconstruction are performed first. The image path for point cloud reconstruction is usually SfM / MVS, and the point cloud path is usually SLAM / fusion. Then, a mesh surface is generated from the dense point cloud, such as Poisson / TSDF surface reconstruction. Finally, multi-view images are projected onto the mesh and stitched to generate texture maps. The texture maps are UV unwrapping or texture atlases, thus obtaining a textured 3D mesh model. The coordinate system and scale are determined and saved by GNSS / RTK control points, LiDAR absolute scale, or known scale calibration objects.
[0039] S2. Perform target segmentation on the 3D mesh model to obtain the target building mesh and the model base mesh, and determine the empty space region by the difference between the target building mesh and the model base mesh;
[0040] Furthermore, when determining the missing space region, the target building mesh and the model base mesh are first geometrically aligned and topologically cleaned to ensure that they do not self-intersect or break surfaces in the same coordinate system. Then, the target building mesh is peeled off from the 3D mesh model to obtain the contact boundary between the building and the base. The contact boundary can be regarded as the footing line of the building on the base. Then, using the contact boundary as a constraint, the corresponding closed boundary loop is extracted on the base surface and a polygonal region is constructed. Finally, the polygonal region is extruded with a certain thickness in the direction of the base normal. The extrusion thickness should not be less than the reconstruction accuracy of the real scene 3D model. For example, the accuracy of UAV oblique photogrammetry modeling is mostly 2cm-50cm. This avoids the 3D spatial expression of the missing region being covered by model noise due to insufficient thickness, which would not provide effective geometric constraints for subsequent registration and generation. It can also be filled in voxels or distance fields to obtain the missing space region. The missing space region can be represented as: a boundary loop / polygonal mask on the base surface, or a voxel mask / 3D volume region with thickness, depending on subsequent needs.
[0041] S3. For the missing texture parts corresponding to the empty space area in the model base mesh, complete the missing texture by generating a model under conditions to obtain the updated base.
[0042] For example, for the model base mesh B, texture completion is performed on the missing texture portion corresponding to the empty space region in its texture map set T, resulting in the completed texture T′ and the updated base mesh B′. Texture completion based on the generative model can be represented as:
[0043]
[0044] Where: T is the original texture map; T′ is the completed texture map; M is the missing mask (missing regions are 1, known regions are 0, obtained by mapping H to UV space); To generate candidate texture results for the missing regions using the generative model; For parameters The distribution of the conditional generation model; C represents the constraint / condition information; Element-wise multiplication; This represents a mask for a known region.
[0045] By performing conditional model completion operations on the missing texture parts in the model base mesh, the texture gaps in areas such as the ground and roof after the target building was removed were filled, allowing the model base to maintain visual continuity and integrity. This laid a seamless scene background foundation for the subsequent backfilling of new buildings and eliminated the visual patchwork feel of the edited scene from the root.
[0046] S4. Using the empty space area as the center, perform multi-channel rendering of the updated base from multiple perspectives to generate a set of rendering results.
[0047] When generating the rendering result set, the rendering viewpoint is first determined. The rendering viewpoint samples a set of camera poses around the empty space region in the horizontal and pitch directions, or selects a pose close to the original acquisition viewpoint, and sets the camera intrinsics, resolution, and near / far clipping planes. Then, the updated base is rendered under each viewpoint to output multiple buffer channels. Finally, the multiple buffer channels of each viewpoint are packaged according to a unified naming and alignment method to form the rendering result set. The buffer channels include RGB channels, empty mask channels, depth channels, and normal and boundary line channels. RGB is typically based on the color rendering result of the updated base texture. The empty mask is typically a binary or soft mask obtained by projecting the empty space region onto screen space. The depth channel is the depth value from each pixel to the camera. The normal and boundary line channels are used to output surface normal maps or boundary lines obtained through normal or depth discontinuity detection.
[0048] By using a multi-channel rendered 3D scene context as a foundation and automatically generating structured prompts based on editing constraints, the generative model is provided with clear style, scale, and structural constraints. At the same time, geometric conditions such as gap masks, depth, and normals can be introduced during the building appearance generation stage, so that the generated new building appearance is highly consistent with the surrounding scene in terms of style, material, and color tone at the 2D level. Then, through graph-generated 3D modeling, the 2D appearance is transformed into a 3D style that conforms to the scene constraints, improving the controllability and consistency of the new building generation.
[0049] S5. Combine the rendering result set with the editing constraint input prompt word generation module to automatically output structured building specifications and prompt words for generating the model;
[0050] Specifically, the prompt word generation module includes a visual parser, a constraint parser, a specification assembler, and a prompt word templater. The visual parser is used to extract key scene information from the rendering result set. The constraint parser is used to normalize editing constraints into computable fields. The specification assembler is used to integrate key scene information and fields to form structured building specifications. The prompt word templater is used to select prompt word templates based on structured building specifications and fill in keywords to generate positive and negative prompt words and output them as prompt words. The prompt word generation module outputs parsable JSON or field tables, such as style, main material, main color, number of layers, window rhythm, and roof type, which facilitates controllable generation and batch editing.
[0051] S6. Use prompt words to generate new building exterior images that are consistent with the surrounding style in the generative image model, and select the best building exterior image.
[0052] During the generation of the new building exterior image, when the geometric constraints of the empty space area need to be strictly met or the requirements for style consistency with the surrounding area are strong, the empty mask, depth, normal and boundary line are used as condition inputs to apply structural and spatial constraints to the generation process; when it is difficult to stably meet the dual objectives of style consistency and constraint satisfaction by generating only once, or when it is necessary to select the best among multiple feasible appearance schemes, multiple candidate images are generated under the same conditions and the consistency index is calculated, and the optimal image is selected as the optimal building exterior image.
[0053] The generative image model is used to generate a building appearance consistent with the scene at the two-dimensional level to provide texture or facade appearance priors for the new building's three-dimensional mesh. This includes scoring candidate images or backfill results for style consistency (e.g., semantic similarity to surrounding areas, hue histogram distance, texture granularity index), selecting the optimal result, or triggering regeneration. Optionally, the comprehensive score is defined as:
[0054] ,
[0055] in, This is the overall score. These represent the scores for semantic consistency, tone consistency, and texture consistency, respectively. These are the corresponding weighting coefficients, used to adjust the contribution of each consistency indicator to the overall score.
[0056] S7. Input the optimal building exterior image into the graphical 3D model, generate a new textured 3D building mesh, and output its material, texture, and geometry.
[0057] The graph-generated 3D model is used to upgrade the 2D appearance to 3D geometry and textured mesh to directly produce the 3D mesh of the new building; the original scene 3D mesh model and the updated base provide coordinate system, scale and placement constraints, so that the 3D mesh of the new building can be aligned and inserted into the empty space area under the same coordinates and scale.
[0058] S8. Solve the similarity transformation T_g to the scene coordinates based on the empty space area and the rendering camera parameters, and complete the placement and registration of the new building mesh in the scene to obtain the scene model after backfilling.
[0059] The similarity transformation is solved by the least squares method to obtain the similarity transformation parameters, which is used to simultaneously align the new building's 3D mesh with the boundary of the empty space region at three levels: scale, orientation, and position.
[0060] Specifically, the similarity transformation T_g includes scale s, rotation R, and translation t, and is expressed as:
[0061] ,
[0062] in, This represents the three-dimensional point coordinates of the new building mesh (or its feature points) in its own coordinate system. This represents the coordinates of the point in the scene coordinate system after the similarity transformation; T_g is the 7DoF similarity transformation that maps the new building's 3D mesh to the scene coordinate system, which is determined by the scale. Rotation With translation The similarity transformation parameters are determined by means of least squares, i.e.:
[0063]
[0064] in, Sampling points on the boundary (or feature point) of the new building. N represents the corresponding point (or matching point pair) on the boundary of the missing region; N is the number of corresponding point pairs participating in the registration. This represents the Euclidean norm.
[0065] S9. Perform fusion processing on the backfilled scene model to output the final deliverable 3D model; fusion processing includes boundary stitching, normal and texture transition, collision detection, terrain fitting, LOD generation and format export.
[0066] This automated editing process for real-world 3D mesh models forms a closed loop, encompassing steps such as 3D mesh model construction, target segmentation and gap extraction, base texture completion, multi-channel rendering, prompt word generation, building appearance generation, image-based 3D modeling, automatic registration and placement, and fusion export. From target segmentation and texture completion to model generation, registration, and fusion, each step requires no manual operation, significantly reducing the editing time for a single scene. It also supports generating multiple candidate building appearances under the same constraints and selecting the best one, enabling the rapid production of various editing solutions to meet the needs of planning comparison, scheme demonstration, and other business applications requiring rapid modeling, thus significantly improving the editing efficiency of real-world 3D mesh models.
[0067] This invention also provides an automatic editing platform for real-scene 3D mesh models, used to implement an automatic editing method for real-scene 3D mesh models based on texture completion and generative backfilling, such as... Figure 2 As shown, the platform includes an input module, a processing module, and an output module. The input end of the processing module is connected to the input module, and the input end of the output module is connected to the output end of the input module. The input module includes an acquisition unit for acquiring raw images or point clouds, an initial 3D mesh model for outputting textures and coordinates, and an editing intent / constraint unit.
[0068] The processing module includes a 3D segmentation and gap extraction unit, a texture completion and consistency constraint unit, a multi-channel rendering and prompt generation unit, a generative modeling service unit, and a placement registration and fusion unit connected in sequence. The acquisition unit is connected to the 3D segmentation and gap extraction unit, the output end of the initial 3D mesh model is connected to the texture completion and consistency constraint unit, and the editing intent / constraint unit is connected to the multi-channel rendering and prompt generation unit. It is used to execute the process of 3D segmentation, texture completion, rendering, prompt generation, image generation, image-generated 3D, registration and fusion.
[0069] The output module includes an updated 3D scene model and a quality report output unit connected to the placement, registration and fusion unit, for outputting the final deliverable 3D model and quality report.
[0070] This invention is not only applicable to the replacement and addition of individual buildings, but can also be seamlessly adapted to multi-object batch editing at the park and street level. It can meet the needs of updating real-scene 3D models and reconstructing scenes in multiple fields such as digital twins, urban planning, park management, and emergency simulation. At the same time, the input and output of each step in the process have standardized features, which can be compatible with existing 3D reconstruction software and generative model platforms without the need for major modifications to the existing technology system. It has strong practical application value and promotion potential.
[0071] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. An automatic editing method for real-world 3D mesh models based on texture completion and generative backfilling, characterized in that: Includes the following steps: S1. Acquire the original image data and point cloud data of the target scene, and construct a textured 3D mesh model through 3D reconstruction, while recording its coordinate system, scale and texture map set. S2. Perform target segmentation on the 3D mesh model to obtain the target building mesh and the model base mesh, and determine the empty space region by the difference between the target building mesh and the model base mesh; S3. For the missing texture parts corresponding to the empty space area in the model base mesh, complete the missing texture by generating a model under conditions to obtain the updated base. S4. Using the empty space area as the center, perform multi-channel rendering of the updated base from multiple perspectives to generate a set of rendering results. S5. Combine the rendering result set with the editing constraint input prompt word generation module to automatically output structured building specifications and prompt words for generating the model; S6. Use prompt words to generate new building exterior images that are consistent with the surrounding style in the generative image model, and select the best building exterior image. S7. Input the optimal building exterior image into the graphical 3D model, generate a new textured 3D building mesh, and output its material, texture, and geometry. S8. Based on the similarity transformation of the empty space area and the rendering camera parameters, the scene coordinates are solved, and the new building mesh is placed and registered in the scene to obtain the scene model after backfilling. S9. Perform fusion processing on the backfilled scene model to output the final deliverable 3D model.
2. The automatic editing method for a real-world 3D mesh model based on texture completion and generative backfilling as described in claim 1, characterized in that: The original image data was obtained through drone aerial photography or ground multi-view panoramic photography, and the point cloud data was obtained through LiDAR scanning.
3. The automatic editing method for a real-world 3D mesh model based on texture completion and generative backfilling as described in claim 2, characterized in that: When constructing the 3D mesh model, camera pose estimation and point cloud reconstruction are performed first, then a mesh surface is generated from the dense point cloud, and finally, multi-view images are projected onto the mesh and stitched together to generate texture maps, thus obtaining a textured 3D mesh model; the coordinate system and scale are determined and saved by GNSS / RTK control points, LiDAR absolute scale or known scale calibration objects.
4. The automatic editing method for a real-world 3D mesh model based on texture completion and generative backfilling as described in claim 1, characterized in that: When determining the vacant space region, the target building mesh and the model base mesh are first geometrically aligned and topologically cleaned to ensure that they do not self-intersect or break surfaces in the same coordinate system. Then, the target building mesh is peeled off from the 3D mesh model to obtain the contact boundary between the building and the base. Using this contact boundary as a constraint, the corresponding closed boundary loop is extracted on the base surface and a polygonal region is constructed. Finally, the polygonal region is extruded with a certain thickness in the direction of the base normal to obtain the vacant space region.
5. The automatic editing method for a real-world 3D mesh model based on texture completion and generative backfilling as described in claim 1, characterized in that: When generating the rendering result set, the rendering viewpoint is first determined, and the camera intrinsics, resolution, and near and far clipping planes are set. Then, the updated base is rendered under each viewpoint to output multiple buffer channels. Finally, the multiple buffer channels of each viewpoint are packaged according to a unified naming and alignment method to form a rendering result set. The buffer channels include RGB channels, empty mask channels, depth channels, and normal and boundary line channels.
6. The automatic editing method for a real-world 3D mesh model based on texture completion and generative backfilling as described in claim 5, characterized in that: The prompt word generation module includes a visual parser, a constraint parser, a specification assembler, and a prompt word templater; the visual parser is used to extract key scene information from the rendering result set; The constraint parser is used to normalize editing constraints into computable fields; the specification assembler is used to merge key scene information and fields to form structured building specifications. The prompt templater is used to select prompt templates based on structured building specifications and fill in keywords, generating positive and negative prompts and outputting them as prompts.
7. The automatic editing method for a real-world 3D mesh model based on texture completion and generative backfilling as described in claim 1, characterized in that: During the generation of the new building exterior image, when the geometric constraints of the empty space area need to be strictly met or the requirements for style consistency with the surrounding area are strong, the empty mask, depth, normal and boundary line are used as condition inputs to apply structural and spatial constraints to the generation process; when it is difficult to stably meet the dual objectives of style consistency and constraint satisfaction by generating only once, or when it is necessary to select the best among multiple feasible appearance schemes, multiple candidate images are generated under the same conditions and the consistency index is calculated, and the optimal image is selected as the optimal building exterior image.
8. The automatic editing method for a real-world 3D mesh model based on texture completion and generative backfilling as described in claim 1, characterized in that: The generative image model is used to produce a building appearance consistent with the scene at the two-dimensional level to provide a priori texture or facade appearance for the new building's three-dimensional mesh. Graph-based 3D models are used to elevate 2D appearances to 3D geometry and textured meshes to directly generate new 3D building meshes; The original scene 3D mesh model and the updated base provide coordinate system, scale and placement constraints, enabling the new building 3D mesh to be aligned and inserted into the empty space area under the same coordinates and scale.
9. The automatic editing method for a real-world 3D mesh model based on texture completion and generative backfilling as described in claim 1, characterized in that: In step S8, the similarity transformation is solved using the least squares method to obtain the similarity transformation parameters, which are used to simultaneously align the new building's 3D mesh with the boundary of the empty space region at three levels: scale, orientation, and position.
10. The automatic editing method for a real-world 3D mesh model based on texture completion and generative backfilling as described in claim 1, characterized in that: In step S9, the fusion process includes boundary stitching, normal and texture transition, collision detection, terrain fitting, LOD generation and format export.