Scenarized modeling method and system for building roof appendages based on instance segmentation

Abstract

The invention provides a scene modeling method and system for building roof appendages based on instance segmentation. The method comprises the following steps: segmenting a vector contour of each roof appendage based on a deep learning model; carrying out superposition analysis on the vector data of each roof appendage, the digital surface model data and the building roof surface model data, calculating the bottom elevation and the top elevation of each roof appendage, and carrying out vertical stretching to generate an outer bounding box of each roof appendage; and performing three-dimensional space affine transformation on the three-dimensional template model of each type of roof appendages, and rotating, zooming and translating to the range position of the outer bounding box of each roof appendage to complete scene modeling of each roof appendage. According to the method, the modeling efficiency of the roof appendage model can be improved, the manual modeling cost is reduced, the construction period of a high-simulation three-dimensional scene is shortened, and favorable conditions are also created for efficient rendering of massive buildings in an ultra-large-range scene by adopting an instantiation rendering mode of the appendage model.

Description

Technical Field

[0001] The present invention relates to the field of real-scene three-dimensional modeling, and more specifically, to a scenario-based modeling method and system for building roof attachments based on instance segmentation. Background Art

[0002] With the rapid development of real-world 3D technology, digital twin cities, and City Information Modeling (CIM), the industry is placing increasingly stringent demands on the accuracy and visual fidelity of 3D models of urban buildings. 3D models of urban buildings primarily consist of three major components: building walls, roofs, and rooftop infrastructure. Rooftop infrastructure includes common types such as solar water heaters, water towers, and air conditioning units. As crucial components of urban building models, their precise and realistic modeling plays a crucial role in ensuring the authenticity and fidelity of 3D building models and, ultimately, the entire 3D urban scene.

[0003] Current single-unit modeling techniques for urban building 3D models primarily rely on OSGB model data generated by oblique photogrammetry and 3D point cloud data as a reference. For standard buildings with regular shapes, automated modeling capabilities for the main building structure have gradually been developed, essentially reproducing the main structural features of the building's walls and roof. However, existing geometric reconstruction algorithms struggle to effectively reconstruct various rooftop ancillary facilities, such as solar water heaters, water towers, and air conditioning units. Firstly, rooftop ancillary facilities are relatively small compared to the main roof structure. During the 3D building geometry reconstruction process, these smaller rooftop ancillary features are often cleaned up and removed to ensure lightweight representation of geometric features and robust algorithm operation. Secondly, rooftop ancillary facilities, such as solar water heaters, water towers, and air conditioning units, require higher precision in geometric form and texture detail than the main building structure. Traditional 3D reconstruction algorithms have significant limitations in reconstructing the 3D morphology of these ancillary facilities. Typically, after completing the modeling of the main urban building structure, manual model placement and modeling operations are required at specific locations on the roof of the 3D building model.

[0004] However, as the demands for low-cost, timely, and wide-area urban scene construction continue to rise, the key trend in the field of 3D scene construction is how to efficiently and cost-effectively reconstruct large-scale scenes quickly and cost-effectively. In this context, traditional manual modeling methods have gradually revealed their inability to meet the demands of rapidly constructing urban 3D scenes when modeling objects attached to rooftops of urban buildings. Therefore, developing technologies that fully automate the modeling of various rooftop attachments has important practical significance and application value for significantly improving the efficiency and update speed of the entire urban 3D scene. Summary of the Invention

[0005] Aiming at the problems of low efficiency in modeling urban building roof attachments and difficulty in achieving fully automated construction of building models, the present invention provides a scenario-based modeling method and system for building roof attachments based on instance segmentation.

[0006] According to a first aspect of the present invention, a scenario-based modeling method for building roof attachments based on instance segmentation is provided, comprising:

[0007] Step S1, reconstructing a building unit according to the tilted model data of the building to be modeled, and obtaining a building unit model of the building to be modeled;

[0008] Step S2, loading the geometric information and texture information of the tilt model data of the building to be modeled, performing vertical grid sampling to generate true orthogonal image data;

[0009] Step S3, segmenting the initial vector outline of each roof attachment from the true orthogonal image data based on a preset roof attachment instance segmentation model;

[0010] Step S4, segmenting the instance to obtain the initial vector outline of each roof attachment, performing regularization processing according to the shape characteristics of different attachments, and obtaining the vector data of each roof attachment after regularization processing;

[0011] Step S5, loading the geometric information of the tilt model data of the building to be modeled, performing elevation value sampling to generate digital surface model data expressing the height fluctuation characteristics of the roof attachment;

[0012] Step S6, loading and reading the geometric information of the building monomer model of the building to be modeled in step S1, and generating building roof surface model data that expresses the undulating characteristics of the roof surface structure;

[0013] Step S7, performing an overlay analysis on the vector data of each roof attachment obtained in step S4, the digital surface model data obtained in step S5, and the building roof surface model data obtained in step S6, calculating the bottom elevation and top elevation of each roof attachment, and vertically stretching the data to generate an outer bounding box for each roof attachment;

[0014] In step S8, a three-dimensional space affine transformation is performed on the three-dimensional template model of each type of roof attachment, and the three-dimensional template model of each type of roof attachment is rotated, scaled, and translated to the range position of the outer bounding box of each roof attachment in step 7, thereby completing the scenario-based modeling of each roof attachment of the building to be modeled.

[0015] According to a second aspect of the present invention, a scenario-based modeling system for building roof attachments based on instance segmentation is provided, comprising:

[0016] A monomer reconstruction module is used to reconstruct a building monomer based on the tilt model data of the building to be modeled, and obtain a building monomer model of the building to be modeled;

[0017] The first sampling module is used to load the geometric information and texture information of the tilt model data of the building to be modeled, and perform vertical grid sampling to generate true orthographic image data;

[0018] An instance segmentation module, configured to segment an initial vector outline of each roof attachment from the true orthogonal image data based on a preset roof attachment instance segmentation model;

[0019] A regularization processing module is used to segment the initial vector outline of each roof attachment from the instance, perform regularization processing according to the shape characteristics of different attachments, and obtain the vector data of each roof attachment after regularization processing;

[0020] The second sampling module is used to load the geometric information of the tilt model data of the building to be modeled, perform elevation value sampling to generate digital surface model data expressing the height fluctuation characteristics of the roof attachment;

[0021] A generation module is used to load and read the geometric information of the building monomer model of the building to be modeled, and generate building roof surface model data that expresses the undulating characteristics of the roof surface structure;

[0022] an overlay analysis module for performing overlay analysis on the vector data of each roof attachment, the digital surface model data, and the building roof surface model data, calculating the bottom elevation and top elevation of each roof attachment, and vertically stretching the data to generate an outer bounding box for each roof attachment;

[0023] The affine transformation module is used to perform three-dimensional space affine transformation on the three-dimensional fine model of each type of roof attachment, rotate, scale, and translate the three-dimensional template model of each type of roof attachment to the range position of the outer bounding box of each roof attachment, and complete the scene-based modeling of each roof attachment of the building to be modeled.

[0024] The present invention provides a scenario-based modeling method and system for building roof attachments based on instance segmentation. This method trains a roof attachment instance segmentation model to automatically segment and extract the outlines and types of building roof attachment objects. By calculating the bounding box of the roof attachment, affine transformations such as rotation, scaling, and translation are performed on common artificial models of the attachments in three-dimensional space based on the bounding box's position, thereby achieving automatic placement of the roof attachment models and scenario-based instance rendering. The technical solution of the present invention not only effectively solves the problem of automated identification and modeling of building roof attachment models, but also adopts an instanced loading and rendering scheme for the same type of attachment models, significantly reducing video memory usage during rendering. This solution also has important application value in optimizing the loading and rendering efficiency of very large scenes, providing strong support for improving the efficiency of urban three-dimensional scene construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A flow chart of a scenario-based modeling method for building roof attachments based on instance segmentation provided by one embodiment of the present invention;

[0026] Figure 2 A structural block diagram of a building roof attachment scenario modeling system based on instance segmentation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be arbitrarily combined with each other to form a feasible technical solution. This combination is not restricted by the sequence of steps and / or structural composition mode, but must be based on the ability of ordinary technicians in this field to implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0028] To address the current contradiction between the high accuracy, fidelity, and timeliness of building modeling in urban 3D scenes and the difficulty of efficiently constructing fine structures such as rooftop appendages using automated algorithms, this paper proposes a scenario-based modeling method for building rooftop appendages based on instance segmentation. This method first loads raw tilted model data and converts it into a true orthogonal image (TDOM) and digital surface model (DSM). It then creates a rooftop appendage annotation dataset based on the TDOM data and trains a rooftop appendage instance segmentation model using the YOLOv8 Instance Segmentation model. The trained instance segmentation model then identifies, extracts, regularizes, and calculates the 3D spatial positions of rooftop appendages, such as solar water heaters, water towers, and air conditioner outdoor units, in the TDOM images. Finally, the standard 3D model of each type of building appendage is subjected to affine transformations such as rotation, translation, and scaling based on the calculated 3D spatial positions. This method ultimately completes the modeling and scene instance rendering of the rooftop appendage model on the main building roof.

[0029] Figure 1 A flow chart of a scenario-based modeling method for building roof attachments based on instance segmentation is provided in one embodiment of the present invention. Figure 1 As shown, the method includes:

[0030] Step S1 : reconstructing a building monomer according to the tilted model data of the building to be modeled, and obtaining a building monomer model of the building to be modeled.

[0031] It is understandable that for buildings to be scenario-based modeled, their tilted model data (OSGB) is loaded for building unit reconstruction. The reconstructed building unit model can only express the main characteristic structures of the roof and wall in terms of geometric accuracy, and cannot represent the auxiliary structures such as solar water heaters, water towers, and air-conditioning outdoor units on the roof.

[0032] The monomer reconstruction of the main structure of the building based on the tilt model data (OSGB) mainly includes the following steps:

[0033] (1) Load the geometric information and texture information of the tilt model data to generate 3D color point cloud data;

[0034] (2) Based on the RandLA-NET deep learning model, semantic segmentation of 3D color point cloud data is performed to separate all building point cloud data.

[0035] (3) Divide the building point cloud data into spatial grids, and use the RANSAC algorithm to fit the three-dimensional plane of the point cloud in each grid to obtain the three-dimensional plane of each grid;

[0036] (4) Calculate the angle between the normal vectors of two adjacent three-dimensional planes, merge the adjacent planes with smaller angles, and obtain the characteristic planes of the building body, including the top surface and facade of the building body;

[0037] (5) Extend and intersect all the main building feature planes in the previous step to obtain the closed roof and wall structure, and output the reconstructed building monomer model data.

[0038] Step S2: Load the geometric information and texture information of the tilted model data of the building to be modeled, and perform vertical grid sampling to generate true orthogonal image data.

[0039] The process involves loading the geometry and texture information of the tilted model data for the building to be modeled, traversing each OSGB model file using the OpenSceneGraph 3D graphics library, and performing vertical grid sampling to generate true orthographic image data (TDOM) at a resolution of 0.05m. This high-resolution TDOM clearly displays the characteristic outlines of the building's roof attachments, providing the data foundation for instance segmentation of these attachments.

[0040] Step S3 : segmenting the initial vector outline of each roof attachment from the true orthogonal image data based on a preset roof attachment instance segmentation model.

[0041] It is understandable that a roof attachment instance segmentation model is pre-trained, and the trained preset roof attachment instance segmentation model is used to instance segment the initial vector outline of each roof attachment from the TDOM image data of the building to be modeled.

[0042] The training process of the roof attachment instance segmentation model is as follows:

[0043] Using TDOM data from different regions, we annotated and compiled a dataset of vector outlines of rooftop attachments, such as solar water heaters, water towers, and air conditioner outdoor units. We then used the YOLOv8 Instance Segmentation algorithm to train a rooftop attachment instance segmentation model on the annotated dataset. The YOLOv8 Instance Segmentation model, a version of YOLOv8 for instance segmentation, uses a mask-based segmentation approach combined with object detection capabilities to quickly and accurately predict the outline of each rooftop attachment from TDOM image data.

[0044] The present invention uses LabelMe data annotation software to create a dataset for instance segmentation of roof attachment contours. The annotated data stores the boundary contour information and accessory type information of each roof attachment. The training set, validation set, and test set are configured in a 6:1:1 ratio for the annotation dataset cost. The roof attachment instance segmentation model is trained based on the training and validation set data, and multiple rounds of iterations are performed until the loss value significantly decreases and converges. The intersection of union (IOU) value of the extracted contour mask and the annotated true value mask is then calculated based on the test set. Results with an IOU value greater than or equal to 0.8 are considered correct. Test analysis confirmed that the proportion of missed and incorrect extractions with an IOU value less than 0.8 was less than 15%. The trained model can meet the basic requirements for instance segmentation of roof attachment contours, completing the preparation of the final roof attachment contour instance segmentation model.

[0045] Based on the trained roof attachment instance segmentation model, the TDOM image data generated in step S2 is subjected to instance segmentation extraction of roof attachment outlines. Instance segmentation extracts the outline of each roof attachment and filters the segmentation results based on their confidence scores. Segmentation results with scores greater than or equal to 0.7 are retained, ultimately yielding the initial vector outline of each roof attachment. The confidence score is a quantitative expression of the accuracy of the predicted roof attachment instance segmentation results by the roof attachment instance segmentation model, ranging from 0 to 1. To ensure the accuracy of the building instance segmentation results, the present invention removes instance segmentation results with scores below 0.7, reducing the probability of incorrectly extracting other roof debris as attachment objects.

[0046] Step S4: segment the instance to obtain the initial vector outline of each roof attachment, perform regularization processing according to the shape features of different attachments, and obtain the vector data of each roof attachment after regularization processing.

[0047] It can be understood that the initial vector outline of each roof attachment segmented from the building is regularized according to the different shape characteristics of the attachment. The solar water heaters, small air-conditioning outdoor units, large air-conditioning outdoor units, etc. with rectangular top-view projection shapes are squared, and the water towers with circular top-view projection shapes are arc fitted to obtain the regularized attachment vector data.

[0048] The specific steps of regularization processing include:

[0049] (1) Regularizing the outlines of rectangular appendages, for example, regularizing the outlines of solar water heaters and small air conditioner outdoor units, includes the following steps:

[0050] a. Traverse one by one to obtain the point set P={(x i ,y i )|i=1,2,…,n}, calculate the average coordinates of all points in the point set as the coordinates of the initial rectangle center point P center ( ).

[0051] b. Translate all point coordinates to return the data mean to zero. The calculation formula is as follows:

[0052] ;

[0053] in, is the coordinate of the point after coordinate transformation, is the mean of the horizontal coordinates of all points, is the mean of the vertical coordinates of all points.

[0054] c. Calculate the covariance matrix C of all points, which describes the degree of change of the points in all directions. The calculation formula is as follows:

[0055] ;

[0056] d. Solve the eigenvalues ​​and eigenvectors of the covariance matrix:

[0057] Assume that the eigenvalues ​​of the covariance matrix are λ1 and λ2, and the corresponding eigenvectors are v1 and v2 respectively. The eigenvalues ​​and eigenvectors satisfy the equation:

[0058] ;

[0059] Solving this equation yields two eigenvalues, λ1 and λ2. Assuming λ1 ≥ λ2 ≥ 0, we substitute this into the equation to find the corresponding eigenvectors, v1 and v2. The principal direction is the direction indicated by the eigenvector v1 corresponding to the largest eigenvalue λ1, and the secondary principal direction is the direction indicated by the eigenvector v2 corresponding to the second-largest eigenvalue λ2. The principal and secondary principal directions are orthogonal. The principal direction corresponds to the long side of the rectangle, and the secondary principal direction corresponds to the short side.

[0060] e. Calculate along the long side and short side directions to find the distance from the center point of the rectangle P. center The farthest two points, calculate the length of the long side and the length of the short side respectively based on the farthest two points;

[0061] f. Determine the outline vector data of each rectangular roof attachment after regularization according to the length of the long side, the length of the short side and the center point of the rectangle.

[0062] (2) Regularize the arcs of the contours of circular appendages, for example, regularize the arcs of the contours of circular water towers, small air-conditioning outdoor units, and large air-conditioning outdoor units:

[0063] Traverse one by one to obtain the point set P={(x i ,y i )|i=1,2,…,n}, and calculate the average coordinates of all points in the point set as the initial circle center coordinates P center ( ), the formula is as follows:

[0064] ;

[0065] ;

[0066] Then calculate the distance from each point in the point set to the initial circle center point P center ( ) is taken as the radius R of the circle. center and radius R to obtain the standard circular contour after regularization.

[0067] After regularizing the outline of each segmented roof attachment, abnormal data are eliminated. The targets corresponding to these abnormal data are most likely not roof attachments such as solar panels and water towers.

[0068] In one embodiment of the present invention, step S4 is to segment the instance to obtain the initial vector outline of each roof attachment, perform regularization processing according to the shape characteristics of different attachments, and obtain the vector data of each roof attachment after regularization processing, and then further includes:

[0069] According to the vector data of each roof attachment after regularization, the area and aspect ratio of each roof attachment are calculated respectively;

[0070] Non-roof attachments are eliminated based on the area and aspect ratio of each roof attachment.

[0071] Specifically, the roof attachment outline vector obtained in step S5 is subjected to calculations of outline area and circumscribed rectangle aspect ratio based on different attachment shape features, and data with abnormal area or aspect ratio is filtered out to obtain the final roof attachment vector data.

[0072] For rectangular roof attachments, the area and aspect ratio of the range enclosed by its rectangular outline are calculated;

[0073] For circular roof attachments, calculate the area and aspect ratio of the circumscribed rectangle of its outline;

[0074] When the area of ​​the roof attachment is within the preset area range and the aspect ratio of the roof attachment is within the preset ratio range, the roof attachment is retained; otherwise, the roof attachment is removed.

[0075] Specifically, in terms of area size, the area of ​​a single building attachment object is between 1m 2 to 10m 2 For less than 1m 2 or more than 10m 2 The appendage extraction results that fall outside this range are directly filtered out. Regarding aspect ratios, the circumscribed rectangles of solar water heaters, circular water towers, and large air conditioner outdoor units generally have ratios between 1:1 and 1:2, while those of small air conditioner outdoor units generally fall between 1:1 and 1:4. Appendage extraction results that fall outside this range are also filtered out.

[0076] Step S5: Load the geometric information of the tilt model data of the building to be modeled, perform elevation value sampling to generate digital surface model data that expresses the height fluctuation characteristics of the roof attachment.

[0077] It is understandable that the digital surface model (DSM) raster data of the building can not only express the fluctuations in the height of all surfaces including the main roof structure and the roof appendage structure, but also facilitate the subsequent overlay analysis with the roof appendage vector contour to calculate the top height of the appendage model.

[0078] Step S6, loading and reading the geometric information of the building monomer model of the building to be modeled in step S1, and generating building roof surface model data that expresses the undulating characteristics of the roof surface structure.

[0079] As can be understood, building roof surface model data (DBSM) is generated based on the geometric information of the individual building models, expressing the undulating characteristics of the roof surface structure. Modeling the roof attachments ultimately requires overlaying and placing the attachment models on the roof of the individual building models to complement the individual building models, which only represent the main roof structure. By directly reading the geometric information of the individual building models and generating the main roof surface model raster data, preparation is made for the calculation of the bottom height position of the attachment models.

[0080] In step S7, the vector data of each roof attachment obtained in step S4 is overlaid with the digital surface model data obtained in step S5 and the building roof surface model data obtained in step S6 to calculate the bottom elevation and top elevation of each roof attachment, and the outer bounding box of each roof attachment is generated by vertical stretching.

[0081] It can be understood that by overlaying and analyzing the outline vector data of each rooftop attachment, the building's digital surface model data, and the building's roof surface model data, the base and top elevations of each rooftop attachment can be calculated. Based on these elevations, vertical stretching is performed to generate the outer bounding box for each rooftop attachment. The outer bounding boxes generated for solar water heaters, small air conditioner outdoor units, and large air conditioner outdoor units are prismatic structures, while the outer bounding box generated for circular water towers is cylindrical.

[0082] In step S8, a 3D spatial radial transformation is performed on the 3D fine model of each type of roof attachment. The 3D template model of each type of roof attachment is rotated, scaled, and translated to the range of the outer bounding box of each roof attachment in step 7, thereby completing the scenario-based modeling of each roof attachment of the building to be modeled.

[0083] It's understandable that a detailed 3D model is prepared for each type of rooftop accessory, such as a solar water heater, water tower, and air conditioning unit. Because these accessories share a high degree of geometric structure, a unified detailed 3D rooftop accessory model is used as a template for each type of rooftop accessory. This provides a foundation for placing accessory model instances on the roof of the building model in the 3D scene.

[0084] The 3D template models of various attachments are subjected to affine transformation in 3D space and placed within the outer bounding box of each roof attachment of the building obtained in step S7 in the order of rotation, scaling, and translation. This completes the modeling and restoration of the 3D template models of the roof attachments on the main building roof and the scene instantiation rendering representation. The specific steps are as follows:

[0085] (1) Calculate the initial position of the three-dimensional template model of the roof attachment.

[0086] The initial position of the 3D template model of the roof attachment is the position of the 3D fine model of the roof attachment in the original local coordinate system. The center point of the outer bounding box of the 3D template model of the roof attachment is taken as the initial position center point P initial (x,y,z), which is the original position of the 3D template model of the roof attachment.

[0087] (2) Calculate the target position of the 3D template model of the roof attachment:

[0088] The target position of the 3D template model of the roof attachment is the final spatial position where the 3D template model of the roof attachment will be placed in the 3D scene, that is, the center point P of the outer bounding box of each roof attachment obtained in step S7. target (x,y,z), the target position of the 3D template model of the roof attachment.

[0089] (3) Rotation of the three-dimensional template model of the roof attachments.

[0090] According to the original position P of the 3D template model of the roof attachment initial (x, y, z) and the target position P of the 3D template model of the roof attachment target (x, y, z), respectively calculate the main direction of the outer bounding box of the initial position of the 3D template model of the roof attachment and the outer bounding box of the target position, determine the rotation matrix R of the 3D template model, and according to the rotation rectangle R and the initial center point P of the 3D template model initial (x, y, z) is rotated, and the main direction of the transformed roof attachment 3D model is rotated to be consistent with the target outer box.

[0091] (4) Scaling of 3D template model.

[0092] The sizes of the outer bounding boxes of the 3D template model and the roof attachment are calculated to determine the scaling matrix S, and the 3D template model is scaled and transformed according to the 3D template model scaling matrix S. The size of the transformed roof attachment 3D model is consistent with the target bounding box.

[0093] (5) Translation of the three-dimensional template model.

[0094] Calculate the original position P initial (x,y,z) and target position P target The translation matrix T between (x, y, z) is calculated, and the position of the three-dimensional template model is translated to complete the transformation and placement of the three-dimensional template model of the roof attachment from the original position to the target space position, thus completing the scenario modeling of the roof attachment of the building.

[0095] See also Figure 2 The present invention also provides a scenario-based modeling system for building roof attachments based on instance segmentation, the system comprising:

[0096] A monomer reconstruction module 201 is used to reconstruct a building monomer based on the tilted model data of the building to be modeled, and obtain a building monomer model of the building to be modeled;

[0097] The first sampling module 202 is used to load the geometric information and texture information of the tilt model data of the building to be modeled, and perform vertical grid sampling to generate true orthogonal image data;

[0098] An instance segmentation module 203 is configured to segment an initial vector outline of each roof attachment from the true orthogonal image data based on a preset roof attachment instance segmentation model;

[0099] A regularization processing module 204 is used to segment the instance to obtain the initial vector outline of each roof attachment, perform regularization processing according to the shape characteristics of different attachments, and obtain the vector data of each roof attachment after regularization processing;

[0100] The second sampling module 205 is used to load the geometric information of the tilt model data of the building to be modeled, perform elevation value sampling to generate digital surface model data expressing the height fluctuation characteristics of the roof attachment;

[0101] A generation module 206 is used to load and read the geometric information of the building monomer model of the building to be modeled, and generate building roof surface model data that expresses the undulating characteristics of the roof surface structure;

[0102] an overlay analysis module 207 for performing overlay analysis on the vector data of each roof attachment, the digital surface model data, and the building roof surface model data, calculating the bottom elevation and top elevation of each roof attachment, and vertically stretching the data to generate an outer bounding box for each roof attachment;

[0103] The affine transformation module 208 is used to perform a three-dimensional affine transformation on the three-dimensional template model of each type of roof attachment, rotate, scale, and translate the three-dimensional template model of each type of roof attachment to the range of the outer bounding box of each roof attachment, and complete the scene-based modeling of each roof attachment of the building to be modeled.

[0104] It can be understood that the building roof attachment scenario modeling system based on instance segmentation provided by the present invention corresponds to the building roof attachment scenario modeling method based on instance segmentation provided in the aforementioned embodiments. The relevant technical features of the building roof attachment scenario modeling system based on instance segmentation can refer to the relevant technical features of the building roof attachment scenario modeling method based on instance segmentation, which will not be repeated here.

[0105] The embodiment of the present invention provides a scenario-based modeling method and system for building roof attachments based on instance segmentation, which has the following beneficial effects:

[0106] (1) The widely used tilt model data is used to convert and derive the true orthogonal image TDOM and the digital surface model DSM. The instance segmentation and extraction of the roof attachment contours in TDOM are performed based on the deep learning model, realizing the automatic acquisition of the roof attachment contour and type information.

[0107] (2) Based on the results of roof attachment contour extraction, the DSM data expressing the original real undulations of the building roof and the DBSM data expressing the undulations of the main structure of the building roof are superimposed to calculate the bottom elevation, top elevation and final bounding box of the attachment, thus realizing the automatic calculation and restoration of the three-dimensional spatial position of each attachment model.

[0108] (3) The 3D model instance rendering technology is used. For the same type of accessory models, only the 3D spatial position is rotated, scaled and translated. Each type of accessory model is rendered using instanced rendering, which greatly reduces the pressure of rendering massive roof accessory models. It effectively alleviates the contradiction between the fine expression of building models and the existing pressure of large-scale scene rendering, and can meet the simulation-level rendering requirements of buildings and their accessory models in super-large city scenes.

[0109] (4) The entire process of object recognition, extraction, three-dimensional spatial position calculation and scene rendering of building roof attachments has been automated without the need for human intervention, which greatly improves the efficiency of building roof attachment modeling and plays a positive role in promoting the refined and low-cost construction of large-scale urban three-dimensional scenes.

[0110] It should be noted that, in the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0111] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0112] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0113] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1The function specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0115] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0116] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A scenario-based modeling method for building roof attachments based on instance segmentation, characterized in that: include: Step S1, reconstructing a building unit according to the tilted model data of the building to be modeled, and obtaining a building unit model of the building to be modeled; Step S2, loading the geometric information and texture information of the tilt model data of the building to be modeled, performing vertical grid sampling to generate true orthogonal image data; Step S3, segmenting the initial vector outline of each roof attachment from the true orthogonal image data based on a preset roof attachment instance segmentation model; Step S4, regularizing the initial vector outline of each roof attachment segmented from the instance according to the shape features of different attachments to obtain vector data of each roof attachment after regularization; Step S5, loading the geometric information of the tilt model data of the building to be modeled, performing elevation value sampling to generate digital surface model data expressing the height fluctuation characteristics of the roof attachment; Step S6, loading and reading the geometric information of the building monomer model of the building to be modeled in step S1, and generating building roof surface model data that expresses the undulating characteristics of the roof surface structure; Step S7, performing an overlay analysis on the vector data of each roof attachment obtained in step S4, the digital surface model data obtained in step S5, and the building roof surface model data obtained in step S6, calculating the bottom elevation and top elevation of each roof attachment, and vertically stretching the data to generate an outer bounding box for each roof attachment; In step S8, a three-dimensional space affine transformation is performed on the three-dimensional template model of each type of roof attachment, and the three-dimensional template model of each type of roof attachment is rotated, scaled, and translated to the range position of the outer bounding box of each roof attachment in step 7, thereby completing the scenario-based modeling of each roof attachment of the building to be modeled.

2. The scenario-based modeling method for building roof attachments according to claim 1, characterized in that: The step S1, reconstructing a building unit according to the tilted model data of the building to be modeled to obtain a building unit model of the building to be modeled, includes: Load the geometric information and texture information of the tilt model data of the building to be modeled to generate three-dimensional color point cloud data; Performing semantic segmentation on the three-dimensional color point cloud data based on a deep learning model to separate all building point cloud data; Divide the building point cloud data into spatial grids, and use the RANSAC algorithm to perform three-dimensional plane fitting of the point cloud in each grid to obtain a three-dimensional plane of each grid; Calculating the angle between the normal vectors of two adjacent three-dimensional planes, merging the adjacent three-dimensional planes with the smaller angle, and obtaining the characteristic plane of the building body, wherein the characteristic plane of the building body includes the top surface and the facade of the building body; The characteristic planes of all building entities are extended and intersected to obtain closed roof plane structures and facade wall structures, and the reconstructed building monomer model is output.

3. The scenario-based modeling method for building roof attachments according to claim 1, characterized in that: The step S4, segmenting the instance to obtain the initial vector outline of each roof attachment, performing regularization processing according to the shape features of different attachments, and obtaining the vector data of each roof attachment after regularization processing, includes: Regularizing the initial vector outline of the rectangular roof attachment to obtain regularized vector data of the rectangular roof attachment; The initial vector outline of the circular class is regularized into an arc to obtain the outline vector data of the circular roof attachment after regularization.

4. The scenario-based modeling method for building roof attachments according to claim 3 is characterized in that: The rectangular-shaped roof attachment initial vector outline is regularized to obtain the regularized outline vector data of the rectangular-shaped roof attachment, including: Traverse one by one to obtain the point set P={(x i ,y i )|i=1,2,…,n}, calculate the average coordinates of all points in the point set as the coordinates of the initial rectangle center point P center ( ); Calculate the covariance matrix C, eigenvalues ​​λ1, λ2 and eigenvectors v1, v2 of all points. The eigenvector corresponding to the larger eigenvalue is in the direction of the long side of the rectangle, and the eigenvector corresponding to the smaller eigenvalue is in the direction of the short side of the rectangle. Calculate along the long side and short side directions to find the distance from the center point of the rectangle P center The farthest two points, calculate the length of the long side and the length of the short side respectively based on the farthest two points; The outline vector data of each rectangular roof attachment after regularization is determined according to the side length in the long side direction, the side length in the short side direction and the center point of the rectangle.

5. The scenario-based modeling method for building roof attachments according to claim 4 is characterized in that: The covariance matrix C, eigenvalues ​​λ1, λ2 and eigenvectors v1, v2 of all points are calculated, where the eigenvector corresponding to the larger eigenvalue is in the direction of the long side of the rectangle and the eigenvector corresponding to the smaller eigenvalue is in the direction of the short side of the rectangle, including: Perform translation transformation on all point coordinates on the initial vector outline of the rectangular roof attachment so that the mean value of all point coordinate data is zero. The calculation formula is as follows: ； in, is the coordinate of the point after coordinate transformation, is the mean of the horizontal coordinates of all points, is the mean of the ordinates of all points; Calculate the covariance matrix C of all points, which describes the degree of change of the points in all directions. The calculation formula is as follows: ； Assume that the eigenvalues ​​of the covariance matrix C are λ1 and λ2, and the corresponding eigenvectors are v1 and v2 respectively. The eigenvalues ​​and eigenvectors satisfy the equation: ； Solving the equation, we obtain two eigenvalues ​​λ1 and λ2. Assuming that λ1≥λ2≥0, we substitute the equation and find the eigenvectors v1 and v2 corresponding to the two eigenvalues ​​λ1 and λ2, where the main direction is the direction indicated by the eigenvector v1 corresponding to the largest eigenvalue λ1, and the secondary main direction is the direction indicated by the eigenvector v2 corresponding to the second largest eigenvalue λ2. The main direction and the secondary main direction are orthogonal, the main direction corresponds to the long side direction of the rectangle, and the secondary main direction corresponds to the short side direction of the rectangle.

6. The scenario-based modeling method for building roof attachments according to claim 3 is characterized in that: The arc regularization of the initial circular vector outline to obtain the regularized outline vector data of the circular roof attachment includes: Traverse one by one to obtain the point set P={(x i ,y i )|i=1,2,…,n}, and calculate the average coordinates of all points in the point set as the initial circle center coordinates P center ( ); Calculate the distance from each point in the point set to the initial circle center point P center ( ) is taken as the radius R of the circle; According to the center point P center The contour vector data of each circular roof-like appendage after regularization is obtained by using the radius R.

7. The scenario-based modeling method for building roof attachments according to claim 1, characterized in that: The step S4 is to segment the instance to obtain the initial vector outline of each roof attachment, and perform regularization processing according to the shape characteristics of different attachments to obtain the vector data of each roof attachment after regularization processing, and then further includes: According to the vector data of each roof attachment after regularization, the area and aspect ratio of each roof attachment are calculated respectively; According to the area and aspect ratio of each roof attachment, non-roof attachments are eliminated; The step of calculating the area and aspect ratio of each roof attachment based on the regularized vector data of each roof attachment comprises: For rectangular roof attachments, calculate the area and aspect ratio of the range enclosed by its rectangular outline; For circular roof attachments, calculate the area and aspect ratio of the circumscribed rectangle of its outline; When the area of ​​the roof attachment is within a preset area range and the aspect ratio of the roof attachment is within a preset ratio range, the vector data of the roof attachment is retained; otherwise, the vector data of the roof attachment is discarded.

8. The scenario-based modeling method for building roof attachments according to claim 1, characterized in that: The step S8, performing a three-dimensional space affine transformation on the three-dimensional template model of each type of roof attachment, rotating, scaling, and translating the three-dimensional template model of each type of roof attachment to the range of the outer bounding box of each roof attachment in step 7, includes: Get the center point P of the outer bounding box of the 3D template model of the roof attachment initial (x, y, z) as the original position, and obtain the center point P of the outer bounding box of the roof attachment of the building to be modeled target (x,y,z) as the target position; According to P initial (x,y,z) and P target (x, y, z), calculate the rotation matrix R of the 3D template model from the original position to the target position; Calculate the scaling matrix S of the 3D template model according to the size of the outer bounding box of the 3D template model and the size of the outer bounding box of the roof appendage; Calculate P initial (x,y,z) and P target The translation matrix T between (x, y, z); The three-dimensional template model is rotated, scaled, and translated according to the rotation matrix R, the scaling matrix S, and the translation matrix T, so as to complete the transformation and placement of the three-dimensional template model from the original position to the target position.

9. A building roof attachment scenario modeling system based on instance segmentation, characterized in that: include: A monomer reconstruction module is used to reconstruct a building monomer based on the tilt model data of the building to be modeled, and obtain a building monomer model of the building to be modeled; The first sampling module is used to load the geometric information and texture information of the tilt model data of the building to be modeled, and perform vertical grid sampling to generate true orthographic image data; An instance segmentation module, configured to segment an initial vector outline of each roof attachment from the true orthogonal image data based on a preset roof attachment instance segmentation model; A regularization processing module is used to perform regularization processing on the initial vector outline of each roof attachment segmented from the instance according to the shape characteristics of different attachments, thereby obtaining vector data of each roof attachment after regularization processing; The second sampling module is used to load the geometric information of the tilt model data of the building to be modeled, perform elevation value sampling to generate digital surface model data expressing the height fluctuation characteristics of the roof attachment; A generation module is used to load and read the geometric information of the building monomer model of the building to be modeled, and generate building roof surface model data that expresses the undulating characteristics of the roof surface structure; an overlay analysis module for performing overlay analysis on the vector data of each roof attachment, the digital surface model data, and the building roof surface model data, calculating the bottom elevation and top elevation of each roof attachment, and vertically stretching the data to generate an outer bounding box for each roof attachment; The affine transformation module is used to perform a three-dimensional affine transformation on the three-dimensional template model of each type of roof attachment, rotate, scale, and translate the three-dimensional template model of each type of roof attachment to the range position of the outer bounding box of each roof attachment, and complete the scene-based modeling of each roof attachment of the building to be modeled.

Images