Building roof appendage scene modeling method and system based on instance segmentation

Abstract

The application provides a kind of based on instance segmentation's building roof accessory scene modeling method and system, based on deep learning model division each roof accessory vector contour;Each roof accessory vector data is overlaid with digital surface model data and building roof surface model data and is analyzed, the bottom elevation and top elevation of each roof accessory are calculated, and the outer bounding box of each roof accessory is generated by vertical stretching;The three-dimensional space affine transformation is carried out to the three-dimensional template model of each kind of roof accessory, and the range position of the outer bounding box of each roof accessory is reached by rotating, scaling and translating, and the scene modeling of each roof accessory is completed.The method of the application can improve the modeling efficiency of roof accessory model, reduce the cost of artificial modeling, shorten the construction cycle of high simulation three-dimensional scene, and create favorable conditions for efficient rendering of super-large range scene mass buildings by using the instantiation rendering method of accessory model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of real three-dimensional modeling, more particularly, to a building roof accessory scene modeling method and system based on instance segmentation. BACKGROUND

[0002] With the rapid development of real three-dimensional technology, digital twin city, and city information model (CIM) fields, the industry has put forward increasingly strict requirements for the construction accuracy and visual fidelity of city building three-dimensional models. City building three-dimensional models are mainly composed of three parts: building walls, roofs, and roof accessories. Roof accessories include common types such as solar water heaters, water towers, and air conditioner outdoor units. As important elements of city building models, the accurate and realistic modeling of these accessories plays an important role in ensuring the authenticity and simulation of building three-dimensional models and even the entire city three-dimensional scene.

[0003] Current single building modeling techniques for city building three-dimensional models mainly rely on OSGB model data generated by oblique photogrammetry and three-dimensional point cloud data as reference benchmarks for construction. For regular-shaped standard buildings, automatic modeling capabilities for building main structures have gradually been developed, and the main structural features of building walls and roofs can be basically restored. However, for various accessories on the roof, such as solar water heaters, water towers, and air conditioner outdoor units, existing geometric reconstruction algorithms are difficult to directly and effectively construct them. On the one hand, roof accessories are relatively small in size compared to the main roof structure. In the process of building three-dimensional geometric reconstruction, to ensure the lightweight expression of geometric features and the stability and robustness of algorithm operation, these small roof accessories are often cleaned and removed. On the other hand, roof accessories such as solar water heaters, water towers, and air conditioner outdoor units have higher requirements for the geometric form and texture detail expression of the model compared to the main building structure. Traditional three-dimensional reconstruction algorithms have obvious limitations in restoring the three-dimensional form features of these accessories, and usually require manual placement and modeling operations on the roof of the building three-dimensional model after completing the modeling of the main building structure.

[0004] However, with the requirements of urban three-dimensional scene construction in low-cost investment, high-time efficiency output and wide coverage, how to quickly and low-cost realize the efficient reconstruction of large-scale scene has become the core development trend in the field of three-dimensional scene construction. Under this background, the traditional manual modeling method has gradually exposed its disadvantages in coping with the modeling task of urban building roof accessories, which cannot meet the needs of rapid construction of urban three-dimensional scene. Therefore, the development and implementation of automatic modeling technology of various accessories on building roof have important practical significance and application value for significantly improving the construction efficiency and updating speed of the whole urban three-dimensional scene. SUMMARY

[0005] In order to solve the problems of low efficiency of urban building roof accessory modeling and difficulty in realizing automatic construction of building model, the application provides a building roof accessory scene modeling method and system based on instance segmentation.

[0006] According to the first aspect of the application, a building roof accessory scene modeling method based on instance segmentation is provided, comprising:

[0007] Step S1, reconstructing a building monomer according to the inclined model data of the building to be modeled, and obtaining a building monomer model of the building to be modeled;

[0008] Step S2, loading the geometric information and texture information of the inclined model data of the building to be modeled, and generating true orthographic image data by vertical direction grid sampling;

[0009] Step S3, instance segmenting an initial vector contour of each roof accessory from the true orthographic image data based on a preset roof accessory instance segmentation model;

[0010] Step S4, regularizing the initial vector contour of each roof accessory instance segmented according to different accessory shape characteristics to obtain the vector data of each roof accessory after regularizing processing;

[0011] Step S5, loading the geometric information of the inclined model data of the building to be modeled, and generating digital surface model data expressing the height fluctuation characteristics of the roof accessory by elevation value sampling;

[0012] Step S6, loading and reading the geometric information of the building monomer model of the building to be modeled in step S1, and generating building roof surface model data expressing the roof surface structure fluctuation characteristics;

[0013] Step S7, superimposing the vector data of each roof accessory obtained in step S4 with the digital surface model data obtained in step S5 and the building roof surface model data obtained in step S6, calculating the bottom elevation and the top elevation of each roof accessory, and vertically stretching to generate the outer bounding box of each roof accessory;

[0014] Step S8, performing three-dimensional space affine transformation on the three-dimensional template model of each type of roof accessory, rotating, scaling and translating the three-dimensional template model of each type of roof accessory to the range position of the outer bounding box of each roof accessory in step 7, completing the scene modeling of each roof accessory of the building to be modeled.

[0015] According to a second aspect of the present application, a building roof accessory scene modeling system based on instance segmentation is provided, comprising:

[0016] The monomer reconstruction module is configured to reconstruct a building monomer according to the inclined model data of the building to be modeled, and obtain a building monomer model of the building to be modeled.

[0017] The first sampling module is configured to load the geometric information and the texture information of the inclined model data of the building to be modeled, perform vertical grid sampling, and generate true orthographic image data.

[0018] The instance segmentation module is configured to instance segment an initial vector contour of each roof accessory from the true orthographic image data based on a preset roof accessory instance segmentation model.

[0019] The regularization processing module is configured to regularize the initial vector contour of each roof accessory instance segmented, according to different accessory shape characteristics, to obtain vector data of each roof accessory after regularization processing.

[0020] The second sampling module is configured to load the geometric information of the inclined model data of the building to be modeled, perform elevation value sampling, and generate digital surface model data expressing the height fluctuation characteristics of the roof accessory.

[0021] The generation module is configured to load and read the geometric information of the building monomer model of the building to be modeled, and generate building roof surface model data expressing the roof surface structure fluctuation characteristics.

[0022] The superimposition analysis module is configured to superimpose the vector data of each roof accessory with the digital surface model data and the building roof surface model data, calculate the bottom elevation and the top elevation of each roof accessory, and vertically stretch to generate the outer bounding box of each roof accessory.

[0023] The affine transformation module is configured to perform three-dimensional space affine transformation on the three-dimensional fine model of each type of roof accessory, and rotate, scale and translate the three-dimensional template model of each type of roof accessory to the range position of the bounding box of each roof accessory, so as to complete the scene modeling of each roof accessory of the building to be modeled.

[0024] The application provides a building roof accessory scene modeling method and system based on instance segmentation, trains a roof accessory instance segmentation model, realizes automatic segmentation and extraction of the contour and type of a building roof accessory object, performs rotation, scaling and translation affine transformation operation on an artificial model of a common accessory in three-dimensional space according to the position of the bounding box of the roof accessory, so as to realize automatic placement and scene instance rendering of the roof accessory model. The technical scheme of the application not only effectively solves the problem of automatic identification and modeling of a building roof accessory model, but also uses an instance loading and rendering scheme for the same type of accessory model, significantly reduces the memory occupation during rendering, has important application value in optimizing the loading and rendering efficiency of a super large scene, and provides strong support for improving the efficiency of city three-dimensional scene construction. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 A building roof accessory scene modeling method based on instance segmentation is provided for an embodiment of the application.

[0026] Figure 2 A structural block diagram of a building roof accessory scene modeling system based on instance segmentation is provided for an embodiment of the application. DETAILED DESCRIPTION

[0027] To make the objectives, technical schemes and advantages of the embodiments of the application clearer, the technical schemes in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are some embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work are within the protection scope of the application. In addition, the technical features in each embodiment or single embodiment provided by the application can be combined with each other to form a feasible technical scheme, and the combination is not restricted by the sequence of steps and / or structure composition mode, but should be based on the implementation by those skilled in the art. When the combination of technical schemes appears contradictory or unfeasible, it should be considered that the combination of technical schemes does not exist, and is not within the protection scope of the application.

[0028] In view of the contradiction that the current urban three-dimensional scene has high requirements for the construction precision, simulation degree and timeliness of the building model, but the fine structure such as roof appendage is difficult to be efficiently constructed by automatic algorithm, the application provides a building roof appendage scene modeling method based on instance segmentation. The method first loads the original oblique model data and converts to generate true orthographic image (TDOM) and digital surface model (DSM), etc., then makes a roof appendage annotation data set based on TDOM data and completes roof appendage instance segmentation model training based on YOLOv8 Instance Segmentation model, then identifies, extracts, regularizes and calculates the three-dimensional space position of the roof appendage outline such as solar water heater, water tower and air conditioner outdoor unit in the TDOM image through the trained instance segmentation model, finally rotates, translates and scales the affine transformation operation of each type of building appendage standard three-dimensional model according to the calculated three-dimensional space position, and finally completes the modeling restoration and scene instantiation rendering of the roof appendage model on the roof of the building main body.

[0029] Figure 1 A building roof appendage scene modeling method based on instance segmentation provided by an embodiment of the application is shown in a flow chart as shown in Figure 1 The method comprises the following steps:

[0030] Step S1, according to the oblique model data of the building to be modeled, the building monomer is reconstructed, and the building monomer model of the building to be modeled is obtained.

[0031] It can be understood that for the building to be scene modeled, the oblique model data (OSGB) thereof is loaded for building monomer reconstruction, and the reconstructed building monomer model can only express the main feature structure of the roof and wall surface in terms of geometric precision, and cannot represent the appendage structure such as solar water heater, water tower and air conditioner outdoor unit on the roof.

[0032] The building main body structure monomerization reconstruction based on the oblique model data (OSGB) mainly comprises the following steps:

[0033] (1) load the geometric information and texture information of the oblique model data to generate three-dimensional color point cloud data;

[0034] (2) based on the RandLA-NET deep learning model, the three-dimensional color point cloud data is subjected to semantic segmentation, and all building point cloud data is separated out.

[0035] (3) the building point cloud data is subjected to spatial grid division, and the RANSAC algorithm is used for three-dimensional plane fitting of the point cloud in each grid to obtain the three-dimensional plane of each grid.

[0036] (4) Calculate the included angle between the normal vectors of two adjacent three-dimensional planes, and merge the adjacent planes with smaller included angles to obtain the feature planes of the building main body, including the top surface and the facade of the building main body;

[0037] (5) Extend and intersect all the building main body feature planes obtained in the previous step to obtain closed roof and wall structures, and output the reconstructed building model data.

[0038] Step S2, load the geometric information and texture information of the oblique model data of the building to be modeled, and perform vertical grid sampling to generate true orthographic image data.

[0039] It can be understood that the geometric information and texture information of the oblique model data of the building to be modeled are loaded, and OpenSceneGraph three-dimensional graphics library is used to traverse each OSGB model file, and vertical grid sampling is performed to generate 0.05m resolution true orthographic image data (TDOM). The 0.05m high-resolution true orthographic image data can clearly display the feature contours of the building roof accessories, providing a data basis for the next building accessory contour instance segmentation.

[0040] Step S3, based on the preset roof accessory instance segmentation model, instance segmenting an initial vector contour of each roof accessory from the true orthographic image data.

[0041] It can be understood that a roof accessory instance segmentation model is pre-trained, and the trained preset roof accessory instance segmentation model is used to instance segment an initial vector contour of each roof accessory from the TDOM image data of the building to be modeled.

[0042] The training process of the roof accessory instance segmentation model is as follows:

[0043] Different regional TDOM data are used for labeling and data set making of the vector contours of roof accessories such as solar water heaters, water towers, and air conditioner outdoor units. The YOLOv8 Instance Segmentation algorithm is used to train the roof accessory instance segmentation model on the labeled data set. The YOLOv8 Instance Segmentation model is the instance segmentation version of YOLOv8, which uses a mask-based (Mask-Based) segmentation method and combines target detection capabilities, enabling fast and accurate prediction of the contours of each roof accessory from TDOM image data.

[0044] The LabelMe data labeling software is used to make a roof accessory contour instance segmentation dataset. The boundary contour information and accessory type information of each roof accessory are stored in the labeled data. The training set, validation set and test set are configured in a ratio of 6:1:1 based on the cost of the labeled dataset. The roof accessory instance segmentation model is trained based on the training set and validation set data, and multiple iterations are performed until the loss value significantly decreases and converges. Then, the intersection of union (IOU) value of the accessory contour extraction mask and the labeled true value mask is calculated based on the test set. The result with an IOU value greater than or equal to 0.8 is determined as the correct result. Through test analysis, it is confirmed that the proportion of missed extraction and incorrect extraction with an IOU value less than 0.8 is less than 15%. The trained model can meet the basic requirements of roof accessory contour instance segmentation, and the final roof accessory contour instance segmentation model is prepared.

[0045] Based on the trained roof accessory instance segmentation model, the TDOM image data generated in step S2 is subjected to roof accessory contour instance segmentation extraction. The contour of each accessory of the building roof is extracted by instance segmentation, and the segmentation results with a confidence score greater than or equal to 0.7 are filtered according to the segmentation result confidence score, so as to finally obtain the initial vector contour of each roof accessory of the building. The confidence score is a quantitative expression of the accuracy of the predicted roof accessory instance segmentation result of the roof accessory instance segmentation model, and the value range is between 0 and 1. In order to ensure the accuracy of the building instance segmentation result, the instance segmentation result with a score less than 0.7 is removed, so as to reduce the probability of extracting other roof debris as accessory objects by mistake.

[0046] In step S4, the initial vector contour of each roof accessory segmented by instance segmentation is subjected to regularization processing according to different accessory shape characteristics, so as to obtain the vector data of each roof accessory after regularization processing.

[0047] It can be understood that the initial vector contour of each roof accessory segmented from the building is subjected to regularization processing according to different accessory shape characteristics. The rectangular shape of the solar water heater, small air conditioner outdoor unit and large air conditioner outdoor unit is subjected to right angle processing, and the circular shape of the water tower is subjected to circular arc fitting, so as to obtain the vector data of the accessory after regularization processing.

[0048] The specific regularization processing steps include:

[0049] (1) The rectangular accessory contour is subjected to right angle regularization, such as the contour of the solar water heater and the small air conditioner outdoor unit, which includes the following steps:

[0050] a. Traverse the point set P = {(x i ,y i )|i=1,2,…,n} on the initial vector profile of the rectangular roof appendage one by one, calculate the average coordinates of all points in the point set as the initial rectangular center point coordinates P center ( ).

[0051] b. Translate all point coordinates so that the data mean is zero, the calculation formula is as follows:

[0052] ;

[0053] Wherein, is the point coordinate after coordinate transformation, is the horizontal coordinate mean of all points, is the vertical coordinate mean of all points.

[0054] c. Calculate the covariance matrix C of all points, which describes the variation of points in each direction, and the calculation formula is as follows:

[0055] ;

[0056] d. Solve the eigenvalues and eigenvectors of the covariance matrix:

[0057] Let the eigenvalues of the covariance matrix be λ1 and λ2, and the corresponding eigenvectors be v1 and v2, respectively. The eigenvalues and eigenvectors satisfy the equation:

[0058] ;

[0059] Solving this equation can obtain two eigenvalues λ1 and λ2, assuming λ1≥λ2≥0, then bring into the equation to solve the corresponding eigenvectors v1 and v2. The principal direction is the direction indicated by the eigenvector v1 corresponding to the maximum eigenvalue λ1, and the secondary principal direction is the direction indicated by the eigenvector v2 corresponding to the second largest eigenvalue λ2, and the principal direction and the secondary principal direction are orthogonal. The principal direction corresponds to the long side direction of the rectangle, and the secondary principal direction corresponds to the short side direction of the rectangle.

[0060] e. Calculate the two points farthest from the rectangular center point P center in the point set along the long side direction and the short side direction respectively, and calculate the long side length and the short side length according to the two farthest points respectively;

[0061] f. According to the long side length, the short side length and the rectangular center point, determine the profile vector data of each regularized rectangular roof appendage.

[0062] (2) Regularization of the circular arc of the circular accessory profile, such as the profile of a circular water tower, a small air conditioner outdoor unit, and a large air conditioner outdoor unit:

[0063] Traverse the point set P={(x i ,y i )|i=1,2,…,n} on the initial vector profile of the circular accessory, and calculate the average coordinates of all points in the point set as the initial center point coordinates P center ( ), as follows:

[0064] ;

[0065] ;

[0066] Then, calculate the average distance of each point in the point set to the initial center point P center ( ) as the radius R of the circle. The regularized standard circular profile is obtained from the center point P center and the radius R.

[0067] After regularizing each roof accessory profile, abnormal data corresponding to a target that is not a solar energy, water tower, or other roof accessory is removed.

[0068] In an embodiment of the present application, step S4, the initial vector profile of each roof accessory is segmented, regularized according to different accessory shape characteristics, and the regularized vector data of each roof accessory is obtained. Then, the following steps are further included:

[0069] According to the regularized vector data of each roof accessory, the area and the aspect ratio of each roof accessory are calculated.

[0070] According to the area and the aspect ratio of each roof accessory, non-roof accessories are removed.

[0071] Specifically, the roof accessory profile vector obtained in step S5 is calculated according to the profile area size and the aspect ratio of the circumscribed rectangle according to different accessory shape characteristics, and data with abnormal area or aspect ratio is filtered out to obtain the final vector data of the roof accessory.

[0072] For rectangular roof accessories, the area of the rectangular profile and the aspect ratio are calculated.

[0073] For circular roof accessories, the area and the aspect ratio of the circumscribed rectangle of the profile are calculated.

[0074] When the area of the roof accessory is within a preset area range and the length-width ratio of the roof accessory is within a preset ratio range, the roof accessory is retained, otherwise, the roof accessory is removed.

[0075] Specifically, in terms of area size, the area of a single building accessory object is between 1 m 2 and 10 m 2 . For accessories smaller than 1 m 2 or larger than 10 m 2 , the extraction result is directly filtered out. In terms of length-width ratio, the length-width ratio of the circumscribed rectangle of a solar water heater, a circular water tower, and a large air conditioner outdoor unit is generally between 1:1 and 1:2, and the length-width ratio of the circumscribed rectangle of a small air conditioner outdoor unit is generally between 1:1 and 1:4. The extraction result of the accessory that exceeds this length-width ratio range is also filtered out.

[0076] Step S5: Load the geometric information of the inclined model data of the building to be modeled, and perform elevation value sampling to generate digital surface model data expressing the height fluctuation characteristics of the roof accessory.

[0077] It can be understood that the digital surface model DSM grid data of the building can express the fluctuation of all surface heights including the roof main structure and the roof accessory structure, and also facilitates subsequent overlay analysis with the roof accessory vector profile to calculate the top height of the accessory model.

[0078] Step S6: Load the geometric information of the building monomer model of the building to be modeled in step S1, and generate building roof surface model data expressing the roof surface structure fluctuation characteristics.

[0079] It can be understood that based on the geometric information of the building monomer model, the building roof surface model data DBSM expressing the roof surface structure fluctuation characteristics is generated. Building roof accessory modeling ultimately needs to superimpose and place the accessory model on the roof of the building monomer model to supplement the monomer building model expressing only the roof main structure. By directly reading the geometric information of the building monomer model and generating building main roof surface model grid data, the bottom height position of the accessory model is prepared for the next step.

[0080] Step S7: Superimpose the vector data of each roof accessory obtained in step S4 with the digital surface model data obtained in step S5 and the building roof surface model data obtained in step S6, calculate the bottom elevation and top elevation of each roof accessory, and vertically stretch to generate the bounding box of each roof accessory.

[0081] It can be understood that the contour vector data of each roof accessory, the digital surface model data of the building and the building roof surface model data are superimposed and analyzed, the bottom elevation and the top elevation of each roof accessory are calculated, and the outer bounding box of each roof accessory of the building is generated according to the bottom elevation and the top elevation. The outer bounding box generated by the solar water heater, the small air conditioner outdoor unit and the large air conditioner outdoor unit is a prism structure, and the outer bounding box generated by the circular water tower is a cylindrical structure.

[0082] In step S8, the three-dimensional space radiation transformation is performed on the three-dimensional fine model of each type of roof accessory, the three-dimensional template model of each type of roof accessory is rotated, scaled and translated to the range position of the outer bounding box of each roof accessory in step 7, and the scene modeling of each roof accessory of the building to be modeled is completed.

[0083] It can be understood that a three-dimensional fine model is prepared for each type of roof accessory such as a solar water heater, a water tower and an air conditioner outdoor unit. Since the solar water heater, the water tower and the air conditioner outdoor unit have high similarity in geometric structure, for each type of roof accessory, a unified three-dimensional fine model of the roof accessory is used as a template model, which provides a basis for placing the accessory model instance object on the building model roof in the three-dimensional scene in the next step.

[0084] The three-dimensional template model of each type of accessory is subjected to three-dimensional space affine transformation, and is placed in the outer bounding box range position of each roof accessory of the building obtained in step S7 in the order of rotation, scaling and translation, and finally the modeling restoration and scene instantiation rendering representation of the three-dimensional template model of the roof accessory on the roof of the building main body are completed. The specific steps are as follows:

[0085] (1) Calculate the initial position of the three-dimensional template model of the roof accessory.

[0086] The initial position of the three-dimensional template model of the roof accessory is the position of the three-dimensional fine model of the roof accessory in the original local coordinate system. The center point of the outer bounding box of the three-dimensional template model of the roof accessory is taken as the initial position center point P initial (x, y, z), as the original position of the three-dimensional template model of the roof accessory.

[0087] (2) Calculate the target position of the three-dimensional template model of the roof accessory:

[0088] The target position of the three-dimensional template model of the roof accessory is the final spatial position of the three-dimensional template model of the roof accessory to be placed in the three-dimensional scene, that is, the center point P target (x, y, z) of the outer bounding box of each roof accessory obtained in step S7, as the target position of the three-dimensional template model of the roof accessory.

[0089] (3) The three-dimensional template model of the roof accessory is rotated.

[0090] According to the original position P initial (x,y,z) of the three-dimensional template model of the roof accessory and the target position P target (x,y,z) of the three-dimensional template model of the roof accessory, the main direction orientation of the three-dimensional template model initial position bounding box and the target position bounding box of the roof accessory is calculated respectively, the rotation matrix R of the three-dimensional template model is determined, and the rotation transformation is performed on the three-dimensional template model according to the rotation matrix R and the three-dimensional template model initial center point P initial (x,y,z), and the main direction orientation of the transformed roof accessory three-dimensional model is rotated to be consistent with the target bounding box.

[0091] (4) The three-dimensional template model is scaled.

[0092] The size of the three-dimensional template model bounding box and the size of the roof accessory bounding box are calculated to determine the scaling matrix S, and the three-dimensional template model is scaled according to the three-dimensional template model scaling matrix S, so that the size of the transformed roof accessory three-dimensional model is consistent with the target bounding box.

[0093] (5) The three-dimensional template model is translated.

[0094] The translation matrix T between the original position P initial (x,y,z) and the target position P target (x,y,z) is calculated, and the position of the three-dimensional template model is translated, so that the three-dimensional template model of the roof accessory is transformed and placed from the original position to the target spatial position, that is, the scene modeling of the building roof accessory is completed.

[0095] Referring to Figure 2 , the application further provides a building roof accessory scene modeling system based on instance segmentation, which comprises:

[0096] The monomer reconstruction module 201 is used for reconstructing the building monomer according to the inclined model data of the building to be modeled, and obtaining the building monomer model of the building to be modeled.

[0097] The first sampling module 202 is used for loading the geometric information and texture information of the inclined model data of the building to be modeled, and performing vertical direction grid sampling to generate real orthographic image data.

[0098] The instance segmentation module 203 is used for instance segmenting the initial vector contour of each roof accessory from the real orthographic image data based on a preset roof accessory instance segmentation model.

[0099] The regularization processing module 204 is configured to perform regularization processing on the initial vector contour of each roof accessory segmented from the instance according to different accessory shape features to obtain vector data of each roof accessory after the regularization processing.

[0100] The second sampling module 205 is configured to load geometric information of the tilt model data of the building to be modeled and perform elevation value sampling to generate digital surface model data expressing the height fluctuation features of the roof accessory.

[0101] The generation module 206 is configured to load geometric information of the building monomer model of the building to be modeled and generate building roof surface model data expressing the roof surface structure fluctuation features.

[0102] The superposition analysis module 207 is configured to perform superposition analysis on the vector data of each roof accessory and the digital surface model data and the building roof surface model data, calculate the bottom elevation and the top elevation of each roof accessory, and perform vertical stretching to generate an outer bounding box of each roof accessory.

[0103] The affine transformation module 208 is configured to perform three-dimensional space affine transformation on the three-dimensional template model of each type of roof accessory, rotate, scale and translate the three-dimensional template model of each type of roof accessory to the range position of the outer bounding box of each roof accessory, and complete the scene modeling of each roof accessory of the building to be modeled.

[0104] It can be understood that the building roof accessory scene modeling system based on instance segmentation provided by the present application corresponds to the building roof accessory scene modeling method based on instance segmentation provided by the above-mentioned embodiments, and the related technical features of the building roof accessory scene modeling system based on instance segmentation can refer to the related technical features of the building roof accessory scene modeling method based on instance segmentation, which will not be repeated here.

[0105] The building roof accessory scene modeling method and system based on instance segmentation provided by the embodiments of the present application have the following beneficial effects:

[0106] (1) The conversion derivation of the true orthophoto image TDOM and the digital surface model DSM is performed by using the widely used tilt model data, the instance segmentation and extraction of the roof accessory contour in the TDOM are performed based on the deep learning model, and the automatic acquisition of the roof accessory contour and type information is realized.

[0107] (2) Based on the extraction result of the roof accessory contour, the DSM data expressing the original true fluctuation of the building roof and the DBSM data expressing the main structure fluctuation of the building roof are superimposed to calculate the accessory bottom elevation, the top elevation and the final bounding box, and the automatic calculation and restoration of the three-dimensional space position of each accessory model are realized.

[0108] (3) Using the three-dimensional model instantiation rendering technology, rotation, scaling and translation of the three-dimensional space position are performed on the same type of accessory model, instantiation rendering is used for each type of accessory model, which greatly reduces the pressure of rendering a large number of roof accessory models, effectively alleviates the contradiction between the fine expression of the building model and the heavy pressure of the existing rendering of the large scene, and can meet the simulation level rendering requirement of the building and its accessory model in the super large city scene.

[0109] (4) The whole building roof accessory object identification, extraction, three-dimensional space position calculation and scene rendering process is realized automatically without manual intervention, which greatly improves the building roof accessory model modeling efficiency and has a positive effect on promoting the fine and low-cost construction of a large range of city three-dimensional scenes.

[0110] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0111] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0112] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices produce a device that implements the flow Figure 1 The function specified in one flow or multiple flows and / or blocks Figure 1 The device that implements the function specified in one block or multiple blocks.

[0113] These computer program instructions can also be stored in a computer-readable memory that can guide the computer or other programmable data processing devices to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including instruction devices, which implement the flow Figure 1 The function specified in one flow or multiple flows and / or blocks Figure 1the function specified in one or more blocks.

[0114] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable data processing devices provide a process for implementing the flowchart Figure 1 one or more flows and / or blocks Figure 1 the function specified in one or more blocks.

[0115] Although preferred embodiments of the application have been described herein, changes and modifications can be suggested to one skilled in the art, and it is intended that the application encompass such changes and modifications as fall within the scope of the appended claims.

[0116] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for instance segmentation based architectural roof appendage scene modeling, characterized in that, The method comprises the following steps: Step S1, reconstructing a building unit according to the inclined model data of a building to be modeled, and obtaining a building unit model of the building to be modeled; Step S2, loading geometric information and texture information of the inclined model data of the building to be modeled, and generating true orthographic image data through vertical direction grid sampling; Step S3, instance segmenting an initial vector contour of each roof accessory from the true orthographic image data based on a preset roof accessory instance segmentation model; Step S4, performing regularization processing on the initial vector contour of each roof accessory instance segmented according to different accessory shape features, and obtaining vector data of each roof accessory after regularization processing; Step S5, loading geometric information of the inclined model data of the building to be modeled, and generating digital surface model data expressing the height fluctuation characteristics of the roof accessory through height value sampling; Step S6, loading and reading geometric information of the building unit model of the building to be modeled in step S1, and generating building roof surface model data expressing the roof surface structure fluctuation characteristics; Step S7, performing overlay analysis on the vector data of each roof accessory obtained in step S4, the digital surface model data obtained in step S5, and the building roof surface model data obtained in step S6, calculating the bottom elevation and top elevation of each roof accessory, and vertically stretching to generate an outer bounding box of each roof accessory; Step S8, performing three-dimensional space affine transformation on the three-dimensional template model of each type of roof accessory, rotating, scaling, and translating the three-dimensional template model of each type of roof accessory to the range position of the outer bounding box of each roof accessory in step S7, and completing the scene modeling of each roof accessory of the building to be modeled.

2. The method of claim 1, wherein, The step S1 comprises the following steps: loading geometric information and texture information of the inclined model data of the building to be modeled, and generating three-dimensional color point cloud data; performing semantic segmentation on the three-dimensional color point cloud data based on a deep learning model, and separating all building point cloud data; performing spatial grid division on the building point cloud data, performing three-dimensional plane fitting on the point cloud in each grid by using a RANSAC algorithm, and obtaining a three-dimensional plane of each grid; calculating the included angle between the normal vectors of two adjacent three-dimensional planes, and performing merging processing on the adjacent three-dimensional planes with smaller included angles to obtain a characteristic plane of a building body, wherein the characteristic plane of the building body comprises a top surface and a facade of the building body; performing extension intersection processing on all characteristic planes of the building body to obtain a closed roof plane structure and a facade wall structure, and outputting the reconstructed building unit model.

3. The method of claim 1, wherein, The step S4 comprises the following steps: performing right-angle regularization on the initial vector contour of the rectangular roof accessory to obtain vector data of the rectangular roof accessory after regularization processing; The initial vector profile of the circular roof accessory is circularly regularized to obtain the profile vector data of the regularized circular roof accessory.

4. The method of claim 3, wherein, The initial vector profile of the rectangular roof accessory is right-angled regularized to obtain the profile vector data of the regularized rectangular roof accessory, including: Traverse the point set P={(x i ,y i )|i=1,2,…,n} on the initial vector profile of the rectangular class roof appendage one by one, calculate the average coordinates of all points in the point set as the initial rectangular center point coordinates ; The covariance matrix C of all points, the eigenvalues λ1 and λ2, and the eigenvectors v1 and v2 are calculated, wherein the eigenvector corresponding to the larger eigenvalue is the long side direction of the rectangle, and the eigenvector corresponding to the smaller eigenvalue is the short side direction of the rectangle; The distance rectangular center point P is found out by calculating along the long side direction and the short side direction respectively center The edge length in the long side direction and the edge length in the short side direction are calculated according to the two farthest points respectively; The profile vector data of each regularized rectangular roof accessory is determined according to the long side length, the short side length, and the rectangular center point.

5. The method of claim 4, wherein, The covariance matrix C of all points, the eigenvalues λ1 and λ2, and the eigenvectors v1 and v2 are calculated, wherein the eigenvector corresponding to the larger eigenvalue is the long side direction of the rectangle, and the eigenvector corresponding to the smaller eigenvalue is the short side direction of the rectangle, including: The coordinates of all points on the initial vector profile of the rectangular roof accessory are translated to make the mean of all coordinate data zero, and the calculation formula is as follows: ； wherein is the x-coordinate of the transformed point, is the mean of the x-coordinates of all points, is the mean of the y-coordinates of all points; The covariance matrix C of all points is calculated, which describes the variation degree of the points in each direction, and the calculation formula is as follows: ; The eigenvalues λ1 and λ2 of the covariance matrix C are set, and the corresponding eigenvectors are v1 and v2, and the eigenvalues and eigenvectors satisfy the equation: ; Two eigenvalues λ1 and λ2 are obtained by solving the equation, assuming λ1≥λ2≥0, and the eigenvectors v1 and v2 corresponding to the two eigenvalues are calculated by substituting the equation, wherein the main direction is indicated by the eigenvector v1 corresponding to the maximum eigenvalue λ1, the secondary main direction is indicated by the eigenvector v2 corresponding to the second maximum eigenvalue λ2, the main direction and the secondary main direction are orthogonal, the main direction corresponds to the long side direction of the rectangle, and the secondary main direction corresponds to the short side direction of the rectangle.

6. The method of claim 3, wherein, The initial vector profile of the circular roof accessory is circularly regularized to obtain the profile vector data of the regularized circular roof accessory, including: The set of points P = {(x i ,y i )|i = 1, 2, …, n} on the initial vector profile of the circular roof appendage is traversed one by one, and the average coordinates of all points in the point set are calculated as the initial center point coordinates ; Calculate the distance from each point in the point set to the initial center point. The average distance is taken as the radius R of the circle; According to the center point P center and the radius R, the profile vector data of each regularized circular roof-like appendage is obtained.

7. The method of claim 1, wherein, The step S4, the initial vector profile of each roof accessory is segmented, and the regularized processing is performed according to different accessory shape characteristics to obtain the vector data of each regularized roof accessory, and then the step S4 further includes: The area and the aspect ratio of each roof accessory are calculated according to the vector data of each regularized roof accessory; The non-roof accessory is removed according to the area and the aspect ratio of each roof accessory. The area and the aspect ratio of each roof accessory are calculated according to the vector data of each regularized roof accessory, including: For the rectangular roof accessory, the area of the rectangular profile range and the aspect ratio are calculated; For the circular roof accessory, the area of the circumscribed rectangle of the profile and the aspect ratio are calculated; When the area of the roof accessory is within the preset area range, and the aspect ratio of the roof accessory is within the preset proportion range, the vector data of the roof accessory is retained, otherwise, the vector data of the roof accessory is removed.

8. The method of claim 1, wherein, The step S8, the three-dimensional space affine transformation is carried out to the three-dimensional template model of each type of roof accessory, and the three-dimensional template model of each type of roof accessory is rotated, scaled and translated to the range position of the outer bounding box of each roof accessory in step S7, including: acquiring a position center point P of an outer bounding box of a three-dimensional template model of a roof appurtenance initial (x,y,z) as a target position; and acquiring a position center point P of an outer bounding box of a roof appurtenance of a building to be modeled target (x,y,z) as a target position; and acquiring a position center point P of an outer bounding box of a roof appurtenance of a building to be modeled According to P initial (x,y,z) and P target (x,y,z), a rotation matrix R of the three-dimensional template model from the original position to the target position is calculated; According to the size of the outer bounding box of the three-dimensional template model and the size of the outer bounding box of the roof accessory, the scaling matrix S of the three-dimensional template model is calculated; Compute P initial (x,y,z) and P target translation matrix T between (x,y,z) and P According to the rotation matrix R, the scaling matrix S and the translation matrix T, the three-dimensional template model is rotated, scaled and translated, and the transformation and placement of the three-dimensional template model from the original position to the target position are completed.

9. A system for instance segmentation based architectural roof appendage scene modeling, the system comprising: Including: The monomer reconstruction module is used for reconstructing the building monomer according to the inclined model data of the building to be modeled, and obtaining the building monomer model of the building to be modeled. The first sampling module is used for loading the geometric information and texture information of the inclined model data of the building to be modeled, performing vertical direction grid sampling to generate real orthographic image data. The instance segmentation module is used for instance segmenting each roof accessory initial vector contour from the real orthographic image data based on a preset roof accessory instance segmentation model. The regularization processing module is used for regularizing the initial vector contour of each roof accessory segmented by the instance according to different accessory shape characteristics, to obtain the vector data of each roof accessory after the regularization processing. The second sampling module is used for loading the geometric information of the inclined model data of the building to be modeled, performing elevation value sampling to generate digital surface model data expressing the height fluctuation characteristics of the roof accessory. The generation module is used for loading and reading the geometric information of the building monomer model of the building to be modeled, and generating building roof surface model data expressing the roof surface structure fluctuation characteristics. The overlay analysis module is used for overlaying the vector data of each roof accessory with the digital surface model data and the building roof surface model data, calculating the bottom elevation and the top elevation of each roof accessory, and vertically stretching to generate the outer bounding box of each roof accessory. The affine transformation module is used for performing three-dimensional space affine transformation on the three-dimensional template model of each type of roof accessory, rotating, scaling and translating the three-dimensional template model of each type of roof accessory to the range position of the outer bounding box of each roof accessory, and completing the scene modeling of each roof accessory of the building to be modeled.

Images