House modeling method, apparatus, device, and product

Abstract

The application discloses a kind of house modeling method, device, equipment and product, relate to building structure analysis technical field, method includes: receiving house picture and the real plot parameter of the house picture;According to the house picture and real plot parameter, building identification and segmentation are carried out by building component three-dimensional perception fusion model, and optimization building component and building logic grid are obtained;The optimization building component and building logic grid are structured, and modeling data are obtained, and modeling is carried out by modeling engine according to the modeling data, and house model is obtained.Therefore, it solves the problem that the modeling effect is poor due to the fact that the equipment cannot obtain accurate modeling data in building component identification in the prior art, improves the accuracy of house modeling.

Description

Technical Field

[0001] This application relates to the field of building structure analysis technology, and in particular to a method, apparatus, equipment and product for building modeling. Background Technology

[0002] With the advancement of digital village and smart city construction, large-scale, low-cost 3D digital modeling of existing rural houses is of great significance. Traditional modeling methods mainly rely on manual on-site surveying and manual CAD (computer-aided design) modeling. This process is not only time-consuming, labor-intensive, and costly, but also requires high professional skills from the operators, making it difficult to meet the needs of large-scale rapid modeling.

[0003] In recent years, 3D reconstruction technology based on photogrammetry has made some progress. However, it usually requires taking images or video sequences from multiple angles, which places high demands on data acquisition. Moreover, the generated models are mostly dense point clouds or mesh models with huge amounts of data, unclear topological structures, and difficulty in parametric editing and semantic understanding. Therefore, it is not suitable for building information models (BIM) or subsequent procedural modeling processes that require parametric input.

[0004] Using deep learning to reconstruct 3D from a single image is a current research hotspot. However, existing methods for building component identification and modeling suffer from insufficient identification and positioning accuracy due to external factors such as lighting, occlusion, and shooting angle. Existing methods also struggle to infer logical relationships between components (such as floor affiliation and vertical bay alignment) and cannot accurately obtain discrete depth hierarchy relationships of different depth structures (such as walls, balconies, and porches).

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main purpose of this application is to provide a method, apparatus, device and product for building modeling, which aims to solve the technical problem that the existing technology cannot obtain accurate modeling data due to equipment factors in the identification of building components, resulting in poor modeling effect.

[0007] To achieve the above objectives, this application proposes a house modeling method, which includes:

[0008] Receive images of houses and the actual land parameters of the houses;

[0009] Based on the house images and real plot parameters, the building is identified and segmented using a 3D perception fusion model of building components, resulting in optimized building components and building logic mesh.

[0010] The optimized building components and building logic grid are structured to obtain modeling data, and a building model is obtained by modeling through a modeling engine based on the modeling data.

[0011] In one embodiment, the 3D perception fusion model for building components includes a target detection model, an image segmentation model, and a depth estimation model. The step of identifying and segmenting buildings based on the house image and real land parameters using the 3D perception fusion model to obtain optimized building components and a building logical mesh includes:

[0012] Based on the real land parcel parameters, the target detection model is used to detect the house image to obtain a set of the main body of the house and building components;

[0013] Based on the main body of the house, a binary mask of the main body of the house is generated by the image segmentation model, and the building component set is removed based on the binary mask to obtain an optimized building component set;

[0014] The optimized set of building components is clustered to obtain horizontal grouping sets and vertical grouping sets;

[0015] The depth estimation model is used to perform edge detection on the house image and the binary mask to obtain a depth edge map;

[0016] The building logic mesh is obtained based on the depth edge map, the horizontal group set, and the vertical group set components.

[0017] In one embodiment, the step of generating a binary mask of the building body based on the building body using the image segmentation model, and then removing components from the building component set based on the binary mask to obtain an optimized building component set includes:

[0018] Based on the house image and the main body of the house, a mask is generated using the image segmentation model to obtain a binary mask of the main body of the house;

[0019] Based on the binary mask, the set of building components is traversed and calculated to obtain the intersection and union ratio set;

[0020] The intersection-union ratio set is compared using an intersection-union ratio threshold;

[0021] By removing building components from the set of building components whose intersection ratio is lower than the intersection ratio threshold, an optimized set of building components is obtained.

[0022] In one embodiment, the step of clustering the optimized set of building components to obtain a horizontal grouping set and a vertical grouping set includes:

[0023] The optimized building component set is filtered based on the first classification component to obtain horizontal components. The vertical axis projection overlap is calculated based on the horizontal components to obtain a horizontal group set.

[0024] The optimized building component set is filtered based on the second category components to obtain vertical components. The horizontal axis projection overlap is calculated based on the vertical components to obtain the vertical group set.

[0025] In one embodiment, the step of performing edge detection on the house image and binary mask using the depth estimation model to obtain a depth edge map includes:

[0026] The non-building areas of the house image are masked using the binary mask to obtain the main building image;

[0027] The depth estimation model is used to perform depth processing on the image of the main building to obtain a grayscale image of the main building;

[0028] The grayscale image of the main building is denoised by the Gaussian blur algorithm to obtain the denoised grayscale image of the main building.

[0029] An edge detection algorithm is used to perform edge detection on the denoised grayscale image of the main building to obtain a depth edge map.

[0030] In one embodiment, the step of structuring the optimized building components and the building logic mesh to obtain modeling data includes:

[0031] The house image is cropped based on an optimized set of building components to obtain component image regions, and the RGB values ​​of the component images are extracted.

[0032] The bounding box coordinates of the optimized building component set are adjusted according to the building logic grid to obtain the adjusted optimized building component set;

[0033] The depth level values ​​of the adjusted optimized building component set are assigned based on the component labels and depth edge map of the optimized building component set, and the assignment results are obtained.

[0034] The allocation results, the adjusted optimized set of building components, and the RGB values ​​are summarized to obtain structured data. The structured data is then converted to a new format to obtain modeling data.

[0035] In one embodiment, the step of assigning depth level values ​​to the adjusted optimized building component set based on the component labels and depth edge map of the optimized building component set to obtain the assignment result includes:

[0036] Calculate the average depth value of the component bounding box of the optimized building component set in the grayscale image of the main building;

[0037] Detect the overlap value of the strong edge contours of the component bounding box in the depth edge map;

[0038] The initial weights of several components in the adjusted optimized building component set are determined based on the component labels of the adjusted optimized building component set.

[0039] The depth level values ​​are assigned to the adjusted optimized building component set based on the average depth value, overlap value, and initial weight, and the assignment results are obtained.

[0040] Furthermore, to achieve the above objectives, this application also proposes a house modeling device, which includes:

[0041] The receiving module is used to receive images of houses and the actual land parameters of the houses.

[0042] The building recognition module is used to identify and segment buildings based on the house images and real plot parameters, and to obtain optimized building components and building logic grids through a 3D perception fusion model of building components.

[0043] The modeling module is used to perform structured processing on the optimized building components and building logic grid to obtain modeling data, and to perform modeling through the modeling engine based on the modeling data to obtain a house model.

[0044] In addition, to achieve the above objectives, this application also proposes a house modeling device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the house modeling method as described above.

[0045] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the house modeling method described above.

[0046] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the house modeling method described above.

[0047] One or more technical solutions proposed in this application have at least the following technical effects:

[0048] This application proposes a method, apparatus, device, and product for house modeling. It receives a house image and the actual land parameters of the house image; based on the house image and the actual land parameters, it performs building identification and segmentation using a 3D perception fusion model of building components to obtain optimized building components and a building logic mesh; it performs structured processing on the optimized building components and the building logic mesh to obtain modeling data; and it performs modeling using a modeling engine based on the modeling data to obtain a house model. Thus, by using a 3D perception fusion model of building components to identify and segment the received house image and actual land parameters to obtain optimized building components and a building logic mesh, it achieves the ability to parse the main body and related parameters of the house from images acquired by the device alone. Subsequently, it performs structured processing on the optimized building components and the building logic mesh to obtain modeling data suitable for modeling, and then uses a modeling engine to perform modeling based on the building data to obtain a house model. This solves the problem in existing technologies where accurate modeling data cannot be obtained due to device factors in building component identification, resulting in poor modeling effects, and improves the accuracy of house modeling. Attached Figure Description

[0049] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating an embodiment of the house modeling method of this application.

[0052] Figure 2 This is a schematic diagram illustrating the detection results of the target detection model involved in the house modeling method of this application;

[0053] Figure 3 This is a schematic diagram illustrating the main building segmentation involved in the building modeling method of this application;

[0054] Figure 4 This is a schematic diagram illustrating the horizontal grouping of components involved in the house modeling method of this application;

[0055] Figure 5 This is a schematic diagram illustrating the vertical grouping of components involved in the house modeling method of this application;

[0056] Figure 6 This is a schematic diagram of the grayscale image of the main building involved in the house modeling method of this application;

[0057] Figure 7 This is a schematic diagram of the depth edge map involved in the house modeling method of this application;

[0058] Figure 8 This is a flowchart illustrating Embodiment 2 of the house modeling method of this application;

[0059] Figure 9 This is a schematic diagram illustrating the depth level allocation involved in the house modeling method of this application;

[0060] Figure 10 A simplified flowchart illustrating the house modeling method provided in Embodiment 2 of this application;

[0061] Figure 11 This is a schematic diagram of the front view of the house model involved in the house modeling method of this application;

[0062] Figure 12 This is a schematic diagram of the perspective view of the house model involved in the house modeling method of this application;

[0063] Figure 13 This is a schematic diagram of the top view of the house model involved in the house modeling method of this application;

[0064] Figure 14 This is a schematic diagram of the modular structure of the house modeling device according to an embodiment of this application;

[0065] Figure 15 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the house modeling method in this application embodiment.

[0066] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0067] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0068] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0069] The main solution of this application embodiment is as follows: Based on the real land parcel parameters, the house image is detected by the target detection model to obtain a set of house main body and building components; based on the house main body, a binary mask of the house main body is generated by the image segmentation model, and the building component set is removed based on the binary mask to obtain an optimized building component set; the optimized building component set is clustered to obtain a horizontal group set and a vertical group set; edge detection is performed on the house image and binary mask by the depth estimation model to obtain a depth edge map; a building logical grid is obtained based on the depth edge map, the horizontal group set, and the vertical group set components. Based on the house image and house main body, a mask is generated by the image segmentation model to obtain a binary mask of the house main body; the building component set is traversed and calculated based on the binary mask to obtain an intersection-union ratio set; the intersection-union ratio set is compared by an intersection-union ratio threshold; building components in the building component set with an intersection-union ratio lower than the intersection-union ratio threshold are removed to obtain an optimized building component set. The optimized building component set is filtered based on the first classification component to obtain horizontal components. The vertical axis projection overlap of these horizontal components is calculated to obtain a horizontal grouping set. Similarly, the optimized building component set is filtered based on the second classification component to obtain vertical components. The horizontal axis projection overlap of these vertical components is calculated to obtain a vertical grouping set. The building image is then masked using a binary mask to obtain a main building image. The main building image is then subjected to depth processing using the depth estimation model to obtain a grayscale image of the main building. The grayscale image is then denoised using a Gaussian blur algorithm to obtain a denoised grayscale image of the main building. Finally, the denoised grayscale image of the main building is subjected to edge detection using an edge detection algorithm to obtain a depth edge map. The house image is cropped based on an optimized set of building components to obtain component image regions, and the RGB values ​​of the component images are extracted. The bounding box coordinates of the optimized set of building components are adjusted according to the building logic grid to obtain an adjusted optimized set of building components. Depth level values ​​are assigned to the adjusted optimized set of building components based on the component labels and depth edge map to obtain the assignment results. The assignment results, the adjusted optimized set of building components, and the RGB values ​​are summarized to obtain structured data, and the structured data is converted to a different format to obtain modeling data.The method involves calculating the average depth value of the bounding boxes of the optimized building component set in the grayscale image of the main building; detecting the overlap value of the strong edge contours of the bounding boxes in the depth edge image; determining the initial weights of several components in the adjusted optimized building component set based on the component labels; and assigning depth level values ​​to the adjusted optimized building component set based on the average depth value, overlap value, and initial weights to obtain the allocation result. This solves the problem of poor modeling results in existing technologies due to equipment limitations in obtaining accurate modeling data for building component recognition, thus achieving accurate building modeling and improving the accuracy of building modeling. Based on this invention, considering the practical reliance on manual on-site surveying and manual CAD (computer-aided design) modeling, which is time-consuming, labor-intensive, costly, and requires high professional skills from operators, making it difficult to meet the needs of large-scale rapid modeling and resulting in low modeling efficiency and accuracy, a building modeling method is designed. The effectiveness of the building modeling method of this invention is verified when modeling a building, and the accuracy of building modeling using the method of this invention is significantly improved.

[0070] In this embodiment, for ease of description, the following description uses a house modeling device as the execution subject.

[0071] Due to inherent limitations in data acquisition, model morphology, and deep learning methods in existing 3D reconstruction technologies, their applicability in structured and parametric architectural modeling (such as BIM and procedural modeling) needs improvement. Firstly, data acquisition requirements are high; photogrammetric-based 3D reconstruction necessitates multi-angle image or video sequences, demanding high equipment accuracy, environmental stability, and standardized operation, increasing implementation difficulty and cost. Secondly, the model output morphology has poor adaptability; the generated dense point cloud or mesh model data is large in volume and has an ambiguous topological structure, making it difficult to directly use for parametric editing and semantic understanding, failing to meet the parametric input requirements of architectural modeling processes. Thirdly, the accuracy of monocular deep learning reconstruction is insufficient, requiring reconstruction from a single image. During construction, the accuracy of identifying and locating building components such as doors and windows is insufficient, and it is easily affected by factors such as lighting, occlusion, and shooting angle, which reduces the accuracy of extracting geometric information of components. Fourthly, there is insufficient inference of structural and depth relationships. Existing methods mostly focus on the geometric shape of individual components and find it difficult to infer the logical relationships between components (such as floor affiliation and vertical bay alignment). Moreover, monocular depth estimation algorithms are prone to depth ambiguity when processing structures of different depths such as walls and balconies, and cannot output discrete procedural modeling depth hierarchy relationships. Therefore, in the scenario of digital building modeling, 3D reconstruction technology has significant bottlenecks in meeting the requirements of structured and parametric methods. If the technical path cannot be optimized in a targeted manner, its application effect in BIM and procedural modeling processes will be greatly reduced.

[0072] This application provides a solution that uses a 3D perception fusion model of building components to identify and segment buildings from received images of houses and real land parameters, resulting in optimized building components and a building logic mesh. This allows the main body of a house and its related parameters to be extracted from images acquired by the device alone. Subsequently, the optimized building components and building logic mesh are subjected to structured processing to obtain modeling data suitable for modeling. A modeling engine is then used to create a house model based on the building data, providing users with a higher quality service.

[0073] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or house modeling device capable of performing the above functions. The following description uses a house modeling device as an example to illustrate this embodiment and the subsequent embodiments.

[0074] Based on this, the embodiments of this application provide a house modeling method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the house modeling method of this application.

[0075] In this embodiment, the house modeling method includes steps S01 to S03:

[0076] Step S01: Receive the house image and the actual land parameters of the house image;

[0077] Before the implementation of this embodiment, it should be clear that with the advancement of digital village and smart city construction, there is an urgent need for 3D digital modeling of rural houses. However, existing technologies have significant bottlenecks. Traditional methods rely on manual on-site surveying and manual CAD modeling, which are time-consuming, labor-intensive, costly, and require high levels of professional skills, making it difficult to meet the needs of large-scale rapid modeling. On the other hand, 3D reconstruction based on photogrammetry requires multi-angle shooting, has high data acquisition requirements, and generates a large amount of dense point cloud or mesh model data with a blurred topological structure, which cannot be directly used for parametric editing and semantic understanding, and is not suitable for BIM or procedural modeling processes. In addition, although deep learning single-image reconstruction is a research hotspot, it has comprehensive problems such as insufficient accuracy in identifying and locating building components (susceptible to lighting, occlusion, and shooting angle), difficulty in inferring logical relationships between components (such as floor ownership and vertical bay alignment), and inability to obtain discrete depth hierarchical relationships (such as walls, balconies, and porches).

[0078] Therefore, in this embodiment, house images can be acquired using communication devices with camera functions or professional camera devices. Then, the house images and the corresponding real land parameters in the house images are input into the system. The system then models the house based on the house images and the real land parameters. The real land parameters include area, planning indicators, spatial range, height restrictions, land properties, and topography, which helps to deduce more accurate house data based on the images and parameters in the subsequent analysis process, resulting in better house modeling.

[0079] Step S02: Based on the house image and real land parameters, the building is identified and segmented using a 3D perception fusion model of building components to obtain optimized building components and building logic mesh.

[0080] To clearly understand the building components and their relationships, this embodiment inputs building images and real-world land parameters into a 3D perception fusion model for building component identification and segmentation. This 3D perception fusion model includes a building component target detection model, an image segmentation model, and a depth estimation model. These three models can detect targets in the image, segment the image based on the detection results, and obtain the depth map corresponding to the building. Furthermore, these three models have been trained before use and are primarily applied to building identification, segmentation, and depth map components. Therefore, by using the aforementioned 3D perception fusion model for building component identification and segmentation, optimized building components can be obtained. And architectural logic grids, where architectural components refer to the basic functional units that make up a building. They are independent components with specific forms, structures and uses, and are the physical carriers for the formation of architectural space and the realization of functions. Common types include walls (load-bearing / enclosure), floors (dividing floors), beams and columns (structural support), doors and windows (lighting and ventilation), stairs (vertical transportation), balconies (extending space), etc. Architectural logic grids are abstract spatial frameworks constructed based on preset rules in architectural design. Through standardized grid units, coordinate benchmarks and related logic, they systematically coordinate the position, size and interrelationship of architectural components (such as door and window positions, wall dimensions), spatial layout (such as room division, circulation organization), and structural systems (such as column grid spacing).

[0081] Step S03: The optimized building components and building logic grid are structured to obtain modeling data, and a house model is obtained by modeling through a modeling engine based on the modeling data.

[0082] After obtaining the optimized building components and building logic mesh of the current house, they cannot be directly applied to modeling. They need to be processed to a certain extent. Therefore, in this embodiment, the optimized building components and building logic mesh are structured to obtain modeling data. Finally, the modeling engine is called based on the modeling data to perform modeling and obtain the final house model.

[0083] The above solution enables the construction of a house model from a house image, solving the problem that existing technologies cannot obtain accurate modeling data due to equipment limitations in the identification of building components, resulting in poor modeling effects.

[0084] Specifically, the aforementioned 3D perception fusion model for building components includes a target detection model, an image segmentation model, and a depth estimation model. Therefore, the step of identifying and segmenting buildings based on the building images and real land parameters using the 3D perception fusion model to obtain optimized building components and building logical mesh includes:

[0085] Step S021: Based on the real land parcel parameters, the target detection model is used to detect the house image to obtain a set of the main body of the house and building components;

[0086] Step S022: Based on the main body of the house, a binary mask of the main body of the house is generated by the image segmentation model, and the building component set is removed based on the binary mask to obtain an optimized building component set;

[0087] Step S023: Cluster the optimized building component set to obtain a horizontal group set and a vertical group set;

[0088] Step S024: Perform edge detection on the house image and binary mask using the depth estimation model to obtain a depth edge map;

[0089] Step S025: Obtain the building logic mesh based on the depth edge map, horizontal group set, and vertical group set components.

[0090] Before processing begins, the system needs to preload three core deep learning models into the memory of the computing device (such as a server with a GPU), including:

[0091] Building component object detection model: The model is a YOLOv8 model that has been fully trained on a dataset containing farmhouses and various building components (such as roofs, walls, doors, windows, balconies, doorposts, eaves, etc.). This model can quickly identify farmhouses and various building elements in the image and output their categories and initial bounding boxes.

[0092] Image segmentation model: The lightweight MobileSAM (Segment Anything Model) model is adopted. This model can perform pixel-level accurate segmentation of specific objects in an image based on given cues (such as a bounding box) and generate their contour masks.

[0093] Depth estimation model: The Depth-Anything-V2 model is used. This model excels at inferring the relative depth information of a scene from a single color image, generating a high-precision depth map the same size as the original image.

[0094] After receiving the house images, the received images are input into the object detection model. Combined with the parameters of the real land parcels, the model performs a forward inference on the entire image, such as... Figure 2 The target detection is performed as shown, and a list containing all identified farmhouses and building components is output. Each item in the list contains the component's category label (such as 'building', 'window', etc.) and its initial bounding box coordinates (x1, y1, x2, y2). Then, all components with the category label 'building' are filtered from the detection results. The area of ​​each 'building' bounding box is calculated as (x2-x1)*(y2-y1), and the one with the largest area is selected as the core target of this processing—the main building (i.e., the main house), and its bounding box house_bbox is recorded. The labels of other categories are the set of building components corresponding to the main house, including doors, windows, and stairs. It should be clear that the house images were obtained by communication devices with camera functions or professional camera devices in the above embodiments. In this embodiment, a mobile phone camera is used as an example.

[0095] Subsequently, in order to improve the correlation between building components and the main body of the house, a binary mask corresponding to the main body of the house is generated through an image segmentation model, and the building component set obtained above is removed based on the binary mask to obtain an optimized building component set.

[0096] Then, the building components are structurally grouped, including horizontal grouping, that is, for the components such as doors and windows in the optimized building component set, clustering is performed based on the degree of overlap of their bounding boxes in the vertical direction to form several horizontal groups, each group corresponding to a floor of the building, and vertical grouping, that is, for the components such as doors, windows, balconies in the optimized building component set, clustering is performed based on the degree of overlap of their bounding boxes in the horizontal direction to form several vertical groups, each group corresponding to a vertical bay of the building.

[0097] To obtain a depth edge map that can characterize the abrupt changes in the depth of a building structure, a binary mask of the generated building body is applied to the original input image to obtain an image containing only the building body. This image is then input into a depth estimation model to generate a pixel-level depth map. Edge detection is then performed on this depth map to obtain the depth edge map.

[0098] Finally, based on the horizontal and vertical group sets obtained above, a parametric grid for the building facade is established. For each horizontal group (floor), the average value of the top Y coordinate and the average value of the bottom Y coordinate of all components in the group are calculated to determine the ceiling line and floor line of that floor. For each vertical group (span), the average value of the center X coordinate of all components in the group is calculated to determine the center vertical line of that span.

[0099] These calculated horizontal and vertical lines are combined to form a logical grid. Finally, all components are traversed, and their bounding box coordinates are "snap" to the nearest grid line to achieve effects such as "aligning the top edges of all windows" and "aligning the centers of components within the same bay," making the component layout more regular.

[0100] The above method can be used to process house images and obtain parameter data for the entire house, including the main body of the house and information on the corresponding building components, thus enabling efficient subsequent house modeling.

[0101] More specifically, step S022 above, which involves generating a binary mask of the building structure based on the building structure using the image segmentation model, and then removing components from the building component set based on the binary mask to obtain an optimized building component set, includes:

[0102] Step S0221: Based on the house image and the main body of the house, a mask is generated using the image segmentation model to obtain a binary mask of the main body of the house;

[0103] Step S0222: Based on the binary mask, perform traversal calculations on the set of building components to obtain the intersection-union ratio set;

[0104] Step S0223: Compare the intersection-union ratio set using the intersection-union ratio threshold;

[0105] Step S0224: Remove building components from the set of building components whose intersection ratio is lower than the intersection ratio threshold to obtain an optimized set of building components.

[0106] To accurately define the scope of the main building and eliminate background interference, the original image and the house main body (house_bbox) determined in the above embodiment are input into the segmentation model MobileSAM. The model then generates an image as shown below. Figure 3The binarized house_mask shown is precisely outlined in the shape of the building.

[0107] Next, the set of building components detected in the above embodiments (including doors, windows, balconies, etc.) is traversed, and the intersection-over-union (IoU) ratio between the bounding box of each component and the house_mask is calculated. A reasonable IoU threshold is set (e.g., 0.6). Any component with an IoU value lower than this threshold is determined to be irrelevant to the main building (such as distant trees, adjacent house parts, etc.) or a false detection, and is removed from the component list to obtain optimized building components.

[0108] Through the above process, a purer set of optimized building components that are strongly related to the main building can be obtained.

[0109] Further, step S023 above, which involves clustering the optimized set of building components to obtain horizontal and vertical grouping sets, includes:

[0110] Step S0231: Based on the first classification components, the optimized building component set is filtered to obtain horizontal components. The vertical axis projection overlap is calculated based on the horizontal components to obtain a horizontal group set.

[0111] Step S0232: Based on the second category of components, the optimized building component set is filtered to obtain vertical components. The horizontal axis projection overlap is calculated based on the vertical components to obtain a vertical group set.

[0112] To understand the building's facade layout, this embodiment logically groups the optimized component set. For horizontal groups (floor identification), it filters out doors and windows. By calculating the overlap of the bounding boxes of any two door / window components projected onto the Y-axis (vertical direction), components with an overlap exceeding a specific threshold (e.g., the overlapping portion accounts for more than 10% of their own height) are grouped into the same horizontal group. This ultimately yields the following... Figure 4 The horizontal grouping shown corresponds to a floor in architecture.

[0113] For vertical grouping (bay recognition), components such as doors, windows, and balconies can be filtered. By calculating the overlap of the bounding boxes of any two components projected onto the X-axis (horizontal direction), components with an overlap exceeding a certain threshold (e.g., the overlapping portion is more than 5% of its own width) are grouped into the same vertical group, ultimately resulting in... Figure 5 The vertical grouping shown corresponds to a bay or vertical functional unit in architecture.

[0114] Furthermore, step S024 above, which involves performing edge detection on the house image and binary mask using the depth estimation model to obtain a depth edge map, includes:

[0115] Step S0241: Obscure the non-building area of ​​the house image according to the binary mask to obtain the main building image;

[0116] Step S0242: Perform depth processing on the main building image using the depth estimation model to obtain a grayscale image of the main building;

[0117] Step S0243: Denoise the grayscale image of the main building by using a Gaussian blur algorithm to obtain a denoised grayscale image of the main building.

[0118] Step S0244: The edge detection algorithm is used to perform edge detection on the denoised grayscale image of the main building to obtain a depth edge map.

[0119] Based on the binary mask `house_mask` generated in the above embodiments, non-building areas in the original color image are masked (e.g., set to black or transparent) to obtain an image containing only the main building. This image is then input into the Depth-Anything model to generate an image like... Figure 6 The grayscale depth map shown depicts a pixel whose grayscale value is inversely proportional to its relative distance in the real world. However, grayscale maps cannot represent the relationships between boundaries or components. Therefore, to capture abrupt changes in depth (such as the boundary between a wall and a protruding balcony), the depth map needs further processing. First, Gaussian blur is used to smooth it and reduce noise. Then, the Canny edge detection algorithm is applied to generate a depth map as shown. Figure 7 The depth_edge map shown clearly indicates the locations where the depth changes drastically.

[0120] This embodiment, through the above-described scheme, specifically receives a house image and the actual land parameters of the house image; based on the house image and the actual land parameters, it performs building identification and segmentation using a 3D perception fusion model of building components to obtain optimized building components and a building logical mesh; it then performs structured processing on the optimized building components and the building logical mesh to obtain modeling data, and finally, based on the modeling data, it performs modeling using a modeling engine to obtain a house model. Thus, by using a 3D perception fusion model of building components to perform building identification and segmentation on the received house image and actual land parameters to obtain optimized building components and a building logical mesh, it achieves the ability to parse the main body and related parameters of a house from images acquired by the device alone. Subsequently, based on the optimized building components and the building logical mesh, it performs structured processing to obtain modeling data suitable for modeling, and then uses a modeling engine to perform modeling based on the building data to obtain a house model. This solves the problem in existing technologies where accurate modeling data cannot be obtained due to device factors in building component identification, resulting in poor modeling effects, and improves the accuracy of house modeling.

[0121] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 8 Step S03 involves structuring the optimized building components and the building logic grid to obtain modeling data. The house modeling method further includes steps S031-S034:

[0122] Step S031: Based on the optimized set of building components, the house image is cropped to obtain the component image region, and the RGB values ​​of the component image are extracted;

[0123] Step S032: Adjust the bounding box coordinates of the optimized building component set according to the building logic grid to obtain the adjusted optimized building component set;

[0124] Step S033: Assign depth level values ​​to the adjusted optimized building component set based on the component labels and depth edge map to obtain the assignment result;

[0125] Step S034: Summarize the allocation results, the adjusted optimized building component set, and the RGB values ​​to obtain structured data, and convert the structured data to obtain modeling data.

[0126] After the geometric positions of the components are aligned and finalized, each component can be traversed. Based on its final bounding box coordinates, the corresponding image region is cropped from the original color input image. For all pixels in the region, the K-Means clustering algorithm (e.g., K=3) is used to cluster colors. The center color of the largest number of pixels, or the average RGB value of all pixels in the region, is used as the representative color of the component and stored in the form of RGB values.

[0127] To convert continuous depth information into discrete levels that can be used for procedural modeling, it is necessary to integrate various information to assign an integer depth level value to each component (the depth level of the most protruding component is always 0, such as 0 representing a protruding balcony, 1 representing a main wall or door with 1 depth, 2 representing a door or window with 2 depth, etc.).

[0128] Finally, a general data structure is created to encapsulate all the parametric information of the house. The parametric information contains basic information (basic_info), including the total width and height of the house in the physical world calculated based on the pixel size and optional initial input, as well as the roof style (such as pitched roof or flat roof) determined based on the roof type detection results.

[0129] In addition, the parametric information also contains floor data (floor_data), an array where each element represents a floor. Within each floor element, there are lists such as Windows, Doors, and Balconies, which record detailed information about each component on that floor, including: parametric width and height, position in the facade grid, RGB color value, and depth level.

[0130] Finally, this complete data structure is serialized into a common format (such as JSON) as the final output (modeling data).

[0131] Specifically, step S033 above, which involves assigning depth level values ​​to the adjusted optimized building component set based on the component labels and depth edge map of the optimized building component set, and obtaining the assignment result, includes:

[0132] Step S0331: Calculate the average depth value of the component bounding box of the optimized building component set in the grayscale image of the main building;

[0133] Step S0332: Detect the overlap value of the strong edge contour of the component bounding box in the depth edge map;

[0134] Step S0333: Determine the initial weights of several components in the adjusted optimized building component set based on the component labels of the adjusted optimized building component set;

[0135] Step S0334: Based on the average depth value, overlap value, and initial weight, the depth level value is assigned to the adjusted optimized building component set to obtain the assignment result.

[0136] To assign depth level values ​​to each component, the average depth value of the area where the component's bounding box is located in the grayscale map (depth_map) of the main building is first calculated. Then, it is checked whether the edge of the component highly coincides with the strong edge contour in the depth_edge map.

[0137] Then, referring to the semantic category of the components themselves, components such as 'balcony', 'porch', and 'eaves' typically have natural protruding attributes and can be assigned higher initial weights. Finally, by weighted fusion of the above information (including average depth value, overlap value, and initial weights), the final result is as follows: Figure 9 The diagram shows that a discrete depth level is determined for each component.

[0138] This embodiment, through the above-described scheme, specifically involves cropping the house image based on an optimized set of building components to obtain component image regions and extracting the RGB values ​​of the component images; adjusting the bounding box coordinates of the optimized set of building components according to the building logic grid to obtain an adjusted optimized set of building components; assigning depth level values ​​to the adjusted optimized set of building components based on the component labels and depth edge maps to obtain an assignment result; summarizing the assignment result, the adjusted optimized set of building components, and the RGB values ​​to obtain structured data; and converting the structured data to obtain modeling data. Thus, by using a 3D perception fusion model of building components to identify and segment the received house image and real land parameters, optimized building components and a building logic grid are obtained. This enables the parsing of the main body of the house and related parameters from images acquired by the device alone. Subsequently, structured processing is performed based on the optimized building components and the building logic grid to obtain modeling data suitable for modeling. A modeling engine then uses this building data to create a house model, solving the problem in existing technologies where accurate modeling data cannot be obtained due to device factors in building component identification, resulting in poor modeling effects and improving the accuracy of house modeling.

[0139] For example, to help understand the implementation process of the house modeling method obtained by combining this embodiment with the first embodiment described above, please refer to... Figure 10 , Figure 10A simplified flowchart of a house modeling method is provided, specifically:

[0140] To achieve the above objectives, the present invention discloses a method, characterized by comprising the following steps:

[0141] First, the system receives a single 2D color image of a building uploaded by the user, along with the initial width and height information of optional associated geographic patches, and loads a pre-trained building component target detection model, image segmentation model, and depth estimation model.

[0142] Then, the target detection model (such as the YOLO series model) is used to process the input image, identify and locate all building components in the image, including but not limited to the main building, doors, windows, roof, balcony, doorposts, etc., and output the category label and initial bounding box of each component.

[0143] Among all the building components obtained above, the bounding box of the main building with the largest area is selected as the region of interest (ROI). This bounding box is used as a prompt and input into an image segmentation model (such as the SAM series model) to generate a pixel-level binary mask of the main building. All components detected in S2 are traversed, and the intersection-over-union (IoU) ratio between their bounding boxes and the binary mask is calculated. Components with an IoU ratio lower than a preset threshold are removed to obtain an optimized set of building components strongly correlated with the main building.

[0144] Subsequently, for the components such as doors and windows in the optimized building component set, clustering is performed based on the degree of overlap of their bounding boxes in the vertical direction to form several horizontal groups, each group corresponding to a floor of the building. For the components such as doors, windows, and balconies in the optimized building component set, clustering is performed based on the degree of overlap of their bounding boxes in the horizontal direction to form several vertical groups, each group corresponding to a vertical bay of the building.

[0145] During the structuring process, the binary mask of the building structure generated in the above steps is simultaneously applied to the original input image to obtain an image containing only the building structure. This image is then input into a depth estimation model (such as the DepthAnything series models) to generate a pixel-level depth map. Edge detection (such as the Canny operator) is performed on this depth map to obtain a depth edge map that can characterize abrupt changes in the depth of the building structure.

[0146] Then, based on the obtained horizontal and vertical grouping results, several horizontal dividing lines representing floor boundaries and several vertical dividing lines representing bay boundaries are calculated and generated, which together constitute the logical grid of the building facade. Based on this grid, the bounding box coordinates of each component in the optimized building component set are adjusted and aligned to conform to the conventional layout of architecture.

[0147] Then, the image region corresponding to each component is cropped from the original color input image. By analyzing the pixel composition of the image and applying the k-means clustering algorithm and color quantization technology, the main color is extracted as the representative color of the component and stored in the form of RGB values. By comprehensively utilizing the obtained depth map, depth edge map, and the category information of the component itself (such as porch, balcony, doorpost), a discrete depth level value is assigned to each aligned component. This value represents the depth level of the component relative to the foremost component, quantizing the continuous depth information into a hierarchy that can be used for parametric modeling.

[0148] Finally, all the above processing results are integrated to generate structured data, which includes: overall building information (such as physical world width, height, depth, and roof type calculated based on initial geographic patch information and pixel size) and floor data. The floor data is organized in array form, and each item contains the type, parametric size, position in the facade grid, and depth level of all components such as doors, windows, and balconies in that floor. Finally, the structured data is converted into the format required by the downstream modeling engine and output.

[0149] After the overall structuring of the data is completed, the modeling engine is called to model the data, resulting in the following: Figure 11 The front view of the house model shown, such as Figure 12 The house model perspective view shown and as follows Figure 13 The house model shown is a top view.

[0150] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the house modeling method of this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0151] This application also provides a house modeling device, please refer to... Figure 14 The house modeling device includes:

[0152] The receiving module 10 is used to receive images of houses and the actual land parameters of the houses;

[0153] Building recognition module 20 is used to identify and segment buildings based on the house images and real plot parameters, and to obtain optimized building components and building logic grids through a three-dimensional perception fusion model of building components.

[0154] The modeling module 30 is used to perform structuring processing on the optimized building components and building logic grid to obtain modeling data, and to perform modeling through the modeling engine based on the modeling data to obtain a house model.

[0155] The house modeling device provided in this application, employing the house modeling method described in the above embodiments, can solve the technical problem in the prior art where accurate modeling data cannot be obtained due to equipment factors in the identification of building components, resulting in poor modeling effects. Compared with the prior art, the beneficial effects of the house modeling device provided in this application are the same as those of the house modeling method provided in the above embodiments, and other technical features in the house modeling device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0156] This application provides a house modeling device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the house modeling method in the first embodiment described above.

[0157] The following is for reference. Figure 15 The diagram illustrates a structural schematic of a house modeling device suitable for implementing embodiments of this application. The house modeling device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 15 The house modeling device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of this application.

[0158] like Figure 15As shown, the house modeling device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the house modeling device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the house modeling device to communicate wirelessly or wiredly with other devices to exchange data. While the figures show house modeling devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0159] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0160] The house modeling device provided in this application, employing the house modeling method described in the above embodiments, can solve the technical problem in the prior art where accurate modeling data cannot be obtained due to equipment limitations in building component identification, resulting in poor modeling effects. Compared with the prior art, the beneficial effects of the house modeling device provided in this application are the same as those of the house modeling method provided in the above embodiments, and other technical features of this house modeling device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0161] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0162] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0163] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the house modeling method in the above embodiments.

[0164] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0165] The aforementioned computer-readable storage medium may be included in the house modeling equipment; or it may exist independently and not be assembled into the house modeling equipment.

[0166] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the house modeling device, the house modeling device causes the house modeling device to: receive a house image and the actual land parameters of the house image; perform building recognition and segmentation based on the house image and the actual land parameters using a 3D perception fusion model of building components to obtain optimized building components and a building logical mesh; perform structured processing on the optimized building components and the building logical mesh to obtain modeling data; and perform modeling based on the modeling data using a modeling engine to obtain a house model.

[0167] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0168] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0169] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0170] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described house modeling method. This solves the technical problem in existing technologies where accurate modeling data cannot be obtained due to equipment limitations in building component identification, resulting in poor modeling performance. Compared with existing technologies, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the house modeling method provided in the above embodiments, and will not be elaborated upon here.

[0171] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the house modeling method described above.

[0172] The computer program product provided in this application can solve the technical problem that the existing technology cannot obtain accurate modeling data in building component identification due to equipment factors, resulting in poor modeling effect. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the house modeling method provided in the above embodiments, and will not be repeated here.

[0173] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for modeling houses, characterized in that, The house modeling method includes: Receive images of houses and the actual land parameters of the houses; Based on the house images and real plot parameters, the building is identified and segmented using a 3D perception fusion model of building components, resulting in optimized building components and building logic mesh. The three-dimensional perception fusion model of the building components includes a target detection model, an image segmentation model, and a depth estimation model. The optimized building components and building logic mesh are structured to obtain modeling data, and a house model is obtained by modeling through a modeling engine based on the modeling data. The step of identifying and segmenting buildings based on the house images and actual land parameters using a 3D perception fusion model of building components to obtain optimized building components and building logical mesh includes: Based on the real land parcel parameters, the target detection model is used to detect the house image to obtain a set of the main body of the house and building components; Based on the main body of the house, a binary mask of the main body of the house is generated by the image segmentation model, and the building component set is removed based on the binary mask to obtain an optimized building component set; The optimized set of building components is clustered to obtain horizontal grouping sets and vertical grouping sets; The depth estimation model is used to perform edge detection on the house image and the binary mask to obtain a depth edge map; The building logic mesh is obtained based on the depth edge map, the horizontal grouping set, and the vertical grouping set components. The step of structuring the optimized building components and building logic mesh to obtain modeling data includes: The house image is cropped based on an optimized set of building components to obtain component image regions, and the RGB values ​​of the component images are extracted. The bounding box coordinates of the optimized building component set are adjusted according to the building logic grid to obtain the adjusted optimized building component set; The depth level values ​​of the adjusted optimized building component set are assigned based on the component labels and depth edge map of the optimized building component set, and the assignment results are obtained. The allocation results, the adjusted optimized set of building components, and the RGB values ​​are summarized to obtain structured data. The structured data is then converted to a new format to obtain modeling data.

2. The house modeling method as described in claim 1, characterized in that, The steps of generating a binary mask of the building structure based on the building structure using the image segmentation model, and then removing components from the building component set based on the binary mask to obtain an optimized building component set include: Based on the house image and the main body of the house, a mask is generated using the image segmentation model to obtain a binary mask of the main body of the house; Based on the binary mask, the set of building components is traversed and calculated to obtain the intersection and union ratio set; The intersection-union ratio set is compared using an intersection-union ratio threshold; By removing building components from the set of building components whose intersection ratio is lower than the intersection ratio threshold, an optimized set of building components is obtained.

3. The house modeling method as described in claim 1, characterized in that, The step of clustering the optimized set of building components to obtain horizontal and vertical group sets includes: The optimized building component set is filtered based on the first classification component to obtain horizontal components. The vertical axis projection overlap is calculated based on the horizontal components to obtain a horizontal group set. The optimized building component set is filtered based on the second category components to obtain vertical components. The horizontal axis projection overlap is calculated based on the vertical components to obtain the vertical group set.

4. The house modeling method as described in claim 1, characterized in that, The step of performing edge detection on the house image and binary mask using the depth estimation model to obtain a depth edge map includes: The non-building areas of the house image are masked using the binary mask to obtain the main building image; The depth estimation model is used to perform depth processing on the image of the main building to obtain a grayscale image of the main building; The grayscale image of the main building is denoised by the Gaussian blur algorithm to obtain the denoised grayscale image of the main building. An edge detection algorithm is used to perform edge detection on the denoised grayscale image of the main building to obtain a depth edge map.

5. The house modeling method as described in claim 4, characterized in that, The step of assigning depth level values ​​to the adjusted optimized building component set based on the component labels and depth edge map of the optimized building component set to obtain the assignment result includes: Calculate the average depth value of the component bounding box of the optimized building component set in the grayscale image of the main building; Detect the overlap value of the strong edge contours of the component bounding box in the depth edge map; The initial weights of several components in the adjusted optimized building component set are determined based on the component labels of the adjusted optimized building component set. The depth level values ​​are assigned to the adjusted optimized building component set based on the average depth value, overlap value, and initial weight, and the assignment results are obtained.

6. A house modeling device, characterized in that, The house modeling device includes: The receiving module is used to receive images of houses and the actual land parameters of the houses. The building recognition module is used to identify and segment buildings based on the house images and real plot parameters, and to obtain optimized building components and building logic grids through a 3D perception fusion model of building components. The three-dimensional perception fusion model of building components includes a target detection model, an image segmentation model, and a depth estimation model. The building recognition module is also used to detect the house image based on the real plot parameters through the target detection model to obtain the house body and the set of building components. Based on the main body of the house, a binary mask of the main body of the house is generated by the image segmentation model, and the building component set is removed based on the binary mask to obtain an optimized building component set; The optimized set of building components is clustered to obtain horizontal grouping sets and vertical grouping sets; The depth estimation model is used to perform edge detection on the house image and the binary mask to obtain a depth edge map; The building logic mesh is obtained based on the depth edge map, the horizontal grouping set, and the vertical grouping set components. The modeling module is used to perform structuring processing on the optimized building components and building logic grid to obtain modeling data, and to perform modeling through the modeling engine based on the modeling data to obtain a house model; The modeling module is also used to crop the house image based on the optimized set of building components to obtain the component image region and extract the RGB values ​​of the component image. The bounding box coordinates of the optimized building component set are adjusted according to the building logic grid to obtain the adjusted optimized building component set; The depth level values ​​of the adjusted optimized building component set are assigned based on the component labels and depth edge map of the optimized building component set, and the assignment results are obtained. The allocation results, the adjusted optimized set of building components, and the RGB values ​​are summarized to obtain structured data. The structured data is then converted to a new format to obtain modeling data.

7. A house modeling device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the house modeling method as described in any one of claims 1 to 5.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the house modeling method as described in any one of claims 1 to 5.

Images