Building programmed modeling method and system based on unmanned aerial vehicle image

Abstract

The invention relates to the field of BIM (Building Information Modeling) model generation and artificial intelligence, and provides an unmanned aerial vehicle image-based programmed building modeling method, which comprises the following steps of: constructing a model training data set; an intelligent building parametric modeling script generation model is constructed and trained, specifically, the intelligent building parametric modeling script generation model uses a ViT model to extract visual features, an MLP multi-layer perceptron carries out cross-modal projection compression, and an LLM large language model generates a building parametric modeling script capable of driving BIM software; building parametric modeling script intelligent generation model reasoning: inputting an unmanned aerial vehicle image, and outputting a building parametric modeling script capable of driving BIM software; and generating a BIM model supporting parameterized linkage updating based on the building parameterized modeling script of the drivable BIM software. According to the invention, end-to-end automatic generation of the BIM model from the unmanned aerial vehicle image is innovatively realized. The invention further provides a building programmed modeling system based on the unmanned aerial vehicle image.

Description

Technical Field

[0001] The present invention relates to the fields of building three-dimensional model generation and artificial intelligence, and in particular to a method and system for programming building modeling based on drone images. Background Art

[0002] Current technologies for generating 3D building models directly from drone imagery (such as generating triangular mesh models based on multi-view stereo vision (MVS)) have the following limitations in achieving component-level semantic expression and programmable output: 1. The extraction of semantic component primitives is lacking and highly dependent on manual labor.

[0003] Existing methods (including traditional computer vision segmentation and shallow neural networks) primarily output continuous triangular mesh surfaces (mesh) or voxel representations. This type of representation is incapable of instance-level identification and parameterized definition of functional building components (such as individual walls, doors, windows, and balconies). For example, a "window" is represented simply as a collection of specific triangular facets within a mesh model, rather than as a distinct object with a type identifier (such as "casement window"), a unique ID, precise geometric parameters (width / height / sill height), and topological relationships (embedded in the wall). Automated extraction of semantically informed component instances and their geometric parameters (size / position) from the mesh model requires extensive manual post-processing: engineers must manually outline component boundaries, classify and label them, measure dimensions, and assign parameters to convert the geometric model into a model containing semantic primitives. This results in single-component processing taking 3-5 hours and an error rate exceeding 15%.

[0004] 2. The output model is non-parametric and has no programmable interface.

[0005] Mesh models (.obj / .stl / .fbx formats) generated by existing technologies are essentially static geometric shells. Component primitives lack parametric properties, preventing users from automatically updating the model by modifying parameters (e.g., adjusting a window width from 1200mm to 1500mm). Any design changes require manual geometric editing (moving vertices / stretching meshes), compromising the semantic integrity of the components. Furthermore, the output lacks directly usable APIs or scripting interfaces, making it impossible to automatically generate script code (Python / C# API calls) or structured data formats (JSON / XML) that can drive BIM software (such as Revit and ArchiCAD). Converting a mesh into an editable BIM model requires complete manual reconstruction or reliance on semi-automated plug-in reverse parsing, resulting in a model conversion efficiency of less than 0.5 units per person-day.

[0006] 3. Insufficient level of intelligence and automation.

[0007] Traditional non-autoregressive methods that output all component parameters simultaneously ignore topological dependencies: over 30% of generated components exhibit spatial conflicts (e.g., windows hanging in the air or doors failing to be embedded in the wall), and design changes fail to update automatically (modifying the window width does not change the wall length). Existing processes rely on a manually pre-set rule base and are unable to autonomously derive component-level logical relationships from imagery. Summary of the Invention

[0008] In view of the shortcomings of the existing technology, the present invention provides a procedural building modeling method based on drone images.

[0009] The purpose of this invention is to automatically parse drone raster data to directly and end-to-end generate a parametric three-dimensional building model with complete BIM semantic information (component type, geometric parameters, spatial relationship).

[0010] The present invention also provides a programmatic building modeling system based on drone images.

[0011] The present invention is implemented by the following technical solutions: A procedural architectural modeling method based on drone images, comprising: Based on the drone building image dataset, a building project collection paired dataset of "drone image ←→ component parameter table ←→ parametric modeling script" is constructed as the training dataset for the intelligent generation model of building parametric modeling script; Build and train an intelligent generation model for architectural parametric modeling scripts: This model includes the ViT model, an MLP multi-layer perceptron, and an LLM large language model. The ViT model extracts visual features, the MLP multi-layer perceptron performs cross-modal projection compression, and the LLM large language model generates architectural parametric modeling scripts that can drive BIM software. The architectural parametric modeling script intelligently generates model reasoning, inputs drone images, and outputs architectural parametric modeling scripts that can drive BIM software; Based on the architectural parametric modeling script that can drive BIM software, BIM software is used to generate a BIM model that supports parametric linkage updates.

[0012] Preferably, the ViT model is an improved ViT model with cross-scale attention gated fusion. The improved ViT model with cross-scale attention gated fusion is to add a cross-scale attention module and a gated fusion module to the standard ViT model. Multi-level feature interactions are established through the cross-scale attention module, and the association between shallow details and deep semantics is established. The cross-scale attention module is combined with the gated fusion module to realize multi-level feature adaptive fusion.

[0013] Preferably, the architectural parametric modeling script intelligently generates a model processing flow including: The ViT model is used to extract multi-level features of building components from drone images, which are converted into structured data through a lightweight parsing layer and output as structured feature tensors. Through the MLP multi-layer perceptron, cross-modal projection is performed to reduce the dimension of the structured feature tensor and output a cross-modal semantic embedding vector; Based on cross-modal semantic embedding vectors, a large language model (LLM) is used to generate architectural parametric modeling scripts that can drive BIM software.

[0014] Preferably, the ViT model outputs a structured feature tensor, which includes three levels of features: Shallow texture: Identify component edge features → Architectural semantics: Doors, windows, openings, and other component boundaries; Middle-level geometry: parsing component size attributes → architectural semantics: window and wall embedding topology; Deep semantics: Aggregate component type identification → Architectural semantics: Component type identification such as window / door / balcony.

[0015] Preferably, the MLP multi-layer perceptron compresses the structured feature tensor into a cross-modal semantic embedding vector through two fully connected layers, where: The first fully connected layer is used to expand the dimension to separate the coupled features of the structured feature tensor and reduce the interference of the wall position on the window width prediction; The second fully connected layer is used to compress redundant background information and strengthen topological relationship encoding.

[0016] Preferably, the large language model (LLM) uses a topology-dependent controlled autoregressive LLM to generate a component modeling script with spatial identity encoding.

[0017] Preferably, a multi-task loss function mechanism is embedded in the building code constraint labels during the training process of the building parametric modeling script intelligent generation model.

[0018] Preferably, the multi-task loss function is a weighted sum of geometric parameter loss, component topology relationship loss, code generation loss and semantic alignment loss.

[0019] A procedural architectural modeling system based on drone imagery, comprising: Dataset construction module: used to construct a paired dataset of building project acquisitions based on the drone building image dataset, which is “drone image ←→ component parameter table ←→ parametric modeling script”, as the training dataset for the intelligent generation model of the building parametric modeling script; Script intelligent generation model construction module: Build and train the architectural parametric modeling script intelligent generation model: The architectural parametric modeling script intelligent generation model includes: ViT model, MLP multi-layer perceptron and LLM large language model, among which: ViT model extracts visual features, MLP multi-layer perceptron performs cross-modal projection compression, and LLM large language model generates architectural parametric modeling scripts that can drive BIM software; Script intelligent generation model reasoning module: Architectural parametric modeling script intelligent generation model reasoning, input drone images, and output architectural parametric modeling scripts that can drive BIM software; BIM model generation module: Based on the architectural parametric modeling script that can drive BIM software, BIM software is used to generate a BIM model that supports parametric linkage updates.

[0020] Preferably, the ViT model is an improved ViT model with cross-scale attention gated fusion. The improved ViT model with cross-scale attention gated fusion is to add a cross-scale attention module and a gated fusion module to the standard ViT model. Multi-level feature interactions are established through the cross-scale attention module, and the association between shallow details and deep semantics is established. The cross-scale attention module is combined with the gated fusion module to realize multi-level feature adaptive fusion.

[0021] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention innovatively realizes the end-to-end automated transformation of drone images → architectural parametric modeling scripts → 3D architectural models through a multimodal architecture.

[0022] 2. In step 21, the ViT model or an improved ViT model extracts component bounding box coordinates (dimensions 1-128) and type probability distributions (dimensions 512-768) from drone imagery. In step 22, MLP cross-modal projection compresses the 768-dimensional visual features into a 512-dimensional semantic vector, encoding the window-wall embedding relationship. In step 23, a topological order-based autoregression method is used to generate the output parametric building modeling script, reducing the spatial conflict rate from 30% to 0.8%. In step 32, a training set of 368 real-world projects is embedded with building code constraint labels. A loss function applies a 2x weight penalty to non-compliant parameters, and component semantic alignment is achieved using Chinese-CLIP. In step 51, a Python parametric building modeling script is generated that can drive BIM software. For components with spatial IDs, the window width error is ≤3.8mm. In step 52, modifying the window width parameter triggers a 300mm shift in the opening position, reducing the design change time from 3 hours to 10 minutes.

[0023] 3. The present invention can also adopt a two-level verification mechanism. The pre-verification mechanism blocks illegal commands in real time, and the post-verification AST parser verifies parameter compliance, and the output script executable rate is ≥99%.

[0024] 4. The present invention is the first to construct a closed-loop automated pipeline of image-parametric modeling script-BIM model throughout the entire process, reducing manual intervention by 90%.

[0025] 5. UAV feature correlation effect: The ViT model significantly improves the robustness of small-scale component recognition; the topology control LLM basically eliminates component spatial conflicts; the dynamic editing response efficiency meets the high-frequency update requirements of UAV inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0027] Figure 1 This is an overview diagram of a method for procedural building modeling according to an embodiment of the present invention; Figure 2 This is a flow chart of a method for procedural building modeling according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a structured feature tensor according to an embodiment of the present invention; Figure 4 A schematic diagram of an MLP multi-layer perceptron according to an embodiment of the present invention; Figure 5 An embodiment of the present invention is a building parametric modeling script that intelligently generates a model architecture diagram; Figure 6 This is a flow chart of autoregressive decoding according to an embodiment of the present invention; Figure 7 This is a BIM model that supports parameterized linkage updates in one embodiment of the present invention. DETAILED DESCRIPTION

[0028] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0029] As used herein, the term "including" and its variations represent open inclusion, i.e., including but not limited to. Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "one example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0030] like Figure 1 As shown in the figure, a procedural building modeling method based on drone images generally includes the following steps: 1. Input raw drone image: Obtain unprocessed drone aerial images of the construction site.

[0031] 2. Image preprocessing: Radiometric correction is performed on the original image to adjust exposure; a fisheye model is used to correct lens distortion; the CLAHE algorithm is used to balance lighting; ArcGIS map data is used to extract building outlines and remove distracting objects such as vehicles in the sky.

[0032] 3. Build a standardized image library: Create a database of preprocessed images for subsequent model training and comparison.

[0033] 4. VIT visual encoding (extracting 768-dimensional features): Use the visual Transformer model to encode the image into a 768-dimensional high-dimensional visual feature vector.

[0034] 5. MLP projection layer (compressed to 512-dimensional semantic vector): The multi-layer perceptron is used to reduce the dimensionality of visual features and generate a 512-dimensional semantic abstract vector.

[0035] 6. LLM decoder generates Python architectural parametric modeling scripts: The large language model receives semantic vectors and outputs Python architectural parametric modeling scripts that can drive BIM software.

[0036] 7. BIM software executes scripts to output models: Python architectural parametric modeling scripts automatically run on BIM platforms (such as Revit) to generate parametric 3D architectural models.

[0037] The following is a detailed description of a method for procedural building modeling based on drone images of the present invention using a specific embodiment: A procedural building modeling method based on drone images, such as Figure 2 Shown, including: S1. Based on the drone building image dataset, a building project collection paired dataset of "drone image ←→ component parameter table ←→ parametric modeling script" is constructed as the training dataset for the intelligent generation model of building parametric modeling script.

[0038] In one embodiment, a paired dataset is collected from 368 real building projects: a paired dataset of real building projects is constructed: "drone image ←→ Revit component parameter table ←→ Python parametric modeling script" as a training dataset for the intelligent generation model of architectural parametric modeling scripts, including: S11. Use drones to collect 1024×1024 high-definition aerial photos of buildings.

[0039] Coverage scenarios: morning (light intensity ≤ 1000 lux), noon (light intensity ≥ 80000 lux), dusk (light intensity ≤ 500 lux) and multiple time periods; Occlusion simulation: 25%-35% vegetation occlusion (achieved by adjusting the pitch angle of the drone).

[0040] In one embodiment, high-definition aerial photos of buildings captured by drones contain a large number of highlight samples. The introduction of highlight samples forces the architectural parametric modeling script to generate a model to learn component inference in overexposed areas.

[0041] In another embodiment, temporary obstruction (crane / construction shed) samples are introduced so that the model can complete the obscured components depending on the context.

[0042] The 368 real projects were expanded to 500,000 training samples through the following methods: random rotation (±10°); scale scaling (0.8-1.2 times); and lighting adjustment (±20% brightness).

[0043] S12. Image standardization processing: Processing chain: The original input image is processed in the following order: Distortion Correction: barrel correction is performed using the camera calibration matrix K and the distortion coefficient d; Illumination Equalization: contrast is enhanced using the CLAHE algorithm; Building Mask Segmentation: building areas are extracted using the UNet model; Standardization: a 1024×1024 region is cropped based on the contour centroid.

[0044] Finally, a standardized image library (stored in PNG format) is generated to provide interference-free visual input for subsequent models.

[0045] Among them: The role of radiation correction is to restore dark details in overexposed areas caused by strong light in drone aerial photography through a linear response function to avoid blurred component edges. The function of fisheye distortion correction is to correct the barrel distortion produced by the wide-angle lens of the drone and ensure the geometric accuracy of the building outline.

[0046] The function of CLAHE lighting balance is to enhance the shadow contrast in low-light areas during dawn and dusk, and improve the recognition ability of small components such as window openings. The role of ArcGIS-assisted segmentation is to integrate geographic information data and accurately remove dynamic interference objects such as the sky and moving vehicles.

[0047] S13. Component parameter marking: Extract component parameter tables from Revit models of 368 real projects, such as: { "WALL_01": {"type": "Wall", "length": 5000, "position": [103,205]}, "WINDOW_03": {"type": "Window", "width": 1200, "parent": "WALL_01"}} S14. Script-constraint pairing: Generate Python architectural parametric modeling script: CreateWall(id="WALL_01", length=5000) AddWindow(parent="WALL_01", width=1200) Embed building code constraints: {"Window": {"min_width": 800}} [Build paired dataset]: Input: Preprocessed drone imagery (1024×1024 PNG).

[0048] Output: Python script generated by the Revit API (Example: CreateWall(length=5000);AddWindow(width=1200)).

[0049] Constraint tags: Embed building codes (e.g. {"min_window_width": 800}).

[0050] Data source: A library of 50,000 Revit models based on real building projects.

[0051] S2. Construct architectural parametric modeling scripts to intelligently generate models.

[0052] The intelligent generation model for architectural parametric modeling scripts in this paper primarily consists of three modules: a ViT (Vision Transformer) model, a Multilayer Perceptron (MLP) layer, and a Large Language Model (LLM). These three modules work together to output architectural parametric modeling scripts that can drive BIM software. The ViT model extracts visual features, the MLP layer performs cross-modal projection compression, and the LLM decoder generates architectural parametric modeling scripts based on topological order, which can drive BIM software.

[0053] In one embodiment, Figure 4 As shown, the ViT model is the InternViT-300M model, and the LLM model is the InternLM2.5-1.8B model.

[0054] In a preferred embodiment, the ViT model is an improved ViT model of cross-scale attention gated fusion. The improved ViT model of cross-scale attention gated fusion adds a cross-scale attention module and a gated fusion module to the standard ViT model. Multi-level feature interactions are established through the cross-scale attention module, and the association between shallow details and deep semantics is established. The cross-scale attention module is combined with the gated fusion module to realize multi-level feature adaptive fusion. While maintaining the global modeling advantages of ViT, the recognition ability of small-scale components is significantly improved, which is particularly suitable for drone image analysis scenarios.

[0055] The improved ViT model with cross-scale attention gated fusion addresses the scale differences of components in drone images. It can achieve the following: shallow feature maps focus on the edges of small components (such as window frames / door gaps); deep feature maps capture large-scale structures (such as wall outlines); bidirectional feature propagation is achieved through the cross-scale attention module: deep features guide shallow semantics, and shallow features enhance deep details; the gating mechanism of the gated fusion module adaptively fuses multi-level features to enhance the ability to recognize small components.

[0056] The ViT model or the improved ViT model outputs a structured feature tensor containing three levels of features: Shallow texture: Identify component edge features → Architectural semantics: boundaries of components such as doors, windows, and openings.

[0057] Middle-level geometry: parsing component size properties → architectural semantics: windows and walls embedded in topology.

[0058] Deep semantics: Aggregate component type identification → Architectural semantics: Component type identification such as window / door / balcony.

[0059] In one embodiment, the process of intelligently generating a model using a building parametric modeling script includes: S21. The ViT model is used to extract multi-level features of building components from drone images, which are converted into structured data through a lightweight parsing layer and output as structured feature tensors.

[0060] In a preferred embodiment, S21 extracts multi-level features of building components from drone images through an improved ViT model with cross-scale attention gate fusion, converts them into structured data through a lightweight parsing layer, and outputs a structured feature tensor.

[0061] In one embodiment, Figure 3 As shown in Figure 2, the ViT model or the improved ViT model with cross-scale attention gating fusion outputs a 768-dimensional structured feature tensor containing three levels of features: Shallow features: bounding box coordinates [x_min, y_min, x_max, y_max] (dimensions 1-128).

[0062] Middle-level features: size parameters {width, height} (dimensions 129-256).

[0063] Deep features: component type probability distribution P(type) and component material (dimensions 512-768).

[0064] S22. Perform cross-modal projection through the MLP multi-layer perceptron to reduce the dimension of the structured feature tensor and output a cross-modal semantic embedding vector.

[0065] In one embodiment, the MLP multi-layer perceptron is as follows: Figure 4 As shown in the figure, the structured feature tensor is reduced in dimension through two fully connected layers and compressed into a 512-dimensional cross-modal semantic embedding vector. The design is based on: The first fully connected layer (768→1024): expands the dimension to separate the ViT model or improves the coupling characteristics of the structured feature tensor output by the ViT model (such as position and size correlation), reducing the interference of the wall position on the window width prediction; The second fully connected layer (1024→512): compresses redundant background information and strengthens the encoding of topological relationships (such as window-wall embedding dependency).

[0066] Input features of MLP: Receive the 768-dimensional structured feature tensor V_vis output by the ViT model or the improved ViT model with cross-scale attention gating fusion (its different dimension intervals encode key information such as component position, size, type, etc.

[0067] The MLP process includes: First fully connected layer (FC1): V_vis first passes through a fully connected layer with 1024 neuron nodes, performs preliminary linear transformation and nonlinear activation, and outputs the feature vector h: h = ReLU(W1 · V_vis + b1) Where W1 is a 768×1024-dimensional weight matrix, and b1 is a bias vector. The ReLU activation function introduces nonlinearity, helping the MLP model learn more complex feature mapping relationships while filtering out unnecessary information.

[0068] The second fully connected layer (FC2): further compresses and projects the 1024-dimensional feature vector h output by the first layer to the target dimension: V_sem = ReLU(W2 h + b2) Where W2 is a 1024×512 dimensional weight matrix, b2 is a bias vector, and the ReLU activation function is used again.

[0069] Output features: The final result is a 512-dimensional cross-modal semantic embedding vector V_sem. This vector V_sem is a compact representation of visual information in the semantic space understandable by the language model (LLM). It integrates and abstracts the component type identifiers (such as "wall" and "window"), precise geometric parameters (such as length and width), spatial position information, and potential topological dependencies (such as the window must be embedded in the wall) from the original visual features.

[0070] S23. Based on the cross-modal semantic embedding vector, a large language model (LLM) is used to generate a building parametric modeling script that can drive BIM software. In one embodiment, the large language model (LLM) receives the cross-modal semantic embedding vector and uses a topology-dependent autoregressive LLM to generate a component modeling script with spatial identity encoding. The script is then output as a Python building parametric modeling script that can drive BIM software.

[0071] 1. Pre-training stage: We fine-tuned the large language model InternLM2 on a 1.2TB Revit API script to achieve end-to-end generation of architectural modeling scripts. Specific functions include: grammatical rule learning: mastering the grammatical structure of the Revit API; component logical reasoning: understanding the topological dependencies between components (for example, the existence of windows depends on walls); and error avoidance mechanisms.

[0072] The learning API call sequence: CreateWall→AddWindow→AddDoor is designed to force the large language model InternLM2 to learn the construction topology order, perform topology dependency control, and solve the "30% component space conflict" problem in the background technology.

[0073] Topology dependency control: Input: Sequence of component types [Wall, Window, Door].

[0074] Output: API calls CreateWall() → AddWindow() → AddDoor() with order constraints.

[0075] Conditional generation mechanism: Autoregression generates enforced topological dependencies: # Generate order constraint pseudocode If the current component == "Window" and the parent wall is not generated: Skip the current token # Generate pseudocode for autoregression for each component in topological order: Generate code token = LLM(semantic vector, historical code) If the constraint is violated (e.g. width < 800) → skip the token Spatial identity encoding mechanisms include: 1. Generate a unique space ID for each component (e.g., NE104J25_W01 = Northeast 104 Block, Building 25, West Wall 01); 2. Enforce dependencies through attention masks during LLM decoding: If the parent wall ID does not appear when generating "AddWindow" → skip the current generation; The window opening position parameters use a relative coordinate ratio (loc_ratio) to ensure that the opening position is adaptively adjusted with the wall size. 3. Output script example: CreateWall(id="NE104J25_W01", length=8000) AddWindow(parent_id="NE104J25_W01", width=1200, loc_ratio=0.35) S3. Train architectural parametric modeling scripts to intelligently generate models.

[0076] A multi-task loss function mechanism that embeds building code constraint labels in the training of intelligent generation models for architectural parametric modeling scripts includes: S31. Design multi-task loss function.

[0077] In one embodiment, the multi-task loss function is a weighted sum of geometric parameter loss, component topology loss, code generation loss, and semantic alignment loss, including: Geometric parameter loss (L vis ): Mean square error loss between the predicted component size parameters (length, width, height) and the true values.

[0078] Code generation loss (L lang ): Cross-Entropy Loss between the predicted code token sequence and the true token sequence, which can give higher weight to key API call tokens (such as CreateWall, AddWindow).

[0079] Semantic consistency loss (L align ): Minimize the L2 distance (Euclidean distance) between the generated 512-dimensional cross-modal embedding vector and the standard embedding vector of the corresponding category in the component knowledge base.

[0080] Component topology loss (L topo ): By calculating the square difference between the cosine similarity and the preset relationship encoding (embedding 0.9 / adjacent 0.3), the spatial connection relationship between building components is enforced.

[0081] Multi-task loss function L total for: L total =1 / 4(L vis +L lang +L align +L topo ) S32. Design rules for building code constraint labels, penalty mechanisms for violations, verification mechanisms, and component semantic alignment methods: The training data embeds building code constraint labels, for example: "constraints": [{"type": "min_window_width", "value": 800}].

[0082] Violation penalty mechanism: When the generated parameters violate the building code constraints (such as window width < 800mm), the loss function weight is ×2.

[0083] Verification mechanism: A two-level verification mechanism is adopted. The pre-verification mechanism blocks illegal commands in real time, and the post-verification AST parser verifies the compliance of script parameters.

[0084] Component Semantic Alignment Method: Chinese-CLIP is used to align model ID text with ViT visual features. The component semantic alignment parsing function aims to achieve cross-modal mapping between visual features and component type labels through Chinese-CLIP, resolving component identification ambiguity (e.g., distinguishing between "casement window" and "sliding window") and ensuring that the visual features extracted by ViT accurately match architectural semantics. The model ID text refers to the type identifier in the component parameter table (e.g., "type": "wall" in the S13 example), and is the component category label extracted from the Revit model (e.g., "wall / window / door"). The ViT visual features are the 768-dimensional structured feature tensor output by S21. Its deep feature dimensions (512-768) carry the component type probability distribution P(type).

[0085] Chinese-CLIP is the Chinese-based version of CLIP (Contrastive Language-Image Pre-training). It is an open-source multimodal pre-training model for aligning Chinese text with visual features. Its core goal is to efficiently understand the semantic relationships between Chinese images and text, enabling tasks such as image-to-Chinese text matching, retrieval, and generation.

[0086] When using the Chinese-CLIP model for semantic alignment, there are usually two main steps involved: first, encoding text into embedding vectors; second, encoding visual features into embedding vectors and ensuring that they are aligned in the semantic space.

[0087] Component semantic alignment is mainly used in the training phase of the intelligent generation model of architectural parametric modeling scripts to align the text embedding vectors generated by Chinese-CLIP (from the model ID text) with the ViT visual features in the semantic space, thereby reducing the feature distance between components of the same category (e.g., clustering the visual features of all "windows").

[0088] S33. Input data: Drone images: Based on 368 real-world projects, expanded to 500,000 training samples through rotation (±10°), scaling (0.8x-1.2x), and illumination adjustment (±20%).

[0089] Component parameter annotation pairs: 50,000 sets (including size / type / topological relationships) of component parameter true values, for example: {"type":"window","width":1200,"height":1500}.

[0090] S34. Model training process As shown in Table 1, the model training process for intelligent generation of architectural parametric modeling scripts includes: 1. Pre-training (500,000 images): Freeze ViT model weights (pre-trained with ImageNet).

[0091] Only train MLP+InternLM2 (learning rate 1e-4).

[0092] 2. Joint fine-tuning (50,000 sets of component parameter annotation pairs): Unfreeze the last 4 layers of the ViT model and fully train the MLP / LLM (learning rate 5e-5); Batch Size=32, Gradient Clipping Threshold=1.0.

[0093] Table 1 Introduction to model training data Model selection evaluation criteria: Script execution rate ≥ 99% (Revit API no errors); Dimensional accuracy: window width / door height error ≤5mm; Rule compliance rate: The proportion of violation parameters is <0.1%.

[0094] S4. Intelligent generation of model reasoning for architectural parametric modeling scripts (end-to-end automation).

[0095] S41. Input preprocessing: Real-time processing chain (same as training phase): original image → building outline segmentation → output 1024×1024 standard image.

[0096] S42, ViT model visual parsing; The structured feature tensor output by the ViT model can accurately analyze parametric components, such as window frame boundary coordinates, window width parameters, and window-wall embedding topology relationships.

[0097] Input: Standard image → Output: 768-dimensional structured feature tensor.

[0098] The different dimensions of the structured feature tensor encode different levels of visual information, such as the overall structure of the building (such as floor height and outer contour), the spatial location of components (such as the bounding box coordinates of doors and windows), and the geometric properties of components (such as width, height, and curvature parameters).

[0099] { "bbox": [103, 205, 603, 555], / / wall coordinates "type": "Wall", / / component type "params": {"length": 5000}, / / mm-level size "confidence": 0.92 / / confidence} S43, MLP cross-modal projection: The cross-modal embedding vector contains the encoding of component type, geometric parameters and other information: h = ReLU(W1 · V_vis + b1)→V_sem = ReLU(W2 · h + b2) Dynamic projection: 768 dimensions → 512 dimensions (fully connected layer + ReLU activation).

[0100] S44,LLM generates Python architectural parametric modeling scripts that can drive BIM software.

[0101] Autoregressive generation process: initialization: <bos>Token; Generate basic wall: CreateWall(id="NE104J25_W01", length=8000); Add doors and windows: AddWindow(parent="NE104J25_W01", width=1200); Finish: <eos>token.

[0102] In a preferred embodiment, the Python architectural parametric modeling script generated by the LLM and capable of driving the BIM software may also be verified using the verification mechanism in S32, and the Python architectural parametric modeling script that passes the verification may be output.

[0103] S5. Based on LLM, a Python architectural parametric modeling script that can drive BIM software is generated, and BIM software is used to generate a BIM model that supports parametric linkage updates.

[0104] S51. Based on BIM software, input the Python architectural parametric modeling script generated by LLM that can drive the BIM software, and output the BIM model and metadata that support parametric linkage updates.

[0105] In one embodiment, based on the BIM software, a verified Python architectural parametric modeling script may also be input.

[0106] Execution: Automatically run in Revit via the PyRevit plugin.

[0107] Output: .rvt file + _metadata.json project file containing the parametric family instance and its complete parametric modeling history.

[0108] .rvt file data structure: Output data structure: { "components": [ { "Type": "Wall", "Space Identity Code": "NE104J25_W01", "parameter": {"length": 8000}, "Material ID": "C35 Concrete" }, { "Type": "Window", "Space Identity Code": "WIN_NE104J25_001", "parameters": {"width": 1200} } ] } JSON example: { "components": [ { "type": "Wall", "id": "W01", "params": {"length": 8000, "height": 3500}, "material_id": "A36" }, { "type": "Window", "id": "WIN_001", "parent_id": "W01", / / Clear parent-child relationship "params": {"width": 1200, "height": 1500} } ] } S52, parameterized linkage update example: ① User operation: set_parameter("WIN_NE104J25_001", "width", 1500) ② Compliance verification: if width < MIN_WINDOW_WIDTH(800): raise ConstraintViolation ③ Topology linkage: update_opening_location( wall_id="NE104J25_W01", window_id="WIN_001", offset_x=300 # Move the hole center 300mm to the right) S53. Overall process summary: Component-level modeling process: like Figure 6 As shown, with the window component as the center, the existence of the wall must be determined before it can be generated. Modifications after generation are also subject to the input rules. The autoregressive decoding process of the present invention realizes the end-to-end conversion from visual features to executable scripts. The topological dependency control method forces the component generation order to follow the architectural logic of "wall → window → door", and accurately binds the parent-child component relationship through spatial identity encoding (such as NE104J25_W01). During the decoding process, the attention mask verifies the dependency relationship in real time. When the parent wall ID is not generated when "AddWindow" is detected, the current token is immediately skipped. At the same time, the relative coordinate ratio parameter (such as loc_ratio=0.35) is used to ensure that the opening position is adaptively offset with the wall size.

[0109] Parametric modeling results: The BIM model that supports parameterized linkage update output by the present invention is as follows: Figure 7 As shown, components are described parametrically, each carrying complete BIM semantics: bounding box coordinates precisely locate window frame positions, geometric parameters record millimeter-level dimensions like window width and door height, and the parent wall ID field clearly defines the window-wall embedding topology. After executing the generated Python script in Revit, the output .rvt model file supports dynamic, linked updates—modifying window width parameters automatically triggers opening position shifts, reducing design change time and ensuring the resulting model fully complies with building code requirements.

[0110] A procedural architectural modeling system based on drone imagery, comprising: Dataset construction module: used to construct a paired dataset of construction project collection based on the drone building image dataset, which is "drone image ←→ component parameter table ←→ parametric modeling script" as the training dataset for the intelligent generation model of the building parametric modeling script.

[0111] Script intelligent generation model construction module: Build and train the architectural parametric modeling script intelligent generation model: The architectural parametric modeling script intelligent generation model includes: ViT model, MLP multi-layer perceptron and LLM large language model, among which: ViT model extracts visual features, MLP multi-layer perceptron performs cross-modal projection compression, and LLM large language model generates architectural parametric modeling scripts that can drive BIM software.

[0112] Script intelligent generation model reasoning module: Architectural parametric modeling script intelligently generates model reasoning, inputs drone images, and outputs architectural parametric modeling scripts that can drive BIM software.

[0113] BIM model generation module: Based on the architectural parametric modeling script that can drive BIM software, BIM software is used to generate a BIM model that supports parametric linkage updates.

[0114] In a preferred embodiment, the ViT model is an improved ViT model of cross-scale attention gated fusion. The improved ViT model of cross-scale attention gated fusion is to add a cross-scale attention module and a gated fusion module to the standard ViT model. Multi-level feature interactions are established through the cross-scale attention module, and the association between shallow details and deep semantics is established. The cross-scale attention module is combined with the gated fusion module to realize multi-level feature adaptive fusion.

[0115] While various embodiments of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.< / eos> < / bos>

Claims

1. A method for procedural building modeling based on drone images, characterized in that: include: Based on the drone building image dataset, a building project collection paired dataset of "drone image ←→ component parameter table ←→ parametric modeling script" is constructed as the training dataset for the intelligent generation model of building parametric modeling scripts; Build and train an intelligent generation model for architectural parametric modeling scripts: This model includes the ViT model, an MLP multi-layer perceptron, and an LLM large language model. The ViT model extracts visual features, the MLP multi-layer perceptron performs cross-modal projection compression, and the LLM large language model generates architectural parametric modeling scripts that can drive BIM software. Intelligent generation of architectural parametric modeling scripts for model reasoning: input drone images and output architectural parametric modeling scripts that can drive BIM software; Based on the architectural parametric modeling script that can drive BIM software, BIM software is used to generate a BIM model that supports parametric linkage updates.

2. The architectural program modeling method according to claim 1, characterized in that: The ViT model is an improved ViT model with cross-scale attention gated fusion. The improved ViT model with cross-scale attention gated fusion adds a cross-scale attention module and a gated fusion module to the standard ViT model. The cross-scale attention module establishes multi-level feature interactions and the association between shallow details and deep semantics. The cross-scale attention module is combined with the gated fusion module to realize multi-level feature adaptive fusion.

3. The architectural program modeling method according to claim 1, characterized in that: The intelligent model generation process of architectural parametric modeling scripts includes: The ViT model is used to extract multi-level features of building components from drone images, which are converted into structured data through a lightweight parsing layer and output as structured feature tensors. Through the MLP multi-layer perceptron, cross-modal projection is performed to reduce the dimension of the structured feature tensor and output a cross-modal semantic embedding vector; Based on cross-modal semantic embedding vectors, a large language model (LLM) is used to generate architectural parametric modeling scripts that can drive BIM software.

4. The architectural program modeling method according to claim 1, characterized in that: The ViT model outputs a structured feature tensor, which includes three levels of features: Shallow texture: Identify component edge features → Architectural semantics: Doors, windows, openings, and other component boundaries; Middle-level geometry: parsing component size attributes → architectural semantics: window and wall embedding topology; Deep semantics: Aggregate component type identification → Architectural semantics: Component type identification such as window / door / balcony.

5. The architectural program modeling method according to claim 1, characterized in that: The MLP multi-layer perceptron compresses the structured feature tensor into a cross-modal semantic embedding vector through two fully connected layers, where: The first fully connected layer is used to expand the dimension to separate the coupled features of the structured feature tensor and reduce the interference of the wall position on the window width prediction; The second fully connected layer is used to compress redundant background information and strengthen topological relationship encoding.

6. The architectural program modeling method according to claim 1, characterized in that: The Large Language Model (LLM) uses topology-dependent controlled autoregressive LLM to generate component modeling scripts with spatial identity encoding.

7. The architectural program modeling method according to claim 1, characterized in that: A multi-task loss function mechanism is used to embed building code constraint labels in the training process of the intelligent generation model of architectural parametric modeling scripts.

8. The architectural program modeling method according to claim 7, characterized in that: The multi-task loss function is a weighted sum of geometric parameter loss, component topology relationship loss, code generation loss and semantic alignment loss.

9. A procedural architectural modeling system based on drone images, characterized in that: include: Dataset construction module: used to construct a paired dataset of building project acquisitions based on the drone building image dataset, which is "drone image ←→ component parameter table ←→ parametric modeling script" as the training dataset for the intelligent generation model of building parametric modeling scripts; Script intelligent generation model construction module: Build and train the architectural parametric modeling script intelligent generation model: The architectural parametric modeling script intelligent generation model includes: ViT model, MLP multi-layer perceptron and LLM large language model, among which: ViT model extracts visual features, MLP multi-layer perceptron performs cross-modal projection compression, and LLM large language model generates architectural parametric modeling scripts that can drive BIM software; Script intelligent generation model reasoning module: Architectural parametric modeling script intelligent generation model reasoning, input drone images, and output architectural parametric modeling scripts that can drive BIM software; BIM model generation module: Based on the architectural parametric modeling script that can drive BIM software, BIM software is used to generate a BIM model that supports parametric linkage updates.

10. The architectural procedural modeling system according to claim 9, characterized in that: The ViT model is an improved ViT model with cross-scale attention gated fusion. The improved ViT model with cross-scale attention gated fusion adds a cross-scale attention module and a gated fusion module to the standard ViT model. The cross-scale attention module establishes multi-level feature interactions and the association between shallow details and deep semantics. The cross-scale attention module is combined with the gated fusion module to realize multi-level feature adaptive fusion.

Images