BIM dynamic modeling method and system based on image intelligent recognition

Abstract

This invention discloses a BIM dynamic modeling method and system based on image intelligent recognition, belonging to the field of modeling technology. The method includes: acquiring multimodal image data of a building scene and performing image intelligent recognition to extract multimodal data of building components; performing depth estimation and 3D geometric reconstruction on the multimodal data to obtain 3D geometric information; constructing a BIM engineering semantic knowledge base and aligning and mapping visual semantic information with the BIM engineering semantic knowledge base to obtain engineering semantic information; generating parameterized BIM components based on 3D geometric information and engineering semantic information, and establishing spatial relationships between components to form a BIM model. This invention solves the problems of BIM dynamic modeling being highly dependent on manual work, difficult to simultaneously obtain component engineering semantics and accurate 3D spatial relationships, and insufficient adaptability to complex scenes and new components.

Description

Technical Field

[0001] This invention relates to the field of modeling technology, and more specifically, to a BIM dynamic modeling method and system based on image intelligent recognition. Background Technology

[0002] Building Information Modeling (BIM), as a crucial digital technology in the field of architectural engineering, can express the geometric information, engineering semantic information, and spatial relationships between building components in a parametric and structured manner. It has been widely applied in architectural design, construction management, and operation and maintenance phases. Existing BIM models are typically generated manually or through semi-automatic modeling methods based on design drawings. This involves professionals creating components one by one in BIM software and setting their parameters and spatial relationships according to two-dimensional construction drawings or design documents. This approach highly relies on human experience and expertise, resulting in low modeling efficiency, long cycles, and difficulty in quickly and accurately constructing BIM models for complex or existing building scenarios.

[0003] With the development of computer vision and deep learning technologies, image- or point cloud-based building information extraction methods have gradually gained attention. Some existing technologies attempt to use single-modal 2D images or 3D point cloud data to perform target detection or geometric reconstruction of building components, thereby assisting BIM modeling. However, these methods typically only focus on the geometric shape or category identification of components, making it difficult to simultaneously obtain complete engineering semantic information and accurate spatial relationship descriptions; especially when relying solely on 2D images, the lack of spatial depth information can easily lead to inaccurate component dimensions and relative positions, while relying solely on point cloud data makes it difficult to effectively distinguish component categories and their engineering semantics.

[0004] Furthermore, many existing deep learning-based BIM-assisted modeling methods employ fixed rules or predefined mapping relationships, directly mapping the visual results obtained from recognition to BIM component types. This approach is highly dependent on training data and existing BIM models, and its adaptability and generalization capabilities are insufficient when facing new components, complex structures, or different engineering specifications, making it difficult to achieve dynamic expansion and updating of engineering semantics. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a BIM dynamic modeling method and system based on image intelligent recognition. By integrating multimodal image intelligent recognition, cross-modal engineering semantic alignment, and graph neural network spatial relationship reasoning, the method solves the problems of BIM dynamic modeling being highly dependent on manual work, having difficulty simultaneously acquiring component engineering semantics and accurate three-dimensional spatial relationships, and being insufficiently adaptable to complex scenes and new components.

[0006] To achieve the above objectives, the present invention provides the following technical solution: Firstly, this application provides a BIM dynamic modeling method based on image intelligent recognition. The method includes: acquiring multimodal image data of a building scene and performing image intelligent recognition to extract multimodal data of building components, wherein the multimodal data includes visual semantic information and spatial feature information; performing depth estimation and three-dimensional geometric reconstruction on the multimodal data to obtain three-dimensional geometric information of the building components; constructing a BIM engineering semantic knowledge base, aligning and mapping the visual semantic information with the BIM engineering semantic knowledge base to obtain engineering semantic information corresponding to the BIM component type; and generating parameterized BIM components based on the three-dimensional geometric information and engineering semantic information, and establishing spatial relationships between components to form a BIM model.

[0007] In one embodiment, multimodal image data of a building scene is acquired and intelligent image recognition is performed to extract multimodal data of building components. Specifically, the multimodal image data is preprocessed to construct a multimodal feature joint input, which is then input into a pre-trained multimodal image intelligent recognition model. The multimodal image data includes two-dimensional visible light images and depth images. Based on the multimodal image intelligent recognition model, target detection and semantic recognition are performed on the building components, outputting initial visual semantic information and pixel-level masks. The initial visual semantic information includes category labels. Based on the model output results, contour extraction is performed on each building component, representing the contour of each component as a set of boundary points to obtain component contour information corresponding to the category labels, forming complete visual semantic information. Based on the component contour information, it is mapped to the depth image to extract three-dimensional spatial feature data corresponding to each building component. Based on the three-dimensional spatial feature data, the spatial relationship between different building components is obtained to form spatial feature information of the building components.

[0008] In one embodiment, based on the model output, a contour extraction operation is performed on each building component, representing the contour of each component as a set of boundary points to obtain component contour information corresponding to the category label. Specifically, based on the model output, a pixel-level mask corresponding to the category label is obtained; based on the pixel-level mask, the pixel region of each building component is processed independently, and the component region of the building component in the two-dimensional visible light image is determined by traversing the mask pixels; according to the component region, boundary detection processing is performed on the pixel-level mask to identify the boundary pixels located at the edge of the component region; the boundary pixels are contour tracked according to the spatial adjacency relationship between the boundary pixels to form a continuous boundary curve; the continuous boundary curve is converted into a set of boundary points composed of multiple boundary points; the set of boundary points is associated and stored with the corresponding category label to form component contour information.

[0009] In one embodiment, depth estimation and 3D geometric reconstruction are performed on multimodal data to obtain the 3D geometric information of building components. Specifically, depth estimation is performed according to different component categories based on visual semantic information, wherein the depth estimation adopts a multi-view depth fusion and spatial consistency optimization method; the depth estimation results are combined with the 2D coordinates of the original image to generate a component-level 3D point cloud; the component-level 3D point cloud is triangulated to generate a 3D mesh model; the parametric geometric information of each component is extracted from the 3D mesh model and integrated to obtain the 3D geometric information.

[0010] In one embodiment, depth estimation is performed according to different component categories based on visual semantic information. The depth estimation employs a multi-view depth fusion and spatial consistency optimization method, specifically: each component instance in the image is separated according to the component category label; a preliminary depth map is generated for each component instance using a multi-view depth estimation method to obtain a preliminary component-level depth value; for each preliminary depth map, depth fusion is performed by combining images from different perspectives to generate a fused component-level depth map; based on the component-level depth map and combined with the spatial feature information between components, global spatial consistency optimization is performed on the component depth.

[0011] In one embodiment, constructing a BIM engineering semantic knowledge base specifically involves: collecting historical BIM models and their corresponding building images; preprocessing the building images and inputting the processed images into a self-supervised learning model; constructing a self-supervised task based on the relationships between components and training the model to extract component feature vectors; inputting images of new building projects into the trained self-supervised learning model to predict component feature vectors and potential semantic information; generating dynamic knowledge base entries based on the predicted component feature vectors and potential semantic information; and comparing and merging the dynamic knowledge base entries with historical BIM models to generate the BIM engineering semantic knowledge base.

[0012] In one embodiment, visual semantic information is aligned and mapped with a BIM engineering semantic knowledge base to obtain engineering semantic information corresponding to BIM component types. Specifically, this involves: calling the BIM engineering semantic knowledge base to obtain BIM component entries; performing feature encoding on the BIM component entries to generate BIM component feature vectors; inputting the visual semantic information into a visual encoder to generate visual feature vectors; inputting the BIM component feature vectors into a BIM encoder to generate BIM feature vectors; mapping the visual feature vectors and BIM feature vectors to a unified embedding space, and using the visual feature vectors and their corresponding BIM component feature vectors as positive samples and non-corresponding features as negative samples for training; after training, mapping the visual feature vectors to be matched to the unified embedding space and calculating the similarity between the visual feature vectors to be matched and the BIM feature vectors; selecting the BIM component entry with the highest similarity as the matching result; and combining the BIM component entry information with the visual semantic information based on the matching result to generate engineering semantic information for each component.

[0013] In one embodiment, parametric BIM components are generated based on three-dimensional geometric information and engineering semantic information, and spatial relationships between components are established to form a BIM model. Specifically, parametric BIM components for each building component are generated based on three-dimensional geometric information and engineering semantic information; spatial relationships are established based on graph neural networks according to the parametric BIM components; and the parametric BIM components and spatial relationships are integrated to form a complete BIM model.

[0014] In one embodiment, spatial relationships are established based on graph neural networks (GNNs) according to parametric BIM components. Specifically, the features of each parametric BIM component are vectorized and used as graph nodes; graph edges are established based on the adjacency, support, and hierarchical relationships between components to form a component spatial relationship graph; the component spatial relationship graph is input into the GNN, and node and edge features are updated and trained using a message passing mechanism; the trained GNN is applied to the component spatial relationship graph of the BIM model to be generated to predict the spatial relationship between each component node and other nodes; the prediction results are used as the preliminary spatial layout between components, and the position, orientation, and hierarchical relationship of the components in three-dimensional space are optimized and adjusted in combination with engineering semantic constraints and geometric constraints; the component spatial layout and topological relationships after GNN inference and optimization are integrated to generate the spatial relationships of the complete BIM model.

[0015] Secondly, this application provides a BIM dynamic modeling system based on image intelligent recognition. The system includes: an image recognition module for acquiring multimodal image data of a building scene and performing image intelligent recognition to extract multimodal data of building components, wherein the multimodal data includes visual semantic information and spatial feature information; a 3D information construction module for performing depth estimation and 3D geometric reconstruction on the multimodal data to obtain the 3D geometric information of the building components; a mapping module for constructing a BIM engineering semantic knowledge base, aligning and mapping the visual semantic information with the BIM engineering semantic knowledge base to obtain engineering semantic information corresponding to the BIM component type; and a BIM model generation module for generating parametric BIM components based on the 3D geometric information and engineering semantic information, establishing spatial relationships between components, and forming a BIM model.

[0016] As can be seen from the above technical solutions, the embodiments of this application have the following advantages: This solution enables fully automated and closed-loop modeling of the entire process, from multimodal building image acquisition, component semantic recognition, 3D geometric reconstruction, engineering semantic alignment to parametric BIM component generation and spatial relationship reasoning, significantly improving the accuracy, consistency, and engineering usability of BIM model generation. It not only deeply integrates visual semantics from 2D visible light images with spatial structural information from depth images to achieve high-precision 3D geometric reconstruction at the component level, but also dynamically constructs and updates the BIM engineering semantic knowledge base through self-supervised learning and cross-modal comparative learning. This allows visual semantics to automatically align to BIM component types and parametric attributes, fundamentally breaking through the limitations of traditional technologies that rely on manual modeling, fixed rules, or static BIM templates. Furthermore, it introduces graph neural networks for intelligent reasoning of adjacency, support, and hierarchical relationships between components, and combines engineering semantic constraints and geometric constraints for global optimization. This ensures that the generated BIM model is highly consistent in topology, spatial layout, and engineering logic, possessing stronger robustness, scalability, and adaptability to complex building scenarios. Therefore, it significantly improves upon conventional technologies in terms of automation, semantic understanding depth, and engineering implementation capability. Attached Figure Description

[0017] Figure 1 A schematic diagram of the BIM dynamic modeling method based on image intelligent recognition provided in the embodiments of this application.

[0018] Figure 2 A schematic diagram of the structure of a BIM dynamic modeling system based on image intelligent recognition provided in an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0020] Reference Figure 1 As shown in the diagram, the BIM dynamic modeling method based on image intelligent recognition provided by this invention includes the following steps: S1. Acquire multimodal image data of the building scene and perform intelligent image recognition to extract multimodal data of building components. The multimodal data includes visual semantic information and spatial feature information. The visual semantic information includes the category label and outline information of the components, and the spatial feature information includes the relative positional relationship of the components.

[0021] The multimodal image data includes two-dimensional visible light images and depth images used to characterize spatial structures. The multimodal image data is acquired through fixed or mobile acquisition devices, which include visible light cameras and depth sensing devices. These devices are capable of acquiring multimodal image data in real time and dynamically, and recording timestamps during acquisition to achieve time synchronization. The multimodal image data is also spatially registered based on preset camera calibration parameters.

[0022] In this embodiment, multimodal image data of the building scene is acquired, and intelligent image recognition is performed to extract multimodal data of building components, specifically as follows: The multimodal image data is preprocessed, including image denoising, scale normalization, distortion correction, and resolution alignment. Based on the processed multimodal image data, a multimodal feature joint input is constructed. The two-dimensional visible light image is used as the main input for visual semantic information, and the depth image is used as the main input for spatial structure information. These inputs are then fed into a pre-trained multimodal image intelligent recognition model. The multimodal image intelligent recognition model includes a convolutional neural network sub-network for processing two-dimensional visible light images to extract low-level visual features of building components, and a convolutional feature extraction sub-network for processing depth images to extract spatial feature representations reflecting the spatial structure of the components. The model includes an input module, a feature fusion module, and an output module. The input module takes a two-dimensional visible light image as input, extracts low-level visual features (texture, edges, color distribution) through convolutional layers, and extracts spatial feature representations of the components from the depth image through convolution, used to assist in classification and mask recognition. The feature fusion module employs feature-level fusion, weighting and fusing low-level visual features and spatial feature representations in an intermediate layer using a Transformer attention mechanism. The output module outputs initial visual semantic information and pixel-level masks (for subsequent contour extraction) corresponding to each building component. The initial visual semantic information includes at least the category label of the building component (e.g., wall, column, beam, window, door). The model training parameters use images of labeled building components and corresponding depth map datasets.

[0023] Based on a multimodal image intelligent recognition model, target detection and semantic recognition are performed on building components in an architectural scene, and the initial visual semantic information and pixel-level mask corresponding to each building component are output. The initial visual semantic information includes the category label of the building component. Based on the model output, a contour extraction operation is performed on each building component, and the contour of each component is represented as a set of boundary points to obtain the component contour information corresponding to the category label, thus forming complete visual semantic information. Based on the component outline information, it is mapped to a depth image, and three-dimensional spatial feature data corresponding to each building component is extracted. The three-dimensional spatial feature data includes centroid coordinates, voxel distribution, and three-dimensional coordinates of boundary points. Based on the component outline information, the pixel coordinates of the boundary point set in the two-dimensional visible light image are first used as input. Combined with the spatially registered depth image, the two-dimensional boundary points are mapped to the corresponding pixel positions in the depth image, thereby obtaining the depth value corresponding to each boundary point. On this basis, the two-dimensional pixel coordinates and the corresponding depth values ​​are converted into three-dimensional spatial coordinates to form a three-dimensional coordinate set of the boundary points of the building component. At the same time, three-dimensional coordinate reconstruction is performed on all effective depth pixels inside the mapped component. Based on the three-dimensional coordinate set, the centroid coordinates of the component are obtained as the position representation of the component in three-dimensional space. By dividing the three-dimensional coordinate point set of the component into a preset spatial grid, its occupancy in each voxel unit is statistically analyzed to obtain the voxel distribution reflecting the spatial shape and occupancy range of the component, thereby completing the extraction of three-dimensional spatial feature data corresponding to each building component.

[0024] Based on three-dimensional spatial feature data, the spatial relationships between different building components are obtained to form spatial feature information of building components. The spatial relationships include the relative positions, distance relationships, vertical relationships and adjacency relationships between components.

[0025] Furthermore, based on the model output, a contour extraction operation is performed on each building component, representing the contour of each component as a set of boundary points to obtain the component contour information corresponding to the category label, specifically: Based on the model output, the pixel-level mask corresponding to the category label of each building component is obtained. The pixel-level mask is used to identify pixel regions belonging to the same building component in a two-dimensional visible light image. Based on pixel-level masks, the pixel region of each building component is processed independently, and the component region in the two-dimensional visible light image is determined by traversing the mask pixels. Based on the determined component region, a boundary detection process is performed on the pixel-level mask. The boundary detection process includes: determining whether there is at least one adjacent pixel in the component region that does not belong to the component region within a preset neighborhood, and identifying the boundary pixel located at the edge of the component region. Contour tracing is performed on the boundary pixels according to the spatial adjacency relationship between the boundary pixels to form a continuous boundary curve that is connected in sequence and closed at both ends, which is used to represent the outer contour of the building component in the two-dimensional visible light image. The resulting continuous boundary curve is converted into a set of boundary points consisting of multiple boundary points, where each boundary point is represented in the form of two-dimensional image coordinates. The set of boundary points is arranged in the outline order to completely describe the two-dimensional outline shape of the building component. The boundary point set is associated with the corresponding category label and stored, establishing a one-to-one correspondence between the category label of each building component and its corresponding boundary point set, thus forming the component outline information of the building component.

[0026] It should be noted that by extracting multimodal data of building components, the visual semantic information in two-dimensional visible light images and the spatial feature information reflected in depth images are organically integrated. This allows each building component to not only have clear category semantics and accurate two-dimensional contour description, but also quantifiable three-dimensional spatial position, shape and relative relationship expression. This effectively overcomes the problem of incomplete semantic understanding or missing spatial information caused by relying on only a single image modality.

[0027] S2 performs depth estimation and 3D geometric reconstruction on multimodal data to obtain the 3D geometric information of building components.

[0028] In this embodiment, depth estimation and 3D geometric reconstruction are performed on the multimodal data to obtain the 3D geometric information of the building components, specifically: Based on visual semantic information, depth estimation is performed according to different component categories. The depth estimation adopts a multi-view depth fusion and spatial consistency optimization method. The depth estimation results (optimized depth map) are combined with the two-dimensional coordinates of the original image to generate a component-level three-dimensional point cloud. Spatial feature information is used to adjust the relative positional relationship of the component-level three-dimensional point cloud to ensure that the topological relationship of each component in the building space is correct. The original image refers to unprocessed two-dimensional image data that is directly photographed or collected from the building scene. Triangulation is performed on the component-level 3D point cloud to generate a 3D mesh model; Among these methods, visual semantic information is used to constrain the grid boundary so that the grid matches the component outline; spatial feature information is used to optimize the connection and alignment between components so as to ensure that the spatial relationship between components conforms to the logic of architectural engineering.

[0029] The parametric geometric information of each component is extracted from the three-dimensional mesh model. The parametric geometric information includes the component size, position, orientation, floor height and spatial topology. Parametric geometric information is integrated to form complete three-dimensional geometric information of building components, which is then used for subsequent BIM modeling.

[0030] Furthermore, based on visual semantic information, depth estimation is performed separately according to different component categories. The depth estimation employs multi-view depth fusion and spatial consistency optimization methods, specifically: Based on the component's category label, separate each component instance in the image; For each component instance, a preliminary depth map is generated using a multi-view depth estimation method to obtain the component-level preliminary depth value; For each preliminary depth map, depth fusion is performed by combining images from different perspectives to generate a fused component-level depth map; Based on the component-level depth map and combined with the spatial feature information between components, global spatial consistency optimization of component depth is performed.

[0031] Global spatial consistency optimization includes correcting depth drift, preserving component topology, and smoothing the boundaries of adjacent components. Correcting depth drift involves treating the depth values ​​of the same component from different viewpoints as the optimization target. By calculating the depth differences between pixels within a component and minimizing these differences, the overall depth of the component remains consistent, thus correcting the drift. Preserving component topology involves introducing spatial constraints during optimization to ensure that the topological relationships between adjacent components are not disrupted. For example, columns are located below beams, walls are perpendicular to the ground, and doors and windows are embedded in walls, preventing misalignment or overlap errors caused by depth adjustments. Smoothing involves smoothing the boundary pixels of adjacent components or within the same component. Filtering reduces abrupt depth changes, making component boundaries continuous and natural, avoiding jagged or irregular edges.

[0032] It should be noted that by closely integrating visual semantic information and spatial feature information, high-precision reconstruction from two-dimensional architectural images to component-level three-dimensional geometric information is achieved: by performing depth estimation, multi-view depth fusion, and spatial consistency optimization according to component categories, not only is depth drift corrected and depth estimation accuracy improved, but the topological relationships and boundary continuity between components are also guaranteed; the optimized depth map is used to generate component-level three-dimensional point clouds, which are then triangulated and meshed. By combining visual semantic constraints and spatial features to optimize mesh boundaries and component alignment, parametric geometric information is extracted and integrated, ultimately forming complete three-dimensional geometric information of architectural components. This significantly improves the accuracy, reliability, and engineering usability of digital architectural modeling, while reducing the workload of manual modeling.

[0033] S3. Construct a BIM engineering semantic knowledge base, align and map visual semantic information with the BIM engineering semantic knowledge base, and obtain engineering semantic information corresponding to BIM component types.

[0034] In this embodiment, the construction of a BIM engineering semantic knowledge base is specifically as follows: Collect historical BIM models and their corresponding building images. The historical BIM models include component categories, geometric parameters, spatial locations, and topological relationships. The architectural images are preprocessed, including image scaling, normalization, data augmentation, and color channel normalization, in order to be input into the self-supervised model; The processed building image is input into a self-supervised learning model, which is a convolutional neural network, a visual Transformer, or a contrastive learning network. Based on the relationships between components, a self-supervised task is constructed. The model is trained by minimizing the loss function (such as MSE) to extract component feature vectors. The self-supervised task includes image reconstruction task, rotation prediction task, and occlusion recovery task. The component feature vector includes component category, shape, texture, and spatial features. In the image reconstruction task, a preprocessed architectural image is input into a self-supervised network structure containing an encoder and a decoder. The encoder performs layer-by-layer feature compression on the input image, extracting latent feature representations reflecting the texture, edges, shape, and spatial layout of the components. The decoder performs pixel-level reconstruction of the original image based on these latent features. By minimizing the pixel difference loss between the reconstructed image and the original image, the model learns the local detail features and overall structural features of the architectural components without manual annotation. In the rotation prediction task, a random rotation transformation (e.g., 0°, 90°, 180°, or 270°) from a preset set of angles is applied to the input architectural image, and the rotated image is input into the model. In the training model, the corresponding rotation angle category is output. By minimizing the classification loss between the predicted rotation angle and the actual rotation angle, the model learns the orientation, geometric symmetry, and spatial structural features of building components, thereby enhancing its ability to distinguish the orientation and morphological differences of components. In the occlusion restoration task, local areas of the input image are occluded according to preset proportions and positional rules to form image samples containing missing regions. These samples are then input into the model, which is required to predict and restore the pixel content of the occluded regions based on the contextual information of the unoccluded regions. By minimizing the reconstruction error between the restored result and the corresponding region in the original image, the model learns the texture continuity of building components and the spatial relationships between components.

[0035] The trained self-supervised learning model is applied to images of new building projects to predict component feature vectors and potential semantic information. Dynamic knowledge base entries are generated based on the predicted component feature vectors and potential semantic information. The dynamic knowledge base entries include component category, parametric geometric attributes, spatial location, floor height, and topological relationship. The generation of dynamic knowledge base entries involves instance differentiation and normalization of the component feature vectors output by the self-supervised model to eliminate the influence of scale, lighting, and viewpoint differences. Subsequently, the component feature vectors are compared with component features in historical BIM models to calculate similarity. Several BIM component categories with similarity higher than a preset threshold are selected as candidate component categories, and potential new components are identified when no matching results are found. Based on this, parametric geometric attributes, including size, spatial location, orientation, and floor height, are extracted by combining the component's corresponding 3D geometric information. Simultaneously, based on the spatial distance, contact relationship, and height order between components, adjacency (spatial distance less than the design distance threshold), support, and hierarchical topological relationships are inferred and encoded. Finally, the component category candidate results, parametric geometric attributes, spatial location, and topological relationships are structurally encapsulated to form complete dynamic knowledge base entries.

[0036] The dynamic knowledge base entries are compared and merged with the historical BIM model to generate a BIM engineering semantic knowledge base. Specifically, by obtaining the cosine similarity between the dynamic knowledge base entries and the component entries in the historical BIM model in terms of category, geometric features, spatial location, and topological relationship, entries with a matching degree higher than a preset threshold are merged and updated, while entries with a matching degree lower than the preset threshold or new entries are added as new entries, thereby generating a complete and consistent BIM engineering semantic knowledge base.

[0037] Specifically, the BIM engineering semantic knowledge base is organized into a structured data format, which includes IFC files, JSON files, or database entries, and can be directly used for BIM parametric modeling or aligned with existing BIM models.

[0038] It should be noted that the constructed BIM engineering semantic knowledge base can automatically align building images with BIM component types, dynamically generating, structuring, and standardizing visual features, geometric parameters, spatial location, and topological relationship information, significantly improving the adaptability and recognition accuracy of new or unknown components. Compared with conventional methods that rely on fixed BIM models and manual annotation, this method automatically extracts component features and generates dynamic knowledge base entries through self-supervised learning. At the same time, it combines cosine similarity comparison and merging optimization to achieve real-time updates and closed-loop expansion of the knowledge base, possessing higher automation, robustness, and scalability, greatly improving the efficiency of BIM parametric modeling and its engineering application value.

[0039] Furthermore, the visual semantic information is aligned and mapped with the BIM engineering semantic knowledge base to obtain engineering semantic information corresponding to the BIM component type, specifically: The BIM engineering semantic knowledge base is invoked to obtain BIM component entries, which include component category, geometric parameters, spatial location and topological relationship; Feature encoding is performed on BIM component items to generate BIM component feature vectors. The BIM component feature vectors include category, geometry, size, spatial relationship and topological features, providing input for cross-modal comparative learning. Visual semantic information is input into the visual encoder to generate visual feature vectors. The visual encoder uses a visual Transformer to generate vectors containing category, shape, texture and spatial features. The feature dimension can be preset to 128 to 512. The BIM component feature vector is input into the BIM encoder to generate a BIM feature vector. The BIM component encoder encodes the component's geometric dimensions and spatial topological relationships into 128 to 512-dimensional feature vectors so that they can be aligned with visual features in a unified embedding space. Map visual feature vectors and BIM feature vectors to a unified embedding space; Visual feature vectors and corresponding BIM component feature vectors are considered positive samples, while non-corresponding features are considered negative samples. A cross-modal contrastive learning model is trained using a cross-modal contrastive learning method to optimize the InfoNCE loss, ensuring that the distance between positive samples and the corresponding BIM component feature vectors in the embedding space is less than a preset distance threshold, while the distance between negative samples and the corresponding BIM component feature vectors is greater than a preset distance threshold. During the training process, batch random sampling of positive and negative samples is adopted, and the gradient descent optimizer is used to iteratively update the model parameters until the embedding space can effectively represent the correspondence between visual semantics and BIM component semantics. Positive and negative samples are sampled by random batch extraction, and the ratio of positive to negative samples in each batch is guaranteed to be within the range of 1:3 to 1:5.

[0040] After training, the visual feature vectors to be matched are mapped to a unified embedding space, and the similarity between the visual feature vectors to be matched and the BIM feature vectors is calculated using cosine similarity or Euclidean distance. Select the BIM component item with the highest similarity as the matching result; Based on the matching results, the BIM component entry information is combined with visual semantic information to generate engineering semantic information for each component. The engineering semantic information includes BIM component type, parametric geometric attributes, spatial location, and topological relationship.

[0041] It should be noted that by performing cross-modal alignment mapping between visual semantic information and the BIM engineering semantic knowledge base in a unified embedding space, each visual component can be automatically mapped to a BIM component type, generating complete parametric geometric attributes, spatial location, and topological relationships in engineering semantic information. This achieves automated and accurate mapping from 2D images to BIM component semantics. Compared with traditional methods that rely on manual annotation or fixed mapping, this method automatically establishes the correspondence between visual and BIM component features through cross-modal contrastive learning, and can dynamically adapt to new or unknown components, improving mapping accuracy, automation, and knowledge base utilization efficiency.

[0042] S4 generates parametric BIM components based on three-dimensional geometric information and engineering semantic information, and establishes spatial relationships between components to form a BIM model.

[0043] In this embodiment, parametric BIM components are generated based on three-dimensional geometric information and engineering semantic information, and spatial relationships between components are established to form a BIM model, specifically: Based on 3D geometric information and engineering semantic information, a parametric BIM component is generated for each building component. The generation process includes: determining the geometric shape and dimensions of the component based on the 3D geometric information; setting the component category and parametric constraints based on the engineering semantic information; and encapsulating the geometric shape, dimensions, category, and parametric constraints into a component object to obtain the parametric BIM component. Spatial relationships are established based on graph neural networks using parametric BIM components; Parametric BIM components are integrated with spatial relationships to form a complete BIM model.

[0044] Furthermore, spatial relationships are established based on graph neural networks according to the parametric BIM components, specifically as follows: Vectorize the features of each parametric BIM component and use them as graph nodes; Graph edges are established based on the adjacency, support, and hierarchical relationships between components to form a component spatial relationship graph. The edge features include the distance between components, connection type, and layer height difference, which are used to assist GNN in reasoning about spatial relationships. The graph edges are established based on the adjacency, support, and hierarchical relationships between components. This can be understood as follows: each parametric BIM component is first regarded as a node in the graph, and its node characteristics include category, geometric dimensions, 3D coordinates, and engineering semantic information. The graph edges between components are generated according to the following rules: for any two components, if the distance between their nearest surfaces in 3D space is less than a preset adjacency threshold, then an adjacency edge is established; if the bottom or support surface of one component overlaps with the top or support surface of another component and the load-bearing direction conforms to engineering specifications, then a support edge is established; if two components have a hierarchical relationship in the vertical direction and the height difference is within a preset range, then a hierarchical edge is established.

[0045] The spatial relationship graph of building components is input into a graph convolutional neural network (GNN), which has 2 to 6 layers and uses ReLU (probability value of prediction result) as the activation function. The node and edge features are updated using a message passing mechanism and trained. During training, the final loss function is defined as the normalized weighted sum of the component adjacency prediction error, support relationship error, and hierarchical order error. During training, positive sample component pairs with relationships and negative sample component pairs without relationships are selected simultaneously in each batch through random batch sampling. The gradient descent optimization algorithm is used to backpropagate and update the network parameters. When the final loss function is less than a preset threshold or reaches the maximum number of training rounds for several consecutive training rounds, the model is considered to have completed training, thus obtaining a graph convolutional network that can stably represent the adjacency, support, and hierarchical spatial relationships of building components. Among them, the adjacency prediction error is the mean square error between the probability value of whether any two components are adjacent as predicted by the GNN and the actual adjacency relationship; the support relationship error is the mean square error between the probability value of the support relationship of the components predicted by the GNN and the actual support state (whether it is load-bearing or in contact); the hierarchical order error is the difference between the hierarchical order of the components predicted by the GNN in the vertical direction and the actual hierarchical order. The three errors are normalized and weighted to form the final loss function, which is used to optimize the GNN parameters.

[0046] The trained GNN is applied to the component spatial relationship diagram of the BIM model to be generated to predict the spatial relationship between each component node and other nodes. The spatial relationship includes adjacency, support and hierarchical relationship. The predicted results are output as a preliminary spatial layout between components; Based on the preliminary spatial layout, and in conjunction with engineering semantic and geometric constraints, the position, orientation, and hierarchical relationship of the components in three-dimensional space are optimized and adjusted. In optimizing the spatial layout of components, the spatial relationships of components predicted by GNN are first used as the initial layout. Then, an objective function is constructed based on preset engineering semantic constraints (such as adjacency requirements, support constraints, floor height limits, and engineering specification requirements specified by component type) and geometric constraints (such as component size, minimum spacing, alignment requirements, and collision constraints). The objective function is represented as the normalized weighted sum of the constraint violation errors. The three-dimensional position, orientation, and level of each component are iteratively adjusted to minimize the constraint violation errors in the objective function while maintaining the topological relationships and engineering logic between components. The optimization method adopts a heuristic search algorithm and sets iteration termination conditions, such as the error being less than a preset threshold or reaching the maximum number of iterations, thereby generating an optimized component spatial layout that satisfies the constraints and has a consistent topology.

[0047] By integrating the spatial layout and topological relationships of components that have been inferred and optimized by GNN, a spatial association of the complete BIM model is generated.

[0048] It should be noted that by combining 3D geometric information, engineering semantic information, and graph neural networks, a closed-loop process from the generation of parametric BIM components to the reasoning and optimization of spatial relationships between components is realized. This enables the automatic and high-precision construction of 3D building models that conform to engineering specifications. By using GNNs to intelligently reason about adjacency, support, and hierarchical relationships, and combining engineering semantic constraints and geometric constraints to optimize component layout, the topological relationships between components are ensured to be reasonable and the spatial layout accurate. At the same time, manual intervention is reduced, the efficiency and repeatability of BIM modeling are improved, and the robustness and applicability of complex building structure modeling are enhanced.

[0049] Reference Figure 2 As shown in the diagram, the BIM dynamic modeling system based on image intelligent recognition provided by this invention includes an image recognition module, a 3D information construction module, a mapping module, and a BIM model generation module, with connections between the modules: The image recognition module is used to acquire multimodal image data of the building scene, perform intelligent image recognition, and extract multimodal data of building components. The multimodal data includes visual semantic information and spatial feature information. The 3D information construction module is used to perform depth estimation and 3D geometric reconstruction on multimodal data to obtain the 3D geometric information of building components. The mapping module is used to build a BIM engineering semantic knowledge base, align and map visual semantic information with the BIM engineering semantic knowledge base to obtain engineering semantic information corresponding to BIM component types; The BIM model generation module is used to generate parametric BIM components based on three-dimensional geometric information and engineering semantic information, and to establish spatial relationships between components to form a BIM model.

[0050] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0051] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0052] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0053] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0054] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A BIM dynamic modeling method based on intelligent image recognition, characterized in that, Includes the following steps: Acquire multimodal image data of architectural scenes, perform intelligent image recognition, and extract multimodal data of architectural components, wherein the multimodal data includes visual semantic information and spatial feature information; Depth estimation and 3D geometric reconstruction are performed on multimodal data to obtain the 3D geometric information of building components; Construct a BIM engineering semantic knowledge base, align and map visual semantic information with the BIM engineering semantic knowledge base, and obtain engineering semantic information corresponding to BIM component types; Based on three-dimensional geometric information and engineering semantic information, parametric BIM components are generated, and spatial relationships between components are established to form a BIM model.

2. The BIM dynamic modeling method based on image intelligent recognition according to claim 1, characterized in that, The process of acquiring multimodal image data of the building scene, performing intelligent image recognition, and extracting multimodal data of building components specifically involves: The multimodal image data is preprocessed to construct a joint input of multimodal features, which is then input into a pre-trained multimodal image intelligent recognition model. The multimodal image data includes two-dimensional visible light images and depth images. Based on a multimodal image intelligent recognition model, target detection and semantic recognition are performed on building components, and initial visual semantic information and pixel-level mask are output. The initial visual semantic information includes category labels. Based on the model output, a contour extraction operation is performed on each building component, and the contour of each component is represented as a set of boundary points to obtain the component contour information corresponding to the category label, thus forming complete visual semantic information. Based on the component outline information, it is mapped to the depth image, and the three-dimensional spatial feature data corresponding to each building component is extracted; Based on three-dimensional spatial feature data, the spatial relationships between different building components are obtained, forming spatial feature information of the building components.

3. The BIM dynamic modeling method based on image intelligent recognition according to claim 2, characterized in that, Based on the model output, a contour extraction operation is performed on each building component, representing the contour of each component as a set of boundary points to obtain the component contour information corresponding to the category label. Specifically: Based on the model output, obtain the pixel-level mask corresponding to the category label; Based on pixel-level masks, the pixel region of each building component is processed independently, and the component region in the two-dimensional visible light image is determined by traversing the mask pixels. Based on the component region, perform boundary detection processing on the pixel-level mask to identify boundary pixels located at the edge of the component region; The boundary pixels are contour-tracked according to their spatial adjacency to form a continuous boundary curve; Transform a continuous boundary curve into a set of boundary points consisting of multiple boundary points; The boundary point set is associated with the corresponding category label and stored to form the component outline information.

4. The BIM dynamic modeling method based on image intelligent recognition according to claim 1, characterized in that, The process of performing depth estimation and 3D geometric reconstruction on multimodal data to obtain the 3D geometric information of building components specifically involves: Based on visual semantic information, depth estimation is performed according to different component categories. The depth estimation adopts a multi-view depth fusion and spatial consistency optimization method. The depth estimation results are combined with the two-dimensional coordinates of the original image to generate a component-level three-dimensional point cloud; Triangulation is performed on the component-level 3D point cloud to generate a 3D mesh model; The parametric geometric information of each component is extracted from the 3D mesh model and integrated to obtain the 3D geometric information.

5. The BIM dynamic modeling method based on image intelligent recognition according to claim 4, characterized in that, The depth estimation based on visual semantic information is performed according to different component categories. Specifically, the depth estimation employs multi-view depth fusion and spatial consistency optimization methods. Based on the component's category label, separate each component instance in the image; For each component instance, a preliminary depth map is generated using a multi-view depth estimation method to obtain the component-level preliminary depth value; For each preliminary depth map, depth fusion is performed by combining images from different perspectives to generate a fused component-level depth map; Based on the component-level depth map and combined with the spatial feature information between components, global spatial consistency optimization of component depth is performed.

6. The BIM dynamic modeling method based on image intelligent recognition according to claim 1, characterized in that, The construction of the BIM project semantic knowledge base specifically involves: Collect historical BIM models and their corresponding building images; The building images are preprocessed, and the processed building images are then input into the self-supervised learning model. Based on the relationships between components, a self-supervised task is constructed, and a model is trained to extract component feature vectors; The images of new building projects are input into a trained self-supervised learning model to predict component feature vectors and potential semantic information. Dynamic knowledge base entries are generated based on the predicted component feature vectors and potential semantic information. The entries in the dynamic knowledge base are compared and merged with the historical BIM model to generate a BIM engineering semantic knowledge base.

7. The BIM dynamic modeling method based on image intelligent recognition according to claim 6, characterized in that, The process of aligning and mapping visual semantic information with the BIM engineering semantic knowledge base to obtain engineering semantic information corresponding to BIM component types specifically involves: Call the BIM project semantic knowledge base to obtain BIM component entries; Feature encoding is performed on BIM component entries to generate BIM component feature vectors; Visual semantic information is input into the visual encoder to generate visual feature vectors; Input the BIM component feature vector into the BIM encoder to generate the BIM feature vector; Visual feature vectors and BIM feature vectors are mapped to a unified embedding space, and visual feature vectors and corresponding BIM component feature vectors are used as positive samples, while non-corresponding features are used as negative samples for training. After training, the visual feature vectors to be matched are mapped to a unified embedding space, and the similarity between the visual feature vectors to be matched and the BIM feature vectors is calculated. Select the BIM component item with the highest similarity as the matching result; Based on the matching results, the BIM component item information is combined with visual semantic information to generate engineering semantic information for each component.

8. The BIM dynamic modeling method based on image intelligent recognition according to claim 1, characterized in that, The process involves generating parametric BIM components based on 3D geometric information and engineering semantic information, establishing spatial relationships between components, and forming a BIM model. Specifically: Based on 3D geometric information and engineering semantic information, generate parametric BIM components for each building component; Spatial relationships are established based on graph neural networks using parametric BIM components; Parametric BIM components are integrated with spatial relationships to form a complete BIM model.

9. The BIM dynamic modeling method based on image intelligent recognition according to claim 8, characterized in that, The establishment of spatial relationships based on graph neural networks for parametric BIM components specifically involves: Vectorize the features of each parametric BIM component and use them as graph nodes; Graph edges are established based on the adjacency, support, and hierarchical relationships between components to form a component spatial relationship graph; Input the component spatial relationship graph into the GNN, update the node and edge features using the message passing mechanism, and then train the GNN. The trained GNN is applied to the component spatial relationship diagram of the BIM model to be generated to predict the spatial relationship between each component node and other nodes. The prediction results are used as the initial spatial layout between components, and the position, orientation and hierarchical relationship of the components in three-dimensional space are optimized and adjusted in combination with engineering semantic constraints and geometric constraints. By integrating the spatial layout and topological relationships of components that have been inferred and optimized by GNN, a spatial association of the complete BIM model is generated.

10. A system using the BIM dynamic modeling method based on image intelligent recognition as described in any one of claims 1-9, characterized in that, include: The image recognition module is used to acquire multimodal image data of the building scene, perform intelligent image recognition, and extract multimodal data of building components. The multimodal data includes visual semantic information and spatial feature information. The 3D information construction module is used to perform depth estimation and 3D geometric reconstruction on multimodal data to obtain the 3D geometric information of building components. The mapping module is used to build a BIM engineering semantic knowledge base, align and map visual semantic information with the BIM engineering semantic knowledge base to obtain engineering semantic information corresponding to BIM component types; The BIM model generation module is used to generate parametric BIM components based on three-dimensional geometric information and engineering semantic information, and to establish spatial relationships between components to form a BIM model.

Images