A BIM model defect detection method based on digital twinning

Abstract

The application discloses a kind of BIM model defect detection methods based on digital twinning, comprising the following steps: constructing BIM digital twinning basic model;Collecting field image or video data and recording acquisition time and acquisition position;Extracting each BIM component attribute, generates text semantic description set;Text semantic description set and field image or video data are input to improved YOLO-World model, and field component candidate set is output;Field component candidate is associated with BIM component and is matched, and component identification mapping relationship is established;Consistency determination is executed based on mapping relationship, and defect component is determined;BIM digital twinning basic model is updated based on defect detection result.The application realizes the automatic detection and dynamic updating of BIM model defect, improves the consistency and defect detection precision of field and model, and is suitable for intelligent management scene of whole process of engineering construction.

Description

Technical Field

[0001] This invention relates to the field of building information modeling and intelligent inspection technology, and in particular to a method for detecting defects in BIM models based on digital twins. Background Technology

[0002] With the increasing demand for digital and refined management in construction projects, technologies for engineering quality control and operation and maintenance management based on Building Information Modeling (BIM) have received widespread attention. In existing technologies, BIM models have been widely used in component modeling and information integration during the design phase. Some research has begun to combine on-site images or video data to verify and update BIM models, in order to achieve defect identification and status assessment during the construction or operation and maintenance phases. However, in practical engineering applications, existing BIM-based defect detection methods still mainly rely on manual comparison, rule matching, or single visual inspection models, and generally suffer from the following problems:

[0003] First, existing methods primarily rely on 2D image detection results, lacking deep integration with the semantic information of BIM components. This leads to confusion in scenarios with densely packed components, similar shapes, or severe occlusion, making it difficult to accurately map components to specific BIM elements. Second, existing technologies typically establish associations between on-site components and BIM components based solely on spatial location or geometric overlap, failing to comprehensively consider multi-dimensional information such as system affiliation and functional attributes. This results in unstable component mapping relationships and a high mismatch rate. Third, some solutions incorporating deep learning object detection models fail to fully utilize the structural priors and system constraints inherent in the BIM model. The model detection results lack constraints on engineering semantics, making it difficult to finely distinguish between various types of defects such as missing components, system errors, or inconsistent attributes. Furthermore, after defect detection, existing technologies often only output the detection results without establishing a BIM digital twin model update mechanism based on defect type. This leads to long-term deviations between the BIM model and the on-site state, hindering the support for continuously evolving engineering management and operation and maintenance decisions.

[0004] Therefore, how to provide a method for detecting defects in BIM models based on digital twins is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a BIM model defect detection method based on digital twins. This invention constructs a BIM digital twin basic model that includes component geometric information, system attribution information, and component association relationships. It then transforms the structured semantic information of BIM components into textual semantic descriptions and introduces them into an improved YOLO-World model. This enables accurate association matching and consistency determination between on-site components and BIM components, thereby achieving automated identification and dynamic updating of BIM model defects such as missing components, inconsistent spatial locations, inconsistent system attribution, and inconsistent component attributes. It has the advantages of high detection accuracy, strong semantic consistency, low reliance on manual intervention, and outstanding model sustainable evolution capabilities.

[0006] A method for detecting defects in a BIM model based on digital twins according to an embodiment of the present invention includes the following steps:

[0007] Step 1: Obtain the BIM model data of the project, construct the basic BIM digital twin model, assign a unique component identifier to each BIM component, and store the component's geometric information, component type information, system affiliation information, and component association relationship;

[0008] Step 2: Collect on-site image or video data corresponding to the BIM digital twin basic model, and record the corresponding collection time and collection location;

[0009] Step 3: Based on the BIM digital twin basic model, extract the component type, system attributes and functional attributes of each BIM component, and generate a textual semantic description set corresponding to each BIM component;

[0010] Step 4: Input the textualized semantic description set as text prompts and on-site image or video data into the improved YOLO-World model for component detection. The improved YOLO-World model includes a BIM structured semantic prompt construction module, a text embedding encoding module, an on-site image multi-scale feature extraction module, a BIM constraint-guided path aggregation module, and a dual-branch component detection output module. The output includes a set of on-site component candidates containing component candidate boxes, component semantic matching results, and key component attribute labels.

[0011] Step 5: Associate and match the on-site component candidates with BIM components to establish a mapping relationship between the on-site component candidates and component identifiers;

[0012] Step 6: Perform consistency determination based on the mapping relationship, identify the corresponding BIM component as a defective component, record the defect type and defect location, and output the BIM model defect detection results;

[0013] Step 7: Based on the BIM model defect detection results, update the BIM digital twin base model and generate the updated BIM digital twin model.

[0014] Optionally, step one specifically includes:

[0015] Obtain BIM model data of the engineering project, parse the BIM model data, extract component-level data, and form a component-level dataset. The component-level dataset includes geometric entity data of each component, component type code, system affiliation code, and component connection relationship data.

[0016] For each component in the component-level dataset, a component identifier is generated, and a correspondence is established between the component identifier and the original component identifier in the BIM model.

[0017] Spatial position parameters of components are extracted based on geometric entity data. These spatial position parameters include the geometric center coordinates, bounding box size, and orientation parameters of the components.

[0018] A set of component association relationships is constructed based on component connection relationship data. The set of component association relationships includes adjacent connection relationships, support dependency relationships, and crossing and avoidance relationships.

[0019] The component identifier, component type code, system affiliation code, spatial location parameters, and component association set are written into the BIM digital twin basic model to form a component information index structure.

[0020] Optionally, step two specifically includes:

[0021] The scope of on-site data collection is determined based on a pre-established BIM digital twin basic model, and the scope of on-site data collection includes the construction area or operation and maintenance area corresponding to the spatial location of BIM components.

[0022] The acquisition equipment acquires on-site images or video data, and during the acquisition process, the image acquisition direction is kept consistent with the spatial coordinate system of the BIM digital twin basic model;

[0023] While acquiring on-site image or video data, record the acquisition time and acquisition location information corresponding to each image or video frame;

[0024] The acquired on-site images or video data are organized in chronological order, and the acquisition time and location information are bound to the corresponding image or video frames.

[0025] Optionally, step three specifically includes:

[0026] Based on the BIM digital twin basic model, the component type code, system affiliation code and functional attribute field of each BIM component are read, and the component type code and system affiliation code are input as discrete semantic tags into the preset component semantic template library;

[0027] Semantic mapping processing is performed on component type codes and system affiliation codes. By matching tags in the component semantic template library, the standard component name and system name corresponding to each BIM component are determined, and component-level semantic tag pairs are generated.

[0028] The component-level semantic tags are concatenated with the functional attribute fields to generate textual semantic description entries according to the preset natural language template.

[0029] Perform deduplication and normalization processing on the generated textual semantic description entries;

[0030] The textual semantic description entries, after being deduplicated and normalized, are collected to form a textual semantic description set corresponding to the BIM component set in the BIM digital twin basic model, and the textual semantic description set is stored as structured text prompt data.

[0031] Optionally, the improved YOLO-World model includes a BIM structured semantic prompting construction module, a text embedding encoding module, a site image multi-scale feature extraction module, a BIM constraint-guided path aggregation module, and a dual-branch component detection output module.

[0032] The BIM structured semantic prompt construction module receives a set of textual semantic descriptions as input, breaks down each textual semantic description entry into a system affiliation field, a component name field, and a functional attribute field, and concatenates the strings to generate a standardized text prompt sequence.

[0033] The text embedding encoding module adopts the CLIP text encoder structure to vectorize the standardized text prompt sequence. The CLIP text encoder includes a word embedding layer, a position encoding layer, and a multi-layer Transformer encoding layer. In the output stage, the word vectors output by the Transformer encoding layer are subjected to average pooling to generate a text embedding vector corresponding to each standardized text prompt. The text embedding vector is then bound to its corresponding BIM component identifier and system attribution field to form a text embedding set.

[0034] The on-site image multi-scale feature extraction module adopts a convolutional backbone network structure to extract features from on-site images or video data. The convolutional backbone network consists of multiple two-dimensional convolutional layers, batch normalization layers, and LeakyReLU activation function layers. It also achieves step-by-step downsampling through stride convolution and outputs multiple sets of image feature maps at different downsampling scales.

[0035] The BIM constraint-guided path aggregation module, based on a path aggregation network structure, performs top-down and bottom-up feature transfer on multi-scale image feature maps. At each feature fusion node, it introduces text embedding vectors from the text embedding set. Specifically, based on the system affiliation and spatial partitioning information of components in the BIM digital twin basic model, it performs candidate filtering on the text embedding set, retaining only text embedding vectors that are consistent with the spatial region and system category corresponding to the current image feature map. It then performs channel-by-channel multiplication on the filtered text embedding vectors and the corresponding scale image feature map, and normalizes the multiplication result in the channel dimension using the Softmax normalization function. Finally, it performs channel-by-channel addition fusion on the normalized semantic modulation result and the image feature map to generate a semantic-visual fusion feature map containing BIM structural constraint information.

[0036] The dual-branch component detection output module receives the fused feature map as input. The first detection branch uses a convolutional regression layer to perform bounding box regression on the fused feature map and normalizes the regression result using the Sigmoid function to output the component candidate box position. At the same time, it outputs the component semantic matching result through the Softmax classification layer. The second detection branch uses an attribute discrimination network composed of two fully connected layers and the ReLU activation function to perform attribute mapping on the fused feature map and output the key attribute labels of the component. The component candidate box, the component semantic matching result, and the key attribute labels of the component together constitute the on-site component candidate set.

[0037] Optionally, step five specifically includes:

[0038] Based on the candidate set of on-site components and the set of BIM components in the BIM digital twin basic model, a candidate matching pair set between on-site component candidates and BIM components is established;

[0039] For each candidate matching pair, a spatial matching score is calculated based on the spatial location parameters of the candidate frame of the on-site component and the spatial location parameters of the BIM component.

[0040] A semantic consistency score is calculated based on the semantic matching results of on-site component candidates and the component type information and system affiliation information of BIM components.

[0041] The consistency score of the attributes is calculated based on the key attribute labels of on-site component candidates and the functional attribute fields of BIM components.

[0042] The spatial matching score, semantic consistency score, and attribute consistency score are weighted and fused according to a preset weight ratio to generate a comprehensive matching score for candidate matching pairs;

[0043] In the candidate matching pair set corresponding to each on-site component candidate, the BIM component with the highest comprehensive matching score is selected as the mapping object of the on-site component candidate. When the comprehensive matching score is greater than the preset mapping threshold, a mapping relationship is established between the on-site component candidate and the BIM component. When the comprehensive matching score is less than or equal to the preset mapping threshold, the on-site component candidate is determined to be an unmapped component candidate.

[0044] Optionally, step six specifically includes:

[0045] Based on the mapping relationship between on-site component candidates and BIM components, consistency determination processing is performed on each BIM component in the BIM digital twin basic model. The consistency determination is carried out in the order of mapping existence determination, spatial consistency determination, system consistency determination and attribute consistency determination.

[0046] For each BIM component, a mapping existence determination is performed. When a BIM component does not correspond to any on-site component candidate in the mapping relationship, the BIM component is determined to be a defective component that has not formed a valid mapping, and the defect type is marked as a component missing defect.

[0047] For BIM components that form a mapping relationship, a spatial consistency determination is performed based on the spatial location parameters of the BIM component and the spatial location parameters of the corresponding on-site component candidate frames.

[0048] Under the premise of passing the spatial consistency judgment, the system consistency judgment is performed based on the system ownership information of BIM components and the system ownership label in the semantic matching result of the candidate on-site components.

[0049] Under the premise of passing the system consistency judgment, the attribute consistency judgment is performed based on the functional attribute fields of BIM components and the key attribute labels of candidate on-site components;

[0050] When a BIM component passes the mapping existence determination, spatial consistency determination, system consistency determination and attribute consistency determination in sequence, the BIM component is determined to be a consistent component and is not recorded as a defective component.

[0051] All BIM components identified as defective are aggregated to form the BIM model defect detection results.

[0052] Optionally, step seven specifically includes:

[0053] Based on the defect detection results of the BIM model, the defective components in the BIM digital twin basic model are updated according to the defect type.

[0054] For components with missing defects, the corresponding component identifier is marked as missing, and the timestamp information corresponding to the missing status is recorded.

[0055] For defective components with inconsistent spatial locations, the geometric center coordinates and bounding box dimensions of the BIM component are updated based on the spatial location parameters of the corresponding on-site component candidate frames.

[0056] For defective components with inconsistent system attribution, update the system attribution information of the BIM component to a system attribution code that is consistent with the semantic matching result of the candidate components on site;

[0057] For defective components with inconsistent component attributes, the functional attribute fields of the BIM components are updated based on the key attribute tags of the candidate components on site.

[0058] After updating various defective components, the updated component information is written into the BIM digital twin base model to generate the updated BIM digital twin model, and a model version identifier is assigned to the updated BIM digital twin model.

[0059] The beneficial effects of this invention are:

[0060] This invention constructs a BIM digital twin basic model that includes component geometric information, component type information, system attribution information, and component association relationships. It transforms the component type, system attributes, and functional attributes of BIM components into a structured textual semantic description set, and introduces an improved YOLO-World model to achieve deep integration of visual inspection and engineering semantics. This effectively solves the problem of false detection and missed detection caused by existing technologies that rely solely on two-dimensional images and lack semantic constraints for component recognition. Simultaneously, by comprehensively considering multi-dimensional information such as spatial location, semantic consistency, and attribute consistency during component association matching, a stable and reliable mapping relationship between on-site components and BIM components is established, achieving precise component-level association and defect location. Furthermore, this invention uses a sequential consistency judgment mechanism to finely distinguish multiple types of defects such as missing components, positional deviations, incorrect system attribution, and inconsistent attributes. Based on this, the BIM digital twin model is classified and updated, enabling the BIM model to continuously reflect the actual on-site state, thereby significantly improving the accuracy of BIM-based engineering quality inspection and the long-term effectiveness of the digital twin model. Attached Figure Description

[0061] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0062] Figure 1 This is a flowchart of a BIM model defect detection method based on digital twin proposed in this invention;

[0063] Figure 2This is a schematic diagram of a BIM model defect detection method based on digital twin proposed in this invention;

[0064] Figure 3 This is a framework diagram of the improved YOLO-World model in the BIM model defect detection method based on digital twin proposed in this invention. Detailed Implementation

[0065] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0066] refer to Figure 1-3 A method for detecting defects in BIM models based on digital twins includes the following steps:

[0067] Step 1: Obtain the BIM model data of the project, construct the basic BIM digital twin model, assign a unique component identifier to each BIM component, and store the component's geometric information, component type information, system affiliation information, and component association relationship;

[0068] Step 2: Collect on-site image or video data corresponding to the BIM digital twin basic model, and record the corresponding collection time and collection location;

[0069] Step 3: Based on the BIM digital twin basic model, extract the component type, system attributes and functional attributes of each BIM component, and generate a textual semantic description set corresponding to each BIM component;

[0070] Step 4: Input the textualized semantic description set as text prompts and on-site image or video data into the improved YOLO-World model for component detection. The improved YOLO-World model includes a BIM structured semantic prompt construction module, a text embedding encoding module, an on-site image multi-scale feature extraction module, a BIM constraint-guided path aggregation module, and a dual-branch component detection output module. The output includes a set of on-site component candidates containing component candidate boxes, component semantic matching results, and key component attribute labels.

[0071] Step 5: Associate and match the on-site component candidates with BIM components to establish a mapping relationship between the on-site component candidates and component identifiers;

[0072] Step 6: Perform consistency determination based on the mapping relationship, identify the corresponding BIM component as a defective component, record the defect type and defect location, and output the BIM model defect detection results;

[0073] Step 7: Based on the BIM model defect detection results, update the BIM digital twin base model and generate the updated BIM digital twin model.

[0074] In this embodiment, step one specifically includes:

[0075] Obtain BIM model data of the engineering project, parse the BIM model data, extract component-level data, and form a component-level dataset. The component-level dataset includes geometric entity data of each component, component type code, system affiliation code, and component connection relationship data.

[0076] For each component in the component-level dataset, a component identifier is generated. The component identifier is generated by sequentially combining the project identifier field, professional field, floor or zone field, component type field, and component serial number field, and a correspondence is established between the component identifier and the original component identifier in the BIM model.

[0077] Spatial position parameters of components are extracted based on geometric entity data. The spatial position parameters include the geometric center coordinates, bounding box size, and orientation parameters of the components. The geometric center coordinates are obtained by averaging the vertex coordinates of the component geometric entity. The bounding box size is obtained by calculating the difference between the maximum and minimum coordinate values ​​of the component geometric entity in the three-dimensional coordinate axis direction. The orientation parameters are determined by extracting the principal axis direction of the component geometric entity.

[0078] A set of component association relationships is constructed based on component connection relationship data. The set of component association relationships includes adjacent connection relationships, support dependency relationships, and crossing and avoidance relationships. Adjacent connection relationships are determined based on the endpoint or surface contact relationships between component geometric entities. Support dependency relationships are determined based on the support or supported attribute fields in the BIM model. Crossing and avoidance relationships are determined based on the spatial intersection detection results between component geometric entities.

[0079] The component identifier, component type code, system affiliation code, spatial location parameters, and component association set are written into the BIM digital twin basic model to form a component information index structure.

[0080] In this embodiment, step two specifically includes:

[0081] The scope of on-site data collection is determined based on a pre-established BIM digital twin basic model, and the scope of on-site data collection includes the construction area or operation and maintenance area corresponding to the spatial location of BIM components.

[0082] The acquisition equipment acquires on-site images or video data, and during the acquisition process, the image acquisition direction is kept consistent with the spatial coordinate system of the BIM digital twin basic model;

[0083] While acquiring on-site image or video data, the acquisition time information and acquisition location information corresponding to each image or video frame are recorded. The acquisition location information includes the spatial location of the image acquisition device in the BIM digital twin basic model coordinate system or the floor and zone identifier to which it belongs.

[0084] The acquired on-site images or video data are organized in chronological order, and the acquisition time and location information are bound to the corresponding image or video frames.

[0085] In this embodiment, step three specifically includes:

[0086] Based on the BIM digital twin basic model, the component type code, system affiliation code and functional attribute field of each BIM component are read, and the component type code and system affiliation code are used as discrete semantic tags and input into the preset component semantic template library. The component semantic template library consists of standard component names, system names and combination relationships in the engineering field.

[0087] Semantic mapping processing is performed on component type codes and system affiliation codes. By matching tags in the component semantic template library, the standard component name and system name corresponding to each BIM component are determined, and component-level semantic tag pairs are generated.

[0088] The component-level semantic tags are concatenated with the functional attribute fields to generate textual semantic description entries according to a preset natural language template, wherein the natural language template is a combination of "system name + component name + functional attribute keywords";

[0089] The generated textual semantic description entries are subjected to deduplication and normalization processing, which includes:

[0090] The textual semantic description entries are standardized by converting synonyms, abbreviations and capitalization differences in different text forms into a preset standard expression.

[0091] The standardized textual semantic description entries are compared for consistency based on their string content. When multiple BIM components have completely identical textual semantic description entries, only one is retained as a shared semantic description, and the association between the shared semantic description and the corresponding BIM component identifier is recorded.

[0092] Field trimming is performed on textual semantic description entries containing repetitive functional attribute keywords or redundant modifiers, retaining the system name, component name, and non-repetitive functional attribute keywords to generate standardized textual semantic description entries.

[0093] The textual semantic description entries, after being deduplicated and normalized, are collected to form a textual semantic description set corresponding to the BIM component set in the BIM digital twin basic model, and the textual semantic description set is stored as structured text prompt data.

[0094] In this embodiment, the improved YOLO-World model includes a BIM structured semantic prompting construction module, a text embedding encoding module, a site image multi-scale feature extraction module, a BIM constraint-guided path aggregation module, and a dual-branch component detection output module.

[0095] The BIM structured semantic prompt construction module receives a set of textual semantic descriptions as input. It splits each textual semantic description entry into a system attribution field, a component name field, and a functional attribute field. The system attribution field is concatenated with the functional attribute field in the order of "system attribution field first, component name field in the middle, and functional attribute field last" to generate a standardized text prompt sequence. During the generation process, multiple text prompts belonging to the same BIM component identifier are merged, and a one-to-one correspondence index relationship between the standardized text prompt sequence and the BIM component identifier is established.

[0096] The text embedding encoding module adopts a CLIP text encoder structure to vectorize the standardized text prompt sequence. The CLIP text encoder includes a word embedding layer, a position encoding layer, and a multi-layer Transformer encoding layer. The word embedding layer maps the lexical units in the text prompt to word vectors of a set dimension. The position encoding layer injects sequence position information into the word vectors using a sinusoidal position encoding method. The Transformer encoding layer calculates the weighted relationship between word vectors through a multi-head self-attention operator and performs linear transformation, ReLU activation function processing, and linear projection operation on the attention output through a feedforward network. In the output stage, the word vectors output by the Transformer encoding layer are subjected to average pooling to generate a text embedding vector corresponding to each standardized text prompt. The text embedding vector is then bound to its corresponding BIM component identifier and system attribution field to form a text embedding set.

[0097] The on-site image multi-scale feature extraction module adopts a convolutional backbone network structure to extract features from on-site images or video data. The convolutional backbone network consists of multiple layers of two-dimensional convolutional layers, batch normalization layers, and LeakyReLU activation function layers. It achieves step-by-step downsampling through stride convolution and outputs multiple sets of image feature maps at different downsampling scales. The shallow image feature map retains the edge and local geometric details of the components, while the deep image feature map retains the overall shape and spatial distribution information of the components.

[0098] The BIM constraint-guided path aggregation module, based on a path aggregation network structure, performs top-down and bottom-up feature transfer on multi-scale image feature maps. At each feature fusion node, it introduces text embedding vectors from the text embedding set. Specifically, based on the system affiliation and spatial partitioning information of components in the BIM digital twin basic model, it performs candidate filtering on the text embedding set, retaining only text embedding vectors that are consistent with the spatial region and system category corresponding to the current image feature map. It then performs channel-by-channel multiplication on the filtered text embedding vectors and the corresponding scale image feature map, and normalizes the multiplication result in the channel dimension using the Softmax normalization function. Finally, it performs channel-by-channel addition fusion on the normalized semantic modulation result and the image feature map to generate a semantic-visual fusion feature map containing BIM structural constraint information.

[0099] The dual-branch component detection output module receives a fused feature map as input. The first detection branch uses a convolutional regression layer to perform bounding box regression on the fused feature map and normalizes the regression result using a sigmoid function to output the component candidate box position. At the same time, it outputs the component semantic matching result through a softmax classification layer. The second detection branch uses an attribute discrimination network composed of two fully connected layers and a ReLU activation function to perform attribute mapping on the fused feature map and output the component key attribute labels. The component key attribute labels include system consistency identifiers and specification range identifiers. The component candidate boxes, component semantic matching results, and component key attribute labels together constitute the on-site component candidate set.

[0100] This implementation introduces an improved YOLO-World model for BIM digital twin scenarios, transforming the system affiliation, component name, and functional attributes of BIM components into structured text semantic prompts. This model is then used for cross-modal joint modeling with on-site image features. Compared to existing target detection models that rely solely on visual features or open-category text, this implementation introduces explicit BIM structure and system constraints during the detection process, effectively reducing the probability of false positives and mixed detections of components from different disciplines under similar appearance conditions. Simultaneously, through a BIM constraint-guided path aggregation mechanism, semantic information is applied only to image feature layers consistent with the current spatial region and system category, avoiding interference from irrelevant semantics and significantly improving the stability and accuracy of component detection in complex engineering scenarios. Furthermore, a dual-branch detection output structure is adopted, simultaneously outputting key component attribute labels while completing component localization and semantic matching. This provides a direct basis for subsequent BIM component consistency judgment, achieving a high degree of alignment between model detection results and BIM defect identification requirements, thus comprehensively improving the accuracy, engineering adaptability, and automation level of BIM model defect detection.

[0101] In this embodiment, step five specifically includes:

[0102] Based on the on-site component candidate set and the BIM component set in the BIM digital twin basic model, a candidate matching pair set between on-site component candidates and BIM components is established, wherein each candidate matching pair consists of an on-site component candidate and a BIM component;

[0103] For each candidate matching pair, a spatial matching score is calculated based on the spatial position parameters of the candidate component frame and the spatial position parameters of the BIM component. The spatial matching score is obtained by comparing the Euclidean distance between the center position of the candidate component frame and the geometric center position of the BIM component, and by weighting the calculation by combining the overlap ratio between the size of the candidate component frame and the size of the bounding box of the BIM component.

[0104] A semantic consistency score is calculated based on the semantic matching results of candidate on-site components and the component type information and system affiliation information of BIM components. The semantic consistency score is determined by judging whether the component name and BIM component type in the semantic matching results are consistent, and whether the corresponding system affiliation label is consistent. When both component name and system affiliation are consistent, a first preset weight is assigned; when only component name is consistent, a second preset weight is assigned; and when neither is consistent, a zero weight is assigned. The first and second preset weights can be set according to the importance of component type and system affiliation. For example, the first preset weight can be set to 1, and the second preset weight can be set to 0.5.

[0105] The attribute consistency score is calculated based on the key attribute labels of candidate on-site components and the functional attribute fields of BIM components. The attribute consistency score is determined by judging whether the specification range identifier and system consistency identifier in the key attribute labels of the components fall into the specification range and system belonging range of the corresponding BIM components, respectively.

[0106] The spatial matching score, semantic consistency score, and attribute consistency score are weighted and fused according to a preset weight ratio to generate a comprehensive matching score for candidate matching pairs. The weight ratios of the spatial matching score, semantic consistency score, and attribute consistency score can be adjusted according to the application scenario, and the sum of the weights is 1. For example, in the construction phase, the weight ratio of the spatial matching score can be increased, and in the operation and maintenance phase, the weight ratios of the semantic consistency score and attribute consistency score can be increased.

[0107] In the candidate matching pair set corresponding to each on-site component candidate, the BIM component with the highest comprehensive matching score is selected as the mapping object of the on-site component candidate. When the comprehensive matching score is greater than the preset mapping threshold, a mapping relationship is established between the on-site component candidate and the BIM component. When the comprehensive matching score is less than or equal to the preset mapping threshold, the on-site component candidate is determined to be an unmapped component candidate. The preset mapping threshold can be set according to the project scale and component density. For example, in densely populated areas, the preset mapping threshold is set to 0.7–0.8, and in sparsely distributed areas, the preset mapping threshold is set to 0.6–0.7.

[0108] In this embodiment, step six specifically includes:

[0109] Based on the mapping relationship between on-site component candidates and BIM components, consistency determination processing is performed on each BIM component in the BIM digital twin basic model. The consistency determination is carried out in the order of mapping existence determination, spatial consistency determination, system consistency determination and attribute consistency determination.

[0110] For each BIM component, a mapping existence determination is performed. When a BIM component does not correspond to any on-site component candidate in the mapping relationship, the BIM component is determined to be a defective component that has not formed a valid mapping, and the defect type is marked as a component missing defect.

[0111] For BIM components that form a mapping relationship, a spatial consistency determination is performed based on the spatial position parameters of the BIM component and the spatial position parameters of the corresponding candidate bounding boxes of on-site components. The spatial consistency determination compares whether the distance between the geometric center of the BIM component and the center of the candidate bounding box is less than a preset spatial offset threshold. When the distance is greater than the preset spatial offset threshold, the BIM component is determined to be a defective component with inconsistent spatial position. The preset spatial offset threshold can be adaptively set according to the actual size of the BIM component. For example, the preset spatial offset threshold can be set to 5%–15% of the maximum side length of the bounding box of the corresponding BIM component.

[0112] Under the premise of passing the spatial consistency judgment, the system consistency judgment is performed based on the system ownership information of BIM components and the system ownership label in the semantic matching result of the candidate components on site. When the system ownership label is inconsistent, the BIM component is judged as a defective component with inconsistent system ownership.

[0113] Under the premise of passing the system consistency judgment, attribute consistency judgment is performed based on the functional attribute fields of BIM components and the key attribute labels of candidate on-site components. The attribute consistency judgment is performed by judging whether the specification interval identifier in the key attribute label of the component falls within the specification interval range corresponding to the BIM component. When the specification interval identifier is not within the specification interval range, the BIM component is judged to be a component attribute inconsistency type defective component.

[0114] When a BIM component passes the mapping existence determination, spatial consistency determination, system consistency determination and attribute consistency determination in sequence, the BIM component is determined to be a consistent component and is not recorded as a defective component.

[0115] All BIM components identified as defective are aggregated to form BIM model defect detection results, which include defective component identification, defect type, and corresponding defect judgment criteria.

[0116] In this embodiment, step seven specifically includes:

[0117] Based on the defect detection results of the BIM model, the defective components in the BIM digital twin basic model are updated according to the defect type.

[0118] For components with missing defects, the corresponding component identifier is marked as missing, and the timestamp information corresponding to the missing status is recorded.

[0119] For defective components with inconsistent spatial locations, the geometric center coordinates and bounding box dimensions of the BIM component are updated based on the spatial location parameters of the corresponding on-site component candidate frames.

[0120] For defective components with inconsistent system attribution, update the system attribution information of the BIM component to a system attribution code that is consistent with the semantic matching result of the candidate components on site;

[0121] For defective components with inconsistent component attributes, the functional attribute fields of the BIM components are updated based on the key attribute tags of the candidate components on site.

[0122] After updating various defective components, the updated component information is written into the BIM digital twin base model to generate the updated BIM digital twin model, and a model version identifier is assigned to the updated BIM digital twin model.

[0123] Example 1:

[0124] To verify the feasibility of this invention in practice, it was applied to the integrated construction and operation management scenario of a large public building project. This project is a comprehensive public building with 18 floors above ground and two floors underground, with a total construction area of ​​approximately 96,000 square meters. It includes various professional components such as HVAC systems, water supply and drainage systems, electrical systems, and fire protection systems. The BIM model contains approximately 180,000 components, with complex component types and dense system overlaps. Furthermore, there were multiple design changes and on-site adjustments during the construction and trial operation phases. Traditional methods relying on manual verification or simple image detection were insufficient to ensure the consistency between the BIM model and the actual on-site conditions.

[0125] In this embodiment, a BIM digital twin basic model is first constructed based on the BIM model data formed during the project design phase. All components in the model are uniformly analyzed to extract their geometric entities, component types, system affiliations, and connection relationships. A unique component identifier is generated for each component. The component identifier is generated using a combination of "Project Number - Professional Category - Floor Zone - Component Type - Serial Number," such as "P01-HVAC-F05-DUCT-01234," to ensure the unique traceability of the component throughout its entire lifecycle. Based on the component's geometric entities, the geometric center coordinates, bounding box dimensions, and orientation parameters are calculated, and the adjacency, support, and crossing relationships between components are established, ultimately forming a complete BIM digital twin basic model.

[0126] During the on-site data acquisition phase, five typical floors that had been completed and entered trial operation were selected as verification areas. On-site images and video data were collected using fixed camera equipment and handheld inspection terminals. During the acquisition process, the position and shooting direction of the camera equipment were aligned with the spatial coordinate system of the BIM digital twin basic model, and the acquisition time, floor, and zone information were recorded for each frame of image or video clip. Throughout the entire implementation cycle, approximately 32,000 on-site images and approximately 420 video clips were collected, covering various components such as HVAC pipes, cable trays, valves, and fan coil units.

[0127] In the semantic construction phase, component type codes, system affiliation codes, and functional attribute fields for each component are extracted based on the BIM digital twin model and mapped to a pre-defined component semantic template library. Through semantic mapping and deduplication normalization, a textual semantic description set corresponding one-to-one with each BIM component is generated, such as "HVAC system - air supply duct - galvanized steel plate" and "water supply and drainage system - shut-off valve - DN50". Statistics show that this project ultimately generated approximately 26,000 standardized textual semantic description entries, significantly reducing semantic redundancy and providing stable textual prompts for subsequent model testing.

[0128] In the component detection phase, a textualized semantic description set and on-site image or video data are input into the improved YOLO-World model. The CLIP text encoder in the model encodes textual prompts into text embedding vectors, while the visual backbone network extracts multi-scale features from the on-site images. In the BIM constraint-guided path aggregation module, the text embeddings are filtered and modulated based on component system affiliation and spatial partitioning information, thereby generating a semantic-visual feature map that integrates BIM structural constraints. Finally, the dual-branch component detection output module outputs candidate component boxes, semantic matching results, and key attribute labels for the components, forming a candidate set of on-site components.

[0129] During the component association and matching phase, for each candidate on-site component, spatial matching score, semantic consistency score, and attribute consistency score are calculated and weighted for fusion. In the spatial matching score, the spatial offset threshold between the center of the candidate component's bounding box and the geometric center of the BIM component is set to 10% of the maximum side length of the component's bounding box. In the semantic consistency score, a value of 1 is assigned when both the component name and system affiliation are consistent, and a value of 0.5 is assigned when only the component name is consistent. In the comprehensive matching score, the weights for the construction phase are set as follows: spatial matching score 0.45, semantic consistency score 0.35, and attribute consistency score 0.20. A mapping relationship is established when the comprehensive matching score is greater than 0.75.

[0130] During the consistency assessment and defect identification phase, all BIM components are sequentially assessed for mapping existence, spatial consistency, system consistency, and attribute consistency to identify defect types such as missing components, positional offsets, system errors, and attribute inconsistencies. This process ultimately generates BIM model defect detection results, which are used to classify and update the basic BIM digital twin model, resulting in an updated version of the BIM digital twin model.

[0131] To verify the effectiveness of the method of this invention, it was compared and analyzed with two other methods: one based on manual inspection and manual comparison of the BIM model, and the other using a general object detection model for component identification without introducing BIM semantic constraints. The detection results of the three methods on the same dataset are shown in Table 1.

[0132] Table 1 Comparison of Component Defect Detection Results Using Different Methods

[0133] Method type Total number of components inspected Correctly identify the number of components False positives Number of missed detections Component recognition accuracy (%) Defect identification accuracy (%) Manual inspection comparison 18200 15340 1240 1620 84.29 79.65 General object detection model 18200 16080 980 1140 88.35 83.12 Method of the present invention 18200 17190 410 600 94.45 92.08

[0134] As shown in Table 1, the method of this invention significantly outperforms the comparative methods in both component identification accuracy and defect identification accuracy. Specifically, compared with the general target detection model, the component identification accuracy is improved by approximately 6.1 percentage points, and the defect identification accuracy is improved by approximately 9.0 percentage points. In particular, in scenarios with dense components and similar appearances, the number of false positives and false negatives is significantly reduced.

[0135] Further statistics on the identification performance of different defect types are shown in Table 2:

[0136] Table 2 Statistical Table of Identification Results for Different Defect Types

[0137] Defect types Actual number of defects Manual method recognition rate (%) General model recognition rate (%) The recognition rate (%) of the method of this invention missing components 210 81.4 85.7 96.2 Position offset 340 76.5 82.1 93.8 System attribution error 180 69.8 74.6 91.1 Inconsistent attributes 260 72.3 78.4 90.5

[0138] As shown in Table 2, the present invention has significant advantages in identifying defect types that are difficult to accurately identify using traditional methods, such as system attribution errors and attribute inconsistencies. This demonstrates that by introducing BIM semantic priors and a multidimensional consistency determination mechanism, the precision of defect identification can be significantly improved.

[0139] As can be seen from the above embodiments, the present invention combines the BIM digital twin model with the improved YOLO-World model to realize a closed-loop process of on-site component inspection, precise component-level mapping, multi-type defect judgment, and dynamic updating of the BIM model. This not only improves the accuracy of component inspection and defect identification, but also effectively reduces the cost of manual verification, enabling the BIM model to continuously reflect the real state of the site. It has significant engineering application value and promotion significance.

[0140] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for defect detection in BIM models based on digital twins, characterized in that, Includes the following steps: Step 1: Obtain the BIM model data of the project, construct the basic BIM digital twin model, assign a unique component identifier to each BIM component, and store the component's geometric information, component type information, system affiliation information, and component association relationship; Step 2: Collect on-site image or video data corresponding to the BIM digital twin basic model, and record the corresponding collection time and collection location; Step 3: Based on the BIM digital twin basic model, extract the component type, system attributes and functional attributes of each BIM component, and generate a textual semantic description set corresponding to each BIM component; Step 4: Input the textualized semantic description set as text prompts and on-site image or video data into the improved YOLO-World model for component detection. The improved YOLO-World model includes a BIM structured semantic prompt construction module, a text embedding encoding module, an on-site image multi-scale feature extraction module, a BIM constraint-guided path aggregation module, and a dual-branch component detection output module. The output includes a set of on-site component candidates containing component candidate boxes, component semantic matching results, and key component attribute labels. Step 5: Associate and match the on-site component candidates with BIM components to establish a mapping relationship between the on-site component candidates and component identifiers; Step 6: Perform consistency determination based on the mapping relationship, identify the corresponding BIM component as a defective component, record the defect type and defect location, and output the BIM model defect detection results; Step 7: Based on the BIM model defect detection results, update the BIM digital twin base model and generate the updated BIM digital twin model.

2. The method for detecting defects in a BIM model based on digital twins according to claim 1, characterized in that, Step one specifically includes: Obtain BIM model data of the engineering project, parse the BIM model data, extract component-level data, and form a component-level dataset. The component-level dataset includes geometric entity data of each component, component type code, system affiliation code, and component connection relationship data. For each component in the component-level dataset, a component identifier is generated, and a correspondence is established between the component identifier and the original component identifier in the BIM model. Spatial position parameters of components are extracted based on geometric entity data. These spatial position parameters include the geometric center coordinates, bounding box size, and orientation parameters of the components. A set of component association relationships is constructed based on component connection relationship data. The set of component association relationships includes adjacent connection relationships, support dependency relationships, and crossing and avoidance relationships. The component identifier, component type code, system affiliation code, spatial location parameters, and component association set are written into the BIM digital twin basic model to form a component information index structure.

3. The method for detecting defects in a BIM model based on digital twins according to claim 1, characterized in that, Step two specifically includes: The scope of on-site data collection is determined based on a pre-established BIM digital twin basic model, and the scope of on-site data collection includes the construction area or operation and maintenance area corresponding to the spatial location of BIM components. The acquisition equipment acquires on-site images or video data, and during the acquisition process, the image acquisition direction is kept consistent with the spatial coordinate system of the BIM digital twin basic model; While acquiring on-site image or video data, record the acquisition time and acquisition location information corresponding to each image or video frame; The acquired on-site images or video data are organized in chronological order, and the acquisition time and location information are bound to the corresponding image or video frames.

4. The method for detecting defects in a BIM model based on digital twins according to claim 1, characterized in that, Step three specifically includes: Based on the BIM digital twin basic model, the component type code, system affiliation code and functional attribute field of each BIM component are read, and the component type code and system affiliation code are input as discrete semantic tags into the preset component semantic template library; Semantic mapping processing is performed on component type codes and system affiliation codes. By matching tags in the component semantic template library, the standard component name and system name corresponding to each BIM component are determined, and component-level semantic tag pairs are generated. The component-level semantic tags are concatenated with the functional attribute fields to generate textual semantic description entries according to the preset natural language template. Perform deduplication and normalization processing on the generated textual semantic description entries; The textual semantic description entries, after being deduplicated and normalized, are collected to form a textual semantic description set corresponding to the BIM component set in the BIM digital twin basic model, and the textual semantic description set is stored as structured text prompt data.

5. The method for detecting defects in a BIM model based on digital twins according to claim 1, characterized in that, The improved YOLO-World model includes a BIM structured semantic prompting construction module, a text embedding encoding module, a site image multi-scale feature extraction module, a BIM constraint-guided path aggregation module, and a dual-branch component detection output module. The BIM structured semantic prompt construction module receives a set of textual semantic descriptions as input, breaks down each textual semantic description entry into a system affiliation field, a component name field, and a functional attribute field, and concatenates the strings to generate a standardized text prompt sequence. The text embedding encoding module adopts the CLIP text encoder structure to vectorize the standardized text prompt sequence. The CLIP text encoder includes a word embedding layer, a position encoding layer, and a multi-layer Transformer encoding layer. In the output stage, the word vectors output by the Transformer encoding layer are subjected to average pooling to generate a text embedding vector corresponding to each standardized text prompt. The text embedding vector is then bound to its corresponding BIM component identifier and system attribution field to form a text embedding set. The on-site image multi-scale feature extraction module adopts a convolutional backbone network structure to extract features from on-site images or video data. The convolutional backbone network consists of multiple two-dimensional convolutional layers, batch normalization layers, and LeakyReLU activation function layers. It also achieves step-by-step downsampling through stride convolution and outputs multiple sets of image feature maps at different downsampling scales. The BIM constraint-guided path aggregation module, based on a path aggregation network structure, performs top-down and bottom-up feature transfer on multi-scale image feature maps. At each feature fusion node, it introduces text embedding vectors from the text embedding set. Specifically, based on the system affiliation and spatial partitioning information of components in the BIM digital twin basic model, it performs candidate filtering on the text embedding set, retaining only text embedding vectors that are consistent with the spatial region and system category corresponding to the current image feature map. It then performs channel-by-channel multiplication on the filtered text embedding vectors and the corresponding scale image feature map, and normalizes the multiplication result in the channel dimension using the Softmax normalization function. Finally, it performs channel-by-channel addition fusion on the normalized semantic modulation result and the image feature map to generate a semantic-visual fusion feature map containing BIM structural constraint information. The dual-branch component detection output module receives the fused feature map as input. The first detection branch uses a convolutional regression layer to perform bounding box regression on the fused feature map and normalizes the regression result using the Sigmoid function to output the component candidate box position. At the same time, it outputs the component semantic matching result through the Softmax classification layer. The second detection branch uses an attribute discrimination network composed of two fully connected layers and the ReLU activation function to perform attribute mapping on the fused feature map and output the key attribute labels of the component. The component candidate box, the component semantic matching result, and the key attribute labels of the component together constitute the on-site component candidate set.

6. The method for defect detection in a BIM model based on digital twins according to claim 1, characterized in that, Step five specifically includes: Based on the candidate set of on-site components and the set of BIM components in the BIM digital twin basic model, a candidate matching pair set between on-site component candidates and BIM components is established; For each candidate matching pair, a spatial matching score is calculated based on the spatial location parameters of the candidate frame of the on-site component and the spatial location parameters of the BIM component. A semantic consistency score is calculated based on the semantic matching results of on-site component candidates and the component type information and system affiliation information of BIM components. The consistency score of the attributes is calculated based on the key attribute labels of on-site component candidates and the functional attribute fields of BIM components. The spatial matching score, semantic consistency score, and attribute consistency score are weighted and fused according to a preset weight ratio to generate a comprehensive matching score for candidate matching pairs; In the candidate matching pair set corresponding to each on-site component candidate, the BIM component with the highest comprehensive matching score is selected as the mapping object of the on-site component candidate. When the comprehensive matching score is greater than the preset mapping threshold, a mapping relationship is established between the on-site component candidate and the BIM component. When the comprehensive matching score is less than or equal to the preset mapping threshold, the on-site component candidate is determined to be an unmapped component candidate.

7. The method for detecting defects in a BIM model based on digital twins according to claim 1, characterized in that, Step six specifically includes: Based on the mapping relationship between on-site component candidates and BIM components, consistency determination processing is performed on each BIM component in the BIM digital twin basic model. The consistency determination is carried out in the order of mapping existence determination, spatial consistency determination, system consistency determination and attribute consistency determination. For each BIM component, a mapping existence determination is performed. When a BIM component does not correspond to any on-site component candidate in the mapping relationship, the BIM component is determined to be a defective component that has not formed a valid mapping, and the defect type is marked as a component missing defect. For BIM components that form a mapping relationship, a spatial consistency determination is performed based on the spatial location parameters of the BIM component and the spatial location parameters of the corresponding on-site component candidate frames. Under the premise of passing the spatial consistency judgment, the system consistency judgment is performed based on the system ownership information of BIM components and the system ownership label in the semantic matching result of the candidate on-site components. Under the premise of passing the system consistency judgment, the attribute consistency judgment is performed based on the functional attribute fields of BIM components and the key attribute labels of candidate on-site components; When a BIM component passes the mapping existence determination, spatial consistency determination, system consistency determination and attribute consistency determination in sequence, the BIM component is determined to be a consistent component and is not recorded as a defective component. All BIM components identified as defective are aggregated to form the BIM model defect detection results.

8. The method for defect detection in a BIM model based on digital twins according to claim 1, characterized in that, Step seven specifically includes: Based on the defect detection results of the BIM model, the defective components in the BIM digital twin basic model are updated according to the defect type. For components with missing defects, the corresponding component identifier is marked as missing, and the timestamp information corresponding to the missing status is recorded. For defective components with inconsistent spatial locations, the geometric center coordinates and bounding box dimensions of the BIM component are updated based on the spatial location parameters of the corresponding on-site component candidate frames. For defective components with inconsistent system attribution, update the system attribution information of the BIM component to a system attribution code that is consistent with the semantic matching result of the candidate components on site; For defective components with inconsistent component attributes, the functional attribute fields of the BIM components are updated based on the key attribute tags of the candidate components on site. After updating various defective components, the updated component information is written into the BIM digital twin base model to generate the updated BIM digital twin model, and a model version identifier is assigned to the updated BIM digital twin model.

Images