BIM component automatic recognition and warehousing method and system based on multi-modal artificial intelligence fusion

Abstract

The application provides a BIM component automatic identification warehousing method and system based on multi-modal artificial intelligence fusion, relates to the field of building information technology, and comprises the following steps: receiving a BIM component model file, and preprocessing an obtained attribute parameter table; inputting information into a multi-modal recognition engine, outputting classification results and confidence through Rapidfuzz fuzzy matching of a file name, OPEN-CV+YOLOv8 image recognition and knowledge graph analysis parameters; a dynamic fusion decision module calculates a final confidence according to a dynamic weight, solves classification conflicts in combination with a dynamic rule base of 'industry specification rule priority', and determines automatic warehousing or triggers hierarchical artificial review according to a confidence threshold; and artificial review data is collected to return to the multi-modal recognition engine, so that incremental knowledge self-learning is realized. Through multi-modal fusion and dynamic decision, the application significantly improves the BIM component recognition accuracy and warehousing efficiency, reduces human errors, and realizes dynamic updating of a component resource library and system self-evolution.

Description

Technical Field

[0001] This invention relates to the field of building information technology, and in particular to a method and system for automatic identification and database entry of BIM components based on multimodal artificial intelligence fusion. Background Technology

[0002] In the field of architectural engineering design, BIM components, as standardized and parametric components for constructing 3D building models, can effectively reduce the workload of designers in repetitive modeling and improve design efficiency. With the popularization of BIM technology, the number and types of component models continue to grow, making the construction and maintenance of an enterprise-level BIM component resource library a key link in improving design quality and promoting team collaboration.

[0003] Currently, the industry generally adopts a multi-stage, manual review process for the management of BIM component libraries. The specific process is as follows: First, designers manually select the professional category of the BIM component model (such as structural, MEP, architectural, etc.); then, the component files are uploaded to the management system and enter a three-level review process within the profession (such as self-review by the designer, second review by the professional head, and third review by the department head); after the review is passed, it needs to be submitted to the company-level BIM experts for further review and standardization (such as parameter format unification and naming standard verification); finally, the BIM resource library administrator manually completes the component entry operation.

[0004] The existing technology has the following significant drawbacks:

[0005] 1. Cumbersome and inefficient process: Multiple manual review steps result in a long component entry cycle. Especially when uploading components in batches, the workload of manual operation increases exponentially, making it difficult to meet the design project's need for rapid access to component resources.

[0006] 2. Reliant on human experience and prone to errors: Key aspects such as component classification and parameter verification rely on the professional experience of the reviewers. Human judgment bias (such as misclassification or missed parameter checks) can lead to inconsistent quality of components entering the warehouse, affecting the accuracy of subsequent model building.

[0007] 3. Difficulty in maintaining the resource library: Due to the lack of an automated update mechanism, the manual review process cannot promptly include new types of components and components with optimized parameters into the resource library, resulting in insufficient timeliness and completeness of the component resource library, making it difficult to support the continuous innovation needs of architectural engineering design. Summary of the Invention

[0008] In view of this, the purpose of this invention is to provide a method and system for automatic identification and database entry of BIM components based on multimodal artificial intelligence fusion, which aims to solve the problems of low efficiency, error susceptibility and difficulty in maintaining the resource database caused by the reliance on manual multi-stage review for BIM component database entry.

[0009] The technical solution adopted by this invention to solve its technical problem is:

[0010] A method for automatic identification and database entry of BIM components based on multimodal artificial intelligence fusion is provided, including the following steps:

[0011] S1: Receive the uploaded BIM component model file, and obtain the file name, shape snapshot and attribute parameter table through preprocessing;

[0012] S2: Input the above information into the multimodal recognition engine, and perform fuzzy matching, image recognition and knowledge graph analysis respectively to obtain the corresponding classification results and confidence scores;

[0013] S3: Input the classification results and confidence scores into the dynamic fusion decision module. Calculate the confidence scores based on the classification results obtained from fuzzy matching, image recognition, and knowledge graph analysis. Obtain the weighted confidence scores of the recognition results. Resolve any conflicts in the classification results and directly store the results without conflicts into the database.

[0014] S4: If the final confidence level is higher than the preset final confidence level threshold, the data will be entered into the database through the real-time interaction interface with the BIM resource library; if it is lower than the preset final confidence level threshold, a manual review prompt will be triggered.

[0015] S5: Collect unidentified BIM component data that has been manually reviewed, and feed it back into the multimodal recognition engine training dataset to achieve incremental knowledge self-learning.

[0016] Preferably, in step S2:

[0017] Fuzzy matching: Based on the Rapidfuzz fuzzy matching library, the filename is matched with the existing component library classification table, and the highest classification result with a similarity higher than a preset similarity threshold is returned. and corresponding confidence level ;

[0018] Image recognition: First, the image of the shape snapshot is binarized and masked using a computer vision library. The number of white pixels in the mask is counted, and the occlusion rate is calculated as the ratio of the number of white pixels to the total number of pixels in the image. Then, a pre-trained deep learning model is used to identify the component category corresponding to the shape snapshot image, and the highest classification result with a confidence level higher than a preset image confidence threshold is returned. and corresponding confidence level ;

[0019] Knowledge graph analysis: Using a pre-trained knowledge graph engine, the completeness and rationality of the parameters in the attribute parameter table are verified. Simultaneously, the corresponding component categories are matched, and the highest classification result with a credibility score higher than the preset knowledge graph credibility threshold is returned. and corresponding confidence level .

[0020] Preferably, in step S3, the confidence level is calculated as follows: A base weight is set: image recognition weight. 0.4, fuzzy matching weight The weight for knowledge graph analysis is 0.3. The initial weight is 0.3; if the occlusion rate is >40%, the weights are adjusted to 0.3 for image recognition and 0.4 for knowledge graph analysis, while the weight for fuzzy matching remains unchanged; and the weight is calculated using the weighted average formula:

[0021]

[0022] Calculate the final confidence score (FinalScore).

[0023] Preferably, in step S3, the specific method for conflict resolution is as follows: calling a dynamic priority rule base, which includes industry standard rules, attribute parameter rules, spatial positioning rules, and component classification rules, with an initial priority order of "industry standard rules > attribute parameter rules > spatial positioning rules > component classification rules"; performing rule matching on each BIM component one by one through the rule base matching engine, sorting them from high to low priority, executing the highest priority rule, and outputting the judgment result.

[0024] Preferably, in step S3, the conflict resolution process further includes a rule optimization step: real-time statistics of the success rate of each rule; if the success rate of a certain rule is <85%, the features of the failed cases of that rule are extracted, new judgment conditions are generated by analyzing the reasons for failure, and updated to the dynamic priority rule base.

[0025] Preferably, in step S2, the computer vision library is the OPEN-CV library, the pre-trained deep learning model is the YOLOv8 model, the preset similarity threshold is 80%, and the preset image credibility threshold and the preset map credibility threshold are both 85%.

[0026] Preferably, in step S1, the preprocessing includes BIM model file format parsing, shape snapshot capture, and attribute parameter extraction; the attribute parameter table includes geometric parameters and professional parameters, wherein the geometric parameters include length, width, height, and elevation, and the professional parameters include flow rate, head, and cooling capacity.

[0027] Preferably, in step S4, the preset final confidence threshold is 90%; if the final confidence is in the range of 70%-90%, the "semi-manual review" mode is triggered, requiring professional reviewers to confirm the classification results without needing to repeatedly verify the attribute parameter table; if the final confidence is <70%, the "full-process manual review" mode is triggered, requiring reviewers to confirm the classification results and verify the completeness and accuracy of the attribute parameter table, and supplement any missing parameters.

[0028] Preferably, in step S5, the manually reviewed unidentified BIM component data includes: the correct classification results of manual annotation, the corrected attribute parameter table, the manual judgment basis for conflict resolution, and the explanation of the reasons for unidentification; after collection, the data needs to be cleaned and then organized according to the association format of "classification result-component feature-parameter table" and fed back to the training dataset of the multimodal recognition engine. Among them, the fuzzy matching library adds new component names / aliases, the pre-trained deep learning model adds corresponding component image samples, and the knowledge graph adds new "component category-attribute parameter" mapping relationship.

[0029] It also provides an automatic BIM component identification and database entry system based on multimodal artificial intelligence fusion, including a preprocessing module, a multimodal recognition engine module, a dynamic fusion decision module, a database entry interaction module, and an incremental learning module;

[0030] The preprocessing module is used to receive single or batch BIM component model files uploaded by users, and automatically obtain the file name, shape snapshot and attribute parameter table of BIM components through format parsing, image cropping and parameter extraction, and synchronously transmit the above information to the multimodal recognition engine module.

[0031] The multimodal recognition engine module includes a built-in Rapidfuzz fuzzy matching submodule, an OPEN-CV+YOLOv8 image recognition submodule, and a pre-trained knowledge graph analysis submodule, which respectively perform fuzzy matching, image recognition, and knowledge graph analysis, and output the corresponding classification results and confidence scores to the dynamic fusion decision module.

[0032] The dynamic fusion decision module includes a confidence calculation unit and a conflict resolution unit. The confidence calculation unit calculates the final confidence score (FinalScore) according to the basic weight and weighted average calculation formula. The conflict resolution unit handles classification conflicts according to the conflict resolution mechanism and outputs the final judgment result and FinalScore to the database interaction module.

[0033] The entry interaction module establishes a real-time data interaction interface with the BIM resource library. If the FinalScore is higher than the preset threshold, the component classification and entry into the library will be completed automatically; otherwise, the manual review interface will be triggered and the review task will be recorded synchronously.

[0034] The incremental learning module is used to collect labeled data generated during the manual review process, and after data cleaning, it feeds back into the training dataset of the multimodal recognition engine module to drive the incremental updates of the fuzzy matching library, YOLOv8 model and knowledge graph.

[0035] The beneficial effects of this invention are:

[0036] This invention provides a method and system for automatic identification and database entry of BIM components based on multimodal artificial intelligence fusion. Through the collaboration of a preprocessing module and a multimodal recognition engine (Rapidfuzz fuzzy matching, OPEN-CV+YOLOv8 image recognition, and pre-trained knowledge graph analysis), it automatically completes component file name matching, shape recognition, and parameter verification, replacing the traditional multi-stage manual review process. This significantly shortens the BIM component database entry cycle, reduces the manual workload during batch uploads, and solves the problem of low efficiency in existing processes. Through a dynamic fusion decision module with occlusion rate-driven weight adjustment (increasing the knowledge graph analysis weight to 0.4 and decreasing the image recognition weight to 0.3 when the occlusion rate is >40%) and a dynamic rule base conflict resolution mechanism prioritizing industry standard rules, combined with an optimization mechanism that sets an 85% success rate threshold for each rule, it reduces classification misjudgments and parameter omissions caused by human experience bias. To ensure the accuracy of component classification and the completeness of attribute parameters, the system addresses the error-prone nature of existing processes that rely on human experience. The incremental learning module collects manually reviewed data on unidentified components (including correct classification results and corrected parameters), cleans it, and feeds it back into the multimodal recognition engine's training dataset. This enables incremental updates to the fuzzy matching library, YOLOv8 model, and knowledge graph, allowing for the timely inclusion of new component types and optimized parameters into the BIM resource library, resolving the timeliness and completeness issues of the existing resource library. Simultaneously, the system employs a "semi-manual review" mode (confirming only classification) triggered by 70%-90% confidence and a "full-process manual review" mode (verifying classification and parameters) triggered by <70% confidence, balancing automation efficiency with review accuracy. The system's self-evolutionary capability continuously improves recognition accuracy, providing efficient, accurate, and sustainable technical support for BIM component digital asset management in the field of architectural engineering design. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the structure of the present invention.

[0038] Figure 2 This is a flowchart illustrating the dynamic fusion decision module of the present invention.

[0039] Figure 3 This is a flowchart illustrating the incremental knowledge learning module of the present invention.

[0040] Figure 4 This is a flowchart illustrating the conflict resolution mechanism of the present invention.

[0041] It should be noted that these accompanying drawings and textual descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art by referring to specific embodiments. Detailed Implementation

[0042] This embodiment takes the automated management needs of an enterprise-level BIM component resource library in the architectural design field as its application scenario. Relying on the method and system of this invention, it replaces the traditional manual, multi-stage review process, achieving intelligent processing of BIM component identification, classification, and storage. Specific implementation details strictly follow the core technical solution and do not involve specific structural data, focusing only on the methodological process and system collaboration logic. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0043] Detailed implementation steps: as follows Figures 1-4 As shown

[0044] Step S1: Receiving and Preprocessing BIM Component Model Files

[0045] File reception: Supports users to upload BIM component model files individually or in batches through the system interface, is compatible with mainstream BIM model formats, and automatically filters invalid or damaged files;

[0046] Format parsing: Extracts metadata and validates the format of uploaded files to ensure they can be read correctly;

[0047] Shape snapshot capture: Automatically captures snapshots of BIM component models from multiple perspectives to generate shape images for image recognition;

[0048] Attribute parameter extraction: Automatically extract the attribute parameter table of the components from the model file, including geometric parameters (such as length, width, height, and elevation) and professional parameters (such as flow rate, head, and cooling capacity), to form structured parameter data.

[0049] Step S2: Multimodal recognition engine processing

[0050] The "filename + shape snapshot + attribute parameter table" obtained in step S1 are synchronously input into the multimodal recognition engine. The three sub-modules execute the recognition task in parallel and output the classification results and corresponding confidence scores respectively:

[0051] Rapidfuzz fuzzy matching: Based on a preset component library classification table, it performs fuzzy matching on component file names, filters out the highest matching category with a similarity higher than a preset threshold (80%), and generates classification results. and confidence level ;

[0052] Open-CV + YOLOv8 Image Recognition: The component outline snapshot is binarized and masked using Open-CV, and the percentage of white pixels is counted to calculate the image occlusion rate. A pre-trained YOLOv8 model is then used to identify features in the snapshot, matching the most suitable component category to generate a classification result. and confidence level (Only results with a confidence level higher than the preset threshold (85%) are retained);

[0053] Knowledge graph analysis: Using a pre-trained knowledge graph engine, the completeness and rationality of the attribute parameter table are verified, and corresponding component categories are matched to generate classification results. and confidence level (Only results with a confidence level higher than the preset threshold (85%) are retained).

[0054] Step S3: Dynamic Fusion Decision

[0055] The identified results are input into the dynamic fusion decision module. The confidence level is calculated based on the classification results obtained from fuzzy matching, image recognition, and knowledge graph analysis. The weighted confidence level of the identification results is obtained. (Multimodal fusion deeply combines "file name semantic information, component shape visual features, and attribute parameter logical constraints", which can compensate for the shortcomings of single modality: fuzzy matching provides a preliminary directional anchor for classification, image recognition verifies physical form through visual features, and knowledge graph verifies the rationality of categories based on parameter constraints. The three work together to greatly reduce the risk of misjudgment caused by the failure of single modality.)

[0056] If the occlusion rate does not exceed the threshold, the basic weights (image recognition 0.4, knowledge graph analysis 0.3, fuzzy matching 0.3) are used. The weights of image recognition and fuzzy matching can ensure basic recognition capabilities and effectively adapt to complex situations commonly encountered in architectural design scenarios such as "non-standard file names, severe image occlusion, and incomplete parameters". If the image occlusion rate exceeds the preset threshold (e.g., 40%) (the effectiveness of visual information is reduced), the image recognition weight is automatically reduced, the knowledge graph analysis weight is increased, and the fuzzy matching weight remains unchanged to avoid a decrease in overall recognition accuracy due to the failure of visual features.

[0057] According to the weighted average formula The final confidence score (FinalScore) for component identification is calculated.

[0058] like Inconsistencies are identified as classification conflicts. When multimodal recognition results conflict (e.g., inconsistencies between fuzzy matching and image recognition classification results), the fused multimodal data can provide a more comprehensive basis for conflict resolution. The system calls the dynamic priority rule base and performs rule matching on each component according to the priority of "industry standard rules > attribute parameter rules > spatial positioning rules > component classification rules". The highest priority matching rule is executed to determine the final classification result. The success rate of each rule is counted in real time. If the success rate of a certain rule is lower than a preset threshold (e.g., 85%), the features of the failed cases of that rule are extracted, the reasons for failure are analyzed, new judgment conditions are generated, and updated to the dynamic rule base.

[0059] Step S4: Inbound Inspection and Manual Review

[0060] If the final confidence score (FinalScore) is higher than the preset threshold (e.g., 90%), the entry interaction module will automatically classify the components into the corresponding category directory through the real-time interface with the BIM resource library and synchronize the attribute parameter table. If the FinalScore is within the preset range (e.g., 70%-90%), the "semi-manual review" mode will be triggered, requiring only professional reviewers to confirm the classification results without repeated verification of attribute parameters. If the FinalScore is lower than the preset lower limit (e.g., 70%), the "full-process manual review" mode will be triggered, requiring reviewers to confirm the classification results and verify the completeness and accuracy of the attribute parameter table, supplementing any missing parameters.

[0061] Step S5: Incremental Knowledge Self-Learning

[0062] Key data generated during the manual review process is collected, including correctly labeled classification results, corrected attribute parameter tables, manual judgment criteria for conflict resolution, and explanations of reasons for non-identification. The collected data is cleaned, invalid or duplicate information is removed, and the data is organized according to the association format of "classification result-component feature-parameter table" to form standardized training data. The standardized data is fed back into the training dataset of the multimodal recognition engine: new component names and aliases are added to the fuzzy matching library; corresponding component shape snapshot samples are added to the YOLOv8 model; and new "component category-attribute parameter" mapping relationships are added to the knowledge graph to achieve incremental updates to the system's recognition capabilities.

[0063] III. System Module Composition and Collaboration

[0064] The system of this invention achieves a closed-loop process for automatic identification and storage of BIM components through the collaboration of five core modules. The functions of each module are as follows:

[0065] Module Name Core Functions Preprocessing module Receive BIM component model files, complete format parsing, shape snapshot capture and attribute parameter extraction, and synchronously transmit data to the multimodal recognition engine. Multimodal recognition engine module It has three built-in sub-modules: fuzzy matching, image recognition, and knowledge graph analysis. These modules perform recognition tasks in parallel and output classification results and corresponding confidence scores. Dynamic Fusion Decision Module The final confidence score is calculated based on the multimodal recognition results. Classification conflicts are handled and rules are optimized through a dynamic rule base, and the final judgment result is output. Inbound interaction module Establish a real-time interactive interface with the BIM resource library to enable automatic data entry or trigger tiered manual review, and synchronously record review tasks and results. Incremental learning module Collect and clean manually reviewed data to feed back into the multimodal recognition engine, driving incremental updates to the fuzzy matching library, deep learning model, and knowledge graph. Specific Implementation

[0066] This embodiment uses the BIM component resource library upgrade project of a large architectural design company (hereinafter referred to as "the Company") as an application scenario. The Company currently has over 5,000 BIM components, covering three major disciplines: structure, mechanical and electrical engineering, and architecture. The traditional manual entry process takes an average of 2 hours to review a single component, and 2 working days for a batch of 100 components, with a classification accuracy of approximately 82%. This embodiment achieves efficient and accurate automatic component identification and entry into the library through the method and system of this invention. The specific implementation details are as follows:

[0067] Implementation Environment Configuration

[0068] The hardware environment is deployed on an application server with multi-format file parsing capabilities, supporting parallel reception and preprocessing of single / batch BIM component files; the multimodal recognition engine module is deployed on an AI server with graphics computing capabilities, meeting the computing power requirements for computer vision processing (image masking, feature recognition) and knowledge graph analysis; the BIM resource library is deployed on a stable cloud database or local database, supporting component classification directory storage, attribute parameter association, and historical data backtracking.

[0069] Software and Model Preparation: Computer Vision Library: OPEN-CV library is used for binarization masking of component shape snapshots; Deep Learning Model: Pre-trained YOLOv8 model is used for feature recognition of snapshots; Fuzzy Matching Library: Based on Rapidfuzz library, importing the standard classification table of BIM components in the building industry (including primary and secondary classifications and common aliases); Knowledge Graph Engine: Built based on graph database, storing the mapping relationship between "component category - attribute parameter" (including parameter integrity constraints and reasonable value ranges).

[0070] Thresholds and rule presets: Recognition thresholds: Fuzzy matching similarity threshold, image recognition confidence threshold, and knowledge graph analysis confidence threshold are all set according to industry-standard accuracy requirements; Weighting rules: The initial weight allocation is image recognition weight > knowledge graph analysis weight = fuzzy matching weight. When the occlusion rate exceeds the preset threshold, the image recognition weight is reduced and the knowledge graph analysis weight is increased; Entry thresholds: The preset final confidence level has three levels: high / medium / low, corresponding to three modes: automatic entry, semi-manual review, and full-process manual review.

[0071] Specific implementation steps

[0072] Step S1:

[0073] Users can upload BIM component model files individually or in batches through the system's interactive interface (such as the web or client). The system automatically filters invalid, damaged, or unsupported format files, ensuring compatibility with mainstream BIM model formats. It extracts metadata and verifies the format of valid files to ensure the model structure and parameters can be read correctly. It automatically captures snapshots of the component model's shape from multiple perspectives, including front, side, and top views, selecting snapshots with sufficient clarity for recognition as image recognition data sources. It automatically extracts a structured attribute parameter table from the model's metadata, including geometric parameters (length, width, height, material, elevation) and professional parameters (such as flow rate and head for electromechanical components, and heat transfer coefficient for building components), and performs preliminary parameter format verification.

[0074] Step S2:

[0075] The "filename + shape snapshot + attribute parameter table" obtained by S1 is input into the multimodal recognition engine, and the three sub-modules process it in parallel:

[0076] Rapidfuzz fuzzy matching: Based on the component library classification table, semantic fuzzy matching is performed on filenames, and the highest matching category with similarity higher than a preset threshold is selected to generate classification results. and reliability If no results meet the threshold, mark them as "pending match" and set the confidence level to 0.

[0077] OPEN-CV+YOLOv8 Image Recognition: Open-CV is used to preprocess the shape snapshot by converting it to grayscale and denoising. Then, a binarized mask is applied, and the percentage of white pixels (number of white pixels / total number of pixels in the image) is calculated to obtain the occlusion rate. The YOLOv8 model is used to extract the shape features of the snapshot, match the most suitable component category, and only retain results with a confidence level higher than a preset threshold to generate the classification result. and confidence level ;

[0078] Knowledge graph analysis: The knowledge graph engine verifies the completeness (whether key parameters are missing) and rationality (whether parameter values ​​are within a reasonable range) of the attribute parameter table, while matching the corresponding component categories, retaining results with a credibility higher than a preset threshold, and generating classification results. and confidence level .

[0079] Step S3:

[0080] If the occlusion rate is less than or equal to the preset threshold, the basic weight is used: fuzzy matching weight. =0.4, image recognition weight =0.3, fuzzy matching weight =0.3; Final confidence level calculation: according to the weighted average formula: ; Obtain the final confidence score (FinalScore) for component recognition; If the occlusion rate is greater than the preset threshold, reduce the image recognition weight, increase the knowledge graph analysis weight, and keep the fuzzy matching weight unchanged;

[0081] Conflict identification and resolution: If , If inconsistencies exist, it is determined to be a classification conflict; the dynamic rule base is invoked: the dynamic priority rule base is invoked, and the rule matching is performed on the components one by one according to the initial priority (such as industry standard rules first checking whether the components conform to the national standard classification); the highest priority matching rule is executed to determine the final classification result;

[0082] Rule optimization: Real-time statistics of the success rate of each rule (the proportion of judgment results that are consistent with the manual review results). If the success rate of a certain rule is lower than the preset optimization threshold, extract the characteristics of the failed cases of that rule (such as component type, conflict scenario, parameter characteristics), analyze the reasons for failure (such as insufficient scope of application of the rule, missing parameter constraints), generate new judgment conditions and update them to the rule base.

[0083] Step S4:

[0084] Automatic entry into the database: If the FinalScore is higher than the preset high threshold, the entry interaction module connects with the BIM resource library through the real-time data interface to automatically classify the components into the corresponding category directory and synchronize the attribute parameter table.

[0085] Semi-manual review: If the FinalScore is within the preset threshold range, the system will trigger the "semi-manual review" interface. Only professional reviewers need to confirm the classification results, without having to repeatedly verify the attribute parameters (reusing the parameters extracted in the preprocessing).

[0086] Full-process manual review: If the FinalScore is lower than the preset low threshold, the system will trigger the "full-process manual review" interface. The reviewer needs to confirm the classification results, verify the completeness and accuracy of the attribute parameter table, and complete the entry into the database after supplementing the missing parameters.

[0087] The system can synchronously record audit task information (component ID, audit mode, auditor, audit result) and support subsequent traceability.

[0088] Step S5:

[0089] Key data generated during the manual review process is collected, including: correctly labeled classification results, corrected attribute parameter tables, manual judgment criteria for conflict resolution, and explanations of reasons for non-identification (such as new types of components, severe occlusion, incomplete parameters). The collected data is preprocessed to remove duplicate data, correct labeling errors, and standardize parameter formats to form standardized training data. This standardized data is then fed back into the multimodal recognition engine according to the association format of "classification result - component features - parameter table."

[0090] Fuzzy matching library: Added new component names and common aliases;

[0091] YOLOv8 model: Add snapshot samples of the shape of unidentified components for incremental training;

[0092] Knowledge Graph: Added new "component category - attribute parameter" mapping relationship (including new parameter constraints);

[0093] After incremental training, the system's recognition accuracy is tested using a validation dataset to ensure that the optimized recognition capability meets the requirements.

[0094] Finally, it should be noted that steps S1-S5 of the above method can be converted into software program instructions. These instructions can be implemented using a system including a processor and memory, or by computer instructions stored in a non-transitory computer-readable storage medium. The aforementioned software functional unit is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the method described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0095] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A BIM component automatic identification and warehousing method based on multi-modal artificial intelligence fusion, characterized in that, The method comprises the following steps: S1: receiving an uploaded BIM component model file, obtaining the file name, shape snapshot and attribute parameter table through preprocessing; S2: inputting the file name, shape snapshot and attribute parameter table information obtained through preprocessing into a multi-modal recognition engine, respectively performing fuzzy matching, image recognition and knowledge graph analysis to obtain corresponding classification results and confidence levels; S3: inputting the classification results and confidence levels into a dynamic fusion decision module, performing confidence level calculation according to the classification results obtained through fuzzy matching, image recognition and knowledge graph analysis, obtaining the confidence level after weighted calculation of the recognition result, and solving the conflict for the classification results without conflict; S4: if the final confidence level is higher than the preset final confidence threshold, completing the storage through the real-time interaction interface with the BIM resource library; if the final confidence level is lower than the preset final confidence threshold, triggering the artificial audit prompt; S5: collecting the un-recognized BIM component data of artificial audit, and feeding back the multi-modal recognition engine training data set to realize incremental knowledge self-learning; in the step S2: Fuzzy match: based on the Rapidfuzz fuzzy matching library, the file name is matched with the existing component library classification table, and the highest classification result with a similarity higher than a preset similarity threshold is returned and the corresponding confidence ; Image recognition: first, the picture of the shape snapshot is processed by binary mask through a computer vision library, the number of white pixels in the mask is counted, and the occlusion rate is calculated by the ratio of the number of white pixels to the total number of pixels; then the pre-trained deep learning model is used to identify the component category corresponding to the shape snapshot picture, and the highest classification result with a confidence higher than a preset image confidence threshold is returned and the corresponding confidence ​ Knowledge graph analysis: through a pre-trained knowledge graph engine, the integrity and rationality of the parameters in the attribute parameter table are checked, and the corresponding component category is matched, and the highest classification result with a confidence higher than a preset graph confidence threshold is returned and the corresponding confidence ; The specific manner of the confidence calculation in the step S3 is: setting a basic weight: an image recognition weight is 0.4, a fuzzy matching weight is 0.3, and a knowledge graph analysis weight is 0.3; if the occlusion rate is greater than 40%, the weights are adjusted as follows: image recognition 0.3, knowledge graph analysis 0.4, and the fuzzy matching weight remains unchanged; and a weighted average calculation formula is used: Compute the final confidence score FinalScore.

2. The BIM component automatic identification warehousing method based on multi-modal artificial intelligence fusion according to claim 1, characterized in that: In the step S3, the specific way of conflict resolution is: calling a dynamic priority rule library, the rule library includes industry specification rules, attribute parameter rules, spatial positioning rules and component classification rules, and the initial priority order is "industry specification rules> attribute parameter rules> spatial positioning rules> component classification rules"; each rule is matched through a rule library matching engine for the BIM component, and after sorting from high to low according to the priority, the highest priority rule is executed and the determination result is output.

3. The BIM component automatic identification warehousing method based on multi-modal artificial intelligence fusion according to claim 2, characterized in that: In the step S3, the rule optimization step is also included in the conflict resolution: the determination success rate of each rule is statistically calculated in real time, if the success rate of a certain rule is < 85%, the features of the failure cases of the rule are extracted, new determination conditions are generated by analyzing the failure reasons, and are updated to the dynamic priority rule library.

4. The BIM component automatic identification warehousing method based on multi-modal artificial intelligence fusion according to claim 1, characterized in that: In the step S2, the computer vision library is the OPEN-CV library, the pre-trained deep learning model is the YOLOv8 model, the preset similarity threshold is 80%, and the preset image confidence threshold and the preset graph confidence threshold are both 85%.

5. The BIM component automatic identification warehousing method based on multi-modal artificial intelligence fusion according to claim 1, characterized in that: In step S1, the preprocessing includes BIM model file format analysis, shape snapshot interception and attribute parameter extraction; the attribute parameter table includes geometric parameters and professional parameters, wherein the geometric parameters include length, width, height and elevation, and the professional parameters include flow, lift and refrigerating capacity.

6. The BIM component automatic identification warehousing method based on multi-modal artificial intelligence fusion according to claim 1, characterized in that: In the step S4, the preset final confidence threshold is 90%; if the final confidence level is in the interval of 70%-90%, the "semi-artificial audit" mode is triggered, and the classification result needs to be confirmed by a professional auditor without repeated verification of the attribute parameter table; if the final confidence level is < 70%, the "full-flow artificial audit" mode is triggered, and the classification result needs to be confirmed by an auditor, and the integrity and accuracy of the attribute parameter table are verified, and the missing parameters are supplemented.

7. The BIM component automatic identification warehousing method based on multi-modal artificial intelligence fusion according to claim 1, characterized in that: The artificial review un-recognized BIM component data in the step S5 includes: artificial labeling correct classification results, corrected attribute parameter table, conflict resolution artificial judgment basis, and un-recognized reason explanation; after collection, data cleaning is required, and then the data is sorted according to the association format of "classification result-component feature-parameter table", and is fed back to the training data set of the multi-modal recognition engine, wherein the fuzzy matching library supplements new component names / aliases, the pre-trained deep learning model adds corresponding component image samples, and the knowledge graph supplements new "component category-attribute parameter" mapping relationship.

8. A BIM component automatic identification and warehousing system based on multi-modal artificial intelligence fusion, characterized in that, The pre-processing module, the multi-modal recognition engine module, the dynamic fusion decision module, the storage interaction module and the incremental learning module for executing the multi-modal artificial intelligence fusion-based BIM component automatic recognition storage method of any one of claims 1-7 are included. The pre-processing module is used for receiving single or batch BIM component model files uploaded by a user, automatically obtaining the file name, shape snapshot and attribute parameter table of the BIM component through format analysis, image extraction and parameter extraction, and synchronously transmitting the above information to the multi-modal recognition engine module. The multi-modal recognition engine module: built-in Rapidfuzz fuzzy matching sub-module, OPEN-CV+YOLOv8 image recognition sub-module and pre-trained knowledge graph analysis sub-module, respectively performing fuzzy matching, image recognition and knowledge graph analysis, outputting corresponding classification results and confidence to the dynamic fusion decision module. The dynamic fusion decision module: contains a confidence calculation unit and a conflict resolution unit, the confidence calculation unit calculates the final confidence FinalScore according to the basic weight and weighted average calculation formula of claim 1, and the conflict resolution unit processes classification conflicts according to the conflict resolution mechanism of any one of claims 2-3, and outputs the final determination result and FinalScore to the storage interaction module. The storage interaction module: establishes a real-time data interaction interface with the BIM resource library, automatically completes component classification storage if the FinalScore is higher than the preset threshold, otherwise triggers the artificial review interface and synchronously records the review task. The incremental learning module: is used for collecting labeled data generated in the artificial review process, and feeding back the training data set of the multi-modal recognition engine module after data cleaning, driving the incremental update of the fuzzy matching library, YOLOv8 model and knowledge graph.

Images