Building component autonomous inspection system and method based on BIM (Building Information Modeling) and VLM (Visual Language Model)

Abstract

The invention discloses a building component autonomous inspection system and method based on a BIM (Building Information Modeling) and a VLM (Visual Language Model), and relates to the technical field of intelligent inspection of building engineering. Generating a spatial reference data set, a semantic tag set and a component semantic embedding vector; the method comprises the following steps: collecting multi-modal visual and spatial data, carrying out semantic segmentation and description generation by utilizing a visual language large model, calculating a semantic ambiguity coefficient and carrying out confidence evaluation; through BIM and visual semantic embedding vector comparison and spatial deviation analysis, calculating a cross-modal consistency index and generating a deviation report; calculating a comprehensive path cost coefficient and performing path optimization; and in combination with semantic recognition, cross-modal consistency and path cost, obtaining inspection performance evaluation indexes, realizing adaptive optimization through reinforcement learning, and writing back to a BIM database to realize continuous improvement of data and decision ability.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection technology for building engineering, specifically to an autonomous inspection system and method for building components based on BIM and Visual Language Model (VLM). Background Technology

[0002] With the continuous improvement of digitalization and intelligence in the construction industry, BIM technology has become an important tool for construction management and intelligent monitoring. BIM technology can digitally model the component information of building projects, recording the geometric attributes, spatial location, material type, construction status, and process information of the components, providing data support for construction management, quality control, and progress monitoring. However, traditional BIM technology can only provide design and static information and cannot reflect the dynamic changes on the construction site in real time, making it difficult to meet the needs for automatic detection and intelligent inspection of building components in complex construction environments.

[0003] Existing intelligent inspection methods for construction sites mainly rely on the following types of means: Manual inspections or fixed camera monitoring involve construction workers personally inspecting the site or remotely monitoring construction progress and quality via fixed cameras. These methods have significant drawbacks: they rely on human experience, are inefficient, are susceptible to human error or environmental factors, and cannot achieve real-time, continuous, and comprehensive inspections.

[0004] Robotic inspection based on point clouds and grid maps converts BIM models into point cloud and grid maps, guiding the robot to acquire real-time point cloud data using LiDAR, and completing localization and obstacle identification based on feature extraction and point cloud registration. While this method reduces human intervention, it lacks flexibility in handling dynamic obstacles in complex environments, relies heavily on point cloud computing, resulting in high computational costs and difficulty adapting to rapid changes on construction sites.

[0005] Automated inspection based on BIM sub-region division and preset inspection paths involves dividing the construction site into several sub-regions and setting inspection points, visual inspection points, and construction statuses in the BIM model. The robot acquires 2D and 3D data according to a preset inspection sequence and performs visual matching and quality inspection through edge computing. However, this method relies on pre-planned inspection paths and static BIM data, making it difficult to handle inconsistencies between design and actual construction or sudden changes in the construction environment.

[0006] The main problems with the existing technical methods mentioned above include: relying on manual intervention or static BIM data, which makes it impossible to achieve real-time intelligent judgment and adaptive adjustment; insufficient adaptability to complex dynamic environments, making it difficult to flexibly respond to obstacles or construction changes; inability to achieve deep semantic understanding and automatic quality assessment, with the system mostly only able to "see" the components but not "understand" the component status and quality issues; and high computational load for point cloud data processing, resulting in high costs and high hardware performance requirements, which is not conducive to large-scale promotion and application. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides an autonomous inspection system and method for building components based on BIM and Visual Language Large Model (VLM) to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: an autonomous inspection method for building components based on BIM and Visual Language Model (VLM), comprising the following steps: Step 1: In the initial stage of construction tasks, extract the spatial location, geometric information, attribute data, and textual semantic features of each component from the BIM model. Through natural language processing and vectorized embedding, generate a spatial benchmark dataset, a set of semantic labels for components, and semantic embedding vectors for components. And construct a set of component BIM semantic vectors; Step 2: Collect multimodal visual and spatial data of components through inspection robots, perform spatial reconstruction and correction using LiDAR and depth cameras, complete semantic segmentation, feature extraction and description generation of components using the Visual Language Model (VLM), calculate the semantic ambiguity coefficient UM and compare it with the semantic confidence threshold Uth to determine whether the image semantic recognition credibility of the component is qualified. If it is not qualified, an appropriate strategy is given. Step 3: By calculating the cosine similarity and spatial coordinate deviation between the BIM semantic embedding vector and the visual language large model semantic embedding vector, the semantic similarity parameters and spatial deviation parameters of the component are obtained; and the cross-modal consistency index CCI is calculated and compared with the cross-modal consistency threshold Cth to determine whether the semantic consistency of the component is qualified. If it is not qualified, a component deviation report is generated. Step 4: Collect robot position and obstacle information in real time using LiDAR, depth camera and positioning sensor, and combine BIM component coordinates and deviation reports to establish a path environment dataset; calculate the spatial distance and detour cost from the robot to the component to form a comprehensive path cost coefficient ZCX, and compare it with the path cost threshold Zth to determine whether the robot inspection path cost is reasonable. If it is not reasonable, perform path replanning and inspection sequence optimization based on deviation information. Step 5: Based on the semantic ambiguity coefficient UM, cross-modal consistency index CCI, and comprehensive path cost coefficient ZCX, obtain the inspection performance evaluation index; achieve adaptive optimization of parameters through reinforcement learning, and write the optimization results back to the BIM database to form a self-learning feedback mechanism to continuously improve data and intelligent decision-making capabilities.

[0009] Preferably, step one includes: S11. During the initialization phase of the construction task, the BIM model of the building project is parsed and component-level semantic information is extracted; by loading the structural identification files of each component into the BIM model, the geometric center coordinates of the components are extracted in real time. A spatial reference dataset is established; an initial pose is provided to the inspection robot by means of positioning equipment and manual specification; based on the initial pose, feature-based SLAM registration and coordinate fusion technology is used to establish the initial association between the robot coordinate system and the BIM global coordinate system. S12. By calling the attribute database interface of the BIM model, extract the material type, installation status, construction stage number, and task priority parameters of the components. The semantics of the components are classified according to their category, functional division and hierarchical structure of work procedures, and a set of semantic tags for the components is generated. S13. By calling the BIM data interface, collect the textual attribute descriptions and design specifications of the components, perform natural language parsing, and use a Transformer-based natural language parsing algorithm to perform semantic segmentation, dependency parsing, and keyword extraction on the collected text data to extract key semantic feature words of the components; and use a vectorized embedding model to numerically encode the semantic feature words to obtain the semantic embedding vector of the components. ; S14, By combining the spatial position parameters of the components Geometric parameters, task priority parameters With component semantic embedding vector Construct a set of component BIM semantic vectors.

[0010] Preferably, step two includes: S21. Collect surface image information of components using high-definition cameras, LiDAR, and depth cameras mounted on the inspection robot. and spatial coordinate parameters This forms a multimodal visual perception dataset; S22. Based on the data from the multimodal visual perception dataset, LiDAR point cloud scanning and depth camera ranging technology are used to reconstruct the spatial structure and correct the image position, and obtain the surface normal vector parameters and spatial depth distribution characteristics of the components. S23. Based on the surface normal vector parameters and spatial depth distribution features of the component, a well-trained visual language model (VLM) is used for semantic segmentation and target recognition to obtain the regional feature parameters, texture feature parameters and boundary feature parameters of the component, forming a semantic feature set of the component. S24. By leveraging the language generation capabilities of the Visual Language Model (VLM), semantic understanding and description generation are performed on the semantic feature set of components to obtain component state description text. The text is then embedded using a semantic encoder to obtain the corresponding semantic embedding vector. .

[0011] Preferably, step two further includes: S25. Semantic embedding vectors based on the Visual Language Model (VLM) By using a similarity matching method, the probability distribution of the semantic tags to which the components belong is obtained, and then the semantic classification probability distribution function is obtained. And calculate and obtain the overall confidence index PLI for image semantic recognition; S26. Based on the overall confidence index PLI of image semantic recognition, calculate the semantic ambiguity coefficient UM of the component image; S27, by setting a preset semantic confidence threshold Uth, and comparing and analyzing the semantic ambiguity coefficient UM of the component image with the semantic confidence threshold Uth, the first evaluation result is obtained, including: When the semantic ambiguity coefficient UM of the component image is less than or equal to the semantic confidence threshold Uth, it indicates that the semantic recognition credibility of the component image is qualified and will be continuously monitored. When the semantic ambiguity coefficient UM of a component image exceeds the semantic confidence threshold Uth, it indicates that the semantic recognition confidence of the component image is unqualified, triggering the first warning instruction and generating the first strategy: to re-plan the viewpoint, using a multi-objective optimization algorithm based on semantic embedding vectors. The system uses the overall confidence index PLI for image semantic recognition to automatically obtain the best suboptimal viewpoint parameter set of the component, re-plan the posture and acquisition angle of the inspection camera, and execute the image re-acquisition command; it also updates the semantic ambiguity coefficient UM value of the component image until it is ≤ semantic confidence threshold Uth.

[0012] Preferably, step three includes: S31, Based on BIM semantic embedding vector and the semantic embedding vector of the Visual Language Model (VLM) By using the cosine similarity calculation method, the consistency between the semantics of BIM components and the semantics recognized by the Visual Language Model (VLM) is analyzed, and the semantic similarity parameters of each component are obtained. ; S32, Geometric center coordinates based on components Visual Language Large Model (VLM) spatial coordinate parameters The spatial deviation between the BIM design location and the actual data acquisition location was analyzed using a three-dimensional Euclidean distance calculation method to obtain the spatial deviation parameters of the components. .

[0013] Preferably, step three further includes: S33. Obtain the semantic similarity parameters of the components. Spatial deviation parameters of components After dimensionless normalization, the cross-modal consistency index (CCI) is calculated and obtained. S34. By setting a cross-modal consistency threshold Cth, and comparing the cross-modal consistency index CCI with the cross-modal consistency threshold Cth, the second evaluation results are obtained, including: When the cross-modal consistency index CCI is greater than or equal to the cross-modal consistency threshold Cth, it indicates that the semantic consistency of the component is qualified and should be continuously monitored. When the cross-modal consistency index CCI is less than the cross-modal consistency threshold Cth, it indicates that the semantic consistency of the component is unqualified, triggering a second warning instruction and generating a second strategy: marking the current component as "semantically inconsistent" and generating a component deviation report.

[0014] Preferably, step four includes: S41. By using the lidar, depth camera, and positioning sensors mounted on the inspection robot, the robot's current position coordinates and the spatial distribution information of surrounding obstacles are collected in real time, combined with the geometric center coordinates of the components. In addition to component deviation reports, spatial mapping and coordinate fusion methods are used to obtain the relative positional relationship between the robot's current position and the components, as well as obstacle distribution information, to form a path environment dataset; S42. Based on the path environment dataset, use the three-dimensional Euclidean distance calculation method to obtain the spatial distance parameters from the robot's current position to the target component. ; S43. Based on the path environment dataset, use the obstacle detour analysis method to evaluate path traversability and detour complexity, and obtain obstacle detour cost parameters in the path. .

[0015] Preferably, step four further includes: S44. Using the spatial distance parameters from the robot's current position to the target component... Obstacle detour cost parameters in the path Combined with task priority parameters After dimensionless processing, the comprehensive path cost coefficient ZCX is calculated and obtained. S45. By using a preset path cost threshold Zth, and comparing the comprehensive path cost coefficient ZCX with the path cost threshold Zth, the third evaluation results are obtained, including: When the comprehensive path cost coefficient ZCX ≤ the path cost threshold Zth, it indicates that the robot inspection path cost is reasonable, and the current inspection order and viewpoint are maintained for continuous monitoring. When the comprehensive path cost coefficient ZCX > the path cost threshold Zth, it indicates that the robot's inspection path cost is unreasonable, and there is a risk of path cost exceeding the standard and path obstruction. This triggers the third warning instruction and generates the third strategy: based on the multi-objective optimization algorithm and combined with the deviation information in the component deviation report, the inspection path is replanned, the inspection sequence and viewpoint parameters are dynamically adjusted, and the task queue is updated and the path is replanned until the comprehensive path cost coefficient ZCX ≤ the path cost threshold Zth.

[0016] Preferably, step five includes: S51. During each inspection, the semantic recognition credibility of the image is obtained based on the semantic ambiguity coefficient UM of the component image, the semantic consistency evaluation result of the component is obtained based on the cross-modal consistency index CCI, and the rationality evaluation result of the inspection path is obtained based on the comprehensive path cost coefficient ZCX. The component recognition success rate, image quality score and number of viewpoint adjustments are combined to form the inspection performance evaluation index. S52. Based on the inspection performance evaluation index, a reinforcement learning method is used for reward feedback to adaptively optimize the inspection parameters. The optimized inspection results and updated parameters are synchronized back to the BIM database to update the BIM semantic information, forming a self-learning feedback mechanism to continuously improve data and intelligent decision-making.

[0017] Preferably, a self-inspection system for building components based on BIM and Visual Model (VLM) includes: The BIM semantic data construction module is used to extract the spatial location, geometric information, attribute data, and textual semantic features of each component from the BIM model during the initialization phase of construction tasks. Through natural language processing and vectorized embedding, it generates a spatial benchmark dataset, a set of semantic labels for components, and semantic embedding vectors for components. And construct a set of component BIM semantic vectors; The visual semantic recognition and evaluation module is used to collect multimodal visual and spatial data of components through inspection robots, perform spatial reconstruction and correction using LiDAR and depth cameras, complete semantic segmentation, feature extraction and description generation of components through the visual language large model (VLM), calculate the semantic ambiguity coefficient UM, and compare it with the semantic confidence threshold Uth to determine whether the image semantic recognition credibility of the component is qualified. If it is not qualified, an appropriate strategy is given. The cross-modal semantic consistency analysis module is used to obtain the semantic similarity parameters and spatial deviation parameters of components by calculating the cosine similarity and spatial coordinate deviation between the BIM semantic embedding vector and the semantic embedding vector of the visual language large model; and to comprehensively calculate the cross-modal consistency index CCI and compare it with the cross-modal consistency threshold Cth to determine whether the semantic consistency of the component is qualified. If it is not qualified, a component deviation report is generated. The path environment modeling and inspection optimization module is used to collect robot position and obstacle information in real time through LiDAR, depth camera and positioning sensor, and combine BIM component coordinates and deviation reports to establish a path environment dataset; calculate the spatial distance and detour cost from the robot to the component to form a comprehensive path cost coefficient ZCX, and compare it with the path cost threshold Zth to determine whether the robot inspection path cost is reasonable. If it is not reasonable, the module performs path replanning and inspection sequence optimization based on the deviation information. The self-learning and intelligent feedback optimization module is used to obtain inspection performance evaluation indicators based on semantic ambiguity coefficient UM, cross-modal consistency index CCI, and comprehensive path cost coefficient ZCX. It achieves parameter adaptive optimization through reinforcement learning and writes the optimization results back to the BIM database to form a self-learning feedback mechanism to continuously improve data and intelligent decision-making capabilities.

[0018] This invention provides an autonomous inspection system and method for building components based on BIM and Visual Model (VLM). It offers the following advantages: (1) This invention relates to an autonomous inspection system and method for building components based on BIM and a Visual Language Model (VLM). Through a BIM semantic data construction module, the system automatically extracts the geometric, attribute, and semantic information of components during the initialization phase of construction tasks. Natural language processing and vectorized embedding are used to generate semantic embedding vectors for the components, forming a unified semantic vector space. This process achieves the semantic structuring and computability of BIM data, providing a highly consistent data foundation for subsequent multimodal comparison and inspection task allocation, significantly improving the accuracy of inspection task initialization and data parsing efficiency.

[0019] (2) This invention proposes the concept of semantic ambiguity coefficient UM in the visual semantic recognition evaluation module. The UM value is calculated by the semantic confidence index PLI of the VLM recognition result and compared with the threshold Uth to realize the quantitative evaluation and dynamic correction of the image recognition credibility. When UM > Uth, the system automatically triggers the viewpoint replanning and re-acquisition strategy, thereby significantly reducing the recognition deviation caused by lighting, angle or texture occlusion and improving the stability and reliability of multimodal image recognition.

[0020] (3) This self-inspection system and method for building components based on BIM and Visual Language Model (VLM) calculates the cosine similarity and spatial deviation parameters between the BIM semantic embedding vector and the VLM semantic embedding vector through a cross-modal semantic consistency analysis module, constructs a cross-modal consistency index (CCI), and compares it with a threshold Cth to determine the comprehensive consistency of semantic matching and spatial mapping. This mechanism can automatically detect and mark semantically inconsistent components, generate deviation reports, and improve the semantic matching accuracy and automated deviation diagnosis at the component level.

[0021] (4) This autonomous inspection system and method for building components based on BIM and Visual Language Model (VLM) calculates the comprehensive path cost coefficient ZCX through the path environment modeling and inspection optimization module, and dynamically optimizes the inspection path in combination with task priority parameters; at the same time, in the self-learning and intelligent feedback optimization module, reinforcement learning algorithm is introduced to feed the inspection performance evaluation index back to the BIM database to achieve adaptive parameter updates and strategy optimization. This mechanism forms a closed-loop self-learning system of "semantic recognition - path planning - performance feedback", which significantly improves the intelligence, adaptability and overall inspection efficiency of the inspection system. Attached Figure Description

[0022] Figure 1 This is a schematic diagram illustrating the steps of an autonomous inspection method for building components based on BIM and a large visual language model (VLM) according to the present invention. Figure 2 This is a flowchart of a building component autonomous inspection system based on BIM and Visual Language Model (VLM) according to the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Example 1 Please see Figure 1 This invention provides a method for autonomous inspection of building components based on BIM and Visual Language Model (VLM), comprising the following steps: Step 1: In the initial stage of construction tasks, extract the spatial location, geometric information, attribute data, and textual semantic features of each component from the BIM model. Through natural language processing and vectorized embedding, generate a spatial benchmark dataset, a set of semantic labels for components, and semantic embedding vectors for components. And construct a set of component BIM semantic vectors; Step 2: Collect multimodal visual and spatial data of components through inspection robots, perform spatial reconstruction and correction using LiDAR and depth cameras, complete semantic segmentation, feature extraction and description generation of components using the Visual Language Model (VLM), calculate the semantic ambiguity coefficient UM and compare it with the semantic confidence threshold Uth to determine whether the image semantic recognition credibility of the component is qualified. If it is not qualified, an appropriate strategy is given. Step 3: By calculating the cosine similarity and spatial coordinate deviation between the BIM semantic embedding vector and the visual language large model semantic embedding vector, the semantic similarity parameters and spatial deviation parameters of the component are obtained; and the cross-modal consistency index CCI is calculated and compared with the cross-modal consistency threshold Cth to determine whether the semantic consistency of the component is qualified. If it is not qualified, a component deviation report is generated. Step 4: Collect robot position and obstacle information in real time using LiDAR, depth camera and positioning sensor, and combine BIM component coordinates and deviation reports to establish a path environment dataset; calculate the spatial distance and detour cost from the robot to the component to form a comprehensive path cost coefficient ZCX, and compare it with the path cost threshold Zth to determine whether the robot inspection path cost is reasonable. If it is not reasonable, perform path replanning and inspection sequence optimization based on deviation information. Step 5: Based on the semantic ambiguity coefficient UM, cross-modal consistency index CCI, and comprehensive path cost coefficient ZCX, obtain the inspection performance evaluation index; achieve adaptive optimization of parameters through reinforcement learning, and write the optimization results back to the BIM database to form a self-learning feedback mechanism to continuously improve data and intelligent decision-making capabilities.

[0025] In this embodiment, a comprehensive evaluation mechanism for inspection performance is constructed by jointly calculating the semantic ambiguity coefficient UM, the cross-modal consistency index CCI, and the comprehensive path cost coefficient ZCX. This mechanism integrates semantic credibility assessment, cross-modal matching analysis, and path optimization control. During the inspection process, this mechanism enables dynamic credibility judgment of component identification results, real-time detection of semantic consistency deviations, and intelligent optimization of inspection path costs. Furthermore, it achieves adaptive parameter updates through reinforcement learning, writing the optimization results back to the BIM database, forming a self-learning closed-loop system of "identification—evaluation—optimization—feedback," thereby significantly improving the system's semantic recognition accuracy, path planning rationality, and overall inspection efficiency.

[0026] Example 2 This embodiment is an explanation based on Embodiment 1. Please refer to it. Figure 1 Specifically, step one includes: S11. During the initialization phase of the construction task, the BIM model of the building project is parsed and component-level semantic information is extracted; by loading the structural identification files of each component into the BIM model, the geometric center coordinates of the components are extracted in real time. A spatial reference dataset is established; an initial pose is provided to the inspection robot by means of positioning equipment and manual specification; based on the initial pose, feature-based SLAM registration and coordinate fusion technology is used to establish the initial association between the robot coordinate system and the BIM global coordinate system. S12. By calling the attribute database interface of the BIM model, extract the material type, installation status, construction stage number, and task priority parameters of the components. The semantics of the components are classified according to their category, functional division and hierarchical structure of work procedures, and a set of semantic tags for the components is generated. S13. By calling the BIM data interface, collect the textual attribute descriptions and design specifications of the components, perform natural language parsing, and use a Transformer-based natural language parsing algorithm to perform semantic segmentation, dependency parsing, and keyword extraction on the collected text data to extract key semantic feature words of the components; and use a vectorized embedding model to numerically encode the semantic feature words to obtain the semantic embedding vector of the components. ; S14, By combining the spatial position parameters of the components Geometric parameters, task priority parameters With component semantic embedding vector Construct a set of component BIM semantic vectors.

[0027] In this embodiment, by performing multi-level parsing and semantic information extraction on the BIM model during the construction task initialization phase, a component BIM semantic vector set integrating spatial geometric parameters, attribute information, and textual semantic features is established. This method not only achieves a unified digital expression of the component's geometric position, construction status, and task priority, but also transforms the textual description information in the BIM model into a computable semantic vector representation through natural language parsing and vectorization embedding technology. This significantly improves the retrievalability and computer understanding capability of the component's semantic information, providing a high-precision semantic data foundation for subsequent cross-modal semantic matching, intelligent recognition, and path planning optimization.

[0028] Example 3 This embodiment is an explanation based on Embodiment 1. Please refer to it. Figure 1 Specifically, step two includes: S21. Based on the spatial reference dataset, component semantic label set, and the initial association between the robot coordinate system and the BIM global coordinate system, the suboptimal view NBV planning method is used to sort the component inspection tasks; random pose sampling is performed on each target component in its surrounding space, and the visible area and structural integrity of the component in the camera's field of view under each candidate observation pose are evaluated through visualization simulation to obtain the optimal observation point with the highest score; the workflow manager calls the global path planner to calculate the path cost of the robot to each observation point and generates the initial inspection sequence according to the shortest path priority principle; based on the initial association between the robot coordinate system and the BIM global coordinate system, a three-dimensional semantic voxel map containing confirmed obstacles and virtual walls that exist in the BIM model but have not yet been detected by sensors is constructed in real time using the SLAM system. A 3D global path planning method is used to construct the initial global inspection path from the robot to each optimal observation point. During the robot's movement, based on the real-time environmental perception capabilities of LiDAR and depth camera, a local obstacle detection method is used to continuously scan the space in front of the robot, identifying and updating the positions of undefined obstacles in the BIM model. Obstacle information is acquired in real time, and a 3D semantic voxel map is updated in real time. A global path replanning algorithm is used to recalculate the feasible path from the robot to the target observation point. The local motion controller generates real-time motion commands based on the planning results to avoid dynamic obstacles. Through the obstacle avoidance-adjusted robot trajectory, an effective path that can safely reach the optimal observation point of the target component is obtained. After adjustment, the surface image information of the component is acquired through a high-definition camera, LiDAR, and depth camera mounted on the inspection robot. and spatial coordinate parameters This forms a multimodal visual perception dataset; S22. Based on the data from the multimodal visual perception dataset, LiDAR point cloud scanning and depth camera ranging technology are used to reconstruct the spatial structure and correct the image position, and obtain the surface normal vector parameters and spatial depth distribution characteristics of the components. S23. Based on the surface normal vector parameters and spatial depth distribution features of the component, a well-trained visual language model (VLM) is used for semantic segmentation and target recognition to obtain the regional feature parameters, texture feature parameters and boundary feature parameters of the component, forming a semantic feature set of the component. S24. By leveraging the language generation capabilities of the Visual Language Model (VLM), semantic understanding and description generation are performed on the semantic feature set of components to obtain component state description text. The text is then embedded using a semantic encoder to obtain the corresponding semantic embedding vector. .

[0029] In this embodiment, by integrating a high-definition camera, LiDAR, and depth camera onto the inspection robot, a multimodal visual perception data acquisition system is constructed, achieving high-precision fusion of component surface images, spatial structure, and depth information. Combined with the semantic segmentation and language generation capabilities of the Visual Language Model (VLM), it can not only perform refined region, texture, and boundary recognition of components, but also automatically generate semantic description text of the component's state and convert it into a computable semantic embedding vector. This process effectively improves the inspection system's accuracy in recognizing the semantic features of components in complex construction environments and enhances its semantic expression capabilities, providing reliable semantic feature support for subsequent cross-modal matching and intelligent decision-making.

[0030] Example 4 This embodiment is an explanation based on Embodiment 1. Please refer to it. Figure 1 Specifically, step two also includes: S25. Semantic embedding vectors based on the Visual Language Model (VLM) By using a similarity matching method, the probability distribution of the semantic tags to which the components belong is obtained, and then the semantic classification probability distribution function is obtained. The overall confidence index (PLI) for image semantic recognition is calculated using the following formula:

[0031] S26. Based on the overall confidence index PLI of image semantic recognition, calculate the semantic ambiguity coefficient UM of the component image, using the following formula:

[0032] S27, by setting a preset semantic confidence threshold Uth, and comparing and analyzing the semantic ambiguity coefficient UM of the component image with the semantic confidence threshold Uth, the first evaluation result is obtained, including: When the semantic ambiguity coefficient UM of the component image is less than or equal to the semantic confidence threshold Uth, it indicates that the semantic recognition credibility of the component image is qualified and will be continuously monitored. When the semantic ambiguity coefficient UM of a component image exceeds the semantic confidence threshold Uth, it indicates that the semantic recognition confidence of the component image is unqualified, triggering the first warning instruction and generating the first strategy: to re-plan the viewpoint, using a multi-objective optimization algorithm based on semantic embedding vectors. The system uses the overall confidence index PLI for image semantic recognition to automatically obtain the best suboptimal viewpoint parameter set of the component, re-plan the posture and acquisition angle of the inspection camera, and execute the image re-acquisition command; it also updates the semantic ambiguity coefficient UM value of the component image until it is ≤ semantic confidence threshold Uth.

[0033] When the inspection robot reaches the vicinity of the optimal observation point, if the line of sight to the original optimal observation point is obstructed due to local material accumulation at the construction site, temporary structural obstruction, or dynamic obstacles, resulting in insufficient visible area, incomplete imaging, or a persistently high semantic ambiguity coefficient (UM) in the currently acquired component image, it indicates that the original planned viewpoint has failed. At this time, while maintaining the original strategy logic, a local secondary viewpoint planning process is triggered based on the occlusion status judgment: by detecting the current component's line of sight accessibility, the proportion of the occluded area, and the locally incomplete area reconstructed by the camera projection, the local replanning of the suboptimal viewpoint is automatically initiated. Random pose sampling and visible area simulation evaluation are re-executed on the local space around the component, and feasible suboptimal observation points that are not obstructed and have a better visible area are prioritized for selection. Subsequently, the posture and acquisition angle of the inspection camera are updated according to the obtained suboptimal viewpoint, and the image re-acquisition command is executed so that the component image can still be restored to the imaging quality that meets the semantic recognition requirements even in the occluded environment. The semantic confidence threshold Uth is obtained by statistically analyzing a large amount of semantic recognition data of component images collected by inspection robots. The overall confidence index (PLI) distribution range under high-confidence and low-confidence recognition states is extracted. Combined with the experience of technical personnel, a reasonable semantic confidence value is determined. Reference is made to Building Information Modeling (BIM) management standards, Visual Language Modeling (VLM) recognition accuracy standards, and relevant industry inspection technical specifications, which typically provide reference ranges for semantic recognition confidence. This threshold is used to effectively distinguish between reliable and unreliable states of component image recognition, ensuring the reliability and data quality of inspection results.

[0034] In this embodiment, the semantic ambiguity coefficient UM of the component image is calculated and compared with a preset semantic confidence threshold Uth to achieve real-time evaluation of the image semantic recognition credibility. When the recognition credibility is insufficient, the system can automatically trigger a viewpoint replanning strategy, use a multi-objective optimization algorithm to generate the best suboptimal viewpoint parameters, dynamically adjust the inspection camera posture and acquisition angle, and perform image re-acquisition, thereby ensuring the semantic recognition accuracy of the component image and the reliability of the inspection data, effectively improving the accuracy and adaptability of the autonomous inspection system.

[0035] Example 5 This embodiment is an explanation based on Embodiment 1. Please refer to it. Figure 1 Specifically, step three includes: S31, Based on BIM semantic embedding vector and the semantic embedding vector of the Visual Language Model (VLM) By using the cosine similarity calculation method, the consistency between the semantics of BIM components and the semantics recognized by the Visual Language Model (VLM) is analyzed, and the semantic similarity parameters of each component are obtained. ; S32, Geometric center coordinates based on components Visual Language Large Model (VLM) spatial coordinate parameters The spatial deviation between the BIM design location and the actual data acquisition location was analyzed using a three-dimensional Euclidean distance calculation method to obtain the spatial deviation parameters of the components. .

[0036] In this embodiment, the consistency between the semantics and spatial location of components is evaluated by calculating the cosine similarity between the BIM semantic embedding vector and the VLM semantic embedding vector, combined with component spatial coordinate deviation analysis. This method can accurately identify the deviation between the component semantics and the actual installation location, promptly detect potential installation or identification anomalies, thereby improving the accuracy and reliability of the inspection process and ensuring the consistency between building component data and the BIM model.

[0037] Example 6 This embodiment is an explanation based on Embodiment 1. Please refer to it. Figure 1 Specifically, step three also includes: S33. Obtain the semantic similarity parameters of the components. Spatial deviation parameters of components After dimensionless normalization, the cross-modal consistency index (CCI) is calculated using the following formula:

[0038] In the formula, w1 and w2 represent weighting coefficients, and Rmax represents the maximum reference deviation value. By statistically analyzing a large amount of spatial measurement data of components at the construction site, the maximum deviation value between the actual position and the design position of the components is extracted, and determined in conjunction with construction experience and accuracy requirements. This deviation is used to... Dimensionless normalization is performed to ensure that the calculation of the cross-modal consistency index (CCI) has a unified dimension and comparability. The semantic similarity of components has a high weight and is a key indicator that directly reflects the degree of consistency between the semantic information of the BIM model and the semantic recognition of the large visual language model. The second highest weighting is given to characterize the influence of component spatial deviation parameters on the cross-modal consistency index (CCI), reflecting the role of the deviation between the actual position of the component and the design position in the BIM model on semantic consistency evaluation.

[0039] S34. By setting a cross-modal consistency threshold Cth, and comparing the cross-modal consistency index CCI with the cross-modal consistency threshold Cth, the second evaluation results are obtained, including: When the cross-modal consistency index CCI is greater than or equal to the cross-modal consistency threshold Cth, it indicates that the semantic consistency of the component is qualified and should be continuously monitored. When the cross-modal consistency index CCI is less than the cross-modal consistency threshold Cth, it indicates that the semantic consistency of the component is unqualified, triggering a second warning instruction and generating a second strategy: marking the current component as "semantically inconsistent" and generating a component deviation report.

[0040] The cross-modal consistency threshold Cth is obtained by statistically analyzing a large amount of comparative data between component BIM semantic vectors and the visual language large model (VLM) semantic vectors. The distribution range of the cross-modal consistency index (CCI) under high and low consistency states is extracted. Combined with the experience of professional technicians and judgment based on the actual construction environment, a reasonable cross-modal consistency judgment value is determined. Reference is made to BIM application standards, inspection robot visual recognition specifications, and building construction quality control standards, which typically provide reference values ​​for semantic consistency assessment. This threshold is used to effectively judge the consistency between component BIM design information and actual visual recognition data, ensuring the matching accuracy between inspection data and model information.

[0041] In this embodiment, a cross-modal consistency index (CCI) is calculated and compared with a preset threshold Cth to achieve an integrated consistency assessment of the semantic and spatial information of building components. When the CCI falls below the threshold, semantic inconsistencies or spatial deviations can be identified in a timely manner, triggering deviation report generation and early warning strategies. This method effectively improves the reliability and accuracy of inspection results, providing a scientific basis for the condition monitoring and maintenance of building components.

[0042] Example 7 This embodiment is an explanation based on Embodiment 1. Please refer to it. Figure 1 Specifically, step four includes: S41. By using the lidar, depth camera, and positioning sensors mounted on the inspection robot, the robot's current position coordinates and the spatial distribution information of surrounding obstacles are collected in real time, combined with the geometric center coordinates of the components. In addition to component deviation reports, spatial mapping and coordinate fusion methods are used to obtain the relative positional relationship between the robot's current position and the components, as well as obstacle distribution information, to form a path environment dataset; S42. Based on the path environment dataset, use the three-dimensional Euclidean distance calculation method to obtain the spatial distance parameters from the robot's current position to the target component. ; S43. Based on the path environment dataset, use the obstacle detour analysis method to evaluate path traversability and detour complexity, and obtain obstacle detour cost parameters in the path. .

[0043] In this embodiment, by collecting real-time information on the inspection robot's position and surrounding obstacles, and combining this information with BIM component coordinates and deviation data, a path environment dataset is established. This enables accurate modeling of the relative position of the robot and the target component, as well as the distribution of obstacles. This method can scientifically assess the spatial distance and detour costs of the inspection path, improve the accuracy of path planning and inspection efficiency, and reduce operational risks.

[0044] Example 8 This embodiment is an explanation based on Embodiment 1. Please refer to it. Figure 1 Specifically, step four also includes: S44. Using the spatial distance parameters from the robot's current position to the target component... Obstacle detour cost parameters in the path Combined with task priority parameters After dimensionless processing, the comprehensive path cost coefficient ZCX is calculated and obtained, as shown in the following formula:

[0045] In the formula, a1, a2, and a3 represent weighting coefficients; The parameter representing the spatial distance from the robot's current position to the target component has a high weighting on the comprehensive path cost coefficient ZCX, directly reflecting the contribution of path length to the inspection cost. The parameter representing the obstacle detour cost in the path has a medium weighting on the comprehensive path cost coefficient ZCX, reflecting the impact of obstacle distribution and detour complexity on inspection efficiency. 2: Characterizes the impact of task priority parameters on the comprehensive path cost coefficient ZCX, which has a minor weight and reflects the auxiliary role of inspection task priority in path planning decisions.

[0046] S45. By using a preset path cost threshold Zth, and comparing the comprehensive path cost coefficient ZCX with the path cost threshold Zth, the third evaluation results are obtained, including: When the comprehensive path cost coefficient ZCX ≤ the path cost threshold Zth, it indicates that the robot inspection path cost is reasonable, and the current inspection order and viewpoint are maintained for continuous monitoring. When the comprehensive path cost coefficient ZCX > the path cost threshold Zth, it indicates that the robot's inspection path cost is unreasonable, and there is a risk of path cost exceeding the standard and path obstruction. This triggers the third warning instruction and generates the third strategy: based on the multi-objective optimization algorithm and combined with the deviation information in the component deviation report, the inspection path is replanned, the inspection sequence and viewpoint parameters are dynamically adjusted, and the task queue is updated and the path is replanned until the comprehensive path cost coefficient ZCX ≤ the path cost threshold Zth.

[0047] The path cost threshold Zth is obtained by statistically analyzing a large amount of path planning data from inspection robots under different construction environments. The distribution range of the comprehensive path cost coefficient ZCX under reasonable and high-cost path cost states is extracted. This, combined with construction task priority, obstacle complexity, and the experience of professional technicians, determines a reasonable path cost judgment value. Reference is made to construction robot operation specifications, building construction safety standards, and the performance indicators of path optimization algorithms. These specifications typically provide an evaluation range for reasonable path planning. This threshold is used to effectively determine whether the inspection path is reasonable, ensuring inspection efficiency and robot operation safety.

[0048] In this embodiment, the rationality of the inspection path can be dynamically evaluated by calculating the comprehensive path cost coefficient ZCX and comparing it with the preset threshold Zth. When the path cost exceeds the limit, a multi-objective optimization algorithm can be automatically triggered to replan the inspection path and adjust the inspection sequence based on the component deviation information, thereby achieving path optimization and dynamic task scheduling, which improves inspection efficiency and reduces path obstruction and operational risks.

[0049] Example 9 This embodiment is an explanation based on Embodiment 1. Please refer to it. Figure 1 Specifically, step five includes: S51. During each inspection, the semantic recognition credibility of the image is obtained based on the semantic ambiguity coefficient UM of the component image, the semantic consistency evaluation result of the component is obtained based on the cross-modal consistency index CCI, and the rationality evaluation result of the inspection path is obtained based on the comprehensive path cost coefficient ZCX. The component recognition success rate, image quality score and number of viewpoint adjustments are combined to form the inspection performance evaluation index. S52. Based on the inspection performance evaluation index, a reinforcement learning method is used for reward feedback to adaptively optimize the inspection parameters. The optimized inspection results and updated parameters are synchronized back to the BIM database to update the BIM semantic information, forming a self-learning feedback mechanism to continuously improve data and intelligent decision-making.

[0050] In this embodiment, the intelligent adjustment and dynamic updating of inspection parameters are achieved through reinforcement learning adaptive optimization based on inspection performance evaluation indicators. The optimization results are written back to the BIM database to form a self-learning feedback mechanism, thereby continuously improving the component recognition accuracy, image acquisition quality and inspection efficiency, and enhancing the system's intelligent decision-making and autonomous optimization capabilities.

[0051] Example 10 A self-inspection system for building components based on BIM and Visual Model (VLM) is described below. Figure 2 Specifically, including: The BIM semantic data construction module is used to extract the spatial location, geometric information, attribute data, and textual semantic features of each component from the BIM model during the initialization phase of construction tasks. Through natural language processing and vectorized embedding, it generates a spatial benchmark dataset, a set of semantic labels for components, and semantic embedding vectors for components. And construct a set of component BIM semantic vectors; The visual semantic recognition and evaluation module is used to collect multimodal visual and spatial data of components through inspection robots, perform spatial reconstruction and correction using LiDAR and depth cameras, complete semantic segmentation, feature extraction and description generation of components through the visual language large model (VLM), calculate the semantic ambiguity coefficient UM, and compare it with the semantic confidence threshold Uth to determine whether the image semantic recognition credibility of the component is qualified. If it is not qualified, an appropriate strategy is given. The cross-modal semantic consistency analysis module is used to obtain the semantic similarity parameters and spatial deviation parameters of components by calculating the cosine similarity and spatial coordinate deviation between the BIM semantic embedding vector and the semantic embedding vector of the visual language large model; and to comprehensively calculate the cross-modal consistency index CCI and compare it with the cross-modal consistency threshold Cth to determine whether the semantic consistency of the component is qualified. If it is not qualified, a component deviation report is generated. The path environment modeling and inspection optimization module is used to collect robot position and obstacle information in real time through LiDAR, depth camera and positioning sensor, and combine BIM component coordinates and deviation reports to establish a path environment dataset; calculate the spatial distance and detour cost from the robot to the component to form a comprehensive path cost coefficient ZCX, and compare it with the path cost threshold Zth to determine whether the robot inspection path cost is reasonable. If it is not reasonable, the module performs path replanning and inspection sequence optimization based on the deviation information. The self-learning and intelligent feedback optimization module is used to obtain inspection performance evaluation indicators based on semantic ambiguity coefficient UM, cross-modal consistency index CCI, and comprehensive path cost coefficient ZCX. It achieves parameter adaptive optimization through reinforcement learning and writes the optimization results back to the BIM database to form a self-learning feedback mechanism to continuously improve data and intelligent decision-making capabilities.

[0052] In this embodiment, through the collaborative work of multiple modules, a closed-loop management system is achieved, from BIM semantic construction to visual semantic recognition, cross-modal consistency analysis, path optimization, and self-learning feedback. This enables the inspection system to automatically perceive, evaluate, and optimize the inspection process, significantly improving the accuracy, efficiency, and intelligence level of autonomous inspection of building components, while enhancing the system's adaptive learning and continuous improvement capabilities.

[0053] The threshold is set to facilitate comparison. The size of the threshold depends on the amount of sample data and the number of bases set by those skilled in the art for each set of sample data; as long as it does not affect the ratio between the parameter and the quantized value, it is acceptable.

[0054] The above formulas are all derived from software simulation using a large amount of data and are selected to be close to the actual values. The coefficients in the formulas are set by those skilled in the art according to the actual situation. The above description is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any equivalent substitutions or changes made by those skilled in the art within the technical scope disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the protection scope of the present invention.

Claims

1. A method for autonomous inspection of building components based on BIM and Visual Language Large Model (VLM), characterized in that, Includes the following steps: Step 1: In the initial stage of construction tasks, extract the spatial location, geometric information, attribute data, and textual semantic features of each component from the BIM model. Through natural language processing and vectorized embedding, generate a spatial benchmark dataset, a set of semantic labels for components, and semantic embedding vectors for components. And construct a set of component BIM semantic vectors; Step 2: Collect multimodal visual and spatial data of components through inspection robots, perform spatial reconstruction and correction using LiDAR and depth cameras, complete semantic segmentation, feature extraction and description generation of components using the Visual Language Model (VLM), calculate the semantic ambiguity coefficient UM and compare it with the semantic confidence threshold Uth to determine whether the image semantic recognition credibility of the component is qualified. If it is not qualified, an appropriate strategy is given. Step 3: By calculating the cosine similarity and spatial coordinate deviation between the BIM semantic embedding vector and the visual language large model semantic embedding vector, the semantic similarity parameters and spatial deviation parameters of the component are obtained; and the cross-modal consistency index CCI is calculated and compared with the cross-modal consistency threshold Cth to determine whether the semantic consistency of the component is qualified. If it is not qualified, a component deviation report is generated. Step 4: Collect robot position and obstacle information in real time using LiDAR, depth camera and positioning sensor, and combine BIM component coordinates and deviation reports to establish a path environment dataset; calculate the spatial distance and detour cost from the robot to the component to form a comprehensive path cost coefficient ZCX, and compare it with the path cost threshold Zth to determine whether the robot inspection path cost is reasonable. If it is not reasonable, perform path replanning and inspection sequence optimization based on deviation information. Step 5: Based on the semantic ambiguity coefficient UM, cross-modal consistency index CCI, and comprehensive path cost coefficient ZCX, obtain the inspection performance evaluation index; achieve adaptive optimization of parameters through reinforcement learning, and write the optimization results back to the BIM database to form a self-learning feedback mechanism to continuously improve data and intelligent decision-making capabilities.

2. The autonomous inspection method for building components based on BIM and Visual Language Large Model (VLM) according to claim 1, characterized in that, Step one includes: S11. During the initialization phase of the construction task, the BIM model of the building project is parsed and component-level semantic information is extracted; by loading the structural identification files of each component into the BIM model, the geometric center coordinates of the components are extracted in real time. A spatial reference dataset is established; an initial pose is provided to the inspection robot by means of positioning equipment and manual specification; based on the initial pose, feature-based SLAM registration and coordinate fusion technology is used to establish the initial association between the robot coordinate system and the BIM global coordinate system. S12. By calling the attribute database interface of the BIM model, extract the material type, installation status, construction stage number, and task priority parameters of the components. The semantics of the components are classified according to their category, functional division and hierarchical structure of work procedures, and a set of semantic tags for the components is generated. S13. By calling the BIM data interface, collect the textual attribute descriptions and design specifications of the components, perform natural language parsing, and use a Transformer-based natural language parsing algorithm to perform semantic segmentation, dependency parsing, and keyword extraction on the collected text data to extract key semantic feature words of the components; and use a vectorized embedding model to numerically encode the semantic feature words to obtain the semantic embedding vector of the components. ; S14, By combining the spatial position parameters of the components Geometric parameters, task priority parameters With component semantic embedding vector Construct a set of component BIM semantic vectors.

3. The autonomous inspection method for building components based on BIM and Visual Language Large Model (VLM) according to claim 1, characterized in that, Step two includes: S21. Collect surface image information of components using high-definition cameras, LiDAR, and depth cameras mounted on the inspection robot. and spatial coordinate parameters This forms a multimodal visual perception dataset; S22. Based on the data from the multimodal visual perception dataset, LiDAR point cloud scanning and depth camera ranging technology are used to reconstruct the spatial structure and correct the image position, and obtain the surface normal vector parameters and spatial depth distribution characteristics of the components. S23. Based on the surface normal vector parameters and spatial depth distribution features of the component, a well-trained visual language model (VLM) is used for semantic segmentation and target recognition to obtain the regional feature parameters, texture feature parameters and boundary feature parameters of the component, forming a semantic feature set of the component. S24. By leveraging the language generation capabilities of the Visual Language Model (VLM), semantic understanding and description generation are performed on the semantic feature set of components to obtain component state description text. The text is then embedded using a semantic encoder to obtain the corresponding semantic embedding vector. .

4. The autonomous inspection method for building components based on BIM and Visual Language Large Model (VLM) as described in claim 1, characterized in that, Step two also includes: S25. Semantic embedding vectors based on the Visual Language Model (VLM) By using a similarity matching method, the probability distribution of the semantic tags to which the components belong is obtained, and then the semantic classification probability distribution function is obtained. And calculate and obtain the overall confidence index PLI for image semantic recognition; S26. Based on the overall confidence index PLI of image semantic recognition, calculate the semantic ambiguity coefficient UM of the component image; S27, by setting a preset semantic confidence threshold Uth, and comparing and analyzing the semantic ambiguity coefficient UM of the component image with the semantic confidence threshold Uth, the first evaluation result is obtained, including: When the semantic ambiguity coefficient UM of the component image is less than or equal to the semantic confidence threshold Uth, it indicates that the semantic recognition credibility of the component image is qualified and will be continuously monitored. When the semantic ambiguity coefficient UM of a component image exceeds the semantic confidence threshold Uth, it indicates that the semantic recognition confidence of the component image is unqualified, triggering the first warning instruction and generating the first strategy: to re-plan the viewpoint, using a multi-objective optimization algorithm based on semantic embedding vectors. The system uses the overall confidence index PLI for image semantic recognition to automatically obtain the best suboptimal viewpoint parameter set of the component, re-plan the posture and acquisition angle of the inspection camera, and execute the image re-acquisition command; it also updates the semantic ambiguity coefficient UM value of the component image until it is ≤ semantic confidence threshold Uth.

5. The autonomous inspection method for building components based on BIM and Visual Language Large Model (VLM) according to claim 1, characterized in that, Step three includes: S31, Based on BIM semantic embedding vector and the semantic embedding vector of the Visual Language Model (VLM) By using the cosine similarity calculation method, the consistency between the semantics of BIM components and the semantics recognized by the Visual Language Model (VLM) is analyzed, and the semantic similarity parameters of each component are obtained. ; S32, Geometric center coordinates based on components Visual Language Large Model (VLM) spatial coordinate parameters The spatial deviation between the BIM design location and the actual data acquisition location was analyzed using a three-dimensional Euclidean distance calculation method to obtain the spatial deviation parameters of the components. .

6. The autonomous inspection method for building components based on BIM and Visual Language Large Model (VLM) according to claim 1, characterized in that, Step three also includes: S33. Obtain the semantic similarity parameters of the components. Spatial deviation parameters of components After dimensionless normalization, the cross-modal consistency index (CCI) is calculated and obtained. S34. By setting a cross-modal consistency threshold Cth, and comparing the cross-modal consistency index CCI with the cross-modal consistency threshold Cth, the second evaluation results are obtained, including: When the cross-modal consistency index CCI is greater than or equal to the cross-modal consistency threshold Cth, it indicates that the semantic consistency of the component is qualified and should be continuously monitored. When the cross-modal consistency index CCI is less than the cross-modal consistency threshold Cth, it indicates that the semantic consistency of the component is unqualified, triggering a second warning instruction and generating a second strategy: marking the current component as "semantically inconsistent" and generating a component deviation report.

7. The autonomous inspection method for building components based on BIM and Visual Language Large Model (VLM) according to claim 1, characterized in that, Step four includes: S41. By using the lidar, depth camera, and positioning sensors mounted on the inspection robot, the robot's current position coordinates and the spatial distribution information of surrounding obstacles are collected in real time, combined with the geometric center coordinates of the components. In addition to component deviation reports, spatial mapping and coordinate fusion methods are used to obtain the relative positional relationship between the robot's current position and the components, as well as obstacle distribution information, to form a path environment dataset; S42. Based on the path environment dataset, use the three-dimensional Euclidean distance calculation method to obtain the spatial distance parameters from the robot's current position to the target component. ; S43. Based on the path environment dataset, use the obstacle detour analysis method to evaluate path drivability and detour complexity, and obtain obstacle detour cost parameters in the path. .

8. The autonomous inspection method for building components based on BIM and Visual Language Large Model (VLM) according to claim 1, characterized in that, Step four also includes: S44. Using the spatial distance parameters from the robot's current position to the target component... Obstacle detour cost parameters in the path Combined with task priority parameters After dimensionless processing, the comprehensive path cost coefficient ZCX is calculated and obtained. S45. By using a preset path cost threshold Zth, and comparing the comprehensive path cost coefficient ZCX with the path cost threshold Zth, the third evaluation results are obtained, including: When the comprehensive path cost coefficient ZCX ≤ the path cost threshold Zth, it indicates that the robot inspection path cost is reasonable, and the current inspection order and viewpoint are maintained for continuous monitoring. When the comprehensive path cost coefficient ZCX > the path cost threshold Zth, it indicates that the robot's inspection path cost is unreasonable, and there is a risk of path cost exceeding the standard and path obstruction. This triggers the third warning instruction and generates the third strategy: based on the multi-objective optimization algorithm and combined with the deviation information in the component deviation report, the inspection path is replanned, the inspection sequence and viewpoint parameters are dynamically adjusted, and the task queue is updated and the path is replanned until the comprehensive path cost coefficient ZCX ≤ the path cost threshold Zth.

9. The autonomous inspection method for building components based on BIM and Visual Language Large Model (VLM) according to claim 1, characterized in that, Step five includes: S51. During each inspection, the semantic recognition credibility of the image is obtained based on the semantic ambiguity coefficient UM of the component image, the semantic consistency evaluation result of the component is obtained based on the cross-modal consistency index CCI, and the rationality evaluation result of the inspection path is obtained based on the comprehensive path cost coefficient ZCX. The component recognition success rate, image quality score and number of viewpoint adjustments are combined to form the inspection performance evaluation index. S52. Based on the inspection performance evaluation index, a reinforcement learning method is used for reward feedback to adaptively optimize the inspection parameters. The optimized inspection results and updated parameters are synchronized back to the BIM database to update the BIM semantic information, forming a self-learning feedback mechanism to continuously improve data and intelligent decision-making.

10. A building component autonomous inspection system based on BIM and Visual Language Model (VLM), applied to the building component autonomous inspection method based on BIM and Visual Language Model (VLM) as described in any one of claims 1 to 9, characterized in that, include: The BIM semantic data construction module is used to extract the spatial location, geometric information, attribute data, and textual semantic features of each component from the BIM model during the initialization phase of construction tasks. Through natural language processing and vectorized embedding, it generates a spatial benchmark dataset, a set of semantic labels for components, and semantic embedding vectors for components. And construct a set of component BIM semantic vectors; The visual semantic recognition and evaluation module is used to collect multimodal visual and spatial data of components through inspection robots, perform spatial reconstruction and correction using LiDAR and depth cameras, complete semantic segmentation, feature extraction and description generation of components through the visual language large model (VLM), calculate the semantic ambiguity coefficient UM, and compare it with the semantic confidence threshold Uth to determine whether the image semantic recognition credibility of the component is qualified. If it is not qualified, an appropriate strategy is given. The cross-modal semantic consistency analysis module is used to obtain the semantic similarity parameters and spatial deviation parameters of components by calculating the cosine similarity and spatial coordinate deviation between the BIM semantic embedding vector and the semantic embedding vector of the visual language large model; and to comprehensively calculate the cross-modal consistency index CCI and compare it with the cross-modal consistency threshold Cth to determine whether the semantic consistency of the component is qualified. If it is not qualified, a component deviation report is generated. The path environment modeling and inspection optimization module is used to collect robot position and obstacle information in real time through LiDAR, depth camera and positioning sensor, and combine BIM component coordinates and deviation reports to establish a path environment dataset; calculate the spatial distance and detour cost from the robot to the component to form a comprehensive path cost coefficient ZCX, and compare it with the path cost threshold Zth to determine whether the robot inspection path cost is reasonable. If it is not reasonable, the module performs path replanning and inspection sequence optimization based on the deviation information. The self-learning and intelligent feedback optimization module is used to obtain inspection performance evaluation indicators based on semantic ambiguity coefficient UM, cross-modal consistency index CCI, and comprehensive path cost coefficient ZCX. It achieves parameter adaptive optimization through reinforcement learning and writes the optimization results back to the BIM database to form a self-learning feedback mechanism to continuously improve data and intelligent decision-making capabilities.

Images