Construction site component level difference detection method, device, equipment and medium

Abstract

The present application relates to the technical field of building information processing, and provides a construction site component level difference detection method, device, equipment and medium, which can generate a component multi-modal priori library according to hierarchical indexes and multi-modal priori, and establishes a correlation basis of BIM design information and field observation data; based on a coarse-to-fine registration mechanism, the field coordinate system and the BIM coordinate system are unified according to the field data and the BIM model, and the field data is reconstructed according to the target coordinate transformation parameters, solving the problem of the difference between the BIM rendering view angle and the field real view angle domain; the cross-modal fusion representation of the expected observation and the field observation is carried out, solving the problem of insufficient robustness of single modal data; the field component instance corresponding to each BIM component is obtained through instance matching, and the component level difference detection is carried out according to the installation plan and the field component instance corresponding to each BIM component, realizing the accurate component level difference detection in the construction site.

Description

Technical Field

[0001] This invention relates to the field of building information processing technology, and in particular to a method, apparatus, equipment and medium for detecting component-level differences at construction sites. Background Technology

[0002] With the development of industrialized and digitalized construction, on-site quality, safety, and schedule management increasingly rely on consistency testing between design and construction. Common methods at present include:

[0003] (1) Manual inspection and photo recording: It relies on experience, is highly subjective, makes it difficult to achieve precise positioning at the component level, and is inefficient;

[0004] (2) Comparison between point cloud and BIM (Building Information Modeling): Point clouds are obtained from the site through ground, vehicle-mounted, and handheld laser scanning, and then geometrically registered with the BIM model before distance threshold judgment is performed. This type of method is highly sensitive to registration accuracy. Occlusion, noise, and sparse scanning can lead to false alarms or missed alarms. Moreover, it is mostly a geometric deviation detection method, which is difficult to identify differences in appearance, material, installation status, etc.

[0005] (3) AI detection based on images or videos: Images are collected using drones or fixed cameras, then target detection or semantic segmentation is performed, and finally compared with the plan. These methods often lack prior constraints on the design side, are not robust enough to scenarios with similar component categories, large changes in perspective, and complex lighting, and are difficult to output difference results that can be directly mapped to BIM components.

[0006] The above methods have the following main drawbacks:

[0007] (1) There is a lack of unified and comparable representation between on-site multimodal data and BIM;

[0008] (2) When relying on only a single modality (such as geometry or image), it is unstable in situations such as occlusion, temporary construction conditions, and complex lighting.

[0009] (3) Existing methods usually only provide regional or grid-level deviations, making it difficult to locate and determine component-level differences;

[0010] (4) There is a domain difference between the BIM perspective rendering and the actual on-site perspective, resulting in a large error in direct comparison. Summary of the Invention

[0011] In view of the above, it is necessary to provide a method, device, equipment and medium for detecting component-level differences at construction sites, in order to solve the problem that multimodal priors cannot be fully utilized to accurately detect component-level differences at construction sites.

[0012] A method for detecting component-level differences at construction sites, the method comprising:

[0013] In response to a component-level difference detection command for the target construction site, a BIM model corresponding to the target construction site is obtained;

[0014] Establish a hierarchical index and multimodal priors based on the BIM model;

[0015] BIM rendering is performed based on the hierarchical index and the multimodal priors to generate a component multimodal prior library;

[0016] Extract candidate camera pose sets from the component multimodal prior library, and collect field data under multimodal observation corresponding to the BIM model perspective;

[0017] Based on the coarse-to-fine registration mechanism, the on-site coordinate system and the BIM coordinate system are unified according to the on-site data and the BIM model to obtain the target coordinate transformation parameters;

[0018] The field data is reconstructed based on the target coordinate transformation parameters to obtain field multimodal observation data;

[0019] For each candidate camera pose in the candidate camera pose set, the desired observation is extracted from the component multimodal prior library, and the field multimodal observation data is determined as the field observation;

[0020] The expected observation and the in-situ observation are subjected to cross-modal fusion characterization to obtain a cross-modal fusion feature map;

[0021] Each BIM component in the BIM model is instance-matched in the cross-modal fusion feature map to obtain the on-site component instance corresponding to each BIM component;

[0022] Obtain the installation plan for each BIM component, and perform component-level difference detection based on the installation plan and the corresponding on-site component instance for each BIM component to obtain the difference data for each differing component.

[0023] A component-level difference detection device for construction sites, the component-level difference detection device comprising:

[0024] The acquisition unit is used to acquire the BIM model corresponding to the target construction site in response to the component-level difference detection command of the target construction site;

[0025] A unit is established to create a hierarchical index and multimodal priors based on the BIM model.

[0026] A rendering unit is used to perform BIM rendering based on the hierarchical index and the multimodal priors to generate a component multimodal prior library.

[0027] The acquisition unit is used to extract a set of candidate camera poses from the multimodal prior library of the components, and to acquire field data under multimodal observation corresponding to the BIM model perspective;

[0028] A unified unit is used to unify the site coordinate system and the BIM coordinate system based on the site data and the BIM model, according to a coarse-to-fine registration mechanism, to obtain the target coordinate transformation parameters.

[0029] The reconstruction unit is used to reconstruct the field data according to the target coordinate transformation parameters to obtain field multimodal observation data;

[0030] The determining unit is used to extract the desired observation from the component multimodal prior library for each candidate camera pose in the candidate camera pose set, and to determine the field multimodal observation data as the field observation.

[0031] The characterization unit is used to perform cross-modal fusion characterization of the expected observation and the on-site observation to obtain a cross-modal fusion feature map;

[0032] The matching unit is used to perform instance matching of each BIM component in the BIM model in the cross-modal fusion feature map to obtain the on-site component instance corresponding to each BIM component.

[0033] The detection unit is used to obtain the installation plan for each BIM component, and perform component-level difference detection based on the installation plan and the on-site component instance corresponding to each BIM component to obtain the difference data of each different component.

[0034] A computer device, the computer device comprising:

[0035] A memory that stores at least one instruction; and a processor that executes the instructions stored in the memory to implement the construction site component-level difference detection method.

[0036] A computer-readable storage medium storing at least one instruction, which is executed by a processor in a computer device to implement the construction site component-level difference detection method.

[0037] As can be seen from the above technical solutions, this invention can generate a component multimodal prior library by performing BIM rendering based on hierarchical indexing and multimodal priors, establishing a foundation for the association between BIM design information and on-site observation data, and transforming the abstract BIM model into a multimodal prior that can be directly compared with on-site data; based on a coarse-to-fine registration mechanism, the on-site coordinate system and the BIM coordinate system are unified according to the on-site data and the BIM model, and the on-site data is reconstructed according to the target coordinate transformation parameters, solving the problem of the gap between the BIM rendering perspective and the actual on-site perspective domain; cross-modal fusion characterization of expected observations and on-site observations solves the problem of insufficient robustness of single-modal data; on-site component instances corresponding to each BIM component are obtained through instance matching, and component-level difference detection is performed according to the installation plan and the on-site component instances corresponding to each BIM component, realizing accurate component-level difference detection within the construction site. Attached Figure Description

[0038] Figure 1 This is a flowchart of a preferred embodiment of the method for detecting component-level differences at construction sites according to the present invention;

[0039] Figure 2 This is a functional block diagram of a preferred embodiment of the construction site component-level difference detection device of the present invention;

[0040] Figure 3 This is a schematic diagram of the structure of a computer device that implements a preferred embodiment of the method for detecting component-level differences at construction sites according to the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] like Figure 1 The diagram shown is a flowchart of a preferred embodiment of the method for detecting component-level differences at construction sites according to the present invention. The order of steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements.

[0043] The method for detecting component-level differences at the construction site is applied to one or more computer devices. The computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0044] The computer device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0045] The computer equipment may also include network equipment and / or user equipment. The network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0046] The server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0047] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0048] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0049] The network in which the computer device is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, and virtual private network (VPN).

[0050] S10, in response to the component-level difference detection command for the target construction site, obtain the BIM model (Building Information Modeling) corresponding to the target construction site.

[0051] In this embodiment, the component-level difference detection command can be triggered according to actual detection needs.

[0052] In this embodiment, the BIM model may include attribute data such as component ID, component category, geometric parameters, material, color, installation schedule, and allowable deviation.

[0053] S11, Establish a hierarchical index and multimodal priors based on the BIM model.

[0054] In this embodiment, the step of establishing a hierarchical index and multimodal priors based on the BIM model includes:

[0055] A hierarchical path is established for each BIM component using floor, area, profession, and process as index dimensions, and the hierarchical path of each BIM component is combined to obtain the hierarchical index.

[0056] Geometric priors, semantic priors, appearance priors, and perspective priors are generated based on the BIM model and used as the multimodal priors.

[0057] Specifically, the predetermined inspection route of the target construction site is discretized by arc length in the BIM coordinate system, and the position of each initial candidate camera is determined by combining the inspection viewpoint height; the camera orientation corresponding to each initial candidate camera position is calculated based on the spatial distribution position of each BIM component in the BIM model and the direction of the outer surface normal; each initial candidate camera position and its corresponding camera orientation are combined to obtain each candidate viewpoint; each candidate viewpoint is screened by a visibility evaluation algorithm and an observation quality score to obtain each effective viewpoint; each effective viewpoint is clustered to obtain the candidate camera pose set including multiple candidate camera poses; the candidate camera pose set is determined as the viewpoint prior.

[0058] For example, the complete attribute information of the BIM model can be read, including but not limited to the unique identifier (component identifier) ​​of each component, its category (such as beam, column, slab, pipe, etc.), geometric dimensions (such as length, width, height, cross-sectional shape, etc.), material type (such as concrete, steel, glass, etc.), surface color, planned installation time window, and construction allowable deviation thresholds (such as position offset allowable ±5cm, dimension deviation allowable ±3cm, etc.). Furthermore, a hierarchical index can be established according to the logic of "floor → area → specialty (such as architecture, structure, MEP, etc.) → process (such as rebar tying, concrete pouring, pipeline installation, etc.)". Each component corresponds to a unique hierarchical path. This hierarchical index can be used as the smallest unit for subsequent difference detection results to be written back to the BIM model to ensure that differences can be accurately associated with specific components. Geometric priors, semantic priors, appearance priors, and view priors can be generated using the following methods:

[0059] Geometric priors: Extract the triangular mesh model of BIM components (for accurately representing the geometric shape of the components), key boundary coordinates (such as edge vertices and corner points of BIM components), and visible surface information (based on the spatial attitude of BIM components to determine the obstructed surfaces).

[0060] Semantic prior: Label each component with category tags (e.g., frame columns, fire pipes), function tags (e.g., load-bearing components, drainage components), and installation stage tags (e.g., main structure stage, decoration and finishing stage);

[0061] Appearance Priority: Based on the material properties of the components (such as the rough texture of concrete and the smooth texture of steel), texture files, surface color parameters, roughness coefficients, etc., a standardized appearance template is generated through material rendering algorithms, thereby ensuring the quantifiable comparison of appearance features.

[0062] Perspective prior, including the following steps 1)-4):

[0063] 1) Path sampling: In the BIM coordinate system, perform discrete sampling of arc length on the predetermined inspection route (the sampling interval can be adjusted according to the component density, such as 5 meters / sampling point), and combine it with the inspection view height (such as the personnel inspection view height or the drone flight height (adjusted according to the building height)) to determine the initial candidate camera position set including multiple initial candidate camera positions.

[0064] 2) Camera orientation calculation: Calculate the optimal orientation for each candidate camera position based on the spatial distribution of the target component in the BIM model and the direction of the outer surface normal (ensure that the camera lens is facing the main visible surface of the BIM component).

[0065] 3) Viewpoint filtering: Through visibility assessment algorithm (used to analyze whether there are other components blocking the view between the component and the camera) and observation quality score (used to remove views with severe occlusion (such as occlusion area exceeding 30%) or poor observation effect (such as component field of view occlusion less than 10%) based on quantitative indicators such as the proportion of the component in the camera's field of view and shooting angle.

[0066] 4) Clustering simplification: The remaining effective viewpoints are clustered according to spatial location (e.g., Euclidean distance clustering) and orientation (e.g., angle clustering, with a clustering threshold of 15°) to merge viewpoints with high similarity, and finally obtain a set of candidate camera poses (including camera 3D coordinates and orientation angles) that covers all key components and has a redundancy of less than 15%, which is used as the viewpoint prior.

[0067] Through the above embodiments, the hierarchical index of components can ensure that the difference results can be accurately written back, avoiding the problem that the regional results cannot be associated with specific components. Furthermore, the optimized filtering of the view prior reduces invalid rendering and subsequent data processing, thereby improving the overall detection efficiency.

[0068] S12, perform BIM rendering based on the hierarchical index and the multimodal priors to generate a component multimodal prior library.

[0069] In this embodiment, the step of performing BIM rendering based on the hierarchical index and the multimodal priors to generate a component multimodal prior library includes:

[0070] At each candidate camera pose, color buffering is performed based on the BIM model and the appearance prior to render a color image consistent with the format acquired by the on-site RGB camera, which is then used as the desired RGB image.

[0071] Based on the aforementioned geometric prior, the distance data from each pixel of each BIM component to the camera is calculated using a depth buffer algorithm to generate the desired depth map.

[0072] Based on the aforementioned geometric prior, the surface normal direction of each BIM component is calculated during the fragment shading stage to generate the desired normal map;

[0073] Based on the semantic prior, the component identifier of each BIM component is mapped to a unique color code, and the desired semantic segmentation map with pixel-level component identifiers is obtained through fill rendering.

[0074] Based on the expected depth map and the expected semantic segmentation map, the number of visible pixels, the proportion of visible area and the number of occlusion layers of each BIM component under the current candidate camera pose are statistically analyzed to obtain the expected visibility and occlusion probability map.

[0075] According to the hierarchical index, the expected RGB image, the expected depth image, the expected normal image, the expected semantic segmentation image, and the expected visibility and occlusion probability image are classified and stored under the hierarchical path of different BIM components to obtain the component multimodal prior library.

[0076] For example: First, configure the rendering environment, that is, set multiple render targets using the rendering engine to achieve simultaneous output of multi-channel data in a single rendering under the same candidate camera pose. Under the same candidate camera pose, perform one or a few off-screen renderings, simultaneously outputting multiple channels. Specifically, under each candidate camera pose, based on the BIM model and its component material information, simultaneously output multi-channel rendering results in the off-screen rendering environment, forming a multimodal "expected observation" corresponding to the on-site sensors. The specific method for generating multimodal expected data is as follows:

[0077] Expected RGB (Red, Green, Blue) image: Based on the color, material, and texture information in the prior appearance, render a color image consistent with the format captured by the on-site RGB camera;

[0078] Desired depth map: The distance data (depth map) from each pixel of the component to the camera is generated by the depth buffer algorithm.

[0079] Expected Normal Map: The surface normal direction of the component is calculated during the fragment shading stage (normal map) for comparing geometric differences;

[0080] Desired semantic segmentation map: Map the component identifier of each component to a unique color code, and achieve pixel-level component identifier labeling through fill rendering to ensure that each pixel can be traced back to the corresponding component;

[0081] Expected Visibility and Occlusion Probability Map: Combining the expected depth map and the expected semantic segmentation map, the number of visible pixels, the proportion of visible area, and the number of occlusion layers (e.g., occluded by 1 other component or occluded by 2 or more other components) of each component under the current view are statistically analyzed to calculate the visibility probability (e.g., 1.0 for fully visible and 0.0 for fully occluded) and occlusion probability distribution of each component.

[0082] All the above rendering results are associated with their corresponding component identifiers and stored according to component hierarchical indexes to form the component multimodal prior library. This library supports fast searching by keywords such as component identifier and viewpoint pose.

[0083] In a single image, all visible components are contained from the corresponding viewpoint.

[0084] In the above embodiments, a foundation for the association between BIM design information and on-site observation data was established, and the abstract BIM model was transformed into a multimodal prior that can be directly compared with on-site data.

[0085] S13, extract the candidate camera pose set from the component multimodal prior library, and collect the field data under multimodal observation corresponding to the BIM model perspective.

[0086] In this embodiment, a data acquisition path can be planned according to the candidate camera pose set, and then the on-site data can be acquired according to the planned data acquisition path to ensure that the acquisition perspective is consistent with the BIM rendering perspective.

[0087] For example, drones, mobile devices (such as tablets with high-definition cameras), or fixed cameras can be used to capture images or videos of the scene in candidate poses (image resolution no less than 1920×1080, video frame rate no less than 30fps) as RGB data. Laser scanners or depth cameras can be used to collect point cloud data of the scene (point cloud density no less than 100 points / cm²). 2Alternatively, depth data can be used as geometric data. Furthermore, if it is necessary to detect temperature-related differences (such as defects in pipe insulation layers), thermal imaging data can be acquired using a thermal imaging camera. If panoramic coverage is required, panoramic images can also be acquired using a panoramic camera as optional data.

[0088] The above embodiments enable standardized collection of multi-source data on site, ensuring that the data format is consistent with the BIM prior library.

[0089] S14. Based on the coarse-to-fine registration mechanism, the on-site coordinate system and the BIM coordinate system are unified according to the on-site data and the BIM model to obtain the target coordinate transformation parameters.

[0090] In this embodiment, the coarse-to-fine registration mechanism, based on the site data and the BIM model, unifies the site coordinate system and the BIM coordinate system to obtain the target coordinate transformation parameters, including:

[0091] Select multiple control points in the BIM model and record the three-dimensional coordinates of each control point in the BIM coordinate system;

[0092] Obtain the three-dimensional coordinates of each control point in the field coordinate system;

[0093] Generate a control point coordinate pair for each control point based on the three-dimensional coordinates of each control point in the BIM coordinate system and the three-dimensional coordinates of each control point in the site coordinate system.

[0094] The least squares algorithm is used to calculate the initial rotation matrix and initial translation vector from the site coordinate system to the BIM coordinate system based on the coordinate pairs of the control points corresponding to each control point.

[0095] Obtain structural feature constraints, and modify the initial rotation matrix and the initial translation vector according to the structural feature constraints to obtain the initial coordinate transformation parameters after coarse registration;

[0096] Generate a field point cloud based on the multiple control points and the field data;

[0097] Extract the component surface mesh within the area corresponding to each control point from the BIM model, and convert the component surface mesh into a BIM reference point cloud;

[0098] Using multi-view constraints and control point constraints as constraints, a preset iterative algorithm is used to iteratively optimize the initial coordinate transformation parameters to obtain the target coordinate transformation parameters after fine registration.

[0099] The control points can be control points with obvious characteristics and that are not easily deformed, such as the center of a structural column, the corner of a floor slab, or the intersection of axes.

[0100] Among these methods, a total station, RTK (Real-Time Kinematic) or visual markers can be used to measure the three-dimensional coordinates of the corresponding control points in the field coordinate system.

[0101] Among them, the rigid body transformation solution method in the sense of least squares is used to obtain the initial rotation matrix and initial translation vector from the site coordinate system to the BIM coordinate system.

[0102] The structural feature constraints may include floor plan, exterior wall surface, column grid direction, etc.

[0103] The point cloud density of the BIM reference point cloud is consistent with that of the on-site point cloud.

[0104] This process involves fusing multiple depth maps or laser scan data to generate a point cloud of the site (noise points, such as isolated points and duplicate points, need to be removed).

[0105] The preset iterative algorithm may include the ICP (Iterative Closest Point) algorithm, the local point cloud-mesh matching algorithm based on planar and line features, etc.

[0106] The multi-view constraint can be used to ensure the consistency of point cloud matching from different perspectives.

[0107] The control point constraints can be used to ensure that the coordinate error of the control points is minimized.

[0108] Among them, multi-view constraints and control point constraints are introduced to construct a global optimization problem, and the camera pose and global rigid body transformation of each frame are jointly adjusted. Finally, the on-site coordinate system and the BIM coordinate system are unified within the centimeter-level or even higher precision range, which provides a reliable pose basis for subsequently projecting or fusing on-site RGB, depth and point cloud data into the BIM rendering perspective to generate on-site multimodal observation.

[0109] In the above embodiments, a coarse-to-fine registration strategy was adopted to achieve centimeter-level unification between the on-site coordinate system and the BIM coordinate system, laying the foundation for subsequent accurate comparison.

[0110] S15, the field data is reconstructed according to the target coordinate transformation parameters to obtain field multimodal observation data.

[0111] In this embodiment, the RGB images, depth data, point cloud data, etc. collected on site can be first transformed and projected according to the unified pose parameters (camera pose in BIM coordinate system).

[0112] Furthermore, the converted field data is projected onto the candidate camera pose viewpoint of the BIM rendering to generate field multimodal observation data consistent with the desired observation data format, thereby achieving alignment with the same viewpoint.

[0113] Specifically, the on-site RGB image in the on-site data can be cropped and corrected according to the BIM rendering perspective to ensure that the image range matches the desired RGB values. Figure 1 For the site depth map in the site data, the site depth data can be mapped to the BIM coordinate system through coordinate transformation, thereby generating depth data with the same resolution and viewpoint as the desired depth map; for the site semantic candidate map in the site data, pixel-level component category candidates can be generated based on the site RGB map and depth map through a preliminary semantic segmentation algorithm, providing a basis for subsequent instance matching.

[0114] Through the above embodiments, the comparison error caused by the difference in perspective can be eliminated based on the reconstruction from the same perspective, thus solving the problem of the gap between the BIM rendering perspective and the actual perspective domain on site.

[0115] S16, for each candidate camera pose in the candidate camera pose set, extract the desired observation from the component multimodal prior library, and determine the field multimodal observation data as the field observation.

[0116] For example, for each candidate viewpoint (i.e., each candidate camera pose), the expected RGB image, expected depth image, expected semantic segmentation image, etc. of the corresponding component can be read from the component multimodal prior library, while the on-site multimodal observation data of the same viewpoint can be read at the same time.

[0117] S17, Perform cross-modal fusion characterization on the expected observation and the on-site observation to obtain a cross-modal fusion feature map.

[0118] In this embodiment, the cross-modal fusion characterization of the expected observation and the in-situ observation to obtain a cross-modal fusion feature map includes:

[0119] The desired observation and the on-site observation are respectively input into a multi-branch encoder that includes multiple modal branches to obtain a first multimodal feature corresponding to the desired observation and a second multimodal feature corresponding to the on-site observation;

[0120] The first multimodal feature and the second multimodal feature are fused by an attention mechanism to map the expected observation and the on-site observation to the same feature space, thereby obtaining an initial fused feature map.

[0121] The initial fused feature map is aligned based on a domain alignment strategy to obtain the cross-modal fused feature map.

[0122] The domain alignment strategy includes image-level coarse alignment to narrow down image styles, feature-level global alignment to make it difficult for the classification network to distinguish between the expected observation and the on-site observation, and local correspondence alignment to narrow down corresponding features of the same component and corresponding features of the same location. For example, when performing image-level coarse alignment, algorithms such as histogram equalization, color mapping, and brightness or contrast adjustment can be used to narrow down the style differences between the expected RGB image (rendered image) and the on-site RGB image (actual image) to reduce interference caused by inconsistencies in color and brightness. When performing feature-level global alignment, a domain classifier (such as a binary classifier to determine whether the feature comes from the rendered data or the actual image) can be introduced, and a back gradient propagation training strategy can be adopted to enable the fusion network to learn domain-independent global features, making it impossible for the domain classifier to distinguish the source of the feature. When performing local correspondence alignment, the geometric or semantic correspondence of BIM (such as the key boundaries of components, pixel-level ID mapping, etc.) can be used, and a feature matching loss function (such as triplet loss) can be used to narrow down the distance between the expected features and the on-site features of the same component and the same location, and widen the distance between the features of different components, thereby ensuring accurate alignment of local features.

[0123] The multi-branch encoder may include RGB branches, depth branches, etc.

[0124] The attention mechanism can include self-attention, cross-attention, etc., and can be implemented through attention fusion structures such as the Transformer structure.

[0125] Specifically, for the image-level coarse alignment, algorithms such as histogram equalization, color mapping, and brightness or contrast adjustment can be used to narrow the style differences between the desired RGB image (rendered image) and the on-site RGB image (actual shot image), thereby reducing interference caused by inconsistencies in color and brightness. For the feature-level global alignment, a domain classifier (such as a binary classifier to determine whether a feature comes from rendered data or actual shot data) can be introduced, and a backpropagation algorithm training strategy can be adopted to enable the fusion network to learn global features that are independent of the domain, making it impossible for the domain classifier to distinguish the source of the features (i.e., the network cannot tell which feature comes from the rendering and which comes from the actual shot). For the local correspondence alignment, the geometric or semantic correspondence of BIM (such as the key boundaries of components and pixel-level ID mapping) can be utilized, and the distance between the desired features and the on-site features of the same component and the same location can be narrowed by using a feature matching loss function (such as triplet loss), while the distance between the features of different components can be widened, thereby ensuring accurate alignment of local features.

[0126] Among them, the domain alignment strategy can make the BIM-rendered map (expected observation) and the map actually captured by the camera (on-site observation) become feature representations of the same style within the network, so as to reduce the difference between the rendering and reality. In this way, when performing difference detection later, the detection results will mainly reflect the construction differences, rather than the difference between the rendering style or the real photo style.

[0127] In the above embodiments, the problem of insufficient robustness of single-modal data is solved by cross-modal fusion and domain alignment strategies, and stable detection can still be achieved in scenarios with occlusion, noise, and changes in illumination.

[0128] S18, perform instance matching on each BIM component in the BIM model in the cross-modal fusion feature map to obtain the on-site component instance corresponding to each BIM component.

[0129] In this embodiment, a component identifier ID mask (keeping only the pixel region corresponding to the component and setting other regions to zero) can be extracted from the desired semantic segmentation map. This mask is then applied to a cross-modal fusion feature map to locate candidate regions for the component in the field observation. Further, instance segmentation is performed on the candidate regions to extract information such as the instance contour and feature vector of the field component, obtaining field component instances. Even further, the similarity between the features of the field component instance and the features of the desired component (e.g., cosine similarity, with a threshold of 0.8) is calculated. If the similarity is higher than the threshold, a successful match is determined, and the field instance is identified as the actual installed entity of the corresponding BIM component.

[0130] In the above embodiments, the precise association between BIM components and on-site entities is achieved through component instance matching, ensuring component-level positioning of discrepancies.

[0131] S19, obtain the installation plan for each BIM component, and perform component-level difference detection based on the installation plan and the on-site component instance corresponding to each BIM component to obtain the difference data for each different component.

[0132] In this embodiment, the installation plan may include an installation time interval, etc.

[0133] In this embodiment, the step of performing component-level difference detection based on the installation plan and the on-site component instance corresponding to each BIM component to obtain the difference data for each differing component includes:

[0134] For each BIM component, if there is no corresponding on-site component instance, the BIM component is determined to be missing; if there is a corresponding on-site component instance, the BIM component and the corresponding on-site component instance are compared point by point to obtain the comparison result, and the geometric deviation distance between the BIM component and the corresponding on-site component instance is calculated based on the comparison result; if the geometric deviation distance is greater than the allowable deviation threshold, and the overall shape of the corresponding on-site component instance is not deformed, the BIM component is determined to have a positional offset; if the geometric deviation distance of the BIM component in a local area is greater than the allowable deviation threshold, and the corresponding on-site component instance has continuous deformation characteristics, the BIM component is determined to have deformation.

[0135] Obtain the RGB image of the on-site component instance as the on-site RGB image, and obtain the expected RGB image of the BIM component from the component multimodal prior library; compare the appearance feature differences between the on-site RGB image and the expected RGB image;

[0136] The theoretical state of the BIM component at the current moment is determined according to the installation plan; statistical results are obtained by performing multi-view statistics on the on-site component instances based on the expected visibility and occlusion probability map; when the statistical results determine that no on-site component instance corresponding to the BIM component is detected in all high visibility views, and the current moment has exceeded the planned installation time, the BIM component is determined to be in an uninstalled state; when the statistical results determine that an on-site component instance corresponding to the BIM component is detected, and the spatial position deviation, orientation deviation, and / or connection relationship deviation of the on-site component instance corresponding to the BIM component exceeds the allowable range, the BIM component is determined to be in an installation error state; when the statistical results determine that an on-site component instance corresponding to the BIM component is detected, the BIM component is not in an installation error state, and the size, appearance, details, and / or design deviation of the on-site component instance corresponding to the BIM component exceeds the limit, the BIM component is determined to be in an installed but non-compliant design state.

[0137] Calculate the confidence level for each type of difference; wherein, the difference types include missing, positional offset, deformation, appearance feature difference, not installed, incorrect installation, and installed but not conforming to the design.

[0138] By integrating the detected difference types and corresponding confidence levels of each BIM component, difference data for each component with differences is obtained.

[0139] For example: if no on-site component instance is found, and no corresponding instance is found from multiple perspectives, it can be determined that the BIM component is missing. If an on-site component instance is found, the on-site depth map or point cloud data can be compared point by point with the expected depth map or the triangular mesh in the geometric prior, and the geometric deviation distance (such as Euclidean distance) of each point can be calculated. If the deviation distance exceeds the allowable deviation threshold (such as ±5cm) and the overall shape is not deformed, it can be determined that there is a positional offset. If the deviation distance of a local area exceeds the threshold and exhibits continuous deformation characteristics (such as column bending), it can be determined that there is deformation. The above determination is a geometric difference determination.

[0140] For example, color histograms, texture features (such as LBP (Local Binary Patterns) features), and roughness features of components can be extracted from the on-site RGB image and compared with the corresponding features in the prior art of the desired appearance. If the color similarity is below a threshold (e.g., 0.7), it is judged as a color anomaly; if the texture feature distance exceeds a set threshold, it is judged as a material anomaly; if the roughness deviation exceeds the allowable range (e.g., ±0.1), it is judged as a surface condition anomaly (e.g., surface cracks, stains). The above judgment is the appearance difference judgment.

[0141] For example, the theoretical state of a component at the current moment can be determined based on its installation schedule (e.g., completed installation, not yet installed). Combining the expected visibility and occlusion probability maps, high visibility angles with theoretical visibility and an occlusion probability below 0.3 are selected. The detection results and geometric and appearance deviations of on-site component instances under these angles are then statistically analyzed. Further, if no on-site component instances are detected in any of the high visibility angles, and the planned installation time has passed, it is determined to be in an uninstalled state. If an on-site component instance is detected, but its spatial position, orientation, or connection to other components deviates beyond the allowable range (e.g., column verticality deviation exceeds 3cm), it is determined to be in an incorrect installation state. If the positional deviation is within the allowable range, but the dimensions, appearance, or details (e.g., node treatment at component connections) deviate from the design limits (e.g., column cross-sectional dimension deviation exceeds ±3cm), it is determined to be in an installed but non-compliant state. The above determinations constitute the installation state difference assessment.

[0142] For example: based on the construction schedule and multi-view visibility priors, the weighted proportion of the corresponding components that were not detected in high visibility views (the weight of high visibility views is higher than that of low visibility views) can be calculated. The higher the proportion, the higher the confidence level (the value range is 0-1.0), thus generating the confidence level of the uninstalled device; normalization evaluation can be performed based on the degree of geometric deviation exceeding the limit (such as the degree to which the geometric deviation of the on-site component instance and the BIM component in the design exceeds the preset tolerance in terms of position, elevation, orientation, etc.) and the consistency of multi-view results (such as two or more of the three views being judged as installation errors). The greater the deviation and the higher the consistency, the higher the confidence level of the installation error; under the premise that the geometric deviation is acceptable, the degree of deviation of appearance features (such as the degree of deviation in appearance features such as color, texture, material and details) and multi-view consistency can be quantified. The more obvious the deviation and the more consistent the observation, the higher the confidence level of the installation being completed but not conforming to the design; the confidence level of the above three anomaly types can be back-calculated by combining the confidence levels (such as 1-maximum anomaly confidence level) or by directly outputting the confidence level of no significant difference from the normal and anomaly discrimination heads.

[0143] The difference data for each component can include the difference type, difference location (such as component local coordinates or BIM global coordinates), difference level (which can be divided into slight, moderate, and severe according to the degree of deviation, such as 5-10cm offset is moderate, and more than 10cm is severe) and corresponding confidence level (such as 0.92).

[0144] In the above embodiments, multi-dimensional difference determination covers non-geometric differences that are currently difficult to detect, enriches the types of differences, and confidence calculation improves the reliability of the results.

[0145] In this embodiment, after obtaining the difference data for each difference component, the method further includes:

[0146] For each difference component, a vote is taken on the difference types under all candidate perspectives, and the difference type with the highest vote rate is selected as the target difference type of the difference component.

[0147] If there are continuous time series field observations, the continuous time series field observations are smoothed.

[0148] Fine-tune the difference data of each difference component based on the target difference type and the smoothed data;

[0149] According to the hierarchical index, the difference data of each differentiated component after fine-tuning is written back to the BIM model.

[0150] For example, the difference determination results (such as difference type and confidence level) of the same component under all candidate views can be collected. A vote is then taken on the difference type, selecting the one with the highest vote rate (e.g., if two out of three views determine it as positional offset and one as having no significant difference, the final difference type is positional offset). If continuous time-series field data exists (such as daily inspection data), Kalman filtering or moving average algorithms are used to remove abnormal results caused by occasional noise (e.g., misjudgments due to temporary obstruction of a certain view). The final difference result after voting and filtering is output to ensure reliability. Furthermore, the difference results (difference type, difference location coordinates, difference level, and confidence level) of each component can be written into the corresponding component attribute fields of the BIM model according to the component level index and component identifier, realizing the BIM model association of design and construction differences, supporting subsequent queries, statistics, and traceability (e.g., filtering all components with severe difference levels using BIM software).

[0151] In the above embodiments, the false alarm rate (such as false positives due to temporary occlusion) is reduced by multi-view consistency verification.

[0152] In this embodiment, before determining component-level differences, an anomaly filtering mechanism based on visibility priors is also included.

[0153] Specifically, the visibility mask of each component to be inspected under the current camera pose can be obtained through the BIM rendering engine. This mask contains pixel-level visibility probability values. In the difference calculation logic, the visibility probability value is used as a judgment weighting factor: when a component is shown to be missing in the on-site observation, but its visibility probability in the BIM prior library is lower than a preset threshold (e.g., 0.5), the component is automatically identified as being in a potential occlusion state rather than a construction omission, and the difference alarm weight of that area is reduced accordingly.

[0154] The above treatment significantly improves the detection accuracy at construction sites (such as in scenarios with dynamic obstructions like scaffolding and material stacking), and avoids the generation of false differences.

[0155] The difference determination in this embodiment is not a simple distance comparison, but is based on visibility prior (such as filtering out false difference points in occluded areas according to the visibility probability map), which significantly improves the accuracy of difference detection.

[0156] Furthermore, components can be labeled with red, yellow, and green in the BIM model, with red indicating severe differences, yellow indicating moderate differences, and green indicating no significant differences.

[0157] Furthermore, based on the difference in location and degree of deviation, a difference heat map of the component surface can be generated (e.g., the positional offset area is represented by different shades of color, with darker colors indicating greater offset).

[0158] Finally, a visual report can be generated, and the report content may include the following:

[0159] (1) Basic information of the project (project name, testing time, testing area, relevant specialties, etc.);

[0160] (2) Summary of differences (the number of differences is counted according to the difference type, difference level, component category, floor or area, etc. For example, 12 components with differences were detected on the 3rd floor, including 5 with positional deviation, 3 with abnormal material, 2 not installed, and 2 with serious differences).

[0161] (3) Detailed list of differences (including component identification, component name, difference type, difference location, difference level, confidence level, handling suggestions, etc.);

[0162] (4) Visual charts (such as screenshots of BIM model annotations, heat maps of differences, statistical bar charts of differences, or pie charts of differences, etc.).

[0163] In the above embodiments, the closed-loop management of design-construction-difference is realized by writing back the difference results to BIM, which facilitates subsequent quality acceptance, progress verification and safety control; the difference information is presented intuitively through visual annotation and reports, which reduces the understanding cost of project managers and improves the efficiency of rectification; the multi-dimensional statistical function supports the overall difference analysis of the project and provides data support for the optimization of the construction process (for example, if there are many abnormal material components in a certain area, the quality problem of raw materials can be traced).

[0164] This embodiment achieves accurate detection of discrepancies across the entire chain from BIM design information to on-site construction, solving problems such as lack of comparable representations, insufficient robustness of single modes, difficulty in component-level positioning, and large errors caused by domain differences in existing technologies. It provides an efficient solution for quality, safety, and schedule management in industrialized and digital construction.

[0165] As can be seen from the above technical solutions, this invention can generate a component multimodal prior library by performing BIM rendering based on hierarchical indexing and multimodal priors, establishing a foundation for the association between BIM design information and on-site observation data, and transforming the abstract BIM model into a multimodal prior that can be directly compared with on-site data; based on a coarse-to-fine registration mechanism, the on-site coordinate system and the BIM coordinate system are unified according to the on-site data and the BIM model, and the on-site data is reconstructed according to the target coordinate transformation parameters, solving the problem of the gap between the BIM rendering perspective and the actual on-site perspective domain; cross-modal fusion characterization of expected observations and on-site observations solves the problem of insufficient robustness of single-modal data; on-site component instances corresponding to each BIM component are obtained through instance matching, and component-level difference detection is performed according to the installation plan and the on-site component instances corresponding to each BIM component, realizing accurate component-level difference detection within the construction site.

[0166] like Figure 2 The diagram shown is a functional block diagram of a preferred embodiment of the construction site component-level difference detection device of the present invention. The construction site component-level difference detection device 11 includes an acquisition unit 110, a creation unit 111, a rendering unit 112, a collection unit 113, a unification unit 114, a reconstruction unit 115, a determination unit 116, a characterization unit 117, a matching unit 118, and a detection unit 119. The module / unit referred to in this invention refers to a series of computer program segments that can be executed by a processor and perform a fixed function, and are stored in memory. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0167] The acquisition unit 110 is used to acquire a BIM model corresponding to the target construction site in response to a component-level difference detection command for the target construction site.

[0168] The establishment unit 111 is used to establish a hierarchical index and multimodal priors based on the BIM model;

[0169] The rendering unit 112 is used to perform BIM rendering based on the hierarchical index and the multimodal priors to generate a component multimodal prior library.

[0170] The acquisition unit 113 is used to extract a set of candidate camera poses from the component multimodal prior library and to acquire field data under multimodal observation corresponding to the BIM model perspective.

[0171] The unification unit 114 is used to unify the site coordinate system and the BIM coordinate system based on the site data and the BIM model according to the coarse-to-fine registration mechanism, so as to obtain the target coordinate transformation parameters.

[0172] The reconstruction unit 115 is used to reconstruct the field data according to the target coordinate transformation parameters to obtain field multimodal observation data.

[0173] The determining unit 116 is used to extract the desired observation from the component multimodal prior library for each candidate camera pose in the candidate camera pose set, and to determine the field multimodal observation data as field observation.

[0174] The characterization unit 117 is used to perform cross-modal fusion characterization on the expected observation and the on-site observation to obtain a cross-modal fusion feature map;

[0175] The matching unit 118 is used to perform instance matching of each BIM component in the BIM model in the cross-modal fusion feature map to obtain the on-site component instance corresponding to each BIM component.

[0176] The detection unit 119 is used to obtain the installation plan for each BIM component, and perform component-level difference detection based on the installation plan and the on-site component instance corresponding to each BIM component to obtain the difference data of each different component.

[0177] As can be seen from the above technical solutions, this invention can generate a component multimodal prior library by performing BIM rendering based on hierarchical indexing and multimodal priors, establishing a foundation for the association between BIM design information and on-site observation data, and transforming the abstract BIM model into a multimodal prior that can be directly compared with on-site data; based on a coarse-to-fine registration mechanism, the on-site coordinate system and the BIM coordinate system are unified according to the on-site data and the BIM model, and the on-site data is reconstructed according to the target coordinate transformation parameters, solving the problem of the gap between the BIM rendering perspective and the actual on-site perspective domain; cross-modal fusion characterization of expected observations and on-site observations solves the problem of insufficient robustness of single-modal data; on-site component instances corresponding to each BIM component are obtained through instance matching, and component-level difference detection is performed according to the installation plan and the on-site component instances corresponding to each BIM component, realizing accurate component-level difference detection within the construction site.

[0178] like Figure 3 The diagram shown is a schematic representation of the computer equipment used in a preferred embodiment of the method for detecting component-level differences at construction sites according to the present invention.

[0179] The computer device 1 may include a memory 12, a processor 13, and a bus (the arrow in the figure represents the bus), and may also include a computer program stored in the memory 12 and executable on the processor 13, such as a construction site component-level difference detection program.

[0180] Those skilled in the art will understand that the schematic diagram is merely an example of computer device 1 and does not constitute a limitation on computer device 1. Computer device 1 can be either a bus topology or a star topology. Computer device 1 may also include more or fewer other hardware or software than shown in the diagram, or different component arrangements. For example, computer device 1 may also include input / output devices, network access devices, etc.

[0181] It should be noted that the computer device 1 described is merely an example. Other existing or future electronic products that are adaptable to this invention should also be included within the scope of protection of this invention and are incorporated herein by reference.

[0182] The memory 12 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the computer device 1, such as a portable hard drive of the computer device 1. In other embodiments, the memory 12 can be an external storage device of the computer device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the computer device 1. Furthermore, the memory 12 can include both internal storage units and external storage devices of the computer device 1. The memory 12 can be used not only to store application software and various types of data installed on the computer device 1, such as the code of a component-level difference detection program at a construction site, but also to temporarily store data that has been output or will be output.

[0183] In some embodiments, the processor 13 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits packaged with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 13 is the control unit of the computer device 1, connecting various components of the computer device 1 via various interfaces and lines. It executes programs or modules stored in the memory 12 (e.g., executing a construction site component-level difference detection program) and calls data stored in the memory 12 to perform various functions of the computer device 1 and process data.

[0184] The processor 13 executes the operating system of the computer device 1 and various installed applications. The processor 13 executes these applications to implement the steps in the above embodiments of the construction site component-level difference detection method, for example... Figure 1 The steps are shown.

[0185] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of performing specific functions, which describe the execution process of the computer program in the computer device 1. For example, the computer program may be divided into an acquisition unit 110, a building unit 111, a rendering unit 112, a collection unit 113, a unification unit 114, a reconstruction unit 115, a determination unit 116, a characterization unit 117, a matching unit 118, and a detection unit 119.

[0186] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, computer equipment, or network device, etc.) or processor to execute portions of the construction site component-level difference detection method described in the various embodiments of this invention.

[0187] If the modules / units integrated in the computer device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware devices. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above.

[0188] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, etc.

[0189] Furthermore, the computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of blockchain nodes, etc.

[0190] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0191] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, in... Figure 3 The bus is represented by only one straight line, but this does not mean that there is only one bus or one type of bus. The bus is configured to enable communication between the memory 12 and at least one processor 13, etc.

[0192] Although not shown, the computer device 1 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 13 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The computer device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0193] Furthermore, the computer device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the computer device 1 and other computer devices.

[0194] Optionally, the computer device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the computer device 1 and to display a visual user interface.

[0195] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0196] It will be understood by those skilled in the art that Figure 3 The structure shown does not constitute a limitation on the computer device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0197] Combination Figure 1 The memory 12 in the computer device 1 stores multiple instructions to implement a method for detecting differences at the component level in a construction site, and the processor 13 can execute the multiple instructions to achieve the following:

[0198] In response to a component-level difference detection command for the target construction site, a BIM model corresponding to the target construction site is obtained;

[0199] Establish a hierarchical index and multimodal priors based on the BIM model;

[0200] BIM rendering is performed based on the hierarchical index and the multimodal priors to generate a component multimodal prior library;

[0201] Extract candidate camera pose sets from the component multimodal prior library, and collect field data under multimodal observation corresponding to the BIM model perspective;

[0202] Based on the coarse-to-fine registration mechanism, the on-site coordinate system and the BIM coordinate system are unified according to the on-site data and the BIM model to obtain the target coordinate transformation parameters;

[0203] The field data is reconstructed based on the target coordinate transformation parameters to obtain field multimodal observation data;

[0204] For each candidate camera pose in the candidate camera pose set, the desired observation is extracted from the component multimodal prior library, and the field multimodal observation data is determined as the field observation;

[0205] The expected observation and the in-situ observation are subjected to cross-modal fusion characterization to obtain a cross-modal fusion feature map;

[0206] Each BIM component in the BIM model is instance-matched in the cross-modal fusion feature map to obtain the on-site component instance corresponding to each BIM component;

[0207] Obtain the installation plan for each BIM component, and perform component-level difference detection based on the installation plan and the corresponding on-site component instance for each BIM component to obtain the difference data for each differing component.

[0208] Specifically, the processor 13's implementation method for the above instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0209] It should be noted that all the data involved in this case was legally obtained.

[0210] If any AI models, software tools, or components not belonging to this company appear in the embodiments of this invention, they are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this invention has been obtained by an entity authorized (with the knowledge and consent) or fully authorized by all parties through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0211] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0212] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0213] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0214] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0215] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0216] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0217] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in this invention can also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0218] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for detecting component-level differences at a construction site, characterized in that, The method for detecting component-level differences at the construction site includes: In response to a component-level difference detection command for the target construction site, a BIM model corresponding to the target construction site is obtained; Establish a hierarchical index and multimodal priors based on the BIM model; BIM rendering is performed based on the hierarchical index and the multimodal priors to generate a component multimodal prior library; Extract candidate camera pose sets from the component multimodal prior library, and collect field data under multimodal observation corresponding to the BIM model perspective; Based on the coarse-to-fine registration mechanism, the on-site coordinate system and the BIM coordinate system are unified according to the on-site data and the BIM model to obtain the target coordinate transformation parameters; The field data is reconstructed based on the target coordinate transformation parameters to obtain field multimodal observation data; For each candidate camera pose in the candidate camera pose set, the desired observation is extracted from the component multimodal prior library, and the field multimodal observation data is determined as the field observation; The expected observation and the in-situ observation are subjected to cross-modal fusion characterization to obtain a cross-modal fusion feature map; Each BIM component in the BIM model is instance-matched in the cross-modal fusion feature map to obtain the on-site component instance corresponding to each BIM component; Obtain the installation plan for each BIM component, and perform component-level difference detection based on the installation plan and the corresponding on-site component instance for each BIM component to obtain the difference data for each differing component.

2. The method for detecting component-level differences at construction sites as described in claim 1, characterized in that, The step of establishing a hierarchical index and multimodal priors based on the BIM model includes: A hierarchical path is established for each BIM component using floor, area, profession, and process as index dimensions, and the hierarchical path of each BIM component is combined to obtain the hierarchical index. Geometric priors, semantic priors, appearance priors, and perspective priors are generated based on the BIM model and used as the multimodal priors. Specifically, the predetermined inspection route of the target construction site is discretized by arc length in the BIM coordinate system, and the position of each initial candidate camera is determined by combining the inspection viewpoint height; the camera orientation corresponding to each initial candidate camera position is calculated based on the spatial distribution position of each BIM component in the BIM model and the direction of the outer surface normal; each initial candidate camera position and its corresponding camera orientation are combined to obtain each candidate viewpoint; each candidate viewpoint is screened by a visibility evaluation algorithm and an observation quality score to obtain each effective viewpoint; each effective viewpoint is clustered to obtain the candidate camera pose set including multiple candidate camera poses; the candidate camera pose set is determined as the viewpoint prior.

3. The method for detecting component-level differences at construction sites as described in claim 2, characterized in that, The step of performing BIM rendering based on the hierarchical index and the multimodal priors to generate a component multimodal prior library includes: At each candidate camera pose, color buffering is performed based on the BIM model and the appearance prior to render a color image consistent with the format acquired by the on-site RGB camera, which is then used as the desired RGB image. Based on the aforementioned geometric prior, the distance data from each pixel of each BIM component to the camera is calculated using a depth buffer algorithm to generate the desired depth map. Based on the aforementioned geometric prior, the surface normal direction of each BIM component is calculated during the fragment shading stage to generate the desired normal map; Based on the semantic prior, the component identifier of each BIM component is mapped to a unique color code, and the desired semantic segmentation map with pixel-level component identifiers is obtained through fill rendering. Based on the expected depth map and the expected semantic segmentation map, the number of visible pixels, the proportion of visible area and the number of occlusion layers of each BIM component under the current candidate camera pose are statistically analyzed to obtain the expected visibility and occlusion probability map. According to the hierarchical index, the expected RGB image, the expected depth image, the expected normal image, the expected semantic segmentation image, and the expected visibility and occlusion probability image are classified and stored under the hierarchical path of different BIM components to obtain the component multimodal prior library.

4. The method for detecting component-level differences at construction sites as described in claim 1, characterized in that, The coarse-to-fine registration mechanism unifies the site coordinate system and the BIM coordinate system based on the site data and the BIM model, resulting in target coordinate transformation parameters including: Select multiple control points in the BIM model and record the three-dimensional coordinates of each control point in the BIM coordinate system; Obtain the three-dimensional coordinates of each control point in the field coordinate system; Generate a control point coordinate pair for each control point based on the three-dimensional coordinates of each control point in the BIM coordinate system and the three-dimensional coordinates of each control point in the site coordinate system. The least squares algorithm is used to calculate the initial rotation matrix and initial translation vector from the site coordinate system to the BIM coordinate system based on the coordinate pairs of the control points corresponding to each control point. Obtain structural feature constraints, and modify the initial rotation matrix and the initial translation vector according to the structural feature constraints to obtain the initial coordinate transformation parameters after coarse registration; Generate a field point cloud based on the multiple control points and the field data; Extract the component surface mesh within the area corresponding to each control point from the BIM model, and convert the component surface mesh into a BIM reference point cloud; Using multi-view constraints and control point constraints as constraints, a preset iterative algorithm is used to iteratively optimize the initial coordinate transformation parameters to obtain the target coordinate transformation parameters after fine registration.

5. The method for detecting component-level differences at construction sites as described in claim 1, characterized in that, The cross-modal fusion characterization of the expected observation and the in-situ observation to obtain the cross-modal fusion feature map includes: The desired observation and the on-site observation are respectively input into a multi-branch encoder that includes multiple modal branches to obtain a first multimodal feature corresponding to the desired observation and a second multimodal feature corresponding to the on-site observation; The first multimodal feature and the second multimodal feature are fused by an attention mechanism to map the expected observation and the on-site observation to the same feature space, thereby obtaining an initial fused feature map. The initial fused feature map is aligned based on a domain alignment strategy to obtain the cross-modal fused feature map. The domain alignment strategies include image-level coarse alignment to bring image styles closer, feature-level global alignment to make it difficult for classification networks to distinguish between the expected observation and the on-site observation, and local correspondence alignment to bring corresponding features of the same component and corresponding features of the same location closer.

6. The method for detecting component-level differences at construction sites as described in claim 2, characterized in that, The step of performing component-level difference detection based on the installation plan and the on-site component instance corresponding to each BIM component to obtain the difference data for each differing component includes: For each BIM component, the difference type is detected based on the installation plan and the corresponding on-site component instance for each BIM component. Calculate the confidence level for each type of difference; wherein, the difference types include missing, positional offset, deformation, appearance feature difference, not installed, incorrect installation, and installed but not conforming to the design. By integrating the detected difference types and corresponding confidence levels of each BIM component, difference data for each component with differences is obtained.

7. The method for detecting component-level differences at construction sites as described in claim 6, characterized in that, After obtaining the difference data for each difference component, the method further includes: For each difference component, a vote is taken on the difference types under all candidate perspectives, and the difference type with the highest vote rate is selected as the target difference type of the difference component. If there are continuous time series field observations, the continuous time series field observations are smoothed. Fine-tune the difference data of each difference component based on the target difference type and the smoothed data; According to the hierarchical index, the difference data of each differentiated component after fine-tuning is written back to the BIM model.

8. A component-level difference detection device for construction sites, characterized in that, Used to perform the on-site component-level difference detection method as described in any one of claims 1-7.

9. A computer device, characterized in that, The computer device includes: A memory that stores at least one instruction; and a processor that executes the instructions stored in the memory to implement the construction site component-level difference detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, which is executed by a processor in a computer device to implement the construction site component-level difference detection method as described in any one of claims 1 to 7.

Images