Construction site real-time guidance method and system based on BIM-MR fusion

Abstract

The invention discloses a construction site real-time guidance method and system based on BIM-MR fusion, and the method mainly comprises the steps: obtaining the spatial data of a construction site through a multi-source sensing device; fusing the multi-source sensing data, and carrying out spatial positioning on the prefabricated part; dynamically loading the BIM model through a lightweight BIM engine; through MR interaction, user gestures and construction tools are recognized, corresponding BIM component operation instructions are triggered, and visual guidance and operation deviation are given; and when the cloud BIM model is updated, comparing a field actual state with the BIM model, and prompting and displaying a change area on the terminal MR equipment. According to the method, high-precision real-time construction guidance and collaborative management can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of building engineering informatization technology, in particular to a construction site real-time guidance method and system based on BIM (Building Information Modeling) and MR (Mixed Reality) technology fusion. BACKGROUND

[0002] At present, the guidance work of the construction site mainly depends on two-dimensional drawings and offline BIM models, and workers need to frequently switch between electronic devices and physical scenes, resulting in low operation efficiency and easy errors. Although the existing MR technology can superimpose virtual models on the real environment, there are still problems such as insufficient positioning accuracy, data collaboration lag, and inconvenient interaction mode, especially in complex construction environments, it is difficult for traditional methods to realize high-precision real-time guidance and collaborative management. SUMMARY

[0003] The present application provides a construction site real-time guidance method and system based on BIM-MR fusion, which can realize real-time construction guidance and collaborative management with centimeter-level precision through multi-source data fusion, lightweight BIM engine, intelligent interaction control and other technologies.

[0004] The construction site real-time guidance method based on BIM-MR fusion provided by the present application mainly includes the following steps: S1, acquiring spatial data of the construction site by using a multi-source perception device; S2, performing fusion processing on the data acquired by the multi-source perception device, and performing spatial positioning on the prefabricated components; S3, dynamically loading a BIM model by using a lightweight BIM engine, and adaptively adjusting the model detail level according to the user's field of view; S4, recognizing user gestures and construction tools through MR interaction, triggering corresponding BIM component operation instructions, and giving visual guidance; S5, when the cloud BIM model is updated, comparing the actual state of the site with the BIM model, and prompting the changed area on the terminal MR device.

[0005] Further, in step S1, the multi-source perception device specifically includes: a UWB base station network for providing macro spatial positioning; a laser SLAM scanner for generating a high-precision point cloud reference surface; a visual recognition system arranged on the MR mobile terminal for recognizing the identification and components of the construction site.

[0006] Further, in step S2, the data fusion adopts an algorithm based on Lie group manifold optimization to dynamically fuse multi-source heterogeneous data and realize centimeter-level precision spatial registration.

[0007] Further, in step S3, the lightweight BIM engine dynamically adjusts model details through adaptive LOD algorithm and reduces data transmission delay through compression technology.

[0008] Further, in step S4, the MR interaction control system identifies construction tools through feature matching and triggers corresponding operation instructions, providing tactile and visual feedback.

[0009] Further, in step S5, the point cloud and BIM model are compared through a non-rigid registration algorithm, and the deviation value is mapped to the color space for visual prompt.

[0010] Further, the method further comprises the following steps: S6, real-time synchronization of construction data through digital twin, risk prediction and model update; Wherein, the digital twin synchronizes construction data in real time based on OPCUA protocol, and performs risk prediction and decision support through LSTM model.

[0011] A BIM-MR fusion-based real-time construction site guidance system applying the above method mainly includes: A multi-source perception module for obtaining spatial data of the construction site; A data fusion module for processing multi-source data and outputting high-precision positioning parameters; A lightweight BIM module for dynamically loading and rendering BIM models; An MR interaction control module for identifying user interaction intent and triggering operation instructions; A real-time guidance module for providing visual guidance and deviation prompt.

[0012] Further, the multi-source perception module includes a UWB positioning unit, a laser SLAM scanning unit and a visual recognition unit.

[0013] Further, the system further includes a digital twin module for synchronizing data and performing risk prediction.

[0014] Compared with the prior art, the beneficial effects of the present disclosure are: ① Through multi-source data fusion, cm-level precision BIM model and physical environment dynamic matching are achieved; ② Through tool recognition and gesture control, a multi-channel interactive real-time construction guidance mechanism is established; ③ Real-time data synchronization ensures that design changes are timely transmitted to the construction site; ④ Eliminate information fault and improve construction quality. BRIEF DESCRIPTION OF DRAWINGS

[0015] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which like reference characters refer to like parts throughout the figures, and in which:

[0016] Figure 1 System architecture and module interaction diagram according to an exemplary embodiment of the present disclosure; Figure 2 Dynamic loading mechanism diagram of lightweight BIM engine; Figure 3 MR dual-mode interaction diagram; Figure 4 Interaction diagram of real-time guidance system and / digital twin;

[0017] Figure 5 Digital twin application flowchart; Figure 6 BIM-MR application flowchart. DETAILED DESCRIPTION

[0018] Preferred embodiments of the present disclosure will be described herein below with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure is more thoroughly and completely conveyed to those skilled in the art, and so that the scope of the present disclosure is fully conveyed to those skilled in the art.

[0019] The present disclosure provides a BIM-MR fusion-based real-time guidance method and system for construction sites, which mainly includes the following parts: Multi-source data acquisition: Deploy a UWB base station network at the construction site to form a spatial positioning framework. High-precision point cloud data is generated by a laser SLAM scanner, and a visual recognition system is used to identify the identification code of prefabricated components. The multi-source perception layer transmits the collected data to the data fusion engine.

[0020] Data fusion processing: The data fusion engine uses extended Kalman filtering and Lie group optimization algorithms to perform real-time fusion processing on multi-source data. Through dynamic weight adjustment, centimeter-level spatial registration parameters are output to ensure high-precision matching of BIM models and physical environments.

[0021] BIM model dynamic loading: The lightweight BIM engine parses the BIM model and dynamically loads the required model parts according to the user's field of view. Adaptive LOD technology is used to optimize rendering efficiency, ensuring smooth model interaction on mobile devices.

[0022] Intelligent Interaction Control: The MR interaction control system recognizes user gestures and construction tools through feature matching algorithms, triggering corresponding operation instructions. For example, when the system detects that a worker is holding a wrench, it automatically displays guidance information for tightening bolts.

[0023] Real-time Guidance and Deviation Prompt: The real-time guidance system compares point cloud data and BIM models in the field through non-rigid registration algorithms, calculates deviation values, and maps them to the HSV color space. At the same time, through spatial audio, it prompts workers for operational deviations.

[0024] Data Synchronization and Risk Prediction: The digital twin synchronizes construction data in real time based on the OPCUA protocol and analyzes historical data through an LSTM model to predict potential risks. For example, when detecting abnormal component stress, the system automatically issues a warning.

[0025] Based on the above ideas, in an exemplary embodiment, a construction site real-time guidance system based on BIM-MR fusion includes the following modules: Multi-source Perception Module: for collecting spatial data of the construction site.

[0026] Data Fusion Module: for processing multi-source data and outputting high-precision positioning parameters.

[0027] Lightweight BIM Module: for dynamically loading and rendering BIM models.

[0028] MR Interaction Control Module: for recognizing user interaction intent and triggering operation instructions.

[0029] Real-time Guidance Module: for providing visual guidance and deviation prompts.

[0030] Digital Twin Module: for synchronizing data and predicting risks.

[0031] The construction site real-time guidance method based on BIM-MR fusion mainly includes the following steps: Multi-source data acquisition: through the UWB base station network, laser SLAM scanner and visual recognition system, spatial positioning data, point cloud data and visual data of the construction site are obtained; Data fusion processing: use data fusion engine to perform real-time fusion processing on multi-source data, output high-precision spatial registration parameters through Lie group optimization algorithm; BIM model dynamic loading: through lightweight BIM engine, dynamically load BIM model according to user's field of view range, and use adaptive LOD technology to optimize rendering efficiency; Intelligent Interaction Control: Through the MR interaction control system, recognize user gestures and construction tools, trigger corresponding operation instructions, and provide tactile and visual feedback; Real-time guidance and deviation prompt: Through real-time guidance system, compare the actual state with BIM model, visualize the deviation value in color space, and provide multi-channel prompt. Data synchronization and risk prediction: Through digital twin, synchronize construction data in real time based on OPCUA protocol, and use LSTM model for risk prediction and decision support.

[0032] The scheme of the embodiment is further described as follows in combination with the drawings.

[0033] As shown in the accompanying Figure 1 The system architecture mainly includes: 1. Multi-source perception layer: In the construction site, deploy ultra-wideband (UWB) base station network to form a spatial positioning framework; combine laser SLAM scanner to generate point cloud reference surface; mobile terminal (such as MR glasses) identifies prefabricated component two-dimensional code label through binocular vision; input the three data into extended Kalman filter, output dynamic positioning compensation parameters, and finally realize <1cm model superposition accuracy.

[0034] Specific implementation: UWB base station network (6-8GHz frequency band) forms a spatial positioning framework, and uses TDOA algorithm to solve terminal coordinates; Mobile laser SLAM scanner (Velodyne VLP-16) generates point cloud reference surface, and reduces noise through ISS key point extraction algorithm; Binocular vision system (global shutter CMOS) identifies prefabricated component two-dimensional code, and uses deep learning OCR decoding; Data flow: Three source data are fused through extended Kalman filter:

[0035]

[0036] Among them: (6DoF pose) .

[0037] 2. Data fusion engine: The data fusion engine uses a multi-source heterogeneous data fusion framework based on Lie group manifold optimization, and through the establishment of a space-time registration model in SE(3) special Euclidean group space, the macro positioning data (accuracy ±10cm) provided by the UWB base station network, the centimeter-level point cloud reference surface (resolution 2cm / pixel) generated by the laser SLAM scanner, and the component-level two-dimensional code recognition information (decoding rate 99.8%) captured by the binocular vision system are fused in real time.

[0038] The engine core runs Lie group optimization algorithm, which performs an iterative calculation every 200 ms through Riemannian gradient descent method Optimize spatial registration parameters while integrating extended Kalman filter for real-time compensation of device jitter errors, and finally output MR rendering instruction stream with registration error <0.8 cm, and maintain sub-centimeter spatial consistency in dynamic construction environment.

[0039] 3. Lightweight BIM engine: The cloud BIM model is split into spatial topology units by the parsing module, and dynamically loaded through adaptive LOD (level of detail) algorithm. When the user's field of view focuses on a specific area, the engine automatically simplifies non-critical structures (such as hidden pipe inner walls) according to the component type, and only retains the outer contour and connection nodes, so that a 200GB level model has a loading delay of less than 300ms under 5G network.

[0040] 4. MR interaction control system: Embed gesture-physical coupling interaction layer in MR interface. As shown in Figure 3 : Capture gesture trajectory by workers wearing touch gloves (with 9-axis IMU sensors built-in); at the same time, scan the morphological characteristics of construction tools (such as wrenches and levels) to establish a tool library; When a worker is detected holding a specific tool, automatically activate the corresponding BIM component operation instruction, such as bolt tightening angle prompt, forming a natural interaction paradigm of "tool as interface".

[0041] Among them, the touch gloves integrate 9-axis IMU (MPU-9250) with a sampling rate of 100Hz; The tool library contains 32 types of construction tool feature templates, which trigger instructions when the matching degree is met; Feedback mechanism: Linear resonant actuator (LRA) generates 200Hz pulse to prompt operation deviation.

[0042] 5. Real-time guidance system: Based on OPC UA protocol to build design change synchronization channel, when the cloud BIM model is updated, the incremental data packet is aligned in space-time stamp by edge computing node (deployed on site server), pushed to the terminal MR device and highlighted the changed area, to ensure that the guidance information is consistent with the latest design version.

[0043] Protocol architecture details: (1) Application layer protocol Core mechanism: Build data distribution service based on OPC UA publish / subscribe mode Data encapsulation: Use JSON serialization format (payload compression rate 62%), containing three layers of metadata structure: metadata header │ space-time stamp <t_loc, t_edge; Business data body │ BIM increment / operation instruction; Check code; Quality of service: QoS = 1 (at least once delivery) ensures key instruction accessibility.

[0044] (2) Transport layer optimization Protocol stack: MQTT 3.1.1 protocol runs on 5G URLLC (Ultra Reliable Low Latency Communication) channel Anti-jitter mechanism: dynamic bandwidth allocation algorithm guarantees key data flow:

[0045] (α = business weight factor, β = burst traffic compensation, τ = 300 ms time window) (3) Network layer routing Addressing scheme: IPv6 segment routing (SRv6) realizes multi-edge node cooperation Time synchronization: integrate gPTP protocol of TSN (Time Sensitive Network), clock synchronization error < 800 ns (4) Physical layer enhancement Dual-mode transmission: Backbone link: 802.11ay millimeter wave (60GHz frequency band, 4x4 MIMO) Redundant link: LoRa 470MHz (10km coverage, 50kbps rate) Adaptive modulation: dynamically switch 256QAM / 64QAM modulation mode according to channel quality Figure 4 For real-time guidance system and / digital twin interaction diagram, wherein: Uplink data flow: operation instruction (average size 5KB) generated by real-time guidance system is published to edge node through OPC UA; Downlink data flow: model change (ΔBIM package) issued by digital twin is pushed to terminal through MQTT, with delay jitter < 15ms; Closed loop control: Lamport logical clock ensures the timing consistency of the three, meets .

[0046] 6. Digital twin: Digital twin constructs a cloud virtual mirror system, which receives real-time construction data flow (including positioning information, quality acceptance record, process video, etc.) from the field through OPC UA protocol, and establishes a four-dimensional digital mapping model based on the space-time stamp <t_loc, t_edge, δ>. Its core functions include: Incremental update mechanism: use improved Delta Lake architecture to synchronize only changed data packages (average size < 50KB) Predictive analytics engine: Based on LSTM network training of historical construction data, predict potential risks (such as the probability of component stress exceeding limits).

[0047] The workflow of the above system is as follows: Figure 6 As shown: 1. Equipment initialization phase During the system startup phase, the multi-source perception layer and the real-time communication layer work together to complete the hardware network construction and software environment deployment.

[0048] UWB base stations automatically form a spatial positioning skeleton through the Zigbee protocol and use a distributed TDOA (Time Difference of Arrival) algorithm to calculate the initial coordinates of the terminal. That At the speed of light, This is a multipath error compensation term (empirical value 0.3ns); At the same time, the lightweight BIM engine loads the spatial topology index (IFC parser output), and the real-time communication layer establishes an OPCUA secure session.

[0049] 2. Spatial calibration stage The LiDAR scanner (Velodyne VLP-16) generates a point cloud reference surface at a rate of 1.2 million points / second. After noise reduction processing using the ISS key point extraction algorithm, the target point cloud density is ≥50pts / m². The vision system identifies the AprilTag mark on the preset calibration board; The data fusion engine performs coarse ICP registration:

[0050] in, LiDAR point cloud coordinates, The theoretical coordinates of the calibration plate are given. The rotation matrix is ​​ultimately solved using SVD decomposition. R Translation vector t This achieves initial coordinate system alignment (error < 2cm).

[0051] Module participation: Multi-source sensing layer: Point cloud generated by LiDAR scanning (ISS key point density ≥ 50 pts / m²) Data fusion engine: Performs initial registration (ICP coarse registration, error <2cm) Coordinate system transformation formula:

[0052] ( Euler angle compensation, solved by SVD decomposition.

[0053] 3. Dynamic registration stage Binocular vision system real-time recognizes component 2D code (decoding rate 99.8%), data fusion engine calls Lie group optimization algorithm:

[0054] Weight matrix

[0055] Dynamic adjustment: (RSSI signal strength), ( p Point cloud density.

[0056] Riemannian gradient descent is performed every 200ms , step ( k number of iterations), output pose compensation parameters.

[0057] 4, Human-computer interaction stage When the worker holds the construction tool, the system performs real-time template matching through improved SURF feature descriptor (Hessian threshold = 500): , where the wrench head feature weight wi=0.7.

[0058] When MatchScore≥0.85, the LRA actuator of the touch finger sleeve (MPU-9250) generates a 200Hz pulse, and superimposes the AR torque guidance to generate a torque guidance progress bar (τ: angular velocity of gyroscope, I: component moment of inertia).

[0059] 5, Deviation guidance stage The real-time guidance system compares point clouds and BIM models through a non-rigid registration algorithm: ε =∥ Pcloud S( Pbim )∥2 where S is a thin plate spline interpolation function (TPS). The MR interaction control layer maps the deviation value δ to the HSV color space: , At the same time, activate the spatial audio prompt (sound pressure level ), realize multi-channel deviation warning.

[0060] 6, Data closed loop stage Acceptance data is uploaded to the digital twin through the OPC UA protocol, and incremental update is performed using the improved Delta Lake architecture (average packet size < 50KB). The prediction engine performs risk analysis based on the LSTM model: where The stress rate of change for the component. The real-time communication layer ensures temporal consistency through Lamport clocks: When the cumulative deviation is detected to be >3mm, the model correction instruction is automatically triggered.

[0061] The process and module correspondence is shown in the following table:

[0062] In this embodiment, through multi-source data fusion, lightweight BIM engine, intelligent interaction control and other technologies, a technical closed loop of "multi-source fusion positioning-BIM lightweight-intelligent interaction" is constructed, realizing high-precision real-time construction guidance and collaborative management: Spatial positioning compensation mechanism: Through the UWB network to provide macro positioning constraint (accuracy ±10cm), LiDAR to build centimeter level point cloud reference surface, visual system to realize component level identification, innovatively use extended Kalman filter to fuse three source heterogeneous data, dynamically compensate sensor error. This mechanism breaks through the technical limitations of traditional single positioning method, and still maintains <1cm positioning accuracy in dense steel structure area.

[0063] Tool interface (TUI) interaction paradigm: Establish a construction tool form feature library (including 32 types of wrenches / levels / welding guns, etc.), use improved SURF feature descriptor for real-time template matching. When the system detects that the worker is holding a specific tool, the corresponding BIM component operation logic is automatically activated, forming a natural interaction mode of "what you see is what you operate". This paradigm eliminates the traditional MR interface menu operation, allowing workers to focus on entity work.

[0064] Space-time driven synchronization protocol: Based on the OPC UA framework, an incremental data synchronization mechanism is designed, and innovatively uses Lamport logical clock to solve the problem of temporal consistency of multiple terminals. The construction method of space-time stamp format <t_loc, t_edge, δ> (t_loc: local time, t_edge: edge node time, δ: change sequence number) ensures reliable transmission of change instructions under 5% packet loss rate.

[0065] Lie group optimization registration algorithm: Establish a registration error model in the SE(3) manifold space, and solve the optimal transformation matrix through the Riemann optimization algorithm. The innovation lies in the adaptive construction method of the weight matrix W: according to the UWB signal strength, LiDAR point cloud density, and visual feature matching degree, the weight distribution is dynamically adjusted, so that the registration accuracy is improved by 83% compared with the traditional ICP algorithm.

[0066] The main beneficial effects are: High-precision positioning: Through multi-source data fusion, the positioning error is reduced from 4-5cm in traditional methods to within 0.8cm.

[0067] Efficient Interaction: Recognize tools and gestures for control, reduce operation interruption, and improve work efficiency.

[0068] Real-time Collaboration: Synchronize data in real-time through digital twins, ensure design changes are timely delivered to the construction site.

[0069] Risk Prediction: Analyze construction data through LSTM model, identify potential problems in advance, and reduce rework and losses.

[0070] The present disclosure is applicable to various construction scenarios, especially suitable for steel structure installation, curtain wall construction, mechanical and electrical pipe gallery integration, and other complex projects.

[0071] The above technical solutions are only exemplary embodiments of the present application. For those skilled in the art, based on the application method and principle disclosed in the present application, various types of improvements or modifications can be easily made, and are not limited to the methods described in the above specific embodiments of the present application. Therefore, the above-described method is only preferred and has no limiting meaning.

Claims

1. A real-time construction site guidance method based on BIM-MR fusion, characterized in that, Includes the following steps: S1, using multi-source sensing devices to acquire spatial data of the construction site; S2, performs fusion processing on the data acquired by multi-source sensing devices, and performs spatial positioning of prefabricated components; S3 dynamically loads BIM models through a lightweight BIM engine and adaptively adjusts the model detail level according to the user's field of view. S4, through MR interaction, recognizes user gestures and construction tools, triggers corresponding BIM component operation commands, and provides visual guidance and operation deviations; S5: When the cloud-based BIM model is updated, it compares the actual site conditions with the BIM model and displays the changed areas on the terminal MR device.

2. The method according to claim 1, characterized in that, In step S1, the multi-source sensing device specifically includes: UWB base station networks are used to provide macroscopic spatial positioning; Laser SLAM scanner is used to generate high-precision point cloud reference surfaces; A visual recognition system installed on MR mobile terminals is used to identify signs and components at construction sites.

3. The method according to claim 1, characterized in that, In step S2, the data fusion adopts an algorithm based on Lie group manifold optimization to dynamically fuse multi-source heterogeneous data and achieve spatial registration with centimeter-level accuracy.

4. The method according to claim 1, characterized in that, In step S3, the lightweight BIM engine dynamically adjusts model details through an adaptive LOD algorithm and uses compression technology to reduce data transmission latency.

5. The method according to claim 1, characterized in that, In step S4, the MR interactive control system identifies the construction tools through feature matching and triggers corresponding operation commands, providing tactile and visual feedback.

6. The method according to claim 1, characterized in that, In step S5, the point cloud and the BIM model are compared using a non-rigid registration algorithm, and the deviation value is mapped to the color space for visualization.

7. The method according to any one of claims 1-6, characterized in that, It also includes the following steps: S6 uses a digital twin to synchronize construction data in real time for risk prediction and model updates; The digital twin synchronizes construction data in real time based on the OPCUA protocol and uses an LSTM model for risk prediction and decision support.

8. A real-time construction site guidance system based on BIM-MR fusion, used to implement the method described in any one of claims 1-7, characterized in that, include: Multi-source sensing module is used to acquire spatial data of the construction site; The data fusion module is used to process multi-source data and output high-precision positioning parameters; A lightweight BIM module for dynamically loading and rendering BIM models; The MR interactive control module is used to identify user interaction intentions and trigger operation commands; The real-time guidance module provides visual guidance and deviation alerts.

9. The system according to claim 8, characterized in that, The multi-source sensing module includes: a UWB positioning unit, a laser SLAM scanning unit, and a visual recognition unit.

10. The system according to claim 8 or 9, characterized in that, Also includes: The digital twin module is used to synchronize data and perform risk prediction.

Images