Engineering management system based on digital twinborn and mixed reality technology and use method thereof

Abstract

The invention relates to the technical field of construction engineering management, in particular to an engineering management system based on digital twinning and mixed reality technologies and a use method thereof, and the engineering management system comprises a digital twinning construction module, a virtual-real mapping module, a natural interaction module and an intelligent decision module. The digital twinning construction module comprises MR equipment, an SLAM technology and a BIM digital twinning model; the MR equipment and the SLAM technology are used for carrying out rapid scanning and spatial positioning on a real engineering environment; the invention relates to a BIM (Building Information Modeling) digital twin model, which takes a BIM model as a geometric and data basis. The method has the advantages of integrating perception, interaction, analysis and decision making; according to the method, cognitive transition from screen three-dimensional to spatial three-dimensional is realized, the high-precision BIM digital twin model and a real engineering environment are subjected to centimeter-level precision spatial registration and superposition through MR equipment and an SLAM technology, a visual barrier between the digital model and a physical entity is thoroughly broken, and a manager can directly see through a building structure.

Description

Technical Field

[0001] This invention relates to the field of construction engineering management technology, specifically to an engineering management system based on digital twin and mixed reality technologies and its usage method. Background Technology

[0002] In the field of construction project management, the successful delivery of a project depends heavily on the effective control of schedule, quality, cost, and safety. However, traditional management methods have many inherent flaws and are difficult to meet the high standards required for modern large-scale and complex projects.

[0003] First, information visualization and interaction methods are outdated. Although current mainstream management systems have introduced 3D BIM models for assistance, they are essentially still just displaying the model on a computer or mobile phone screen. This is a kind of "screen 3D" detached from the real environment. Managers cannot make intuitive and accurate spatial comparisons between virtual models and physical entities. They rely heavily on personal spatial imagination and engineering experience, which can easily lead to cognitive biases and decision-making errors.

[0004] Secondly, the data collection and updating mechanisms are inefficient and fragmented. Recording of on-site progress, quality, and safety issues still heavily relies on manual methods, such as paper forms and taking photos with a mobile phone before secondary data entry into a computer. This results in significant data lag, preventing digital models from accurately reflecting the rapidly changing construction site and creating a severe disconnect between the "digital virtual model" and the "physical entity." Digital models often cease updating after the construction phase, becoming static archives and failing to realize their core value throughout the entire project lifecycle.

[0005] Furthermore, the phenomenon of "information silos" is widespread. BIM design data, real-time environmental data collected by IoT sensors, schedule data, and quality and safety report data are usually stored in independent systems and platforms with different data formats, making it difficult to conduct effective correlation analysis and in-depth mining. This results in management decisions being based on isolated and outdated information, lacking data-driven scientific basis.

[0006] Therefore, there is an urgent need for an engineering management system based on digital twin and mixed reality technologies, as well as its usage methods, to solve the above problems. Summary of the Invention

[0007] The purpose of this invention is to provide an engineering management system and its usage method based on digital twin and mixed reality technologies, which has the advantages of integrating perception, interaction, analysis and decision-making, and solves the problems mentioned in the background technology.

[0008] To achieve the above objectives, the present invention provides the following technical solution: an engineering management system based on digital twin and mixed reality technologies, comprising: a digital twin construction module, a virtual-real mapping module, a natural interaction module, and an intelligent decision-making module.

[0009] The digital twin building block includes MR devices and SLAM technology, as well as a BIM digital twin model. MR devices and SLAM technology are used for rapid scanning and spatial positioning of real-world engineering environments.

[0010] A BIM digital twin model is a dynamic digital model that uses a BIM model as its geometric and data foundation and brings it to life through the Internet of Things and big data technologies. It can evolve synchronously with the real building and support simulation and decision-making.

[0011] The virtual-real mapping module includes a project progress display module, a project progress reporting module, a quality and safety positioning and information uploading module, and a quality and safety display module.

[0012] The project progress display module uses the currently projected digital twin image to dynamically display the overall construction progress of the project.

[0013] The project progress reporting module allows on-site personnel to directly manipulate the virtual model within the MR environment, thereby updating the actual project progress data.

[0014] The quality and safety positioning and information uploading module allows personnel to directly "click" on the problematic area in the digital twin model using gestures when quality defects or safety hazards are discovered on-site, achieving spatial positioning with centimeter-level accuracy.

[0015] In the quality and safety display module, all reported quality and safety issues are marked with prominent virtual labels in the digital twin model.

[0016] The natural interaction module includes a voice reporting module, a gesture interaction mechanism module, and a gesture interaction library; The voice reporting module automatically identifies key information, converts it into structured data, and automatically updates it to the backend database.

[0017] Gesture interaction mechanism module, used for spatial pointing and operation.

[0018] Gesture interaction library, a collection of predefined gesture commands.

[0019] The intelligent decision-making module includes a multi-source data integration module and an intelligent analysis and decision support module.

[0020] The multi-source data integration module can access and integrate various information from on-site engineering projects.

[0021] The intelligent analysis and decision support module performs machine learning and rule-based reasoning on the fused data.

[0022] Furthermore, as a preferred embodiment of the present invention, the MR device has a built-in depth camera and an inertial measurement unit.

[0023] Furthermore, as a preferred embodiment of the present invention, the project progress display module uses color status coding, with unconstructed parts displayed in gray, parts under construction displayed in yellow, and completed parts displayed in green.

[0024] Furthermore, as a preferred embodiment of the present invention, the quality and safety positioning and information uploading module includes an MR camera and a microphone, which are used to collect images, videos and descriptions of the problem scene.

[0025] Furthermore, as a preferred embodiment of the present invention, the gesture interaction mechanism module inputs gesture information into the gesture interaction library, analyzes the predefined gesture instructions in the gesture interaction library, and then transmits the correct instructions.

[0026] Furthermore, as a preferred embodiment of the present invention, the gesture interaction mechanism module includes a depth camera, an inertial measurement unit, and a radar system.

[0027] Furthermore, as a preferred embodiment of the present invention, the multi-source data integration module is electrically connected to the digital twin construction module, the quality and safety positioning and information uploading module.

[0028] This invention discloses a method for using an engineering management system based on digital twin and mixed reality technologies, characterized in that: Step 1: The digital twin building module is the foundation of the entire system. Among them, the MR device plays a key role. After the MR device is started, its built-in depth camera and inertial measurement unit will start working. The depth camera will acquire depth point cloud data of the surrounding environment by emitting and receiving infrared light, and quickly build a three-dimensional geometric space of the scene. The IMU will continuously track the device's own degrees of freedom of movement through accelerometers and gyroscopes. SLAM technology enables the MR device to fuse visual data with IMU data and run simultaneous localization and mapping algorithms. This allows the device to calculate its own position and attitude in real time in an unknown environment and build an environmental map at the same time. The system will perform coordinate system unification and precise alignment between the pre-created high-precision BIM model and the real environment map generated by SLAM technology. This process ensures that a column in the virtual model can be accurately superimposed on its corresponding position in the real world. The static BIM model is connected to the backend database through an interface. Real-time data collected by IoT sensors, status data updated by the project progress reporting module, and problem data reported by the quality and safety positioning and information uploading module are continuously "injected" into the corresponding components of the model. For example, a pile foundation component not only has its geometric dimensions and material information, but is also associated with its concrete strength sensor readings and the timestamp of completion of pouring. By combining with the real engineering environment information obtained by MR equipment, the BIM digital twin model can evolve synchronously with the real building, not only presenting the geometric shape of the building, but also containing rich data information, becoming a dynamic digital model that supports simulation and decision-making, providing data and model support for subsequent virtual-real mapping, interaction and decision-making.

[0029] Step Two: The Virtual-Real Mapping Module achieves a close connection between reality and virtuality through multiple sub-modules. The Project Progress Display Module uses color-coding to show unconstructed areas in gray, areas under construction in yellow, and completed areas in green, dynamically displaying the overall construction progress and allowing managers to intuitively understand the project's progress. The Project Progress Reporting Module allows on-site personnel to directly manipulate the virtual model within the MR environment, such as modifying construction progress information. The system will update this operation to the actual project progress data, achieving synchronization between reality and virtuality in terms of project progress. The Quality and Safety Positioning and Information Upload Module includes an MR camera and microphone. When quality defects or safety hazards are found on-site, personnel can directly "click" on the problematic area in the digital twin model using gestures. The MR camera and microphone capture images, videos, and voice descriptions of the problem site, achieving centimeter-level spatial positioning accuracy and accurately recording the problem information. All reported quality and safety issues are marked with prominent virtual labels in the digital twin model. The Quality and Safety Display Module monitors quality issue records in the database; when a new record is generated, it instantiates a prominent virtual label at the corresponding spatial coordinates. When an administrator gazes at the tag, the system triggers a query that renders the associated photo and voice details in the MR field of view.

[0030] Step 3: The natural interaction module provides users with a convenient human-computer interaction method. The voice reporting module can automatically identify key information in the user's voice and convert it into structured data, which is then automatically updated to the background database for convenient information storage and management. The gesture interaction mechanism module includes a depth camera, an inertial measurement unit, and a radar system. These devices work together to capture the skeletal joint data, motion trajectory, and speed of the user's hand. The gesture interaction mechanism module will capture key features of the gesture, such as finger opening and closing, palm orientation, and motion trajectory, and input the extracted feature vectors into the gesture interaction library for comparison. When the current gesture feature highly matches a template in the library, the gesture interaction mechanism module will output a corresponding standardized command, enabling the user to perform spatial pointing and operation on the system through gestures, such as selecting and moving virtual objects.

[0031] Step Four: The intelligent decision-making module relies on the multi-source data integration module and the intelligent analysis and decision support module to achieve scientific decision-making. The multi-source data integration module is electrically connected to the digital twin construction module and the quality and safety positioning and information uploading module. It can access and integrate various information from the on-site project, including geometric and attribute information of the BIM model obtained from the digital twin construction module; problem reports and multimedia data obtained from the quality and safety positioning and information uploading module; and IoT sensor data, schedule data, and weather data obtained from external systems. Then, it cleans, converts, and aligns the format of this data to form a unified and standardized dataset. The intelligent analysis and decision support module uses machine learning and rule reasoning on the integrated data to uncover potential patterns and correlations in the data, providing intelligent analysis and decision support for project managers, such as predicting project risks and optimizing construction plans, helping managers make more scientific and reasonable decisions.

[0032] Beneficial effects: The technical solution of this application has the following technical effects: This invention has the advantages of integrating perception, interaction, analysis and decision-making.

[0033] This invention achieves a cognitive leap from "screen 3D" to "spatial 3D". Through MR equipment and SLAM technology, high-precision BIM digital twin models are spatially registered and superimposed with real engineering environments with centimeter-level precision. This completely breaks down the visual barrier between digital models and physical entities. Managers can directly see through the building structure and compare the future design state with the current construction state in the same field of vision, which greatly improves the intuitiveness and accuracy of spatial cognition and reduces the over-reliance on personal experience.

[0034] This invention constructs a closed-loop management system for automatic data flow, which greatly improves management efficiency and changes the traditional manual and delayed data reporting mode. Through voice reporting and gesture interaction, on-site personnel can complete data entry work that used to take several minutes or even tens of minutes in seconds. Progress, quality and safety information is updated in minutes or even seconds from discovery to entry into the digital model. The analysis results of the intelligent decision-making module guide on-site operations in reverse, forming an intelligent closed loop of "perception-analysis-decision-execution", which fundamentally solves the problems of data gaps and delays.

[0035] This invention provides an ultimate natural interactive experience, significantly reducing the technical barrier to entry. Through a predefined gesture interaction library and an optimized voice recognition engine, the system transforms complex software operations into gesture pointing, grasping, and voice commands that conform to human instincts. Staff can start operating the system without complicated training, which powerfully promotes the application of high technology in traditional industries.

[0036] This invention breaks down "information silos" and achieves deep integration and intelligent insight of all-element data. Through a multi-source data integration module, the system cleans, aligns, and integrates the originally isolated BIM geometric data, IoT sensor data, progress reporting data, and quality and safety data. This enables the intelligent analysis engine to perform machine learning and rule reasoning on a unified and standardized dataset, and to proactively discover patterns and risks that are difficult for the human eye to detect from massive amounts of data.

[0037] This invention enables traceable and quantifiable precise management of quality and safety issues. Through the quality and safety positioning and information uploading module, any problem can be accurately anchored on three-dimensional spatial coordinates and associated with multimedia evidence to form a complete "problem data package". This ensures that every problem can be tracked, processed and closed-loop, greatly improving the quality and safety management level of the project.

[0038] This invention improves the efficiency and immersiveness of cross-regional collaborative work. Project participants located in different places can share the same MR session to conduct real-time discussions and decisions around the same virtual model, breaking the limitations of geographical space and constructing a brand-new engineering collaboration paradigm. Attached Figure Description

[0039] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of the system modules of the present invention; Figure 2 This is a hierarchical diagram of the system architecture of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. To better understand the technical content of the present invention, specific embodiments are provided and described in conjunction with the accompanying drawings. Various aspects of the present invention are described in this disclosure with reference to the accompanying drawings, which illustrate numerous illustrative embodiments. It should be understood that the various concepts and embodiments described above, as well as those described in more detail below, can be implemented in any of many ways. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0041] As attached Figure 1 To be continued Figure 2 As shown: This embodiment provides an engineering management system based on digital twin and mixed reality technologies, including: a digital twin construction module, a virtual-real mapping module, a natural interaction module, and an intelligent decision-making module.

[0042] The digital twin building block includes MR devices and SLAM technology, as well as a BIM digital twin model. MR devices and SLAM technology are used for rapid scanning and spatial positioning of real-world engineering environments.

[0043] A BIM digital twin model is a dynamic digital model that uses a BIM model as its geometric and data foundation and brings it to life through the Internet of Things and big data technologies. It can evolve synchronously with the real building and support simulation and decision-making.

[0044] The virtual-real mapping module includes a project progress display module, a project progress reporting module, a quality and safety positioning and information uploading module, and a quality and safety display module.

[0045] The project progress display module uses the currently projected digital twin image to dynamically display the overall construction progress of the project.

[0046] The project progress reporting module allows on-site personnel to directly manipulate the virtual model within the MR environment, thereby updating the actual project progress data.

[0047] The quality and safety positioning and information uploading module allows personnel to directly "click" on the problematic area in the digital twin model using gestures when quality defects or safety hazards are discovered on-site, achieving spatial positioning with centimeter-level accuracy.

[0048] In the quality and safety display module, all reported quality and safety issues are marked with prominent virtual labels in the digital twin model.

[0049] The natural interaction module includes a voice reporting module, a gesture interaction mechanism module, and a gesture interaction library; The voice reporting module automatically identifies key information, converts it into structured data, and automatically updates it to the backend database.

[0050] Gesture interaction mechanism module, used for spatial pointing and operation.

[0051] Gesture interaction library, a collection of predefined gesture commands.

[0052] The intelligent decision-making module includes a multi-source data integration module and an intelligent analysis and decision support module.

[0053] The multi-source data integration module can access and integrate various information from on-site engineering projects.

[0054] The intelligent analysis and decision support module performs machine learning and rule-based reasoning on the fused data.

[0055] Specifically, MR devices have a built-in depth camera and inertial measurement unit.

[0056] Specifically, the project progress display module uses color-coded status: unconstructed areas are displayed in gray, areas under construction are displayed in yellow, and completed areas are displayed in green.

[0057] Specifically, the quality and safety positioning and information uploading module includes an MR camera and a microphone, which are used to collect images, videos, and descriptions of the childcare at the problem site.

[0058] Specifically, the gesture interaction mechanism module inputs gesture information into the gesture interaction library, analyzes the predefined gesture commands in the library, and then transmits the correct commands.

[0059] Specifically, the gesture interaction mechanism module includes a depth camera, an inertial measurement unit, and a radar system.

[0060] Specifically, the multi-source data integration module is electrically connected to the digital twin construction module, the quality and safety positioning and information uploading module.

[0061] This invention discloses a method for using an engineering management system based on digital twin and mixed reality technologies, characterized in that: Step 1: The digital twin building module is the foundation of the entire system. Among them, the MR device plays a key role. After the MR device is started, its built-in depth camera and inertial measurement unit will start working. The depth camera will acquire depth point cloud data of the surrounding environment by emitting and receiving infrared light, and quickly build a three-dimensional geometric space of the scene. The IMU will continuously track the device's own degrees of freedom of movement through accelerometers and gyroscopes. SLAM technology enables the MR device to fuse visual data with IMU data and run simultaneous localization and mapping algorithms. This allows the device to calculate its own position and attitude in real time in an unknown environment and build an environmental map at the same time. The system will perform coordinate system unification and precise alignment between the pre-created high-precision BIM model and the real environment map generated by SLAM technology. This process ensures that a column in the virtual model can be accurately superimposed on its corresponding position in the real world. The static BIM model is connected to the backend database through an interface. Real-time data collected by IoT sensors, status data updated by the project progress reporting module, and problem data reported by the quality and safety positioning and information uploading module are continuously "injected" into the corresponding components of the model. For example, a pile foundation component not only has its geometric dimensions and material information, but is also associated with its concrete strength sensor readings and the timestamp of completion of pouring. By combining with the real engineering environment information obtained by MR equipment, the BIM digital twin model can evolve synchronously with the real building, not only presenting the geometric shape of the building, but also containing rich data information, becoming a dynamic digital model that supports simulation and decision-making, providing data and model support for subsequent virtual-real mapping, interaction and decision-making.

[0062] Step Two: The Virtual-Real Mapping Module achieves a close connection between reality and virtuality through multiple sub-modules. The Project Progress Display Module uses color-coding to show unconstructed areas in gray, areas under construction in yellow, and completed areas in green, dynamically displaying the overall construction progress and allowing managers to intuitively understand the project's progress. The Project Progress Reporting Module allows on-site personnel to directly manipulate the virtual model within the MR environment, such as modifying construction progress information. The system will update this operation to the actual project progress data, achieving synchronization between reality and virtuality in terms of project progress. The Quality and Safety Positioning and Information Upload Module includes an MR camera and microphone. When quality defects or safety hazards are found on-site, personnel can directly "click" on the problematic area in the digital twin model using gestures. The MR camera and microphone capture images, videos, and voice descriptions of the problem site, achieving centimeter-level spatial positioning accuracy and accurately recording the problem information. All reported quality and safety issues are marked with prominent virtual labels in the digital twin model. The Quality and Safety Display Module monitors quality issue records in the database; when a new record is generated, it instantiates a prominent virtual label at the corresponding spatial coordinates. When an administrator gazes at the tag, the system triggers a query that renders the associated photo and voice details in the MR field of view.

[0063] Step 3: The natural interaction module provides users with a convenient human-computer interaction method. The voice reporting module can automatically identify key information in the user's voice and convert it into structured data, which is then automatically updated to the background database for convenient information storage and management. The gesture interaction mechanism module includes a depth camera, an inertial measurement unit, and a radar system. These devices work together to capture the skeletal joint data, motion trajectory, and speed of the user's hand. The gesture interaction mechanism module will capture key features of the gesture, such as finger opening and closing, palm orientation, and motion trajectory, and input the extracted feature vectors into the gesture interaction library for comparison. When the current gesture feature highly matches a template in the library, the gesture interaction mechanism module will output a corresponding standardized command, enabling the user to perform spatial pointing and operation on the system through gestures, such as selecting and moving virtual objects.

[0064] Step Four: The intelligent decision-making module relies on the multi-source data integration module and the intelligent analysis and decision support module to achieve scientific decision-making. The multi-source data integration module is electrically connected to the digital twin construction module and the quality and safety positioning and information uploading module. It can access and integrate various information from the on-site project, including geometric and attribute information of the BIM model obtained from the digital twin construction module; problem reports and multimedia data obtained from the quality and safety positioning and information uploading module; and IoT sensor data, schedule data, and weather data obtained from external systems. Then, it cleans, converts, and aligns the format of this data to form a unified and standardized dataset. The intelligent analysis and decision support module uses machine learning and rule reasoning on the integrated data to uncover potential patterns and correlations in the data, providing intelligent analysis and decision support for project managers, such as predicting project risks and optimizing construction plans, helping managers make more scientific and reasonable decisions.

[0065] It should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0066] While the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. An engineering management system based on digital twin and mixed reality technologies, characterized in that: include: Digital twin construction module, virtual-real mapping module, natural interaction module, intelligent decision-making module; The digital twin building module includes MR devices and SLAM technology, and BIM digital twin models; MR equipment and SLAM technology are used for rapid scanning and spatial positioning of real-world engineering environments; A BIM digital twin model is a dynamic digital model that uses a BIM model as its geometric and data foundation and brings it to life through the Internet of Things and big data technologies. It can evolve synchronously with the real building and support simulation and decision-making. The virtual-real mapping module includes a project progress display module, a project progress reporting module, a quality and safety positioning and information uploading module, and a quality and safety display module. The project progress display module uses the currently projected digital twin image to dynamically display the overall construction progress of the current project. The project progress reporting module allows on-site personnel to directly manipulate the virtual model in the MR environment, thereby updating the actual project progress data. The quality and safety positioning and information uploading module allows personnel to directly "click" on the problematic area in the digital twin model using gestures when quality defects or safety hazards are discovered on-site, achieving spatial positioning with centimeter-level accuracy. In the quality and safety display module, all reported quality and safety issues are marked with prominent virtual labels in the digital twin model; The natural interaction module includes a voice reporting module, a gesture interaction mechanism module, and a gesture interaction library; The voice reporting module automatically identifies key information, converts it into structured data, and automatically updates it to the backend database. Gesture interaction mechanism module, used for spatial pointing and operation; Gesture interaction library, a collection of predefined gesture commands; The intelligent decision-making module includes a multi-source data integration module and an intelligent analysis and decision support module; The multi-source data integration module can access and integrate various information from on-site engineering projects; The intelligent analysis and decision support module performs machine learning and rule-based reasoning on the fused data.

2. The engineering management system based on digital twin and mixed reality technologies according to claim 1, characterized in that: MR devices have a built-in depth camera and inertial measurement unit.

3. The engineering management system based on digital twin and mixed reality technologies according to claim 1, characterized in that: The project progress display module uses color-coded status: unconstructed areas are displayed in gray, areas under construction are displayed in yellow, and completed areas are displayed in green.

4. The engineering management system based on digital twin and mixed reality technologies according to claim 1, characterized in that: The quality and safety positioning and information uploading module includes an MR camera and a microphone, which are used to collect images, videos, and descriptions of the problem scene.

5. The engineering management system based on digital twin and mixed reality technologies according to claim 1, characterized in that: The gesture interaction mechanism module inputs gesture information into the gesture interaction library, analyzes the predefined gesture commands in the gesture interaction library, and then transmits the correct commands.

6. The engineering management system based on digital twin and mixed reality technology according to claim 1, characterized in that: The gesture interaction mechanism module includes a depth camera, an inertial measurement unit, and a radar system.

7. An engineering management system based on digital twin and mixed reality technologies according to claim 1, characterized in that: The multi-source data integration module is electrically connected to the digital twin construction module, the quality and safety positioning and information uploading module.

8. A method for using an engineering management system based on digital twin and mixed reality technologies, characterized in that: Step 1: The digital twin building module is the foundation of the entire system. Among them, the MR device plays a key role. After the MR device is started, its built-in depth camera and inertial measurement unit will start working. The depth camera will acquire depth point cloud data of the surrounding environment by emitting and receiving infrared light, and quickly build a three-dimensional geometric space of the scene. The IMU will continuously track the device's own degrees of freedom of movement through accelerometers and gyroscopes. SLAM technology enables the MR device to fuse visual data with IMU data and run simultaneous localization and mapping algorithms. This allows the device to calculate its own position and attitude in real time in an unknown environment and build an environmental map at the same time. The system will perform coordinate system unification and precise alignment between the pre-created high-precision BIM model and the real environment map generated by SLAM technology. This process ensures that a column in the virtual model can be accurately superimposed on its corresponding position in the real world. The static BIM model is connected to the backend database through an interface. Real-time data collected by IoT sensors, status data updated by the project progress reporting module, and problem data reported by the quality and safety positioning and information uploading module are continuously "injected" into the corresponding components of the model. For example, a pile foundation component not only has its geometric dimensions and material information, but is also associated with its concrete strength sensor readings and the timestamp of completion of pouring. By combining with the real engineering environment information obtained by MR equipment, the BIM digital twin model can evolve synchronously with the real building, not only presenting the geometric shape of the building, but also containing rich data information, becoming a dynamic digital model that supports simulation and decision-making, providing data and model support for subsequent virtual-real mapping, interaction and decision-making. Step Two: The Virtual-Real Mapping Module achieves a close connection between reality and virtuality through multiple sub-modules. The Project Progress Display Module uses color-coding to show unconstructed areas in gray, areas under construction in yellow, and completed areas in green, dynamically displaying the overall construction progress and allowing managers to intuitively understand the project's progress. The Project Progress Reporting Module allows on-site personnel to directly manipulate the virtual model within the MR environment, such as modifying construction progress information. The system will update this operation to the actual project progress data, achieving synchronization between reality and virtuality in terms of project progress. The Quality and Safety Positioning and Information Upload Module includes an MR camera and microphone. When quality defects or safety hazards are discovered on-site, personnel can directly "click" on the problematic area in the digital twin model using gestures. The MR camera and microphone capture images, videos, and voice descriptions of the problem site, achieving centimeter-level spatial positioning accuracy and accurately recording the problem information. All reported quality and safety issues are marked with prominent virtual labels in the digital twin model. The Quality and Safety Display Module monitors quality issue records in the database; when a new record is generated, it instantiates a prominent virtual label at the corresponding spatial coordinates. When an administrator gazes at the tag, the system triggers a query, rendering the associated photo and voice details in the MR field of view; Step 3: The natural interaction module provides users with a convenient human-computer interaction method. The voice reporting module can automatically identify key information in the user's voice and convert it into structured data, automatically updating it to the background database for convenient information storage and management. The gesture interaction mechanism module includes a depth camera, an inertial measurement unit, and a radar system. These devices work together to capture the skeletal joint data, motion trajectory, and speed of the user's hand. The gesture interaction mechanism module will capture key features of the gesture, such as finger opening and closing, palm orientation, and motion trajectory, and input the extracted feature vector into the gesture interaction library for comparison. When the current gesture feature highly matches a template in the library, the gesture interaction mechanism module will output a corresponding standardized command, enabling the user to perform spatial pointing and operation on the system through gestures, such as selecting and moving virtual objects. Step Four: The intelligent decision-making module relies on the multi-source data integration module and the intelligent analysis and decision support module to achieve scientific decision-making. The multi-source data integration module is electrically connected to the digital twin construction module and the quality and safety positioning and information uploading module. It can access and integrate various information from the on-site project, including geometric and attribute information of the BIM model obtained from the digital twin construction module; problem reports and multimedia data obtained from the quality and safety positioning and information uploading module; and IoT sensor data, schedule data, and weather data obtained from external systems. Then, it cleans, converts, and aligns the format of this data to form a unified and standardized dataset. The intelligent analysis and decision support module uses machine learning and rule reasoning on the integrated data to uncover potential patterns and correlations in the data, providing intelligent analysis and decision support for project managers. For example, it can predict project risks, optimize construction plans, and help managers make more scientific and reasonable decisions.

Images