Building robot multi-machine collaborative operation optimizing and monitoring method and system based on digital twinning

Abstract

The invention discloses a building robot multi-machine collaborative operation optimizing and monitoring method and system based on digital twinning, and belongs to the technical field of building construction intellectualization. The method comprises the following steps: constructing a digital twinborn body comprising an environment perception model and a kinematics / dynamics model based on a building information model and robot physical parameters; performing multi-machine task allocation and path planning containing static / dynamic obstacle avoidance in the virtual environment; synchronizing physical environment data in real time through a multi-mode sensor; dynamically optimizing operation parameters by adopting a genetic algorithm or a particle swarm algorithm and realizing closed-loop control; and generating a safety early warning and emergency scheme based on machine learning. According to the method, Markov decision path planning of reinforcement learning and multi-agent game task allocation are creatively fused, laser radar-vision-inertial navigation multi-source data fusion is adopted, the technical problems that in a traditional method, digital twinning precision is insufficient, and dynamic cooperation efficiency is low are solved, and the method is suitable for large-scale popularization and application. And the construction efficiency, the safety and the man-machine interaction experience are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent building construction technology, specifically to a method and system for optimizing and monitoring multi-robot collaborative operations of building robots based on digital twins. Background Technology

[0002] The statements in this section are provided only as background information in relation to this disclosure and may not constitute prior art.

[0003] With the rapid development of the construction industry, multi-robot collaborative operation and real-time monitoring technologies in construction have gradually become important means to improve construction efficiency, accuracy, and safety. However, existing technical solutions still have many shortcomings in areas such as digital twin model construction, collaborative optimization capabilities, real-time monitoring mechanisms, and human-computer interaction experience, making it difficult to meet the needs of complex construction environments. The main problems are as follows: 1. Lack of high-precision digital twin models Existing technologies typically employ simple simulation models or static environment models to model the operational state of construction robots and the construction environment. These models cannot reflect the dynamic changes of the physical robot and the construction environment in real time, resulting in significant discrepancies between the virtual and physical environments. Specifically, they cannot update dynamic changes at the construction site in real time (such as equipment movement, personnel activity, and material stacking), leading to inaccurate path planning and task allocation. The lack of high-precision modeling of robot kinematics, dynamics, and environmental perception makes it difficult to support complex collaborative operations and optimization analyses; existing models largely rely on offline data and cannot synchronize the state information of the physical robot and the environment in real time through sensor networks. These problems limit the application effectiveness of digital twin technology in the field of construction robots, making it difficult to achieve accurate virtual mapping and real-time optimization.

[0004] 2. Insufficient collaborative optimization capability Multi-robot collaborative task allocation and path planning are core issues in collaborative construction robot operations. Existing technologies mostly employ static algorithms for task allocation and path planning, which struggle to cope with dynamic changes in complex construction environments. Specifically, they cannot dynamically adjust task allocation strategies based on real-time task priorities, robot states, and environmental changes, leading to low resource utilization efficiency; they cannot avoid dynamic obstacles (such as equipment movement and personnel activity) in real time, resulting in path conflicts and construction delays. Existing algorithms often focus on local optimization, making it difficult to achieve global collaboration and efficient scheduling of multi-robot systems. These problems result in low efficiency in multi-robot collaborative operations, failing to meet the demands of complex construction projects.

[0005] 3. Inadequate real-time monitoring and feedback mechanisms Real-time monitoring and feedback are crucial for ensuring the safe and efficient operation of construction robots. Existing monitoring systems often rely on single sensor data (such as position and speed sensors), lacking multi-source data fusion and real-time analysis capabilities, making accurate status monitoring and fault prediction difficult. Specifically, they fail to comprehensively reflect the robot's operating status and environmental information, resulting in insufficient monitoring accuracy; existing systems mostly use offline data analysis, unable to detect abnormal states (such as collision risks or equipment malfunctions) in real time; and they lack efficient feedback mechanisms, making it difficult to adjust robot operating parameters and task allocation strategies promptly. These problems lead to high safety risks during construction and hinder accurate real-time monitoring and fault prevention.

[0006] 4. Poor human-computer interaction experience Human-computer interaction is a crucial component of construction robot systems. Existing systems primarily utilize two-dimensional interfaces for status display and operational control, lacking immersive visualization capabilities. This makes it difficult for operators to intuitively understand the construction status and robot operation. Specifically, the inability to visually display the three-dimensional construction environment and robot operating status hinders operators' ability to quickly locate problems and make decisions. Existing systems largely rely on keyboard and mouse operation, lacking natural interaction methods (such as gesture control and voice control). Furthermore, they cannot provide an immersive monitoring experience through virtual reality (VR) or augmented reality (AR) technologies, reducing operational efficiency and user experience. These issues limit the effectiveness of human-computer interaction and fail to meet the demands of complex construction environments.

[0007] In summary, existing technologies for multi-robot collaborative operation and monitoring in construction have significant shortcomings in areas such as high-precision digital twin model construction, dynamic collaborative optimization, real-time monitoring and feedback mechanisms, and human-machine interaction. These problems lead to low construction efficiency, insufficient accuracy, poor safety, and a subpar user experience, making it difficult to meet the high-efficiency, precise, and safe demands of modern construction. Therefore, with the rapid development of digital twins, the Internet of Things, artificial intelligence, and 5G communication technologies, multi-robot collaborative operation and monitoring technologies for construction are evolving towards higher precision, dynamism, intelligence, and immersive experiences. Summary of the Invention

[0008] The purpose of this invention is to provide a method and system for optimizing and monitoring multi-robot collaborative operations of construction robots based on digital twins, thereby solving the problems existing in the prior art.

[0009] The technical solution of the present invention is as follows: A method for optimizing and monitoring multi-robot collaborative operations in construction using digital twins includes: Step S1: Construct a digital twin based on the building information model and the physical parameters of the building robot; the digital twin includes an environmental perception model and the geometric model, kinematic model, and dynamic model of the building robot; Step S2: Perform multi-machine collaborative task allocation and path planning in the virtual environment constructed by the digital twin; the path planning includes static path planning and dynamic obstacle avoidance path planning based on real-time sensor data; Step S3: Collect status and environmental data in real time through the sensor modules on the construction robot and update them synchronously to the digital twin; Step S4: Simulate multi-machine collaborative operation based on digital twin, dynamically adjust robot operation parameters through optimization algorithm and feed them back to the construction robot; Step S5: Analyze the digital twin data using machine learning algorithms to trigger security alerts and generate contingency plans that include task reassignment and path replanning.

[0010] Furthermore, the sensor module includes a lidar, a vision sensor, a force sensor, and an inertial measurement unit, enabling real-time data synchronization between the construction robot and its digital twin via the Internet of Things.

[0011] Furthermore, the construction robot includes: The mobile chassis module supports one or more combinations of wheeled, tracked, and legged mobility methods. The robotic arm module adopts a multi-degree-of-freedom biomimetic structure or a multi-axis linkage robotic arm; The end effector module allows for quick replacement of actuators according to construction tasks, including: bricklaying tools, rebar tying tools, concrete spraying tools, roughening equipment, and squeegee equipment.

[0012] Furthermore, the path planning is modeled as a Markov decision process using a reinforcement learning algorithm, and the reward function guides the robot to select the shortest path and avoid obstacles.

[0013] Furthermore, the task allocation adopts a multi-agent collaborative optimization algorithm, which models the task allocation as a distributed optimization problem and implements task allocation through game theory or auction algorithms.

[0014] Furthermore, in step S4, a genetic algorithm or a particle swarm optimization algorithm is used to dynamically adjust the robot's operating parameters; In this context, the genetic algorithm encodes the task parameters into chromosomes for iterative optimization; Particle swarm optimization (PSO) represents task parameters as particle positions for swarm intelligence optimization.

[0015] Furthermore, the security early warning uses an isolated forest or support vector machine algorithm for anomaly detection. When an anomaly is detected, an early warning mechanism is triggered and an emergency plan is generated.

[0016] Furthermore, it also includes building a visual human-computer interaction interface through VR / AR technology, displaying the status information of the digital twin in real time, and supporting remote control command input.

[0017] Furthermore, the visual human-computer interaction interface supports gesture control and voice control interaction methods, and has the functions of three-dimensional visualization of construction progress and abnormal status location.

[0018] This invention also proposes a multi-robot collaborative operation optimization and monitoring system for construction robots based on digital twins, comprising: Digital twin building blocks are used to build and update digital twins for construction robots; The multi-machine collaborative optimization module is used for task allocation, path planning, and collaborative operation optimization. The real-time monitoring module is used for data acquisition, status monitoring, and security alerts. The human-computer interaction module is used for visualization and remote control.

[0019] Compared with existing technologies, the advantages of this invention are: 1. Improve construction efficiency: Dynamic task allocation and path planning: Through digital twins and optimization algorithms, efficient scheduling of multi-robot collaborative operations is achieved, reducing task conflicts and resource waste; Real-time optimization and feedback: Dynamically adjust robot operation parameters based on real-time data to improve construction efficiency.

[0020] 2. Improve construction precision: A high-fidelity virtual environment is constructed using data detected by building information models and sensor modules to ensure the accuracy of robot operations; Multi-source data fusion, combining data from lidar, visual sensors, and other sources, enables precise positioning and environmental perception.

[0021] 3. Enhance construction safety: Real-time monitoring and safety early warning: Through digital twins and machine learning algorithms, the status of construction robots and environmental changes are monitored in real time, and potential risks are warned in advance; Fault prediction and emergency response: Based on historical and real-time data, faults are predicted and emergency plans are generated to reduce the accident rate.

[0022] 4. Reduce construction costs: Predictive maintenance: By monitoring equipment status in real time through digital twins, faults can be predicted and maintenance can be carried out in advance, reducing equipment downtime and maintenance costs; Resource optimization: Resources are allocated rationally through collaborative optimization algorithms to reduce material waste and labor costs.

[0023] 5. Improve human-computer interaction experience: Immersive visual interface: Provides an intuitive and immersive monitoring experience through VR / AR technology, reducing the difficulty of operation; Remote monitoring and control: Supports remote access and control, facilitating real-time intervention and adjustment by operators. Attached Figure Description

[0024] Figure 1 A block diagram illustrating the optimization and monitoring method and system principle of multi-robot collaborative operation of construction robots based on digital twins. Detailed Implementation

[0025] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0026] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0027] Example 1 Please see Figure 1 A method for optimizing and monitoring multi-robot collaborative operations in construction based on digital twins, including: Step S1: Construct a digital twin of the construction robot; A digital twin is constructed based on Building Information Modeling (BIM) and the physical parameters of the building robot; the digital twin includes an environmental perception model and the geometric, kinematic, and dynamic models of the building robot. Step S2: Multi-machine collaborative task allocation and path planning; Multi-machine collaborative task allocation and path planning are performed in a virtual environment constructed by a digital twin; the path planning includes static path planning and dynamic obstacle avoidance path planning based on real-time sensor data; Step S3: Real-time data acquisition and status monitoring; The sensor modules mounted on the construction robot collect status and environmental data in real time and update them synchronously to the digital twin. Step S4: Optimization of collaborative operations based on digital twins; Multi-robot collaborative operation simulation is performed based on digital twins, and the robot operation parameters are dynamically adjusted through optimization algorithms and fed back to the construction robot. Step S5: Safety warning and fault diagnosis; By using machine learning algorithms to analyze digital twin data, security alerts can be triggered and emergency plans including task reassignment and path replanning can be generated.

[0028] In this embodiment, it should be noted that the sensor module of the construction robot includes LiDAR, vision sensor, force sensor and inertial measurement unit, etc., and the real-time data synchronization between the construction robot and the digital twin is realized through the Internet of Things. This involves connecting construction robots and their digital twins in real time via the Internet of Things (IoT) to achieve data synchronization.

[0029] In this embodiment, it should be noted that step S2 specifically includes: assigning tasks and planning paths for multiple construction robots based on a virtual environment constructed using a digital twin; Path planning includes static path planning and dynamic obstacle avoidance path planning. Dynamic obstacle avoidance path planning updates the environmental perception model in real time using data detected by sensor modules. Task allocation is dynamically adjusted based on the capabilities of the construction robot, task priority, and construction progress.

[0030] In this embodiment, it should be noted that the virtual environment is used to simulate a construction site and the operating state of a construction robot in the physical world, and specifically includes the following construction steps: Building structural information and construction task information are extracted based on building information modeling. Geometric, kinematic, and dynamic models are constructed based on the physical parameters of construction robots. The environmental perception model is updated in real time using the detection data from the sensor module to construct a virtual environment.

[0031] In this embodiment, specifically, the path planning is modeled as a Markov decision process using a reinforcement learning algorithm, and the reward function guides the robot to select the shortest path and avoid obstacles. Specifically, reinforcement learning algorithms are machine learning methods that learn optimal policies through the interaction between an agent and its environment. When applied to task allocation, the task allocation problem is modeled as a Markov decision process, where each robot selects a task based on its current state, and a reward function guides the robot to choose an efficient and low-conflict task allocation strategy. When applied to path planning, the path planning problem is modeled as a sequential decision problem, where the robot selects a direction of movement based on its current position and target position, and a reward function guides the robot to choose the shortest path and avoid obstacles.

[0032] In this embodiment, specifically, the task allocation adopts a multi-agent cooperative optimization algorithm, which models the task allocation as a distributed optimization problem and implements task allocation through game theory or auction algorithm; Specifically, multi-agent cooperative optimization algorithms achieve a globally optimal solution through cooperation and competition among agents. When applied to task allocation, the task allocation problem is modeled as a distributed optimization problem. Each construction robot, as an agent, selects a task based on its own capabilities and task requirements. Game theory or auction algorithms are used to allocate tasks, ensuring fairness and efficiency. When applied to path planning, the path planning problem is modeled as a multi-agent path planning problem. Each construction robot, as an agent, plans its own path and avoids conflicts. Conflict search algorithms or distributed optimization algorithms are used to resolve path conflict problems.

[0033] In this embodiment, step S3 specifically includes: The sensor modules of the construction machine collect real-time status and environmental data of the construction robot. The collected data is transmitted to the digital twin to update the virtual environment and the status of the construction robot; Real-time monitoring of the position, posture, operating status, and construction progress of construction robots based on digital twins.

[0034] In this embodiment, step S4 specifically includes: Using digital twins to simulate multi-machine collaborative operations, predict potential conflicts and efficiency bottlenecks; By dynamically adjusting robot operation parameters (such as speed, acceleration, and operation sequence) through optimized algorithms, the efficiency of collaborative operations can be improved. The optimization results (adjusted operating parameters of the construction robot) are fed back to the physical robot to achieve closed-loop control.

[0035] In this embodiment, specifically, in step S4, a genetic algorithm or a particle swarm optimization algorithm is used to dynamically adjust the robot's operating parameters; In this context, the genetic algorithm encodes the task parameters into chromosomes for iterative optimization; Particle swarm optimization (PSO) represents task parameters as particle positions for swarm intelligence optimization.

[0036] In this embodiment, the genetic algorithm specifically includes the following application steps: (1) Encode the robot's operation parameters (velocity, acceleration, operation sequence) into chromosomes and randomly generate a set of initial solutions; (2) Design a fitness function to simulate the selection, crossover, and mutation process of the construction robot's operation parameters encoded as chromosomes; (3) Repeat the selection, crossover and mutation process until the maximum number of iterations or fitness value converges.

[0037] In this embodiment, the particle swarm optimization algorithm specifically includes the following application steps: (1) Represent the robot's operation parameters (velocity, acceleration, operation sequence) as the position of the particles, and randomly initialize the position and velocity of the particle swarm; (2) Design a fitness function to change the speed and position of the construction robot operation parameters, which are represented as particles; (3) Repeat the velocity and position update process until the maximum number of iterations or fitness value converges.

[0038] In this embodiment, step S5 specifically includes: Based on real-time data from the digital twin, machine learning algorithms are used to analyze the robot's operating status and predict potential failures. When an abnormal state is detected (such as collision risk, equipment overheating, or path deviation), a safety warning mechanism is triggered, and an emergency plan is generated through a digital twin. The contingency plan includes task reallocation, route replanning, and robot shutdown protection.

[0039] Specifically, a fault prediction model can be trained based on the historical and real-time data of the digital twin, and fault prediction can be performed through the fault prediction model. Simultaneously, anomaly detection algorithms (isolated forest or support vector machine) are used to identify robot malfunctions; When an anomaly is detected, an early warning mechanism is triggered and an emergency response plan is generated.

[0040] In this embodiment, specifically, the construction robot includes: The mobile chassis module supports one or more combinations of wheeled, tracked, and footed mobility methods to adapt to different construction environments. When using multiple combinations, the corresponding mobility method is deployed according to the construction environment for stable movement. The robotic arm module adopts a multi-degree-of-freedom biomimetic structure or a multi-axis linkage robotic arm, which has the characteristics of multiple degrees of freedom and high flexibility. The end effector module allows for quick replacement of actuators according to construction tasks, including: bricklaying tools, rebar tying tools, concrete spraying tools, roughening equipment, and squeegee equipment.

[0041] In this embodiment, specifically, it also includes constructing a visual human-computer interaction interface using VR / AR technology to display the status information of the digital twin in real time and support remote control command input.

[0042] In this embodiment, specifically, the visual human-computer interaction interface supports gesture control and voice control interaction methods, and has the functions of three-dimensional visualization of construction progress and abnormal status location. It provides a visual human-computer interaction interface that displays real-time status information, construction progress, and optimization results of the digital twin; the human-computer interaction interface can support remote control and task adjustment, and provides virtual reality and augmented reality visualization functions.

[0043] This embodiment also proposes a multi-robot collaborative operation optimization and monitoring system for construction robots based on digital twins, including: Digital twin building blocks are used to build and update digital twins for construction robots; The multi-machine collaborative optimization module is used for task allocation, path planning, and collaborative operation optimization. The real-time monitoring module is used for data acquisition, status monitoring, and security alerts. The human-computer interaction module is used for visualization and remote control.

[0044] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

[0045] This background section is provided to generally present the context of the invention. The work of the currently named inventors, the work to the extent described in this background section, and aspects of this section that did not constitute prior art at the time of application are neither expressly nor impliedly acknowledged as prior art to the invention.

Claims

1. A method for optimizing and monitoring multi-robot collaborative operations in construction based on digital twins, characterized in that, include: Step S1: Construct a digital twin based on the building information model and the physical parameters of the building robot; the digital twin includes an environmental perception model and the geometric model, kinematic model, and dynamic model of the building robot; Step S2: Perform multi-machine collaborative task allocation and path planning in the virtual environment constructed by the digital twin; the path planning includes static path planning and dynamic obstacle avoidance path planning based on real-time sensor data; Step S3: Collect status and environmental data in real time through the sensor modules on the construction robot and update them synchronously to the digital twin; Step S4: Simulate multi-machine collaborative operation based on digital twin, dynamically adjust robot operation parameters through optimization algorithm and feed them back to the construction robot; Step S5: Analyze the digital twin data using machine learning algorithms to trigger security alerts and generate contingency plans that include task reassignment and path replanning.

2. The method for optimizing and monitoring multi-robot collaborative operation of construction robots based on digital twins according to claim 1, characterized in that, The sensor module includes a lidar, a vision sensor, a force sensor, and an inertial measurement unit, which enables real-time data synchronization between the construction robot and its digital twin via the Internet of Things.

3. The method for optimizing and monitoring multi-robot collaborative operation of construction robots based on digital twins according to claim 1, characterized in that, The construction robot includes: The mobile chassis module supports one or more combinations of wheeled, tracked, and legged mobility methods. The robotic arm module adopts a multi-degree-of-freedom biomimetic structure or a multi-axis linkage robotic arm; The end effector module allows for quick replacement of actuators according to construction tasks, including: bricklaying tools, rebar tying tools, concrete spraying tools, roughening equipment, and squeegee equipment.

4. The method for optimizing and monitoring multi-robot collaborative operation of construction robots based on digital twins according to claim 1, characterized in that, The path planning is modeled as a Markov decision process using a reinforcement learning algorithm, and the reward function guides the robot to choose the shortest path and avoid obstacles.

5. The method for optimizing and monitoring multi-robot collaborative operation of construction robots based on digital twins according to claim 1, characterized in that, The task allocation adopts a multi-agent collaborative optimization algorithm, which models the task allocation as a distributed optimization problem and implements task allocation through game theory or auction algorithm.

6. The method for optimizing and monitoring multi-robot collaborative operation of construction robots based on digital twins according to claim 1, characterized in that, In step S4, a genetic algorithm or particle swarm optimization algorithm is used to dynamically adjust the robot's operating parameters. In this context, the genetic algorithm encodes the task parameters into chromosomes for iterative optimization; Particle swarm optimization (PSO) represents task parameters as particle positions for swarm intelligence optimization.

7. The method for optimizing and monitoring multi-robot collaborative operation of construction robots based on digital twins according to claim 1, characterized in that, The security early warning uses an isolated forest or support vector machine algorithm for anomaly detection. When an anomaly is detected, the early warning mechanism is triggered and an emergency plan is generated.

8. The method for optimizing and monitoring multi-robot collaborative operation of construction robots based on digital twins according to claim 1, characterized in that, It also includes building a visual human-computer interaction interface through VR / AR technology, displaying the status information of the digital twin in real time, and supporting remote control command input.

9. The method for optimizing and monitoring multi-robot collaborative operation of construction robots based on digital twins according to claim 8, characterized in that, The visual human-computer interaction interface supports gesture control and voice control interaction methods, and has the functions of three-dimensional visualization of construction progress and abnormal status location.

10. A multi-robot collaborative operation optimization and monitoring system for construction robots based on digital twins, characterized in that: include: Digital twin building blocks are used to build and update digital twins for construction robots; The multi-machine collaborative optimization module is used for task allocation, path planning, and collaborative operation optimization. The real-time monitoring module is used for data acquisition, status monitoring, and security alerts. The human-computer interaction module is used for visualization and remote control.

Images