A construction robot virtual simulation teaching platform construction method

Abstract

This invention discloses a method for constructing a virtual simulation teaching platform for construction robots, relating to the field of construction robot teaching technology. This invention establishes an end-to-cloud collaborative bidirectional data interaction architecture, deploying cloud resources and local components while optimizing interaction performance. It constructs a five-layer functional module comprising a 3D visualization unit, a virtual simulation engine, a construction scene control and perception unit, a cross-platform communication unit, and a visual programming unit. The virtual simulation engine employs Newton-Euler dynamics algorithms for high-precision simulation, and the construction scene control and perception unit uses multiple algorithms to complete sensor simulation and scene modeling. A two-dimensional functional platform is constructed, covering the entire simulation cycle and a three-level progressive interaction. This invention allows users to select robot types and configure the simulation environment from the cloud, design control algorithms through visual programming, drive the robot to complete tasks, and observe the effects through 3D visualization, thus meeting the simulation, interaction, and algorithm development needs in construction robot teaching.

Description

Technical Field

[0001] This invention relates to the field of teaching technology for construction robots, and specifically to a method for constructing a virtual simulation teaching platform for construction robots. Background Technology

[0002] With the rapid development of industrialized and intelligent construction, construction robots, with their advantages of efficiency, safety, and precision, have been widely used in various scenarios such as pouring, welding, and hoisting in construction. However, the training of construction robot operators faces many practical difficulties: physical equipment is expensive, often costing hundreds of thousands or even millions of yuan, making it difficult for educational institutions to purchase on a large scale; real construction environments pose safety hazards such as falls from heights and mechanical injuries, making them unsuitable for repeated practice by beginners; in addition, the conditions on construction sites are varied, making it difficult for trainees to experience a sufficiently rich range of working scenarios within limited training time. Furthermore, traditional theoretical teaching is too abstract, making it difficult for students to develop an intuitive understanding of core concepts such as robot motion control, path planning, and human-robot collaboration. Therefore, virtual simulation teaching platforms have become a key vehicle for cultivating professional operators and R&D talents. Virtual simulation teaching platforms can avoid the pain points of high equipment costs, high operational risks, and limited scenario coverage in physical robot teaching. By constructing virtual construction environments and robot models, they can realize full-process teaching and R&D practice, including robot operation, algorithm debugging, and construction process simulation. Therefore, the research and development and patent layout of related technologies have attracted much attention.

[0003] Chinese patent (publication number CN212084426U) discloses a robot teaching and training platform. This patent achieves teaching functions through physical robot demonstrations. The core is to assist in robot teaching and training by demonstrating the robot's appearance and usage process through physical demonstrations. However, relying on physical robots for teaching demonstrations results in high equipment costs, making it difficult to widely adopt for teaching. Moreover, the level of concreteness of teaching knowledge is low, and it is impossible to intuitively demonstrate the robot's internal mechanical structure, motion principles, and the core logic of control algorithms. Students are easily distracted by physical operation and lose focus. In addition, the lack of targeted teaching interaction design makes it impossible to achieve progressive teaching from basic operation to algorithm development, and it cannot meet the learning needs of students at different levels.

[0004] Chinese patent (publication number CN121389274A) discloses a dynamic simulation method and system for unmanned tower cranes based on virtual simulation technology. This method constructs a BIM model and a point cloud data simulation model, develops a customized physical engine and a multibody dynamics model, and performs coupled simulation with a wind field model to achieve unmanned tower crane path planning and risk monitoring, aiming to solve the problems of scene distortion and rough simulation in virtual pre-simulation. However, this method does not focus on the targeted design of teaching scenarios, lacks a resource system and interaction mode adapted to teaching needs, and cannot achieve the efficient combination of centralized management of teaching resources and local lightweight simulation. It relies heavily on single terminal deployment, making it difficult to support simultaneous access and parallel simulation by multiple users. Furthermore, it does not integrate core resources required for teaching, such as building robot model libraries, control algorithm libraries, and teaching resource libraries, and cannot provide comprehensive resource support for teaching.

[0005] In summary, current technologies cannot meet the needs of high-efficiency and precise training of professionals in construction robots. Therefore, developing a construction robot virtual simulation teaching platform that can solve the above-mentioned deficiencies and has an edge-cloud collaborative architecture, high-precision simulation capabilities, progressive teaching interaction, and full life-cycle scenario coverage has become an urgent technical problem to be solved. Summary of the Invention

[0006] Based on the above-mentioned technical problems, this application discloses a method for constructing a virtual simulation teaching platform for construction robots, specifically including:

[0007] Establish a two-way data interaction architecture that integrates cloud and local collaboration. Deploy construction robot models, construction scene materials, control algorithms, and teaching resources in the cloud, and deploy simulation engines, data transmission plugins, and visual programming platforms on the local end.

[0008] Based on a two-way data interaction architecture, a five-layer functional module is constructed that is connected and works collaboratively from top to bottom, including a 3D visualization unit, a virtual simulation engine, a building scene control and perception unit, a cross-platform communication unit, and a visualization programming unit.

[0009] Based on five functional modules, a two-dimensional functional platform is constructed, including full life cycle simulation scenarios and three-level progressive interaction. The simulation scenarios cover the entire life cycle of digital exploration, intelligent production, intelligent construction, and intelligent operation and maintenance. The three-level progressive interaction includes keyboard interaction, algorithm interaction, and algorithm development.

[0010] Based on an edge-cloud collaborative architecture, a five-layer core module system, and a dual-dimensional functional system, users can select the appropriate type of construction robot from the cloud resource library; build and configure the simulation operation environment through the five-layer functional modules; and design control algorithms through the visual programming module to drive the robot to complete tasks and observe the operation effect through the three-dimensional visualization unit.

[0011] The preferred bidirectional data interaction architecture is as follows: based on the types of resources required for teaching construction robots, a construction robot model library, motion parameter model, construction scene material library, control algorithm library, and teaching resource library are classified and constructed.

[0012] Based on local terminal hardware configuration, a lightweight deployment of the simulation engine, data transfer plugin, and visual programming platform is implemented.

[0013] A dedicated data transmission plugin based on the TCP / IP protocol was developed, integrating data sending, data receiving, data parsing, and abnormal retransmission.

[0014] Simulate scenarios where multiple users simultaneously access cloud resources and perform parallel simulations on the local end to verify the architecture's concurrent processing capabilities. Based on the debugging results, optimize the response speed of edge-cloud data interaction and resource caching strategies to finalize the architecture.

[0015] Preferably, the five functional modules are: a 3D visualization unit based on motion data from a virtual simulation engine and environmental data from a building scene control and perception unit, which outputs a visual interactive interface;

[0016] The virtual simulation engine uses control algorithms from visual programming units and environmental parameters from building scene control perception units to output robot motion simulation results.

[0017] The building scene control and perception unit, based on construction scene materials collected by sensors and user-configured working condition parameters, outputs scene modeling data and sensor simulation data through a dual-path modeling algorithm of large model generation and digital twin.

[0018] The cross-platform communication unit outputs external system interaction data based on the interaction requests from the other four layers of modules and the feedback data from the external system.

[0019] The visual programming unit is based on a local end-to-end environment with an edge-cloud architecture. It develops a graphical, modular programming interface, encapsulates robot control commands into visual components, and outputs robot control algorithms based on user programming operations.

[0020] Preferably, the virtual simulation engine constructs a multibody dynamics model of the construction robot using the Newton-Euler dynamics algorithm, and numerically solves the robot's motion differential equations using the fourth-order Runge-Kutta method. It also combines the penalty function method of the contact force model to realize collision detection and force feedback simulation between the robot and construction components and the environment.

[0021] Preferably, the robot multibody dynamics model is based on the mechanical structure of the construction robot, constructing the motion constraint relationships and force transmission paths between each component, and outputting the robot's motion differential equations, including the component center of mass motion equations, component posture motion equations, and the robot's overall motion differential equations. The component center of mass motion equation is:

[0022]

[0023] in, For the first The mass of each component For the first The linear acceleration of the center of mass of each component. For the action in the External forces on each component Adjacent components For the The resultant force of the forces acting on each component For the first A set of adjacent components;

[0024] The equations of motion for the component's attitude are:

[0025]

[0026] in, For the first The inertial tensor of a component about its center of mass. For the first The angular velocity of each component For the first The angular acceleration of each component, For the action in the External torque on each component Adjacent components For the The resultant force of the torques acting on each component This refers to the cross product operation of vectors.

[0027] The differential equation of motion for the robot as a whole is:

[0028]

[0029] For the robot's inertia matrix, For robot joint angle vectors, The joint angular velocity vector. The joint angular acceleration vector. The matrix of Coriolis force and centrifugal force. The gravity vector This is the joint driving torque vector. This is the transpose of the Jacobian matrix. The collision contact force vector experienced by the end effector.

[0030] Preferably, the fourth-order Runge-Kutta method is used to numerically solve the robot's motion differential equations. Specifically, the second-order motion differential equations of the robot are transformed into a system of first-order differential equations. By calculating four intermediate time steps, the angles, angular velocities, and angular accelerations of the robot joints at each time step are obtained, driving the robot to complete continuous and smooth virtual motion. The formula is as follows:

[0031]

[0032] in, For the first The state vector at time t, To solve for the time step, There are four intermediate increments.

[0033] Preferably, the building scene control perception unit uses a Gaussian noise simulation algorithm to perform accuracy and delay simulations on various sensor data. Then, for the visual sensor simulation, it generates environmental visual data based on a perspective projection transformation algorithm, with the following formula:

[0034]

[0035] in, The world coordinates of a point in a 3D construction scene. Let these be the homogeneous image coordinates of the points in space. These are the two-dimensional pixel coordinates mapped from spatial points. For the camera intrinsic parameter matrix, For rotation matrix, It is a translation vector;

[0036] For distance sensor simulation, the detection distance is calculated based on the TOF time-of-flight algorithm; for collision sensor simulation, the collision signal is triggered based on the bounding box intersection detection algorithm.

[0037] Based on sensor data, using an LLM large model and cloud-based scene materials, a B-spline curve fitting algorithm is used to achieve smooth modeling of terrain and building outlines. The formula is as follows:

[0038]

[0039] in, For points on the B-spline curve, Let B be the degree of the B-spline curve. To control the number of vertices, Let n be the basis functions of the B-spline. To control the vertices;

[0040] The three-dimensional spatial division and component layout positioning of the construction scene are completed based on the spatial subdivision algorithm, and the scene switching process is smoothly transitioned through the scene interpolation algorithm.

[0041] Preferably, keyboard interaction transmits basic control signals to the virtual simulation engine via a Socket communication link through a preset keyboard and mouse command set, directly driving the robot to complete actions. The mapping formula between keyboard and mouse commands and robot actions is as follows:

[0042]

[0043] This refers to the mapped robot joint drive torque command. This is the instruction gain coefficient. For instruction mapping functions, For keyboard input signal values, The minimum driving torque for the robot joints, This represents the maximum driving torque of the robot's joints.

[0044] The formula for socket communication signal transmission is:

[0045]

[0046] in, for The actual control signals are constantly transmitted to the virtual simulation engine. Standardize the initial keyboard input signal. The signal attenuation coefficient is... For signal transmission time, For transmitting noise.

[0047] Preferably, the algorithm interaction retrieves a matching algorithm from a pre-set algorithm library through algorithm matching logic, and sets the core parameters of the algorithm through the algorithm parameter configuration interface. The standardized algorithm call command is then transmitted to the virtual simulation engine via a Socket communication link. After parsing the command, the engine integrates the algorithm parameters into a Newton-Euler dynamics model, driving the robot to execute corresponding complex construction tasks. The formula is:

[0048]

[0049] in, The joint driving torque corresponding to the algorithm call command. The weight matrix is ​​the algorithm parameter. This refers to the vector of core algorithm parameters set via the parameter configuration interface. This provides the basic driving torque for the robot.

[0050] Preferably, algorithm development involves dragging and dropping algorithm modules, configuring module parameters, and connecting module logic to complete the design of a custom algorithm. After the unit's built-in debugging tool completes syntax verification, logic debugging, and simulation testing to ensure that the algorithm can run normally, the unit's built-in compiler compiles the graphical algorithm logic into machine code that can be recognized by the virtual simulation engine. A two-way communication between the visual programming unit and the virtual simulation engine is established through a Socket communication link, and the compiled algorithm code is sent to the engine in real time. After parsing the code, the engine integrates the custom algorithm logic into the Newton-Euler dynamics model, driving the robot to execute the corresponding custom construction task.

[0051] Compared with the prior art, the technical solution of this application has the following technical effects:

[0052] This invention achieves centralized management of cloud-based teaching resources and efficient collaboration between local lightweight simulations through an edge-cloud collaborative two-way data interaction architecture. It optimizes interactive response speed and concurrent processing capabilities through a dedicated data transmission plugin, supports simultaneous access by multiple users and parallel simulation, and significantly improves the utilization rate of teaching resources and the flexibility of teaching.

[0053] This invention is based on five functional modules. It achieves high-precision solution of the robot's motion differential equations by combining the Newton-Euler dynamics algorithm with the fourth-order Runge-Kutta method. It also uses the penalty function method to complete collision detection and force feedback simulation, realistically reproducing the robot's working state. Through the large model of the building scene control and perception unit and the dual-path modeling of digital twins, as well as the accurate simulation algorithm of sensor data, the scene simulation error is controlled within a reasonable range. The cross-platform communication unit breaks down data silos and realizes efficient interaction between various modules and external systems.

[0054] This invention is based on a three-level progressive interactive mode. It enables basic operation teaching through keyboard interaction, pre-set algorithm application teaching through algorithm interaction, and custom algorithm design and debugging teaching through algorithm development. The graphical block-style programming interface lowers the learning threshold of algorithms, allowing students to intuitively grasp the robot control logic, improve the depth and flexibility of teaching, and help cultivate digital operation talents.

[0055] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0056] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0058] Based on the description of the figures and their corresponding technical content in the document, the titles of the figures are as follows:

[0059] Figure 1 This is a flowchart illustrating the construction method of a virtual simulation teaching platform for building robots.

[0060] Figure 2 This is a general architecture diagram of a construction method for a virtual simulation teaching platform for building robots;

[0061] Figure 3 This is an architecture diagram of the five functional modules in this application;

[0062] Figure 4 This is a diagram of the three-level progressive interaction architecture in this application;

[0063] Figure 5 Graphs showing the recorded joint motion angles for each method;

[0064] Figure 6 A comparison chart of response time data for each method;

[0065] Figure 7 To obtain data for the application of each method and to assess the data probability density plot;

[0066] Figure 8 This is a comparison chart of the comprehensive performance indicators of the various methods. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0068] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0069] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0070] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0071] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0072] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0073] Example 1 mainly describes a method for constructing a virtual simulation teaching platform for construction robots, such as... Figure 1 , Figure 2 As shown, it specifically includes:

[0074] Establish a two-way data interaction architecture that integrates cloud and local collaboration. Deploy construction robot models, construction scene materials, control algorithms, and teaching resources in the cloud, and deploy simulation engines, data transmission plugins, and visual programming platforms on the local end.

[0075] Based on a two-way data interaction architecture, a five-layer functional module is constructed that is connected and works collaboratively from top to bottom, including a 3D visualization unit, a virtual simulation engine, a building scene control and perception unit, a cross-platform communication unit, and a visualization programming unit.

[0076] Based on five functional modules, a two-dimensional functional platform is constructed, including full life cycle simulation scenarios and three-level progressive interaction. The simulation scenarios cover the entire life cycle of digital exploration, intelligent production, intelligent construction, and intelligent operation and maintenance. The three-level progressive interaction includes keyboard interaction, algorithm interaction, and algorithm development.

[0077] Based on an edge-cloud collaborative architecture, a five-layer core module system, and a dual-dimensional functional system, users can select the appropriate type of construction robot from the cloud resource library; build and configure the simulation operation environment through the five-layer functional modules; and design control algorithms through the visual programming module to drive the robot to complete tasks and observe the operation effect through the three-dimensional visualization unit.

[0078] Furthermore, the two-way data interaction architecture specifically includes: a categorized construction robot model library (3D models and motion parameter models of robots for exploration, construction, and inspection), a construction scene material library (3D materials for building frames, construction sites, and component layouts), a control algorithm library (AI algorithms such as pre-set path planning and motion control), and a teaching resource library (operation manuals, teaching cases, and practical training tasks).

[0079] Based on local terminal hardware configuration, the three core components are deployed in a lightweight manner: simulation engine (physical engine adapted to motion simulation of construction robots), data transmission plug-in (core carrier of edge-cloud data interaction), and visual programming platform (graphical algorithm editing / debugging environment). Redundant functions are eliminated to ensure the real-time performance of local simulation.

[0080] A dedicated data transmission plugin based on the TCP / IP protocol was developed, integrating data sending, data receiving, data parsing, and abnormal retransmission.

[0081] Simulate scenarios where multiple users simultaneously access cloud resources and perform parallel simulations on the local end to verify the architecture's concurrent processing capabilities. Based on the debugging results, optimize the response speed of edge-cloud data interaction and resource caching strategies to finalize the architecture.

[0082] Furthermore, such as Figure 3 The diagram shows the architecture of the five functional modules. Specifically, the five functional modules are: a 3D visualization unit based on motion data from a virtual simulation engine, an environmental data unit for building scene control and perception, and an output visualization interactive interface.

[0083] The virtual simulation engine uses control algorithms from visual programming units and environmental parameters from building scene control perception units to output robot motion simulation results.

[0084] The building scene control and perception unit, based on construction scene materials collected by sensors and user-configured working condition parameters, outputs scene modeling data and sensor simulation data through a dual-path modeling algorithm of large model generation and digital twin.

[0085] The cross-platform communication unit outputs external system interaction data based on the interaction requests from the other four layers of modules and the feedback data from the external system.

[0086] The visual programming unit is based on a local end-to-end environment with an edge-cloud architecture. It develops a graphical, modular programming interface, encapsulates robot control commands into visual components, and outputs robot control algorithms based on user programming operations.

[0087] Furthermore, the virtual simulation engine constructs a multibody dynamics model of the construction robot using the Newton-Euler dynamics algorithm, and numerically solves the robot's motion differential equations using the fourth-order Runge-Kutta method. Combined with the penalty function method of the contact force model, it realizes collision detection and force feedback simulation between the robot and construction components and the environment.

[0088] Furthermore, the multibody dynamics model of the robot is based on the mechanical structure of the construction robot, constructing the motion constraint relationships and force transmission paths between various components, and outputting the robot's motion differential equations, including the component center of mass motion equations, component posture motion equations, and the robot's overall motion differential equations. Among them, the component center of mass motion equation is:

[0089]

[0090] in, For the first The mass of each component For the first The linear acceleration of the center of mass of each component. For the action in the External forces on each component Adjacent components For the The resultant force of the forces acting on each component For the first A set of adjacent components;

[0091] The equations of motion for the component's attitude are:

[0092]

[0093] in, For the first The inertial tensor of a component about its center of mass. For the first The angular velocity of each component For the first The angular acceleration of each component, For the action in the External torque on each component Adjacent components For the The resultant force of the torques acting on each component This refers to the cross product operation of vectors.

[0094] The differential equation of motion for the robot as a whole is:

[0095]

[0096] For the robot's inertia matrix, For robot joint angle vectors, The joint angular velocity vector. The joint angular acceleration vector. The matrix of Coriolis force and centrifugal force. The gravity vector This is the joint driving torque vector. This is the transpose of the Jacobian matrix. The collision contact force vector experienced by the end effector.

[0097] Furthermore, the robot's motion differential equations are numerically solved using the fourth-order Runge-Kutta method. Specifically, the second-order motion differential equations of the robot are transformed into a system of first-order differential equations, as shown in the following formula:

[0098]

[0099]

[0100] By calculating four intermediate time steps, the angle, angular velocity, and angular acceleration of the robot joints at each time step are obtained, driving the robot to complete continuous and smooth virtual motion. The formula is as follows:

[0101]

[0102] in, For the first The state vector at time t, To solve for the time step, There are four intermediate increments. , , , , , For the first The time of moment, It is the right-hand function of a system of first-order differential equations.

[0103] Furthermore, the building scene control perception unit uses a Gaussian noise simulation algorithm to perform accuracy and delay simulations on various sensor data. Then, for the visual sensor, it simulates the generation of environmental visual data based on a perspective projection transformation algorithm, using the following formula:

[0104]

[0105] in, The world coordinates of a point in a 3D construction scene. Let these be the homogeneous image coordinates of the points in space. These are the two-dimensional pixel coordinates mapped from spatial points. For the camera intrinsic parameter matrix, For rotation matrix, It is a translation vector;

[0106] For distance sensor simulation, the detection range is calculated based on the Time-of-Flight (TOF) algorithm, and the formula is:

[0107]

[0108] in, This represents the detection distance between the sensor and the target object. At the speed of light, The time of flight of the light signal;

[0109] This paper aims to simulate collision signals triggered by a bounding box intersection detection algorithm based on collision sensors.

[0110] Based on sensor data, using an LLM large model and cloud-based scene materials, a B-spline curve fitting algorithm is used to achieve smooth modeling of terrain and building outlines. The formula is as follows:

[0111]

[0112] in, For points on the B-spline curve, Let B be the degree of the B-spline curve. To control the number of vertices, Let n be the basis functions of the B-spline. To control the vertices;

[0113] The construction scene is divided into three-dimensional spaces and components are positioned using a spatial partitioning algorithm. Scene transitions are smoothed using a scene interpolation algorithm. The three-dimensional space of the construction scene is recursively divided into eight subspaces. The existence of components is checked in each subspace, achieving efficient spatial partitioning and precise component positioning, facilitating subsequent scene rendering and collision detection. The boundary of the current spatial node is defined as follows:

[0114]

[0115] The center point of the space is:

[0116]

[0117] Then the boundaries of the 8 subspaces are:

[0118]

[0119] The partitioning termination conditions are: subspace side length ≤ 0.1m; component positioning accuracy ≤ 0.005m; spatial partitioning efficiency ≥ 1000 spatial nodes / second.

[0120] Furthermore, such as Figure 4 The diagram shows a three-level progressive interaction architecture. Keyboard interaction transmits basic control signals to the virtual simulation engine via a Socket communication link through a preset keyboard and mouse command set, directly driving the robot to complete actions. The mapping formula between keyboard and mouse commands and robot actions is as follows:

[0121]

[0122] This refers to the mapped robot joint drive torque command. This is the instruction gain coefficient. For instruction mapping functions, For keyboard input signal values, The minimum driving torque for the robot joints, This represents the maximum driving torque of the robot's joints.

[0123] The formula for socket communication signal transmission is:

[0124]

[0125] in, for The actual control signals are constantly transmitted to the virtual simulation engine. Standardize the initial keyboard input signal. The signal attenuation coefficient is... For signal transmission time, For transmitting noise.

[0126] Furthermore, the algorithm interaction retrieves a matching algorithm from a pre-set algorithm library through algorithm matching logic, and completes the core parameter settings of the algorithm through the algorithm parameter configuration interface. The standardized algorithm call command is transmitted to the virtual simulation engine via a Socket communication link. After parsing the command, the engine integrates the algorithm parameters into a Newton-Euler dynamics model, driving the robot to execute corresponding complex construction tasks. The formula is:

[0127]

[0128] in, The joint driving torque corresponding to the algorithm call command. The weight matrix is ​​the algorithm parameter. This refers to the vector of core algorithm parameters set via the parameter configuration interface. This provides the basic driving torque for the robot.

[0129] Furthermore, algorithm development involves dragging and dropping algorithm modules, configuring module parameters, and connecting module logic to complete the design of custom algorithms. After syntax verification, logic debugging, and simulation testing are completed using the unit's built-in debugging tools to ensure that the algorithm can run normally, the graphical algorithm logic is compiled into machine code that can be recognized by the virtual simulation engine through the unit's built-in compiler. A two-way communication between the visual programming unit and the virtual simulation engine is established through a Socket communication link, and the compiled algorithm code is sent to the engine in real time. After parsing the code, the engine integrates the custom algorithm logic into the Newton-Euler dynamics model, driving the robot to execute the corresponding custom construction task.

[0130] This embodiment details a method for constructing a virtual simulation teaching platform for building robots. It establishes an end-to-cloud collaborative bidirectional data interaction architecture, deploys cloud resources and local components, and optimizes interactive performance. The platform comprises five functional modules: a 3D visualization unit, a virtual simulation engine, a building scene control and perception unit, a cross-platform communication unit, and a visual programming unit. The virtual simulation engine uses Newton-Euler dynamics algorithms to achieve high-precision simulation, while the building scene control and perception unit uses multiple algorithms to complete sensor simulation and scene modeling. Finally, a dual-dimensional functional platform is constructed, covering the entire lifecycle of simulation scenarios and three levels of progressive interaction.

[0131] Example 2, based on Example 1, describes in detail the process of using this method in conjunction with a construction robot masonry teaching scenario to conduct virtual simulation teaching of a robot for laying irregularly shaped refractory bricks, as follows:

[0132] A cloud-ground collaborative two-way data interaction architecture is established. On the cloud, a robot model library containing models of irregular refractory brick laying robots, a construction scene material library (covering four aspects: digital exploration, intelligent production, intelligent construction, and intelligent operation and maintenance, including scene materials such as coke oven masonry, building leveling, and pipeline laying), a control algorithm library (pre-built with 50 commonly used control algorithms), and a teaching resource library (containing at least 200 resources such as operation tutorials and algorithm explanation videos) are deployed. On the local end, a Unity simulation engine, a dedicated data transmission plugin, and a graphical visualization programming platform are deployed. The dedicated data transmission plugin is developed based on the TCP / IP protocol and integrates data sending, receiving, parsing, and abnormal retransmission functions.

[0133] The system comprises five functional modules: a 3D visualization unit based on the Unity engine, outputting a 1080P immersive visual interactive interface; a virtual simulation engine using the Newton-Euler dynamics algorithm to construct a multibody dynamics model of the masonry robot, numerically solving the robot's motion differential equations using the fourth-order Runge-Kutta method, and implementing collision detection and force feedback simulation using the penalty function method; a building scene control and perception unit simulating sensor data using a Gaussian noise simulation algorithm, generating environmental visual data based on a perspective projection transformation algorithm, and achieving smooth modeling of the coke oven masonry scene using an LLM large model combined with a B-spline curve fitting algorithm, with smooth transitions achieved through scene interpolation algorithms; a cross-platform communication unit enabling efficient interaction with BIM systems and intelligent sensors; and a visual programming unit developing a modular programming interface that encapsulates robot control commands into at least 30 visual components.

[0134] A dual-dimensional functional platform is constructed, with full lifecycle simulation scenarios covering four stages: digital exploration (coke oven site exploration, refractory brick parameter acquisition), intelligent production (refractory brick processing simulation), intelligent construction (irregular refractory brick masonry), and intelligent operation and maintenance (robot fault diagnosis and maintenance simulation). The three-level progressive interaction includes keyboard interaction (16 sets of preset keyboard and mouse commands, mapping robot grasping, grouting, masonry and other actions), algorithm interaction (retrieving masonry path planning algorithms from the algorithm library, with configurable core parameters such as path accuracy and speed), and algorithm development (dragging and dropping algorithm modules to design custom masonry control algorithms, with built-in debugging tools to complete syntax verification and simulation testing).

[0135] 150 architecture students were selected and divided into 3 groups. They underwent 10 days of practical training using the platform constructed using this method, as well as the UTDSM (CN121389274A) and CRTPP (CN212084426U) methods described in the background of this application. Each day's training lasted 4 hours. After the training, operational assessments and performance statistics were conducted, and the core performance data of the three methods were recorded. The data are shown in Tables 1-3 below.

[0136] Table 1 Comparison of scenario simulation test data for each method

[0137] According to Table 1 and Figure 5As shown in the joint motion angle recording data graphs for each method, this method, using the Newton-Euler dynamics algorithm combined with the fourth-order Runge-Kutta method, achieves a joint motion angle error of 0.4° for the masonry robot and a scene simulation error of only 2.8%, covering 4 full life cycle stages and 22 sub-scenes, and can comprehensively reproduce the operational scenarios of each stage of the construction robot. The joint motion angle error of UTDSM is 2.3°, and the scene simulation error is 4.8%, covering only 1 construction stage and 3 sub-scenes, with limited scene coverage and insufficient simulation accuracy. CRTPP, based on physical demonstration, has a joint motion angle error of 5.8° and a scene simulation error of 18.5%, lacks sub-scenes, and can only be demonstrated in a fixed location, resulting in extremely poor scene simulation effects and failing to meet the needs of multi-scenario teaching.

[0138] Table 2 Response time test data for each method

[0139] According to Table 2 and Figure 6 The comparison chart of response time data for each method shows that the dedicated data transmission plugin of this method optimizes the efficiency of cloud-to-ground data interaction, with a collision detection response time of only 8.6ms, keyboard interaction response time of 38.5ms, algorithm call response time of 76.2ms, and cloud-to-ground data transmission latency of 47.8ms. It supports concurrent access by 50 users with a concurrent response latency of 52.4ms and a 100% concurrency success rate, demonstrating high efficiency and stability. UTDSM has a collision detection response time of 35.3ms, a keyboard interaction response time of 156.2ms, no algorithm call function, and a cloud-to-ground data transmission latency of 125.3ms. It only supports operation by one user, has no concurrency capability, and has low interaction efficiency. CRTPP is a physical operation method with a collision detection response time of 821.8ms and a keyboard interaction response time of 517.1ms. It lacks algorithm call function and cloud-to-ground collaborative architecture, supports operation by only one student / device, has no concurrency capability, and exhibits extremely poor operational flexibility and severe interaction lag.

[0140] Table 3. Evaluation and test data for each method

[0141] According to Table 3 and Figure 7The probability density plots of the application mastery data and assessment pass data for each method show that, for students using this method platform, the pass rate for the masonry robot operation assessment was 96.0%, and the pass rate for the control algorithm design assessment was 92.0%. Among them, 85.0% of the students could independently complete the design and debugging of custom masonry algorithms, mastering the application of algorithms in just 3.2 days, and were able to accurately master the core technology of robot motion control. Students using UTDSM, due to the lack of teaching adaptability and algorithm development functions, had a pass rate of 68.0% for the operation assessment and a pass rate of 0.0% for the control algorithm design assessment, and had no algorithm application ability. Students using CRTPP could only master the basic operation of the robot, with a pass rate of 70.0% for the operation assessment and a pass rate of 0.0% for the control algorithm design assessment, and had no algorithm application ability. Moreover, due to the limitations of physical operation, their learning efficiency was low.

[0142] Table 4. Overall performance indicators of each method

[0143] According to Table 4 and Figure 8 The comparison chart of comprehensive performance indicators for each method shows that the virtual simulation teaching platform for construction robots constructed by this method is significantly superior to CRTPP and UTDSM in terms of scene simulation, response timeliness, assessment effect, and overall performance. Based on the specific data related to comprehensive performance in Table 4, the simulation accuracy compliance rate of this method reaches 99.2%, the response timeliness compliance rate reaches 98.7%, the teaching suitability compliance rate reaches 97.9%, the multi-user concurrent stability rate reaches 100.0%, the platform's continuous operation stability reaches 99.5%, and the average training cost per student is only 85.6 yuan / day, far lower than UTDSM's 218.3 yuan / day and CRTPP's 562.8 yuan / day.

[0144] This embodiment details the virtual simulation teaching platform for construction robots constructed using this method. Tests comparing it with existing methods CRTPP and UTDSM in terms of scene simulation, response time, assessment effectiveness, and overall performance demonstrate that this platform significantly outperforms existing methods in core performance aspects such as joint motion angle error, scene simulation error, and response time. It boasts a high pass rate for student operation and algorithm design assessments, high learning efficiency, superior overall performance, and lower training costs. It effectively addresses the shortcomings of existing patents, such as reliance on physical objects, insufficient simulation accuracy, and limited scene coverage. This platform can meet the needs of efficient and precise training of construction robot professionals and is applicable to various training scenarios.

[0145] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter changes made to these embodiments within the spirit and principles of the present invention, without departing from the principles and spirit of the present invention, through conventional substitutions or to achieve the same function, fall within the scope of protection of the present invention.

Claims

1. A method for constructing a virtual simulation teaching platform for construction robots, characterized in that, include: Establish a two-way data interaction architecture that integrates cloud and local collaboration. Deploy construction robot models, construction scene materials, control algorithms, and teaching resources in the cloud, and deploy simulation engines, data transmission plugins, and visual programming platforms on the local end. Based on a two-way data interaction architecture, a five-layer functional module is constructed that is connected and works collaboratively from top to bottom, including a 3D visualization unit, a virtual simulation engine, a building scene control and perception unit, a cross-platform communication unit, and a visualization programming unit. Based on five functional modules, a two-dimensional functional platform is constructed, including full life cycle simulation scenarios and three-level progressive interaction. The simulation scenarios cover the entire life cycle of digital exploration, intelligent production, intelligent construction, and intelligent operation and maintenance. The three-level progressive interaction includes keyboard interaction, algorithm interaction, and algorithm development. Based on an edge-cloud collaborative architecture, a five-layer core module system, and a dual-dimensional functional system, users can select the appropriate type of construction robot from the cloud resource library; build and configure the simulation operation environment through the five-layer functional modules; and design control algorithms through the visual programming module to drive the robot to complete tasks and observe the operation effect through the three-dimensional visualization unit.

2. The method for constructing a virtual simulation teaching platform for construction robots according to claim 1, characterized in that, The bidirectional data interaction architecture is specifically as follows: based on the types of resources required for teaching construction robots, a construction robot model library, motion parameter model, construction scene material library, control algorithm library, and teaching resource library are classified and constructed. Based on local terminal hardware configuration, a lightweight deployment of the simulation engine, data transfer plugin, and visual programming platform is implemented. A dedicated data transmission plugin based on the TCP / IP protocol was developed, integrating data sending, data receiving, data parsing, and abnormal retransmission. Simulate scenarios where multiple users simultaneously access cloud resources and perform parallel simulations on the local end to verify the architecture's concurrent processing capabilities. Based on the debugging results, optimize the response speed of edge-cloud data interaction and resource caching strategies to finalize the architecture.

3. The method for constructing a virtual simulation teaching platform for construction robots according to claim 1, characterized in that, The five functional modules are as follows: the three-dimensional visualization unit outputs a visual interactive interface based on motion data from the virtual simulation engine and environmental data from the building scene control and perception unit; The virtual simulation engine uses control algorithms from visual programming units and environmental parameters from building scene control perception units to output robot motion simulation results. The building scene control and perception unit, based on construction scene materials collected by sensors and user-configured working condition parameters, outputs scene modeling data and sensor simulation data through a dual-path modeling algorithm of large model generation and digital twin. The cross-platform communication unit outputs external system interaction data based on the interaction requests from the other four layers of modules and the feedback data from the external system. The visual programming unit is based on a local end-to-end environment with an edge-cloud architecture. It develops a graphical, modular programming interface, encapsulates robot control commands into visual components, and outputs robot control algorithms based on user programming operations.

4. The method for constructing a virtual simulation teaching platform for construction robots according to claim 3, characterized in that, The virtual simulation engine constructs a multibody dynamics model of the construction robot using the Newton-Euler dynamics algorithm, and numerically solves the robot's motion differential equations using the fourth-order Runge-Kutta method. It also combines the penalty function method of the contact force model to realize collision detection and force feedback simulation between the robot and construction components and the environment.

5. The method for constructing a virtual simulation teaching platform for construction robots according to claim 4, characterized in that, The robot multibody dynamics model is based on the mechanical structure of the construction robot, constructing the motion constraint relationships and force transmission paths between various components, and outputting the robot's motion differential equations, including the component center of mass motion equations, component posture motion equations, and the robot's overall motion differential equations. The component center of mass motion equation is as follows: in, For the first The mass of each component For the first The linear acceleration of the center of mass of each component. For the action in the External forces on each component Adjacent components For the first The resultant force of the forces acting on each component For the first A set of adjacent components; The equations of motion for the component's attitude are: in, For the first The inertial tensor of a component about its center of mass. For the first The angular velocity of each component For the first The angular acceleration of each component, For the action in the External torque on each component Adjacent components For the first The resultant force of the torques acting on each component This refers to the cross product operation of vectors. The differential equation of motion for the robot as a whole is: For the robot's inertia matrix, For robot joint angle vectors, The joint angular velocity vector. The joint angular acceleration vector. The matrix of Coriolis force and centrifugal force. The gravity vector This is the joint driving torque vector. This is the transpose of the Jacobian matrix. The collision contact force vector experienced by the end effector.

6. The method for constructing a virtual simulation teaching platform for construction robots according to claim 4, characterized in that, The numerical solution of the robot's motion differential equations using the fourth-order Runge-Kutta method involves: transforming the robot's second-order motion differential equations into a system of first-order differential equations; calculating the angles, angular velocities, and angular accelerations of the robot joints at each time step using four intermediate steps; and driving the robot to complete continuous and smooth virtual motion. The formula is as follows: in, For the first The state vector at time t, To solve for the time step, There are four intermediate increments.

7. The method for constructing a virtual simulation teaching platform for construction robots according to claim 3, characterized in that, The building scene control and perception unit uses a Gaussian noise simulation algorithm to perform accuracy and delay simulations on various sensor data. Then, for the visual sensor, it simulates and generates environmental visual data based on a perspective projection transformation algorithm, using the following formula: in, The world coordinates of a point in a 3D construction scene. Let these be the homogeneous image coordinates of the points in space. These are the two-dimensional pixel coordinates mapped from spatial points. For the camera intrinsic parameter matrix, For rotation matrix, It is a translation vector; For distance sensor simulation, the detection distance is calculated based on the TOF time-of-flight algorithm; for collision sensor simulation, the collision signal is triggered based on the bounding box intersection detection algorithm. Based on sensor data, using an LLM large model and cloud-based scene materials, a B-spline curve fitting algorithm is used to achieve smooth modeling of terrain and building outlines. The formula is as follows: in, For points on the B-spline curve, Let B be the degree of the B-spline curve. To control the number of vertices, Let n be the basis functions of the B-spline. To control the vertices; The three-dimensional spatial division and component layout positioning of the construction scene are completed based on the spatial subdivision algorithm, and the scene switching process is smoothly transitioned through the scene interpolation algorithm.

8. The method for constructing a virtual simulation teaching platform for construction robots according to claim 1, characterized in that, The keyboard interaction transmits basic control signals to the virtual simulation engine via a Socket communication link through a preset keyboard and mouse command set, directly driving the robot to complete actions. The mapping formula between keyboard and mouse commands and robot actions is as follows: This refers to the mapped robot joint drive torque command. This is the instruction gain coefficient. For instruction mapping functions, For keyboard input signal values, The minimum driving torque for the robot joints, This represents the maximum driving torque of the robot's joints. The formula for socket communication signal transmission is: in, for The actual control signals are constantly transmitted to the virtual simulation engine. Standardize the initial keyboard input signal. The signal attenuation coefficient is... For signal transmission time, For transmitting noise.

9. The method for constructing a virtual simulation teaching platform for construction robots according to claim 1, characterized in that, The algorithm interaction retrieves a matching algorithm from a pre-set algorithm library through algorithm matching logic, and sets the core parameters of the algorithm through the algorithm parameter configuration interface. The standardized algorithm call command is transmitted to the virtual simulation engine via a Socket communication link. After parsing the command, the engine integrates the algorithm parameters into a Newton-Euler dynamics model, driving the robot to execute corresponding complex construction tasks. The formula is: in, The joint driving torque corresponding to the algorithm call command. The weight matrix is ​​the algorithm parameter. This refers to the vector of core algorithm parameters set via the parameter configuration interface. This provides the basic driving torque for the robot.

10. The method for constructing a virtual simulation teaching platform for construction robots according to claim 1, characterized in that, The algorithm development process involves dragging and dropping algorithm modules, configuring module parameters, and connecting module logic to complete the design of a custom algorithm. After the built-in debugging tool in the unit completes syntax verification, logic debugging, and simulation testing to ensure that the algorithm can run normally, the graphical algorithm logic is compiled into machine code that can be recognized by the virtual simulation engine through the built-in compiler in the unit. A two-way communication between the visual programming unit and the virtual simulation engine is established through a Socket communication link, and the compiled algorithm code is sent to the engine in real time. After parsing the code, the engine integrates the custom algorithm logic into the Newton-Euler dynamics model to drive the robot to execute the corresponding custom construction task.

Images