Digital twin system of multi-functional quadruped robot in coal mine and operation method thereof

Abstract

The application discloses a kind of coal mine underground multifunctional four-foot robot digital twin systems and operating method, belong to digital twin technical field.The system of the present application, including robot virtual space and robot physical entity, robot physical entity is four-foot structure, contains autonomous perception module, autonomous decision module and autonomous control module, robot virtual space contains four sub-systems of robot virtual simulation, robot virtual planning, robot virtual debugging, robot virtual monitoring.Robot virtual simulation, virtual planning, virtual debugging subsystem provides information and reference for the design and operation of robot physical entity;Robot virtual monitoring subsystem provides three-dimensional operating state monitoring and remote visual man-machine interface for robot physical entity.Through the real-time bidirectional information interaction mapping of physical and virtual space, a digital twin solution is provided for the design, operation and maintenance, and monitoring of coal mine robots, and the intelligent level of coal mines is improved.

Description

Technical Field

[0001] This invention belongs to the field of digital twin technology, specifically relating to a digital twin system and operation method for a multifunctional quadruped robot in underground coal mines. Background Technology

[0002] Intelligentization has become a core technological support for the high-quality development of the coal industry and represents its future. Over 60% of coal mine workers are engaged in dangerous and arduous work; the development and application of coal mine robots can effectively improve the safety of coal mine production.

[0003] Researching coal mine robots is technically challenging, time-consuming, and costly. Some key technologies remain unresolved, and the design and application of coal mine robots still have many shortcomings, mainly in the following aspects:

[0004] (1) At present, existing coal mine robots are only used in fields related to detection such as inspection and rescue, and there is no substantial progress in operations under complex conditions;

[0005] (2) Due to the complexity and unpredictability of the underground environment in coal mines, coal mine robots are more difficult to debug on-site than ground robots, and are more unpredictable, which can easily lead to a waste of human and material resources.

[0006] (3) At present, coal mine robots are still difficult to replace workers to achieve fully autonomous operation. They still need to rely on human-robot collaboration. The human-machine interaction channel between operators and coal mine robots needs to be studied.

[0007] (4) The terrain in coal mines is complex and varied, but coal mine robots lack effective three-dimensional visualization remote monitoring methods, making it difficult to intuitively and comprehensively obtain the robot's real-time pose status.

[0008] With the continuous advancement and maturation of digital twin technology, combined with the widespread application and rapid development of modern communication technologies, a bridge has been built between the physical and virtual worlds. Digital twin technology can map a digital model of the physical world onto a virtual space, enabling real-time perception, diagnosis, and prediction of the state of physical entities within that space, and allowing for optimization and command-based control of these entities. This provides new ideas and important insights for the design, operation, and monitoring of coal mine robots.

[0009] In the prior art, patent document CN113050649A discloses a remote control system and method for a digital twin-driven inspection robot, which includes a physical entity of the inspection robot, a data sensing and transmission module, a virtual entity of the inspection robot, a control module, and twin data. It utilizes digital twin technology to establish a virtual remote control platform for the inspection robot in a fully mechanized coal mining face, realizing simultaneous virtual and physical movement, status monitoring, and remote control of the inspection robot. Patent document CN114398773A discloses a coal mine scheduling robot system based on artificial intelligence technology, which uses digital twin technology to construct a virtual space scheduling twin model that maps to the underground physical space scheduling system. This model integrates and coordinates real-time data acquired during the mine production process with the physical entity to achieve three-dimensional visualization of coal mine production command and scheduling.

[0010] However, in the two aforementioned solutions, digital twin technology is only used for remote status monitoring and control scheduling of coal mine robots. It fails to fully leverage the powerful analytical and predictive capabilities of digital twin technology in virtual space, and does not apply it to preliminary stages such as virtual simulation, virtual planning, and virtual debugging of coal mine robots to provide information and reference for the autonomous execution of the robot's physical system. Furthermore, it does not address human-machine interaction methods that connect the physical and virtual spaces during robot status monitoring and remote control.

[0011] In the prior art, the patent document with publication number CN113128109A discloses a testing and evaluation method for an intelligent fully mechanized mining robot production system. This method involves constructing a virtual offline operation system for the fully mechanized mining face, reproducing the virtual operation and mining situation of the face, determining the initial simulation data and virtual scene operation data, constructing an AI robot analysis system, inputting equipment and geological exploration methods according to the parameters for future intelligent development, constructing an evaluation system for the fully mechanized mining face operation, simulating the future operation of the fully mechanized mining robot, determining development trends, and testing the robot's operational performance.

[0012] However, in the above scheme, the system is mainly used for offline simulation and reproduction of the mining operation of the fully mechanized mining face, predicting and analyzing future operating conditions to optimize equipment operation paths and ensure the safe and efficient operation of the working face. Although it can also access real-time equipment operation data and conduct virtual-real fusion monitoring in the virtual space, it lacks real-time interaction between offline simulation and online monitoring, and cannot verify the accuracy and reliability of the offline simulation system's mechanism model through real-time operation data.

[0013] In summary, existing technologies have not fully utilized the characteristics and advantages of digital twin technology, applying it only to a specific process in the entire lifecycle of coal mine robots, resulting in a limited application scenario. On-site debugging is difficult, there is a lack of effective 3D visualization and remote monitoring methods, the digital twin model is incomplete, and the operational mechanisms such as real-time interaction, iterative optimization, and dynamic correction between physical and virtual spaces need further development, leading to low system integration. Summary of the Invention

[0014] The technical problem to be solved by this invention is to provide a digital twin system for a multifunctional quadruped robot in coal mines and its operation method. Through real-time two-way information interaction and mapping between physical and virtual spaces, it provides a digital twin solution for the design, operation and maintenance, and monitoring of underground coal mine robots, thereby improving the level of intelligence in coal mines.

[0015] To address the above technical problems, according to one aspect of the present invention, a digital twin system for a multifunctional quadruped robot in underground coal mines is provided, comprising a virtual space for the robot and a physical entity of the robot.

[0016] The robot virtual space includes a robot virtual simulation subsystem, a robot virtual planning subsystem, a robot virtual debugging subsystem, and a robot virtual monitoring subsystem;

[0017] The robot virtual simulation subsystem was developed using Unity3d and serves as a testing and verification platform for the early stages of robot body design and kinematic modeling.

[0018] The robot virtual planning subsystem was developed using Unity3d and is used to complete path planning for a multi-functional quadruped robot in a virtual environment.

[0019] The robot virtual debugging subsystem was developed by Webots and is used for testing and optimizing the robot virtual planning subsystem.

[0020] The robot virtual monitoring subsystem was developed using Unity3d and includes a robot virtual monitoring module and a teleoperation module.

[0021] The robot physical entity adopts a quadruped structure and includes an autonomous perception module, an autonomous decision-making module, and an autonomous control module; the robot physical entity and the robot virtual debugging system communicate with the robot virtual monitoring system Unity3d host computer through a wireless serial communication module;

[0022] The autonomous perception module is used to perceive the robot's own state and the information of its environment.

[0023] The autonomous decision-making module is used to complete three types of adaptation for unknown and complex terrain: robot body height adaptation, body posture adaptation, and robot gait planning.

[0024] The autonomous control module is used to receive control commands from the Unity3d host computer and convert the control commands into robot actions.

[0025] Furthermore, in the robot virtual monitoring subsystem, the virtual monitoring module reads the robot joint angle information sent by the autonomous sensing module through the serial port, decomposes the joint data according to the sending pattern, and assigns the data to the joint angles of the robot's digital twin, realizing three-dimensional visualization monitoring of the robot's real-time operating status. It can observe the robot's pose status in real time in the virtual environment, and can also read the environmental map around the robot sent by the autonomous sensing module, and observe the robot's surrounding environment in real time in the virtual environment.

[0026] Furthermore, in the robot virtual monitoring subsystem, the teleoperation module remotely controls the robot's physical entity or the robot's digital twin in the robot virtual debugging system through the UGUI human-computer interaction panel in Unity3d to complete complex tasks with uncertainties.

[0027] Furthermore, the autonomous perception module includes a three-dimensional lidar, a monocular camera, a fuselage attitude sensor, a servo motor internal angle sensor, and a foot-to-ground contact sensor.

[0028] Three-dimensional LiDAR, monocular camera and body attitude sensor are used to build environmental map in robot virtual planning system, and at the same time use environmental information to perform autonomous localization. The fusion of the three can improve the robustness and accuracy of the system.

[0029] The internal angle sensor of the servo motor is used to provide feedback on the angle information of each joint of the robot;

[0030] Foot contact sensors are used to provide feedback signals when the robot's feet touch the ground, causing the feet to stop falling, in order to adapt to unknown and complex terrain.

[0031] Furthermore, in addition to building environmental maps and autonomous localization, the robot's posture sensor is also used to perceive the robot's posture information in real time in order to maintain the stability of the robot's posture.

[0032] According to another aspect of the present invention, a method for operating the above-described multifunctional quadruped robot digital twin system in coal mines is provided, comprising the following steps:

[0033] S1: Construct a digital twin of a multi-functional underground coal mine robot in the Unity3D environment within the robot virtual simulation subsystem. The specific process is as follows:

[0034] S101: Design the robot dimensions and create a robot model in modeling software;

[0035] S102: Import the robot model into the 3D rendering software, adjust the coordinate axes of each component, render the model, and import it into the robot virtual simulation subsystem; the adjustment of the coordinate axes of each component is to adjust the local coordinate system of each component of the robot model to an appropriate position to ensure that the model imported into Unity3D does not have local coordinate axis rotation.

[0036] S103: In the robot virtual simulation subsystem, establish parent-child relationships for the robot model and add Transform components to each part to enable each part to have basic translation and rotation transformation capabilities.

[0037] S2: The robot's structure, kinematic model, and gait model are verified and optimized in the robot virtual simulation subsystem. The specific process is as follows:

[0038] S201: Establish a kinematic model of a multifunctional quadruped robot in underground coal mines, write a C# script for the kinematic model, and mount it onto the robot's digital twin;

[0039] S202: Perform gait planning for a multi-functional quadruped robot in underground coal mines, write the planning results into a gait planning C# script, and attach it to the robot's digital twin;

[0040] S203: The robot digital twin runs in the robot virtual simulation subsystem to test and optimize the robot's structure, kinematic model and gait;

[0041] S3: Import the optimized digital twin of the multi-functional quadruped robot in the coal mine into the robot virtual planning subsystem, and build the robot's physical entity based on the robot's digital twin;

[0042] S4: Construct a digital twin of a multi-functional quadruped robot for underground coal mines under the Webots environment within the robot virtual debugging subsystem. The specific process is as follows:

[0043] S401: Import the robot model into Webots and create the main nodes of the robot model in the scene tree;

[0044] S402: Add inertial units, position sensors, rotational motors, and touch sensors to the robot model and set appropriate parameters;

[0045] S403: Establish the main controller program and kinematics program for the robot model, and build the interface between the main controller and the inertial unit, position sensor, rotary motor, and contact sensor;

[0046] S5: Import offline maps of underground coal mines into the robot virtual planning subsystem and the robot virtual debugging subsystem, respectively;

[0047] S6: The multi-functional quadruped robot for underground coal mines is debugged in the robot virtual debugging subsystem. The robot path is planned through the robot virtual planning subsystem, and the planning results are executed in the robot virtual debugging subsystem. The robot semi-autonomous control and manual control of the robot are performed through the robot virtual monitoring subsystem. The specific process is as follows:

[0048] S601: Initial pose of the robot digital twin in the synchronous robot virtual planning subsystem and robot virtual debugging subsystem;

[0049] S602: Establish a path planning model for a multi-functional quadruped robot in underground coal mines based on the A* algorithm, write a C# script for path planning, and attach it to the robot's digital twin in the robot virtual planning subsystem.

[0050] S603: Perform robot path planning in the robot virtual planning subsystem and synchronize the planning results to the robot virtual debugging subsystem, and complete the movement according to the path planning results;

[0051] S604: The robot virtual monitoring subsystem enables semi-autonomous and manual control of the digital twin in the robot virtual debugging subsystem, allowing the robot digital twin to complete the specified tasks.

[0052] S7: Compare and analyze the planning results of the robot virtual planning subsystem with the execution results of the robot digital twin in the robot virtual debugging subsystem, and optimize the robot virtual planning subsystem until the robot virtual planning subsystem can accurately plan the path for the robot.

[0053] S8: Place the physical robot entity underground in a coal mine. Through the coordinated operation of the physical robot entity, the robot virtual planning subsystem, and the robot virtual monitoring subsystem, the multi-functional quadruped robot in the coal mine achieves adaptive walking and complex operations underground. The specific process is as follows:

[0054] S801: Install autonomous perception, autonomous decision-making and autonomous control modules on the physical robot entity, establish a multi-functional quadruped robot software system for underground coal mines, and place it in an actual underground coal mine.

[0055] S802: The robot's physical entity operates underground in a coal mine. It constructs a map of the environment around the robot in real time through the autonomous perception module and synchronizes the map to the robot's virtual planning subsystem.

[0056] S803: Perform robot path planning in the robot virtual planning subsystem and synchronize the planning results to the autonomous control module of the robot physical entity. Follow the path planning results to complete the walking and autonomously complete repetitive fixed tasks such as inspection.

[0057] S804: The robot physical entity is semi-autonomously and manually controlled through the teleoperation module in the robot virtual monitoring subsystem, enabling the robot physical entity to complete complex tasks with uncertainties, and the robot's real-time pose status and surrounding environment are monitored in the virtual monitoring module.

[0058] S805: Verify the accuracy and reliability of the kinematic model, gait model and path planning model in the virtual simulation system and virtual planning system based on the real-time monitoring data of the virtual monitoring system, and perform iterative optimization and dynamic correction of the model.

[0059] Furthermore, in step S201, the kinematic model of the multifunctional quadruped robot in the coal mine includes a forward kinematic model and an inverse kinematic model.

[0060] The robot's forward kinematics model refers to the calculation of the foot position based on the robot's body posture, link length, and the rotation angle of each joint.

[0061] The robot inverse kinematics model refers to the calculation of the rotation angles of each joint based on the robot's body posture, link length, and foot position.

[0062] Further, in step S201, the kinematic model C# script includes a forward kinematics calculation script, an inverse kinematics calculation script, and an interpolation calculation script;

[0063] The forward kinematics model of the robot is incorporated into the forward kinematics calculation script, which can be used to calculate the motion space of the robot's feet and determine whether the robot's structure is reasonable.

[0064] The inverse kinematics model of the robot is incorporated into the inverse kinematics calculation script, which can calculate the angles of each joint and input the interpolation calculation script, and is the foundation for realizing robot control.

[0065] The interpolation calculation script calls the Mathf.Lerp() function in the Update() function. Each frame, it obtains the target joint angle of the robot from the inverse kinematics calculation script and moves the robot from the existing angle to the target angle.

[0066] Furthermore, in step S202, the gait planning of the multi-functional quadruped robot in the coal mine adopts a static gait planning method, including robot stepping sequence planning and robot center of gravity trajectory planning;

[0067] The gait planning C# script takes the robot's desired movement speed, attitude angle, and body height as input and outputs the desired foot coordinates.

[0068] Furthermore, in step S5, the offline map of the coal mine specifically refers to a complete underground map of the coal mine pre-constructed based on geological exploration and 3D mapping, which is used to test the path planning function of the robot virtual planning subsystem.

[0069] Compared with existing technologies, the digital twin system and operation method for a multifunctional quadruped robot in coal mines provided by this invention have the following beneficial effects:

[0070] (1) It fully leverages the characteristics and advantages of digital twin technology, constructs a complete digital twin system, and refines the robot virtual space into a robot virtual simulation system, a virtual planning system, a virtual debugging system, and a virtual monitoring system. The division of labor among the systems is clear and highly integrated, which can reduce costs and increase efficiency in multiple stages of the entire life cycle of the multi-functional quadruped robot in coal mines.

[0071] (2) The robot virtual simulation subsystem provides a platform for the verification and optimization of the structure, kinematic model and gait model of the multi-functional quadruped robot in coal mines. It can fully utilize the robot digital twin to optimize the robot before the robot physical entity is put into production and manufacturing, so that the robot can reach the ideal level without repeated processing and manufacturing, thus shortening the development and iteration cycle of the robot.

[0072] (3) The robot's path planning is completed in the robot virtual planning system based on Unity3d, which avoids using the robot's own autonomous decision-making module to build a large scene dense 3D point cloud map obtained by the autonomous perception module. Instead, it uses the more powerful computing power of the Unity3d host computer to process it, which reduces the computing pressure on the robot's physical entity and avoids the lag in the path planning process.

[0073] (4) The robot virtual planning subsystem was optimized using the robot virtual debugging system. By importing the offline coal mine map into the robot virtual planning system and the virtual debugging system respectively, the process of the robot physical entity establishing the coal mine map in the robot virtual planning system based on the actual coal mine environment was effectively simulated. The highly realistic characteristics of the Webots platform were effectively utilized. The robot virtual planning subsystem was optimized by comparing and analyzing the path planning results with the results executed by the robot digital twin, so as to achieve accurate path planning.

[0074] (5) The virtual space of the robot was optimized before the physical robot was put into the mine. Finally, through the coordinated operation of the physical robot, the virtual robot planning subsystem, and the virtual robot monitoring subsystem, the multi-functional quadruped robot in the coal mine can achieve adaptive walking and complex operations in the mine. Furthermore, the mechanism model in the virtual simulation system and the virtual planning system can be verified and iteratively optimized based on the real-time monitoring data of the virtual monitoring system.

[0075] (6) This invention provides a relatively complete solution for the design, manufacturing, debugging and operation of multifunctional quadruped robots in coal mines, enhances the application depth of digital twin technology in the field of coal mine robots, and can also provide a reference for digital twins of robots in other fields. Attached Figure Description

[0076] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention.

[0077] Figure 1 This invention provides an architecture diagram of a digital twin system for a multifunctional quadruped robot in underground coal mines.

[0078] Figure 2 This is a schematic diagram of the operation method of a digital twin system for a multifunctional quadruped robot in a coal mine, provided by the present invention.

[0079] Figure 3 This is a parent-child relationship diagram of the robot model in the robot virtual simulation subsystem;

[0080] Figure 4 This is a flowchart of the forward calculation script in the robot virtual simulation subsystem;

[0081] Figure 5 This is a flowchart of the inverse kinematics calculation script in the robot virtual simulation subsystem;

[0082] Figure 6 This is a diagram of the program file structure of a multi-functional quadruped robot software system for underground coal mines. Detailed Implementation

[0083] A typical embodiment of the present invention provides a digital twin system for a quadruped robot in an underground coal mine, referenced... Figure 1 The system includes a virtual space for robots and a physical entity for robots.

[0084] Robot Virtual Space

[0085] The robot virtual space includes a robot virtual simulation subsystem, a robot virtual planning subsystem, a robot virtual debugging subsystem, and a robot virtual monitoring subsystem.

[0086] The robot virtual simulation subsystem, developed using Unity3d, serves as a testing and verification platform for early-stage robot body design and kinematic modeling. It is capable of performing robot structural design, gait planning, and motion sequence design.

[0087] The robot virtual planning subsystem, developed using Unity3d, is used to complete path planning for a multi-functional quadruped robot in a virtual environment.

[0088] The robot virtual debugging subsystem, developed by Webots, is used for testing and optimizing the robot virtual planning subsystem.

[0089] The robot virtual debugging subsystem includes autonomous control mode, semi-autonomous control mode and manual control mode.

[0090] The autonomous control mode is used for repetitive fixed tasks such as robot inspection. The operator issues task instructions, and the robot performs the task autonomously without human intervention.

[0091] Semi-autonomous control mode is used for specific area operation tasks. These tasks are uncertain due to the different maintenance operations. The robot needs to arrive at the operation area first. The system issues the instruction, and then the operator manually selects the specific task to complete the operation.

[0092] The manual control mode is used for uncertain tasks. The operator manually controls the robot based on the robot's environment and state, manually sets the robot's posture and foot endpoints, and sends control commands to the robot's physical entity to complete the control after confirming that everything is correct.

[0093] The robot virtual monitoring subsystem, developed using Unity3d, includes a robot virtual monitoring module and a teleoperation module.

[0094] The virtual monitoring module can read the angle information of each joint of the robot sent by the autonomous sensing module through the serial port, decompose the data of each joint according to the sending pattern, and assign the data to the angle of each joint of the robot's digital twin, so as to realize the three-dimensional visualization monitoring of the robot's real-time running status, observe the robot's pose status in real time in the virtual environment, and also read the environmental map around the robot sent by the autonomous sensing module, observe the robot's surrounding environment in real time in the virtual environment.

[0095] The teleoperation module can remotely control the physical entity of the robot or the digital twin of the robot in the robot virtual debugging system to complete complex tasks with uncertainties through the UGUI human-computer interaction panel in Unity3D.

[0096] Robot physical entity

[0097] The robot's physical entity adopts a quadruped structure and includes an autonomous perception module, an autonomous decision-making module, and an autonomous control module. The robot has multiple functions such as inspection, picking, carrying, drilling, and pressing.

[0098] The physical robot and the virtual robot debugging system communicate with the Unity3D host computer of the robot virtual monitoring system via a wireless serial communication module. The HC-12 multi-channel embedded wireless data transmission module is preferably used as the wireless serial communication module.

[0099] The autonomous perception module is used to perceive the robot's own state and the information of its environment.

[0100] The autonomous sensing module includes a 3D LiDAR, a monocular camera, a fuselage attitude sensor, an internal angle sensor for the servo motor, and a foot contact sensor.

[0101] LVI-SAM is adopted as the combined inertial / visual / LiDAR SLAM algorithm. This algorithm has two modules: inertial / visual odometry and inertial / LiDAR odometry, which can construct dense 3D maps with color information.

[0102] Three-dimensional LiDAR, a monocular camera, and a body attitude sensor are used to build environmental maps in the robot's virtual planning system, and to perform autonomous localization using environmental information. The fusion of these three components improves the system's robustness and accuracy. In addition to building environmental maps and autonomous localization, the body attitude sensor is also used to perceive the robot's posture information in real time to maintain the robot's stability.

[0103] More specifically, the 3D LiDAR model is Velodyne Puck Hi-Res; the monocular camera model is FLIRBFS-U3-04S2M-CS; and the robot posture sensor uses the MPU6050 chip. This module can simultaneously detect three-axis acceleration, three-axis angular velocity, and module temperature, and the data accuracy meets the requirements of motion control.

[0104] LVI-SAM is adopted as the combined inertial / visual / LiDAR SLAM algorithm. This algorithm has two modules: inertial / visual odometry and inertial / LiDAR odometry, which can construct dense 3D maps with color information.

[0105] The angle sensor inside the servo motor is used to provide feedback on the angle information of each joint of the robot.

[0106] Foot contact sensors provide feedback signals when the robot's foot touches the ground, stopping the foot from falling and adapting to unknown and complex terrain. Specifically, an RP-C resistive thin-film pressure sensor is used as the foot contact sensor.

[0107] The autonomous decision-making module is used to perform three types of adaptation for unknown and complex terrain: robot body height adaptation, robot body posture adaptation, and robot gait planning.

[0108] The autonomous control module receives control commands from the Unity3D host computer and converts these commands into robot actions. Preferably, the autonomous control module uses a Raspberry Pi 4B as the control host.

[0109] In the above system, the robot virtual simulation system and the robot virtual debugging system are used for testing and optimizing the robot physical entity and the robot virtual planning system, respectively, so as to achieve the parallel operation and real-time interaction of the robot virtual monitoring system, the robot virtual planning system and the robot physical entity.

[0110] The real-time monitoring data of the virtual monitoring system can verify the accuracy and reliability of the mechanism models in the virtual simulation system and virtual planning system, and perform iterative optimization and dynamic correction.

[0111] Another typical embodiment of the present invention provides a method for operating the above-mentioned digital twin system for a quadruped robot in an underground coal mine, as described above. Figure 2 The method includes the following steps.

[0112] S1: Construct a digital twin of a multi-functional quadruped robot for underground coal mines in the Unity3D environment within the robot virtual simulation subsystem. The specific process is as follows:

[0113] S101: Design the robot dimensions and create a robot model in modeling software, saving it in a specific format;

[0114] The modeling software used was Solidworks, and the specific format mentioned above was STEP.

[0115] S102: Import the model into the 3D rendering software, adjust the coordinate axes of each component, render the model, save it in a specific format, and import it into the Unity3d robot virtual simulation subsystem.

[0116] Specifically, adjusting the coordinate axes of each component refers to adjusting the local coordinate system of each component of the robot model to an appropriate position to ensure that the model imported into Unity3D does not have any local coordinate axis rotation.

[0117] The 3D rendering software used is 3ds Max, and the specific format mentioned above is FBX.

[0118] S103: Create the robot model in the Unity3d robot virtual simulation subsystem as shown in the appendix. Figure 3The parent-child relationship shown allows you to add Transform components to each part, giving each part basic translation and rotation transformation capabilities. For example... Figure 3 As shown, the robot's body includes hips, thighs, calves, and foot tips numbered 1-4.

[0119] S2: The structure, kinematics model, and gait model of the multi-functional quadruped robot for underground coal mines are verified and optimized in the robot virtual simulation subsystem. The specific process is as follows:

[0120] S201: Establish a kinematic model of a multifunctional quadruped robot in underground coal mines, write a C# script for the kinematic model, and mount it onto the robot's digital twin;

[0121] The kinematic model of the multifunctional quadruped robot in underground coal mines includes the robot's forward kinematic model and the robot's inverse kinematic model.

[0122] Among them, the robot's forward kinematics model specifically refers to calculating the foot position based on the robot's body posture, link length, and rotation angle of each joint.

[0123] Among them, the robot inverse kinematics model specifically refers to calculating the rotation angle of each joint based on the robot's body posture, link length, and foot position;

[0124] The kinematic model C# script includes forward kinematics calculation script, inverse kinematics calculation script, and interpolation calculation script;

[0125] The forward kinematics calculation script incorporates a robot forward kinematics model, which can be used to calculate the robot's foot motion space and determine whether the robot's structure is reasonable. Its flowchart is attached. Figure 4 As shown;

[0126] The inverse kinematics calculation script incorporates the robot's inverse kinematics model, enabling it to calculate the angles of each joint and input the interpolation calculation script. This is the foundation for robot control, and its flowchart is attached. Figure 5 As shown;

[0127] The interpolation calculation script calls the Mathf.Lerp() function in the Update() function. Each frame, it obtains the target joint angle of the robot from the inverse kinematics calculation script and moves the robot from the existing angle to the target angle.

[0128] S202: Perform gait planning for a multi-functional quadruped robot in underground coal mines, write the planning results into a gait planning C# script, and attach it to the robot's digital twin;

[0129] The gait planning of the multi-functional quadruped robot in the coal mine adopts a static gait planning method, which includes robot step sequence planning and robot center of gravity trajectory planning.

[0130] The gait planning C# script takes the robot's desired movement speed, attitude angle, and body height as input and outputs the desired foot coordinates as output.

[0131] S203: The robot digital twin runs in the robot virtual simulation subsystem to test and optimize the robot's structure, kinematic model, and gait.

[0132] S3: Import the optimized digital twin of the multi-functional quadruped robot in the coal mine into the robot virtual planning subsystem, and build the robot's physical entity based on the robot's digital twin.

[0133] S4: Construct a digital twin of a multi-functional quadruped robot for underground coal mines under the Webots environment within the robot virtual debugging subsystem. The specific process is as follows:

[0134] S401: Import the robot model into Webots and create the main nodes of the robot model in the scene tree;

[0135] S402: Add inertial units, position sensors, rotational motors, and touch sensors to the robot model and set appropriate parameters;

[0136] S403: Establish the main controller program and kinematics program for the robot model, and build the interface between the main controller and the inertial unit, position sensor, rotary motor, and contact sensor;

[0137] The programming language for the main controller program and the kinematics program is Python.

[0138] S5: Import offline maps of underground coal mines into the robot virtual planning subsystem and the robot virtual debugging subsystem, respectively;

[0139] The offline coal mine map specifically refers to a complete underground coal mine map pre-constructed based on geological exploration and 3D mapping, used to test the path planning function of the robot's virtual planning subsystem.

[0140] S6: The multi-functional quadruped robot for underground coal mines is debugged in the robot virtual debugging subsystem. The robot path is planned through the robot virtual planning subsystem, and the planning results are executed in the robot virtual debugging subsystem. The robot semi-autonomous control and manual control of the robot are performed through the robot virtual monitoring subsystem. The specific process is as follows:

[0141] S601: Initial pose of the robot digital twin in the synchronous robot virtual planning subsystem and robot virtual debugging subsystem;

[0142] S602: Establish a path planning model for a multi-functional quadruped robot in underground coal mines based on the A* algorithm, write a C# script for path planning, and attach it to the robot's digital twin in the robot virtual planning subsystem.

[0143] S603: Perform robot path planning in the robot virtual planning subsystem and synchronize the planning results to the robot virtual debugging subsystem, and complete the movement according to the path planning results;

[0144] S604: The robot virtual monitoring subsystem enables semi-autonomous and manual control of the digital twin in the robot virtual debugging subsystem, allowing the robot digital twin to complete designated tasks.

[0145] S7: Compare and analyze the planning results of the robot virtual planning subsystem with the execution results of the robot digital twin in the robot virtual debugging subsystem, and optimize the robot virtual planning subsystem until it can accurately plan the path for the robot.

[0146] S8: Place the physical robot entity underground in a coal mine. Through the coordinated operation of the physical robot entity, the robot virtual planning subsystem, and the robot virtual monitoring subsystem, the multi-functional quadruped robot in the coal mine achieves adaptive walking and complex operations underground. The specific process is as follows:

[0147] S801: Install autonomous perception, autonomous decision-making, and autonomous control modules on the physical robot entity to establish a multi-functional quadruped robot software system for underground coal mines. Place the robot in an actual underground coal mine. The robot software system program file structure is attached. Figure 6 As shown;

[0148] S802: The robot's physical entity operates underground in a coal mine. It constructs a map of the environment around the robot in real time through the autonomous perception module and synchronizes the map to the robot's virtual planning subsystem.

[0149] S803: Perform robot path planning in the robot virtual planning subsystem and synchronize the planning results to the autonomous control module of the robot physical entity. Follow the path planning results to complete the walking and autonomously complete repetitive fixed tasks such as inspection.

[0150] S804: The robot physical entity is semi-autonomously and manually controlled through the teleoperation module in the robot virtual monitoring subsystem, enabling the robot physical entity to complete complex tasks with uncertainties, and the robot's real-time pose status and surrounding environment are monitored in the virtual monitoring module.

[0151] S805: Verify the accuracy and reliability of the kinematic model, gait model and path planning model in the virtual simulation system and virtual planning system based on the real-time monitoring data of the virtual monitoring system, and perform iterative optimization and dynamic correction of the model.

Claims

1. A digital twin system for a multifunctional quadruped robot in underground coal mines, comprising a virtual space for the robot and a physical entity of the robot, characterized in that: The robot virtual space includes a robot virtual simulation subsystem, a robot virtual planning subsystem, a robot virtual debugging subsystem, and a robot virtual monitoring subsystem; The robot virtual simulation subsystem was developed using Unity3d and serves as a testing and verification platform for the early stages of robot body design and kinematic modeling. The robot virtual planning subsystem was developed using Unity3d and is used to complete path planning for a multi-functional quadruped robot in a virtual environment. The robot virtual debugging subsystem was developed by Webots and is used for testing and optimizing the robot virtual planning subsystem. The robot virtual monitoring subsystem was developed using Unity3d and includes a robot virtual monitoring module and a teleoperation module. The robot physical entity adopts a quadruped structure and includes an autonomous perception module, an autonomous decision-making module, and an autonomous control module; the robot physical entity and the robot virtual debugging subsystem communicate with the robot virtual monitoring subsystem Unity3d host computer through a wireless serial communication module; The autonomous perception module is used to perceive the robot's own state and the information of its environment. The autonomous decision-making module is used to complete three types of adaptation for unknown and complex terrain: robot body height adaptation, body posture adaptation, and robot gait planning. The autonomous control module is used to receive control commands from the Unity3d host computer and convert the control commands into robot actions.

2. The digital twin system for a multi-functional quadruped robot in underground coal mines according to claim 1, characterized in that: In the robot virtual monitoring subsystem, the virtual monitoring module reads the robot joint angle information sent by the autonomous sensing module through the serial port, breaks down the joint data according to the sending pattern, and assigns the data to the joint angles of the robot's digital twin, realizing three-dimensional visualization monitoring of the robot's real-time operating status. The robot's pose status can be observed in real time in the virtual environment. It can also read the environmental map around the robot sent by the autonomous sensing module and observe the robot's surrounding environment in real time in the virtual environment.

3. The digital twin system for a multi-functional quadruped robot in underground coal mines according to claim 2, characterized in that: In the robot virtual monitoring subsystem, the teleoperation module remotely controls the robot's physical entity or the robot's digital twin in the robot virtual debugging subsystem through the UGUI human-computer interaction panel in Unity3d to complete complex tasks with uncertainties.

4. The digital twin system for a multi-functional quadruped robot in underground coal mines according to claim 1 or 3, characterized in that: The autonomous perception module includes a 3D lidar, a monocular camera, a fuselage attitude sensor, an internal angle sensor of the servo motor, and a foot contact sensor. Three-dimensional LiDAR, monocular camera and body attitude sensor are used to build environmental map in robot virtual planning subsystem, and at the same time use environmental information to perform autonomous localization. The fusion of the three can improve the robustness and accuracy of the system. The internal angle sensor of the servo motor is used to provide feedback on the angle information of each joint of the robot; Foot contact sensors are used to provide feedback signals when the robot's feet touch the ground, causing the feet to stop falling, in order to adapt to unknown and complex terrain.

5. The digital twin system for a multi-functional quadruped robot in underground coal mines according to claim 4, characterized in that: In addition to building environmental maps and autonomous localization, the robot's posture sensor is also used to perceive the robot's posture information in real time in order to maintain the stability of the robot's posture.

6. The operation method of the digital twin system for a multifunctional quadruped robot in underground coal mines as described in claim 5, characterized in that, Includes the following steps: S1: Construct a digital twin of a multi-functional underground coal mine robot in the Unity3D environment within the robot virtual simulation subsystem. The specific process is as follows: S101: Design the robot dimensions and create a robot model in modeling software; S102: Import the robot model into the 3D rendering software, adjust the coordinate axes of each component, render the model, and import it into the robot virtual simulation subsystem; the adjustment of the coordinate axes of each component is to adjust the local coordinate system of each component of the robot model to an appropriate position to ensure that the model imported into Unity3D does not have local coordinate axis rotation. S103: In the robot virtual simulation subsystem, establish parent-child relationships for the robot model and add Transform components to each part to enable each part to have basic translation and rotation transformation capabilities. S2: The robot's structure, kinematic model, and gait model are verified and optimized in the robot virtual simulation subsystem. The specific process is as follows: S201: Establish a kinematic model of a multifunctional quadruped robot in underground coal mines, write a C# script for the kinematic model, and mount it onto the robot's digital twin; S202: Perform gait planning for a multi-functional quadruped robot in underground coal mines, write the planning results into a gait planning C# script, and attach it to the robot's digital twin; S203: The robot digital twin runs in the robot virtual simulation subsystem to test and optimize the robot's structure, kinematic model and gait; S3: Import the optimized digital twin of the multi-functional quadruped robot in the coal mine into the robot virtual planning subsystem, and build the robot's physical entity based on the robot's digital twin; S4: Construct a digital twin of a multi-functional quadruped robot for underground coal mines under the Webots environment within the robot virtual debugging subsystem. The specific process is as follows: S401: Import the robot model into Webots and create the main nodes of the robot model in the scene tree; S402: Add inertial units, position sensors, rotational motors, and touch sensors to the robot model and set appropriate parameters; S403: Establish the main controller program and kinematics program for the robot model, and build the interface between the main controller and the inertial unit, position sensor, rotary motor, and contact sensor; S5: Import offline maps of underground coal mines into the robot virtual planning subsystem and the robot virtual debugging subsystem, respectively; S6: The multi-functional quadruped robot for underground coal mines is debugged in the robot virtual debugging subsystem. The robot path is planned through the robot virtual planning subsystem, and the planning results are executed in the robot virtual debugging subsystem. The robot semi-autonomous control and manual control of the robot are performed through the robot virtual monitoring subsystem. The specific process is as follows: S601: Initial pose of the robot digital twin in the synchronous robot virtual planning subsystem and robot virtual debugging subsystem; S602: Establish based on The algorithm is used to develop a path planning model for a multi-functional quadruped robot in underground coal mines. A C# script for path planning is written and attached to the robot's digital twin in the robot virtual planning subsystem. S603: Perform robot path planning in the robot virtual planning subsystem and synchronize the planning results to the robot virtual debugging subsystem, and complete the movement according to the path planning results; S604: The robot virtual monitoring subsystem enables semi-autonomous and manual control of the digital twin in the robot virtual debugging subsystem, allowing the robot digital twin to complete the specified tasks. S7: Compare and analyze the planning results of the robot virtual planning subsystem with the execution results of the robot digital twin in the robot virtual debugging subsystem, and optimize the robot virtual planning subsystem until the robot virtual planning subsystem can accurately plan the path for the robot. S8: Place the physical robot entity underground in a coal mine. Through the coordinated operation of the physical robot entity, the robot virtual planning subsystem, and the robot virtual monitoring subsystem, the multi-functional quadruped robot in the coal mine achieves adaptive walking and complex operations underground. The specific process is as follows: S801: Install autonomous perception, autonomous decision-making and autonomous control modules on the physical robot entity, establish a multi-functional quadruped robot software system for underground coal mines, and place it in an actual underground coal mine. S802: The robot's physical entity operates underground in a coal mine. It constructs a map of the environment around the robot in real time through the autonomous perception module and synchronizes the map to the robot's virtual planning subsystem. S803: Performs robot path planning in the robot virtual planning subsystem and synchronizes the planning results to the autonomous control module of the robot physical entity. It then completes the walking according to the path planning results and autonomously completes the inspection task. S804: The robot physical entity is semi-autonomously and manually controlled through the teleoperation module in the robot virtual monitoring subsystem, enabling the robot physical entity to complete complex tasks with uncertainties, and the robot's real-time pose status and surrounding environment are monitored in the virtual monitoring module. S805: Verify the accuracy and reliability of the kinematic model, gait model and path planning model in the virtual simulation system and virtual planning system based on the real-time monitoring data of the virtual monitoring system, and perform iterative optimization and dynamic correction of the model.

7. The operation method of the digital twin system for a multifunctional quadruped robot in underground coal mines according to claim 6, characterized in that: In step S201, the kinematic model of the multifunctional quadruped robot in the coal mine includes the robot's forward kinematic model and the robot's inverse kinematic model; The robot's forward kinematics model refers to the calculation of the foot position based on the robot's body posture, link length, and the rotation angle of each joint. The robot inverse kinematics model refers to the calculation of the rotation angle of each joint based on the robot's body posture, link length, and foot position.

8. The operation method of the digital twin system for a multifunctional quadruped robot in underground coal mines according to claim 7, characterized in that: In step S201, the kinematic model C# script includes a forward kinematics calculation script, an inverse kinematics calculation script, and an interpolation calculation script; The forward kinematics model of the robot is incorporated into the forward kinematics calculation script, which can be used to calculate the motion space of the robot's feet and determine whether the robot's structure is reasonable. The inverse kinematics model of the robot is incorporated into the inverse kinematics calculation script, which can calculate the angles of each joint and input the interpolation calculation script, and is the foundation for realizing robot control. The interpolation calculation script calls the Mathf.Lerp() function in the Update() function. Each frame, it obtains the target joint angle of the robot from the inverse kinematics calculation script and moves the robot from the existing angle to the target angle.

9. The operation method of the digital twin system for a multifunctional quadruped robot in underground coal mines according to claim 6 or 8, characterized in that: In step S202, the gait planning of the multi-functional quadruped robot in the coal mine adopts a static gait planning method, which includes robot stepping sequence planning and robot center of gravity trajectory planning. The gait planning C# script takes the robot's desired movement speed, attitude angle, and body height as input and outputs the desired foot coordinates.

10. The operation method of the digital twin system for a multifunctional quadruped robot in underground coal mines according to claim 9, characterized in that: In step S5, the offline coal mine map specifically refers to a complete underground coal mine map pre-constructed based on geological exploration and 3D mapping, which is used to test the path planning function of the robot virtual planning subsystem.

Images