Table tennis robot implementation method and system based on digital twinning
Through digital twin technology, combined with three-dimensional modeling and simulation software, a virtual table tennis robot model was created, which solved the problem of difficulty in capturing ball speed and rotation in the existing technology, realized accurate prediction and real-time control of table tennis trajectory, and improved training Efficiency and skill level.
Patent Information
- Application Number
- CN202410920472.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-10
- Publication Date
- 2024-11-08
AI Technical Summary
Due to technical limitations, the existing table tennis robot training system is difficult to effectively capture the ball speed, position and rotation method, which makes it impossible to clearly determine the relationship between the ball touch method and table tennis movement, and cannot update the technology through data analysis and self-learning, which is limited. Training level.
Using digital twin technology, a virtual table tennis robot model is created through three-dimensional modeling software and simulation software, and real-time monitoring and optimization are carried out in combination with actual scene data, so as to realize the prediction and real-time human manipulation of table tennis trajectory, and the data interaction is used by the MQTT protocol, and the optimization is carried out. Digital twin models to improve training efficiency.
Overcoming the limitations of traditional training methods, the precise prediction and real-time control of the trajectory of table tennis is achieved, the training functions are enriched, and the training efficiency and skill level of table tennis players are improved.
Smart Images

Figure CN118903798A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of digital twin robots, and in particular to a method and system for realizing a table tennis robot based on digital twins. Background Art
[0002] Digital twin is an emerging technology that combines the digital world with the real world. It digitally replicates the physical systems of the real world, collects, analyzes and simulates data in real time for experiments, optimization, monitoring and decision support, and provides great potential for applications in various fields. Digital twin technology brings new possibilities to table tennis training through digital simulation and emulation methods.
[0003] With the development of science and technology and the increasing demand for human-computer interaction training robots, various sports robots that can interact with people for training have appeared at home and abroad. Due to the limitations of robot technology, there is a certain upper limit to the training technology of robots. During the training process, due to the fast ball speed, the speed, position and rotation of table tennis balls during the movement are difficult to capture and study. The lack of detailed data recording and analysis makes it impossible to clearly determine the relationship between the way of touching the ball and the movement of table tennis, which makes it impossible for students and coaches to have a deep and detailed understanding of the weaknesses and improvement opportunities of skills. In addition, the current human-computer training system is usually based on fixed algorithms and programs. The lack of real data in the game makes it impossible for table tennis robots to update their own technology through data analysis and self-learning. Therefore, the upper limit of robot technology is limited and it is impossible to fully simulate the training of some high-level players and unique technologies. Summary of the invention
[0004] In view of the deficiencies in the prior art, the object of the present invention is to provide a table tennis robot system and implementation method based on digital twin technology.
[0005] The present invention aims to apply digital twin technology to table tennis robots to improve robot training technology, create virtual twins corresponding to real training scenarios in virtual space, connect physical entity modules with digital twin model modules through communication modules, and interact with data, and change the corresponding parameters of digital twin models for deduction, continuously optimize the model, and finally achieve a higher level of auxiliary training through the verified model, overcome the limitations of traditional training methods, and improve the training efficiency and skill level of table tennis players. Through digital twin technology, the system will provide table tennis players and coaches with more powerful tools to achieve a higher level of training and competition performance.
[0006] SolidWorks is a 3D modeling software widely used in engineering design and manufacturing. It can efficiently and accurately create, edit and analyze complex 3D models. SolidWorks supports parametric modeling, which can quickly modify models by adjusting dimensions and parameters to improve design flexibility. In the assembly function, it can build and simulate the assembly architecture of multiple parts. In addition, the software also provides detailed engineering drawings and rendering functions, supporting high-fidelity scene modeling.
[0007] Unreal Engine, a powerful game engine, also plays an important role in the field of digital twins. Unreal Engine is feature-rich in the field of digital twins, providing users with powerful tools and resources to achieve highly realistic and interactive virtual environments. Its advanced graphics engine and rendering technology enable it to create highly realistic virtual environments, providing excellent visual effects for digital twin tasks. Unreal Engine supports real-time interaction and simulation, allowing users to simulate and observe various scenarios in digital twin systems to optimize design, testing and training. Its powerful physics engine and multi-platform support also make it an ideal choice for simulating real-world behavior.
[0008] The technical solution of the present invention is:
[0009] A method for implementing a table tennis robot based on digital twins, comprising:
[0010] Complete their respective functions in the actual scenario, i.e., the table tennis robot training scenario, collect the position information of the table tennis ball, as well as the operation status data and control data of the table tennis robot, which include the position state of the table tennis robot in space during its movement, the data of the robot controller, the data of the sensor, the torque and angular acceleration information at each action joint, and the motion trajectory;
[0011] Through 3D modeling software and simulation software, a virtual twin model system of the table tennis robot system corresponding to the physical entity in the actual scene is created according to the data information;
[0012] The actual scene is connected to the digital twin model module for communication and data exchange, and the actual scene is monitored in real time through the digital twin virtual environment;
[0013] Predict the trajectory of table tennis through simulation;
[0014] The posture of the table tennis ball is changed accordingly according to its trajectory, and real-time human control is performed in the digital twin module.
[0015] Preferably, according to the present invention, completing respective functions in an actual scene, collecting position information of a table tennis ball, and operation status data and control data of a table tennis robot, includes:
[0016] Build a table tennis robot training scene in the real world, including: table tennis robot, table tennis equipment and binocular camera;
[0017] In this training scenario, the table tennis robot hits the table tennis ball, and the binocular camera measures the position of the table tennis ball.
[0018] The collected information includes the size, material properties, matching relationship, posture state in space, and position information of the table tennis robot, and collects data of the robot controller, sensor data, and image data of the table tennis ball.
[0019] Preferably, according to the present invention, a virtual twin model system of a table tennis robot system corresponding to a physical entity in an actual scene is created by three-dimensional modeling software and simulation software according to data information; comprising:
[0020] The collected data includes the position information of the ping-pong ball and the operating status data and control data of the ping-pong robot. The scene device is modeled using SolidWorks 3D modeling software, and the obtained model is imported into UnrealEngine to build a virtual environment. The simulation software is then used to set parameters for simulation to create a digital twin model corresponding to the physical entity in the real physical space. The constructed digital twin model is simulated using historical data.
[0021] Further preferably, the historical data is the operating data saved during the operation of the table tennis robot; the historical data is the operating data with a unified data structure and data type after data cleaning and data conversion; the implementation process of historical data acquisition is: installing a corresponding sensor on the device that needs to collect data, transmitting the collected temporary data to a computer, identifying and repairing missing values, outliers and duplicate records for the temporary data, and performing data conversion including normalization and standardization, encoding and mapping, aggregation and merging, discretization, and feature engineering operations, so as to obtain historical data.
[0022] Preferably, according to the present invention, the physical entity module is connected to the digital twin model module for communication, and data is exchanged, and the physical entity module is monitored in real time through the digital twin virtual environment; including:
[0023] MQTT is used to realize data interaction between the actual scene and the digital twin model. Unreal engine is used as the client and the table tennis robot controller is used as the server. The table tennis robot system is decomposed to establish multiple clients, which are connected to the corresponding servers. Each client is responsible for receiving and displaying data from the server; the data includes the posture status of the robot in the space, the position information of the table tennis ball, the data of the robot controller and the data of the sensor.
[0024] Preferably, according to the present invention, prediction of the motion trajectory of a table tennis ball is achieved by simulation, including:
[0025] Import historical data into the digital twin model for deduction, compare the output of the digital twin model with the actual observation data, and analyze the differences; if there are differences, further optimize the digital twin model to finally obtain a verified digital twin model; according to the verification results, iteratively optimize the digital twin model, including: changing the structure of the digital twin model, adjusting parameters, and improving the learning algorithm of the digital twin model;
[0026] Through the real-time deduction of the digital twin model, the table tennis robot can be monitored, including the prediction of the trajectory of the table tennis ball and the real-time response decision of the table tennis robot;
[0027] The method for predicting the trajectory of a table tennis ball is as follows: using a dual-driven method of a physical model and a data-driven model to predict the movement of the table tennis ball;
[0028] The physical model refers to the twinning of the actual table tennis racket, table tennis ball and environment, which enables the prediction of the trajectory of the table tennis ball in the simulation software;
[0029] The data-driven model collects a large amount of table tennis trajectory data and uses a machine learning algorithm to train the data-driven model; the data-driven model learns the movement patterns of the table tennis ball and uses it to predict future trajectories; the digital twin model completes the response decisions of the robotic arm based on the predicted trajectory, and transmits the instructions to the robotic arm controller through the communication protocol to achieve motion control of the robotic arm.
[0030] Preferably, according to the present invention, the posture change is performed accordingly according to the motion trajectory of the table tennis ball and real-time human control is performed in the digital twin module, including:
[0031] Through the visualization technology of digital twins, the human-computer interaction system interface is established and the robot's movement is mapped in real time; using the Web development platform, Vue.js is selected as the front-end framework, and HTML, CSS and JavaScript are used to design the user interface. HTML is used to define the page structure, CSS is used for style design, and JavaScript is used to achieve interaction and dynamic effects. The CSS framework is used to support responsive design to ensure that the user interface can adapt to different devices and screen sizes. The state management tool Vuex is used to manage the state and data flow between components, and Material-UI is used to develop UI components. The GUI control panel is designed to display the various data of the table tennis robot system in real time and switch the table tennis robot operation mode, and finally realize the setting of the system interface.
[0032] Further preferably, the specific implementation method is: by connecting the computer to the actual robotic arm, using sensors to provide feedback, and utilizing the built interface to realize the use of a computer to control the precise movement of the robotic arm; calculating the joint angles through an inverse kinematics algorithm, transmitting instructions to the robotic arm controller through a communication protocol, thereby realizing motion control of the robotic arm.
[0033] A table tennis robot implementation system based on digital twins includes a physical entity module, a digital twin model module, a communication module, a table tennis trajectory prediction module, and a control module. The communication module is an important medium for realizing the combination of virtual and real.
[0034] The physical entity module is used to: complete the respective functions in the actual scene, collect the position information of the table tennis ball, and the operation status data and control data of the table tennis robot;
[0035] The digital twin model module is used to: create a virtual twin model system of the table tennis robot system corresponding to the physical entity in the actual scene according to the data information through three-dimensional modeling software and simulation software;
[0036] The communication module is used to: communicate and connect the actual scene with the digital twin model module, and exchange data, and monitor the actual scene in real time through the digital twin virtual environment;
[0037] The table tennis ball trajectory prediction module is used to: measure the three-dimensional coordinates of the table tennis ball using a binocular camera; by simultaneously capturing the images of the table tennis ball in two cameras, and based on the parallax information between the images, calculate the three-dimensional coordinates of the table tennis ball;
[0038] The control module is used to: perform corresponding posture changes according to the motion trajectory of the table tennis ball and perform real-time human control in the digital twin module;
[0039] Preferably, according to the present invention, the table tennis robot implementation system also includes a data storage module for storing the position information of the table tennis ball, the operating status data and control data of the table tennis robot, and real-time data deduced by the digital twin model, and storing the data of the robot's operating status, motor, sensor, and each joint angle, angular velocity, angular acceleration, and joint torque.
[0040] Further preferably, in the data storage module, a MySQL database system is selected, data tables are designed to store sensor data, robot configuration and user information, data acquisition and storage codes are written, data access interfaces are provided to support query and retrieval, data is backed up regularly and operation logs are recorded.
[0041] Preferably, according to the present invention, the table tennis robot implementation system also includes a visualization module, including a visualization display panel of a real-time monitoring system, an operation panel of a control module, various data information in the actual scene, table tennis ball position information, table tennis ball predicted trajectory and robot posture state data.
[0042] The beneficial effects of the present invention are:
[0043] The present invention uses digital twin technology to highly simulate and emulate the physical system of the real world. By twinning the actual table tennis robot training scene, it overcomes the limitations of traditional training methods that are inconvenient to record the training process and the upper limit of the robot's training level. It optimizes autonomous decision-making and realizes training monitoring through the deduction ability of the digital twin model. It allows real-time data collection, simulation prediction and remote control, enriches and improves the training function of the table tennis robot, and improves the training efficiency and skill level of table tennis players. Through digital twin technology, the system will provide table tennis players and coaches with more powerful tools to achieve higher levels of training and competition performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 This is a flow chart of a method for implementing a table tennis robot based on digital twins according to an embodiment of the present invention.
[0045] Figure 2 This is a schematic diagram of the overall architecture of a table tennis robot system based on digital twins according to an embodiment of the present invention.
[0046] Figure 3 This is a schematic diagram of the construction framework of a table tennis robot system based on digital twins according to an embodiment of the present invention. DETAILED DESCRIPTION
[0047] The present invention will be further defined below in conjunction with the accompanying drawings and embodiments, but is not limited thereto.
[0048] Example 1
[0049] A method for implementing a table tennis robot based on digital twins, such as Figure 1 As shown, including:
[0050] S1, complete their respective functions in the actual scene, i.e., the table tennis robot training scene, collect the position information of the table tennis ball, as well as the operation status data and control data of the table tennis robot, the operation status data and control data of the table tennis robot include the position state of the table tennis robot in space during the movement, the data of the robot controller, the data of the sensor, the torque and angular acceleration information at each action joint and the movement trajectory;
[0051] S2, using 3D modeling software and simulation software, based on data information, creates a virtual twin model system of the table tennis robot system corresponding to the physical entity in the actual scene;
[0052] S3, connects the actual scene with the digital twin model module and exchanges data, and monitors the actual scene in real time through the digital twin virtual environment;
[0053] S4, prediction of table tennis trajectory through simulation;
[0054] S5, performs corresponding posture changes according to the movement trajectory of the table tennis ball and performs real-time human control in the digital twin module.
[0055] Example 2
[0056] The method for implementing a table tennis robot based on digital twins according to Embodiment 1 is different in that:
[0057] Complete their respective functions in the actual scene, collect the position information of the table tennis ball, as well as the operation status data and control data of the table tennis robot, including:
[0058] Build a table tennis robot training scene in the real world, including: table tennis robot, table tennis equipment and binocular camera;
[0059] In this training scenario, the table tennis robot hits the table tennis ball, and the binocular camera measures the position of the table tennis ball.
[0060] The principle of using binocular cameras to measure three-dimensional coordinates is to simultaneously capture the images of the table tennis ball in two cameras, and then calculate the three-dimensional position coordinates of the ball based on the parallax information between the images. This process includes camera calibration, feature matching, parallax calculation and three-dimensional reconstruction, so as to achieve accurate table tennis position measurement and tracking. The collected information includes the size, material properties, matching relationship, posture state in space, and position information of the table tennis robot, and collects data from the robot controller, sensor data, and image data of the table tennis ball.
[0061] The size and material properties of the table tennis robot refer to the size and material of the parts, and the parts of the robot are modeled based on these. The matching relationship refers to the matching setting between these parts. By modeling the parts based on the matching, high-fidelity modeling of the table tennis robot can be achieved; the position state of the robot in space refers to the position and posture of the joints at each degree of freedom when completing a certain action. The position information of the table tennis ball refers to the three-dimensional coordinates of the table tennis ball during the operation after completing a certain hitting action. This task is achieved through the binocular camera method; the robot controller records the output signals and joint angles of the joint motors, receives, processes and integrates information from various sensors, and plans the robot's motion path based on the status of the robot and the table tennis ball to achieve corresponding decisions; the image data of the table tennis ball during the human-machine training process is used to train the deep learning model to recognize the table tennis ball in the actual scene.
[0062] Through 3D modeling software and simulation software, based on data information, a virtual twin model system of the table tennis robot system corresponding to the physical entity in the actual scene is created; including:
[0063] The collected data includes the location information of the table tennis ball and the running status data and control data of the table tennis robot. The scene device is modeled using solidworks 3D modeling software, and the obtained model is imported into Unrealengine to build a virtual environment. Then the simulation software is used to set parameters for simulation to create a digital twin model corresponding to the physical entity in the real physical space. The method for creating the digital twin model includes: according to the size, material properties, and matching relationship of the table tennis robot, the table tennis ball and the real environment are modeled with high fidelity using solidworks, and the material properties of each component of the table tennis robot are set. First, the components are modeled, and then the robot is assembled according to the actual matching relationship. After the material properties and sizes of the racket of the robot end effector and the table tennis ball are defined according to the real definition, the real hitting action can be simulated. Secondly, the environment is twinned according to the training environment, including the table tennis table and the temperature field; the constructed digital twin model is simulated through historical data. The specific verification can be carried out through the historical data saved in the data storage module.
[0064] Historical data is the operation data saved during the operation of the table tennis robot; historical data is the operation data with unified data structure and data type after data cleaning and data conversion; the implementation process of historical data acquisition is: install the corresponding sensor on the device that needs to collect data, transmit the collected temporary data to the computer, identify and repair missing values, abnormal values and duplicate records of the temporary data, and perform data conversion including normalization and standardization, encoding and mapping, aggregation and merging, discretization, and feature engineering operations to obtain historical data. To ensure data quality, consistency and applicability, and provide a reliable foundation for subsequent data analysis and modeling.
[0065] The physical entity module is connected to the digital twin model module for communication and data exchange, and the physical entity module is monitored in real time through the digital twin virtual environment; including:
[0066] MQTT is used to realize data interaction between the actual scene and the digital twin model. Unreal engine is used as the client and the table tennis robot controller is used as the server. The table tennis robot system is decomposed to establish multiple clients, which are connected to the corresponding servers. Each client is responsible for receiving and displaying data from the server; the data includes the posture status of the robot in the space, the position information of the table tennis ball, the data of the robot controller and the data of the sensor.
[0067] The table tennis robot system includes a robot device, a control system, a sensor and a user interface. The robot device is the entity of the robot, which realizes the movement of each joint under the control of the robot control system, and the sensor realizes the collection of force data and speed data.
[0068] In Unreal regine, C++ and blueprints are used together for UI programming. C++ is used to write high-performance low-level UI components and functions, while blueprints are used to create the visual part of the user interface, interaction logic, and event processing. Utilize the MQTT protocol, which is a lightweight, publish-subscribe communication protocol for data transmission between devices in the Internet of Things (IoT) and distributed systems. MQTT is simple in design and has a small protocol header, making it suitable for devices with limited resources and low-bandwidth networks.
[0069] The prediction of table tennis trajectory is achieved through simulation, including:
[0070] Import historical data into the digital twin model for deduction, compare the output of the digital twin model with the actual observation data, and analyze the differences; if there are differences, further optimize the digital twin model to finally obtain a verified digital twin model; make the model simulate the real scene as much as possible to achieve consistency between virtual and real. According to the verification results, iteratively optimize the digital twin model, including: changing the structure of the digital twin model, adjusting parameters, and improving the learning algorithm of the digital twin model; in actual applications, continuously collect and update real-time data through sensor networks, data acquisition systems, and communication modules to continuously improve the accuracy and realism of the digital twin model.
[0071] Through the real-time deduction of the digital twin model, the table tennis robot can be monitored, including the prediction of the trajectory of the table tennis ball and the real-time response decision of the table tennis robot;
[0072] The method for predicting the trajectory of a table tennis ball is as follows: using a dual-driven method of a physical model and a data-driven model to predict the movement of the table tennis ball;
[0073] The physical model refers to the twinning of the actual table tennis racket, table tennis ball and environment, and the prediction of the trajectory of the table tennis ball is realized in the simulation software; the specific implementation process is: select the PhysX physics engine of Unreal Engine to create physical objects of table tennis, table tennis table and table tennis racket in the simulation environment. These physical objects will simulate the movement and collision of real objects. Define material properties for table tennis, table tennis table and table tennis racket, including elasticity and friction. These properties will affect the collision response between objects. Use the collision detection function provided by the physics engine to detect the collision between the table tennis ball and table tennis racket, and between the table tennis ball and table tennis table. When a collision occurs, the physics engine calculates the force and direction of the collision. After the collision is detected, simulate the movement of the table tennis ball according to the force and direction of the collision. This involves adjusting the speed and rotation parameters of the table tennis ball. Use the motion tracking function provided by the physics engine to track the position of the table tennis ball in real time and draw the trajectory. Complete the setting of air resistance and gravity parameters according to the environment to adjust the authenticity of the simulation.
[0074] The data-driven model collects a large amount of table tennis trajectory data and uses machine learning algorithms (such as regression, neural networks, etc.) to train the data-driven model; the data-driven model learns the movement patterns of table tennis and uses them to predict future trajectories; the digital twin model completes the response decisions of the robotic arm based on the predicted trajectory, and transmits the instructions to the robotic arm controller through the communication protocol to realize the motion control of the robotic arm.
[0075] According to the trajectory of the table tennis ball, the corresponding posture changes are carried out and real-time human control is performed in the digital twin module, including:
[0076] Through the visualization technology of digital twins, the human-computer interaction system interface is established and the robot's movement is mapped in real time; using the Web development platform, Vue.js is selected as the front-end framework, and HTML, CSS and JavaScript are used to design the user interface. HTML is used to define the page structure, CSS is used for style design, and JavaScript is used to achieve interaction and dynamic effects. The CSS framework is used to support responsive design to ensure that the user interface can adapt to different devices and screen sizes. The state management tool Vuex is used to manage the state and data flow between components, and Material-UI is used to develop UI components. The GUI control panel is designed to display the various data of the table tennis robot system in real time and switch the table tennis robot operation mode, and finally realize the setting of the system interface.
[0077] The specific implementation method is: by connecting the computer to the actual robotic arm, using sensors to provide feedback, and using the built interface to use the computer to control the precise movement of the robotic arm; calculating the joint angles through the inverse kinematics algorithm, and transmitting the instructions to the robotic arm controller through the communication protocol to achieve motion control of the robotic arm.
[0078] Example 3
[0079] A table tennis robot implementation system based on digital twins, such as Figure 2 As shown, it includes a physical entity module, a digital twin model module, a communication module, a table tennis trajectory prediction module, and a control module. The communication module is an important medium for realizing the combination of virtual and real.
[0080] The physical entity module is used to: complete the respective functions in the actual scene, collect the position information of the table tennis ball, and the operation status data and control data of the table tennis robot;
[0081] The digital twin model module is used to: create a virtual twin model system of the table tennis robot system corresponding to the physical entity in the actual scene based on data information through three-dimensional modeling software and simulation software;
[0082] Twin space is a digital mapping of the real scene, containing all the information of various devices in the real scene. The creation of virtual scenes usually starts with 3D modeling software such as SolidWords and 3DsMax to model and design the physical system or scene to be simulated, then import the collection model into the virtual world, and add appropriate text to the scene to make it look more like the real world. Next, the modeling of the physical behavior of objects and systems is completed. This includes using a physics engine to simulate physical effects such as collision, gravity, and movement between objects. This can be achieved using physics engines such as Unreal Engine's PhysX and Bullet. In order to realize the interaction between the digital twin model and the real world, it is also necessary to simulate the behavior of sensors. This can be achieved through virtual sensors, simulating the working mode and data output of sensors. The data collected by the actual scene is transmitted to the virtual scene to realize the driving operation of the array twin model module, compare the actual data, adjust the model parameters and algorithms, so as to improve the accuracy and reliability of the model and achieve consistent movement with the real workstation.
[0083] The communication module is used to: connect the actual scene with the digital twin model module for communication and data exchange, and monitor the actual scene in real time through the digital twin virtual environment; select the MQTT protocol to realize data transmission and control command interaction between the virtual scene and the real scene, decompose the table tennis robot system to establish multiple clients, connect to the corresponding server, and each client is responsible for receiving and displaying data from the server.
[0084] The table tennis ball trajectory prediction module is used to: measure the three-dimensional coordinates of the table tennis ball using a binocular camera; by simultaneously capturing the images of the table tennis ball in two cameras, and based on the parallax information between the images, calculate the three-dimensional coordinates of the table tennis ball;
[0085] The control module is used to: perform corresponding posture changes according to the motion trajectory of the table tennis ball and perform real-time human control in the digital twin module; by connecting the computer to the actual robotic arm, using sensors (such as encoders and laser rangefinders) to provide feedback, and using control software, the computer can be used to control the precise movement of the robotic arm. The joint angle is calculated through the inverse kinematics algorithm, and then the command is transmitted to the robotic arm controller through the communication protocol to realize the motion control of the robotic arm.
[0086] The table tennis robot implementation system also includes a data storage module, which is used to store the position information of the table tennis ball, the operating status data and control data of the table tennis robot, and the real-time data deduced by the digital twin model, and to store the robot's operating status, motor, sensor, and data on each joint angle, angular velocity, angular acceleration, and joint torque.
[0087] In the data storage module, the MySQL database system is selected, data tables are designed to store sensor data, robot configuration and user information, data acquisition and storage codes are written, data access interfaces are provided to support query and retrieval, security, real-time and scalability are ensured, data is backed up regularly and operation logs are recorded, so as to effectively manage and utilize data related to robot operation and performance.
[0088] The table tennis robot implementation system also includes a visualization module, including a visualization display panel of the real-time monitoring system, an operation panel of the control module, data information of various parts in the actual scene, table tennis position information, table tennis predicted trajectory and robot posture state data. Select appropriate visualization tools and techniques, and then visualize the data obtained from the robot sensor, control unit and simulator in the form of graphics, animation or virtual reality on the user interface to reflect the real-time state and motion trajectory of the robot, while providing user interaction functions such as controlling robot actions, adjusting simulation parameters or recording and replaying table tennis games, so as to achieve the purpose of real-time monitoring of robot performance and simulation training.
[0089] Specifically, Figure 3 This is a schematic diagram of the framework of a table tennis robot system based on digital twins in this manual, including:
[0090] Physical entity level: The goal of the table tennis robot is to simulate the movements of a human table tennis player, so that it can play table tennis with human players or provide practice opportunities. This includes the physical structure of the table tennis robot, size and assembly relationship, controller data collection, etc. The physical structure specifically includes the robotic arm, racket device, control unit, sensor system, power supply system, etc.
[0091] Twin space level: Import the device model into Unreal Engine to build a virtual table tennis robot training scene, perform virtual-to-real mapping of the position, geometric dimensions, material properties, assembly relationships, and forward and inverse kinematic characteristics of the table tennis robot, import the collected data into the digital twin model for deduction and simulation, compare and analyze the deduction results with the real-world data, continuously optimize the model, and improve the accuracy and reliability of the model by adjusting model parameters and algorithms.
[0092] Data transmission level: Connect the physical entity module with the digital twin model module, and exchange data, and monitor the physical entity module in real time through the digital twin virtual environment. Select the MQTT protocol to realize the data transmission and control command interaction between the virtual scene and the real scene, decompose the table tennis robot system to establish multiple clients, connect to the corresponding server, and each client is responsible for receiving and displaying the data from the server.
[0093] The invention is beneficial in that it uses digital twin technology to highly simulate and emulate the physical system of the real world. By twinning the actual table tennis robot training scene, it overcomes the limitations of traditional training methods that are inconvenient to record the training process and the upper limit of the robot's training level, and optimizes autonomous decision-making and realizes training monitoring through the deduction ability of the digital twin model. It allows real-time data collection, simulation prediction and remote control, enriches and improves the training function of the table tennis robot, and improves the training efficiency and skill level of table tennis players. Through digital twin technology, the system will provide table tennis players and coaches with more powerful tools to achieve higher levels of training and competition performance.
Claims
1. A method for implementing a table tennis robot based on digital twins, characterized in that: include: Complete their respective functions in the actual scenario, i.e., the table tennis robot training scenario, collect the position information of the table tennis ball, as well as the operation status data and control data of the table tennis robot, which include the position state of the table tennis robot in space during its movement, the data of the robot controller, the data of the sensor, the torque and angular acceleration information at each action joint, and the motion trajectory; Through 3D modeling software and simulation software, a virtual twin model system of the table tennis robot system corresponding to the physical entity in the actual scene is created according to the data information; The actual scene is connected to the digital twin model module for communication and data exchange, and the actual scene is monitored in real time through the digital twin virtual environment; Predict the trajectory of table tennis through simulation; The posture of the table tennis ball is changed accordingly according to its trajectory, and real-time human control is performed in the digital twin module.
2. The method for implementing a table tennis robot based on digital twin according to claim 1, characterized in that: Complete their respective functions in the actual scene, collect the position information of the table tennis ball, as well as the operation status data and control data of the table tennis robot, including: Build a table tennis robot training scene in the real world, including: table tennis robot, table tennis equipment and binocular camera; In this training scenario, the table tennis robot hits the table tennis ball, and the binocular camera measures the position of the table tennis ball. The collected information includes the size, material properties, matching relationship, posture state in space, and position information of the table tennis robot, and collects data of the robot controller, sensor data, and image data of the table tennis ball.
3. The method for implementing a table tennis robot based on digital twin according to claim 1, characterized in that: Through 3D modeling software and simulation software, based on data information, a virtual twin model system of the table tennis robot system corresponding to the physical entity in the actual scene is created; including: The collected data includes the location information of the table tennis ball and the operation status data and control data of the table tennis robot. The scene device is modeled using solidworks 3D modeling software, and the obtained model is imported into Unreal Engine to build a virtual environment. Then the simulation software is used to set parameters for simulation to create a digital twin model corresponding to the physical entity in the real physical space; the constructed digital twin model is simulated using historical data; Further preferably, the historical data is the operating data saved during the operation of the table tennis robot; the historical data is the operating data with a unified data structure and data type after data cleaning and data conversion; the implementation process of historical data acquisition is: installing a corresponding sensor on the device that needs to collect data, transmitting the collected temporary data to a computer, identifying and repairing missing values, outliers and duplicate records for the temporary data, and performing data conversion including normalization and standardization, encoding and mapping, aggregation and merging, discretization, and feature engineering operations, so as to obtain historical data.
4. The method for implementing a table tennis robot based on digital twin according to claim 1, characterized in that: The physical entity module is connected to the digital twin model module for communication and data exchange, and the physical entity module is monitored in real time through the digital twin virtual environment; include: MQTT is used to realize data interaction between the actual scene and the digital twin model. Unreal engine is used as the client and the table tennis robot controller is used as the server. The table tennis robot system is decomposed to establish multiple clients, which are connected to the corresponding servers. Each client is responsible for receiving and displaying data from the server; the data includes the posture status of the robot in the space, the position information of the table tennis ball, the data of the robot controller and the data of the sensor.
5. The method for realizing a table tennis robot based on digital twin according to claim 1, characterized in that: The prediction of table tennis trajectory is achieved through simulation, including: Import historical data into the digital twin model for deduction, compare the output of the digital twin model with the actual observation data, and analyze the differences; if there are differences, further optimize the digital twin model to finally obtain a verified digital twin model; according to the verification results, iteratively optimize the digital twin model, including: changing the structure of the digital twin model, adjusting parameters, and improving the learning algorithm of the digital twin model; Through the real-time deduction of the digital twin model, the table tennis robot can be monitored, including the prediction of the trajectory of the table tennis ball and the real-time response decision of the table tennis robot; The method for predicting the trajectory of a table tennis ball is as follows: using a dual-driven method of a physical model and a data-driven model to predict the movement of the table tennis ball; The physical model refers to the twinning of the actual table tennis racket, table tennis ball and environment, which enables the prediction of the trajectory of the table tennis ball in the simulation software; The data-driven model collects a large amount of table tennis trajectory data and uses a machine learning algorithm to train the data-driven model; the data-driven model learns the movement patterns of the table tennis ball and uses it to predict future trajectories; the digital twin model completes the response decisions of the robotic arm based on the predicted trajectory, and transmits the instructions to the robotic arm controller through the communication protocol to achieve motion control of the robotic arm.
6. The method for realizing a table tennis robot based on digital twin according to claim 1, characterized in that: According to the trajectory of the table tennis ball, the corresponding posture changes are carried out and real-time human control is performed in the digital twin module, including: The human-computer interaction system interface is established through the visualization technology of digital twins and the movement of the robot is mapped in real time. Using the Web development platform, Vue.js is selected as the front-end framework, and the user interface is designed using HTML, CSS and JavaScript.
7. The method for implementing a table tennis robot based on digital twin according to claim 6, characterized in that: The specific implementation method is: by connecting the computer to the actual robotic arm, using sensors to provide feedback, and using the built interface to use the computer to control the precise movement of the robotic arm; calculating the joint angles through the inverse kinematics algorithm, and transmitting the instructions to the robotic arm controller through the communication protocol to achieve motion control of the robotic arm.
8. A table tennis robot implementation system based on digital twin, characterized in that: It includes physical entity module, digital twin model module, communication module, table tennis trajectory prediction module and control module; The physical entity module is used to: complete the respective functions in the actual scene, collect the position information of the table tennis ball, and the operation status data and control data of the table tennis robot; The digital twin model module is used to: create a virtual twin model system of the table tennis robot system corresponding to the physical entity in the actual scene according to the data information through three-dimensional modeling software and simulation software; The communication module is used to: communicate and connect the actual scene with the digital twin model module, and exchange data, and monitor the actual scene in real time through the digital twin virtual environment; The table tennis ball trajectory prediction module is used to: measure the three-dimensional coordinates of the table tennis ball using a binocular camera; by simultaneously capturing the images of the table tennis ball in two cameras, and based on the parallax information between the images, calculate the three-dimensional coordinates of the table tennis ball; The control module is used to: perform corresponding posture changes according to the movement trajectory of the table tennis ball and perform real-time human control in the digital twin module.
9. A table tennis robot implementation system based on digital twin according to claim 8, characterized in that: The table tennis robot implementation system also includes a data storage module for storing the position information of the table tennis ball, the operation status data and control data of the table tennis robot, and the real-time data derived from the digital twin model, and storing the robot operation status, motor, sensor, and data of each joint angle, angular velocity, angular acceleration, and joint torque; Further preferably, in the data storage module, a MySQL database system is selected, data tables are designed to store sensor data, robot configuration and user information, data acquisition and storage codes are written, data access interfaces are provided to support query and retrieval, data is backed up regularly and operation logs are recorded.
10. A table tennis robot implementation system based on digital twin according to claim 8, characterized in that: The table tennis robot implementation system also includes a visualization module, including a visualization display panel of a real-time monitoring system, an operation panel of a control module, various data information in an actual scene, table tennis ball position information, table tennis ball predicted trajectory, and robot posture state data.
Citation Information
Patent Citations
Table tennis table with table tennis track recognition and statistics functions
CN116808548A
Table tennis VR simulation training system and device and safety behavior tracking optimization method
CN117504257A
Table tennis scoring device
US20180243634A1
Generating simulation scenarios and digital media using digital twins
US20230334773A1
Cited By
Data processing method and system based on game platform
CN119939486A
