Robot control, monitoring and simulation system
Through ROS communication middleware and integrated interface, the existing robot control software is solved inadequate flexibility and high cost problems, and a cross-platform, low-cost robot control and simulation system is realized, improving operational convenience and monitoring efficiency.
Patent Information
- Application Number
- CN202510708741.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-29
- Publication Date
- 2025-08-29
AI Technical Summary
The existing robot control and simulation software has problems such as insufficient flexibility, not supporting a general ROS framework, not supporting Linux, limitations on offline programming, high costs, inconsistent interface, lack of robot-specific programming and simulation functions, and insufficient kinematic planning.
The ROS communication middleware is used to realize cross-platform, open source, and modular communication of the robot system, integrate simulation modules, control modules, data monitoring modules and point table modules, supports Linux system, connects to the actuator through USB serial port and ROS communication middleware, provides three-dimensional interaction, monitoring synchronization and simulation preview functions, uses URDF files to describe the robot structure model, generate control strategies, integrates QT and Rviz visualization platforms, provides integrated interfaces and flexible control methods.
It improves the efficiency of robot control and task orchestration, reduces the operating threshold, enhances the flexibility and scalability of the system, reduces costs, realizes real-time monitoring and data analysis, and improves operational convenience and monitoring efficiency.
Smart Images
Figure CN120552094A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of software and information technology, and in particular to a robot control, monitoring and simulation system. Background Art
[0002] With the development of industrial automation and intelligent manufacturing, robots are increasingly used in production and service fields. As a complex mechatronic system, robots are usually composed of hardware (such as robotic arms, sensors, actuators) and software (such as control systems, host computer software). Among them, the host computer software, as the core module for human-computer interaction and function realization, undertakes key tasks such as robot control, task programming, and status monitoring; the existing technology solves the problem by using a simulation and programming tool called Fanuc Roboguide, which is powerful and widely used in Fanuc robot programming and simulation; vofa-plus is a software focusing on industrial control and data acquisition, often used for embedded system debugging and real-time monitoring; in addition, ABB RobotStudio is a comprehensive robot simulation and programming tool, widely used in the configuration and debugging of ABB robots in the industrial field; The above prior art has the following problems: Existing technical issues with Fanuc Roboguide and ABB RobotStudio include: 1. Lack of flexibility: They do not support the universal ROS (Robot Operating System) framework and cannot be seamlessly integrated with non-Fanuc or ABB ecosystems; 2. They do not support Linux and cannot be used under Linux systems; 3. Offline programming limitations: Although offline programming is supported, its functionality is biased towards the hardware characteristics of Fanuc and ABB robots, making it difficult to flexibly adapt to the robotic arm structure of custom robots; 4. High cost: The software is relatively expensive. Existing technical issues with vofa-plus include: 1. Single-purpose positioning: Primarily used for signal monitoring and debugging, it lacks robot-specific programming, simulation, and kinematic planning capabilities; 2. Unsuitable for complex control scenarios: Insufficient support for kinematic modeling and path planning for multi-degree-of-freedom robots, making it unable to meet the demands of complex robotic tasks; 3. Inconsistent interface: The integration with the robot system is relatively loose, requiring users to perform additional development work to achieve efficient integration. Based on the above technical problems, we propose a robot control, monitoring and simulation system. Summary of the Invention
[0003] The main purpose of the present invention is to provide a robot control, monitoring and simulation system that can effectively solve the problems in the background technology.
[0004] To achieve the above object, the technical solution adopted by the present invention is: A robot control, monitoring, and simulation system includes a host computer and actuators, wherein the host computer is connected to the actuators via ROS communication middleware. ROS communication middleware is a distributed communication framework for robot software development that provides communication, coordination, and resource management functions for different nodes in the robot system. It is cross-platform, open source, and modular. The host computer includes a simulation module, a control module, a data monitoring module, and a point table module. The control module and data monitoring module are connected to the actuator via ROS communication middleware. The point table orchestration module has two connection methods: one is to connect to the actuator via ROS communication middleware, and the other is to connect to the actuator via a USB serial port.
[0005] The actuator includes a USB serial communication module, a data acquisition module, and an embedded control module. The data acquisition module and the joint servo module collect robot data and control the robot through the USB serial communication module, and report data and receive control instructions from the host computer through the ROS communication middleware. The simulation module includes three-dimensional interaction, monitoring synchronization and simulation preview functions. It is developed based on Linux, Rviz visualization platform, QT framework and ROS robot operating system, and uses Rviz to realize three-dimensional interaction, simulation preview and monitoring of the robotic arm. The three-dimensional interaction function refers to supporting users to affect the robot posture through mouse interaction in a three-dimensional view. The simulation preview function is used to preview the execution process of the point table in real time in the Rviz environment, including the overall simulation execution and the preview of single-point targets. The monitoring synchronization function refers to synchronizing the real robot data collected in real time to the simulation module to render the same posture; The robot modeling method uses a URDF file to describe the robot structure model, including the parameter configuration of multiple links and joints; the URDF model is verified by a simulation tool, and a control strategy is generated based on the model to realize robot motion control.
[0006] Preferably, the control module uses the Rviz visualization tool to achieve three-dimensional interaction, providing two modes: joint control and Cartesian control. Joint control achieves the target posture by adjusting the angles of each joint, while Cartesian control operates by directly setting the spatial position and posture of the end effector. Users set the target posture through an interface developed based on QT, and a smooth motion trajectory is generated by combining the planning control algorithm and the velocity interpolation algorithm. Finally, users can select the control method and preview the motion process in real time through the Rviz visualization tool, or send instructions to the actuator through the ROS communication middleware to complete the actual operation task.
[0007] Preferably, the data monitoring module is based on QT and the ROS robot operating system, and utilizes the ROS communication middleware to obtain data collected by the data acquisition module in the actuator. This data includes joint values, joint speeds, Cartesian coordinate values of the robotic arm, joint power-on status, and brake status. The system displays this information to the user through a user interface (UI) to facilitate inspection and abnormal recovery. At the same time, the collected data is synchronized in real time to the Rviz visualization tool, which renders the robot's three-dimensional model posture based on the joint angle information, thereby intuitively displaying the robot's motion state and spatial position. This process enables efficient monitoring of the robot's trajectory, posture changes, and joint status. Among them, the ROS middleware relies on wireless communication technology (such as Wi-Fi), so it is necessary to ensure that the host computer for human-machine interaction and the actuator are in the same local area network.
[0008] Preferably, the point table module includes the following functions: point table compilation, import and export, and point table execution. The point table is composed of multiple point sets with a hierarchical nested relationship, and each point set contains the following parameters: sequence number, description, joint angle, motion type, speed, acceleration, and dwell time.
[0009] Preferably, the point table arrangement is used to systematically add and re-arrange the point table, and its characteristics include three methods: users obtaining robot posture in three-dimensional interaction in the simulation module, manually inputting robot posture and monitoring real machine posture to assist users in determining the content of the point table, thereby reducing the difficulty of point table arrangement.
[0010] Preferably, the point table control function includes: a loop execution function, which loops through each point in the point table according to the hierarchical nesting relationship of the point table; a single-point execution mode, which allows the independent execution of any point in the point table. The point table control modes include: an online control mode, which controls and operates the robot in real time based on the point table; and an offline control mode, which simulates the point table control process in a simulation environment. The point table layout module implements user interface interaction through QT and implements robot model motion simulation and execution preview through Rviz.
[0011] Preferably, the USB serial communication module is designed to enable data transmission between the actuator and the embedded device (robot), enabling efficient and stable data transmission. This module connects to the embedded device via a standardized USB interface, acting as a bridge for data exchange. The data acquisition module is used to obtain the actuator's operating status data, including but not limited to joint position, velocity, acceleration, current, and other information. This module collects data from the embedded device (robot) via the USB serial communication module.
[0012] Preferably, the embedded control module is used to precisely control the movement of the robot joints, and can drive the embedded device to perform precise positioning, speed and torque control according to the control instructions sent by the USB serial communication module.
[0013] Compared with the prior art, the present invention has the following beneficial effects: 1. In the present invention, the efficiency of robot control and task scheduling is improved: Integrated interface: robot control, task programming, motion simulation, status monitoring and data analysis functions are integrated into a unified interface, simplifying the user's operation process.
[0014] 2. High flexibility and low cost: Built on the ROS ecosystem, the system is highly flexible and scalable. Developers can easily add new features or replace hardware components as needed, reducing system update and maintenance costs. Its open source nature and modular design further reduce initial investment and long-term maintenance costs.
[0015] 3. This invention improves monitoring and troubleshooting efficiency: The system synchronizes the robot's status in real time and provides rapid operational feedback through data analysis. Users can view the robot's joints, motion paths, and status at any time, identifying problems and making adjustments promptly.
[0016] 4. This invention simplifies robot operation: The system provides an intuitive UI panel, lowering the technical barriers to robot control. Through a simple graphical interface, users can perform task scheduling, robot control, and monitoring without a deep programming background, expanding the popularity and application of robotics technology and improving operational convenience and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 This is a system structure diagram of a robot control, monitoring and simulation system of the present invention; Figure 2 This is an interactive diagram of the simulation preview function modules of a robot control, monitoring and simulation system of the present invention; Figure 3 This is an interaction diagram of a monitoring module of a robot control, monitoring and simulation system of the present invention; Figure 4 This is a point table module of a robot control, monitoring and simulation system of the present invention; Figure 5 A deployment flow chart of a robot control, monitoring and simulation system of the present invention; Figure 6 An online control module interaction diagram of a robot control, monitoring and simulation system of the present invention; Figure 7 This is an interaction diagram of an offline control module of a robot control, monitoring and simulation system of the present invention; Figure 8 This is an overview diagram of a control module of a robot control, monitoring and simulation system of the present invention; Figure 9 This is a detailed interaction diagram of the control modules of a robot control, monitoring and simulation system of the present invention. DETAILED DESCRIPTION
[0018] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below in conjunction with specific implementation methods.
[0019] like Figure 1-9 As shown in the figure, a robot control, monitoring, and simulation system first deploys ROS2 middleware on the robot embedded system and configures the ROS domain. First, deploy ROS middleware (such as third-party middleware like Fast DDS and Cyclone DDS) to enable data communication and information exchange between the robot system and the host computer. Based on the specific robot application requirements, appropriate middleware is selected to ensure efficient data transmission and real-time performance. Next, the ROS domain value is configured. This configuration ensures that the robot embedded system can correctly join the specific ROS network domain and ensures unimpeded communication between the robot and other devices. Actuator programs are deployed on the robot embedded system. Within the control system of the robot embedded system, actuators are deployed. These programs are responsible for converting robot control commands into mechanical motion. This process includes, but is not limited to, adjusting the actuator's motor control logic, kinematic modeling, and sensor data collection and processing to ensure that the robot can execute precise movements according to host computer commands. It also seamlessly integrates with the ROS system's middleware to ensure accurate and timely data communication. On the client computer, the host computer and middleware (consistent with the middleware on the robot embedded system) are installed, ensuring the same ROS domain value and being on the same network segment. This ensures smooth two-way communication between the client and the robot's embedded system, allowing the robot to receive control commands from the host computer and feedback real-time status information. By configuring a consistent ROS domain, the client and the robot can work together in the same network environment, thereby achieving efficient system communication and real-time data interaction; Figure 3As shown, the UI panel monitors and synchronizes the robot status in real time: In the host computer software, users monitor and control the robot in real time through an intuitive UI panel. When the user clicks the motor synchronization button, the system will synchronize the current status of each joint of the robot in real time, including joint values, joint speeds, Cartesian coordinate values of the robotic arm, joint power-on status, and brake status. This information is transmitted to the host computer in real time through the ROS2 middleware to ensure that the user can immediately see the current accurate status of the robot in the UI panel. In addition, the robot posture in the simulation module view will also be displayed synchronously, and the user can intuitively observe the robot's movement performance in the real environment to ensure the real-time and accuracy of the operation. Through this process, the host computer can not only provide rich real-time data, but also ensure that the status between the robot and the host computer is synchronized during execution. The user can more effectively monitor the robot's operating status and adjust the control strategy in a timely manner. The real-time data synchronization function of the UI panel enables users to maintain full control of the robot even in complex tasks and dynamic environments, reduce operational risks, and improve the robot's work efficiency and accuracy; as shown Figure 6As shown, precise robot motion control: In the control system of this invention, users can conveniently interact through a highly integrated graphical user interface (UI) to precisely adjust the robot's motion control, covering two main operating modes: joint control and Cartesian control. Specifically, users can achieve joint control by adjusting the target values of each robot joint. In this mode, the host computer automatically calculates the motion trajectory between the starting joint angle and the target joint angle based on the user-set joint target position. Using an interpolation algorithm, it smoothly transitions according to the set speed and acceleration, thereby achieving smooth and precise motion of the robot arm. In addition to joint control, this system also provides an intuitive and easy-to-use Cartesian control mode, which is particularly suitable for operators to accurately position the end effector in three-dimensional space. Users can intuitively adjust the target position and posture of the robot end effector by dragging the robot end effector in the simulation view. The system updates the motion trajectory in real time, calculates the corresponding relationship between the robot's joint angles, and automatically adjusts the posture and position of the robot arm to ensure that the target position set in the simulation view is accurately achieved. To optimize the robot's motion control, this system uses a velocity-based interpolation algorithm, which can provide high-precision motion scheduling under different operating conditions. The system will dynamically adjust the interpolation step size and execution speed according to the distance between the target point and the initial point to ensure that the robot can complete various operation tasks in an optimal path, efficient and stable manner under the premise of meeting the kinematic constraints; Point table programming: In the control system of the present invention, users can implement offline programming by arranging point tables. Users can add points in three ways: three-dimensional interactive acquisition of robot posture in the simulation module, manual input of robot posture, and monitoring of real machine posture, and modify and delete points with the help of UI. By selecting different functions and modes of the point table, user business needs can be met. Among them, the point table control function includes: loop execution function, which loops through each point in the point table according to the hierarchical nesting relationship of the point table; single point execution mode, which allows any point in the point table to be executed separately. The point table control mode includes: online control mode, which controls and operates the robot in real time based on the point table; offline control mode, which simulates the control process of the point table in a simulation environment; such as Figure 7 As shown, point table offline control: In the present invention, the point table module has the function of exporting the point table in YAML format and can export the exported point table file to the system where the actuator is located. The actuator receives and parses the point information in the point table file, exchanges data with the embedded system through its internal USB serial port communication module, and controls the embedded system to perform corresponding operations according to the instructions in the point table, thereby achieving precise control of the target device and task execution, and realizing the effect of offline control; The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.
Claims
1. A robot control, monitoring and simulation system, characterized by: The system includes a host computer and actuators, wherein the host computer is connected to the actuators via ROS communication middleware. ROS communication middleware is a distributed communication framework for robotics software development that provides communication, coordination, and resource management functions for different nodes in the robotics system. It is cross-platform, open source, and modular. The host computer includes a simulation module, a control module, a data monitoring module, and a point table module. The control module and the data monitoring module are connected to the actuator via the ROS communication middleware. The point table arrangement module has two connection methods: one is to connect to the actuator via the ROS communication middleware, and the other is to connect to the actuator via the USB serial port. The actuator includes a USB serial communication module, a data acquisition module, and an embedded control module. The data acquisition module and the joint servo module collect robot data and control the robot through the USB serial communication module, and report data and receive control instructions from the host computer through the ROS communication middleware. The simulation module includes three-dimensional interaction, monitoring synchronization and simulation preview functions. It is developed based on Linux, Rviz visualization platform, QT framework and ROS robot operating system, and uses Rviz to realize three-dimensional interaction, simulation preview and monitoring of the robotic arm. The three-dimensional interaction function refers to supporting users to affect the robot posture through mouse interaction in a three-dimensional view. The simulation preview function is used to preview the execution process of the point table in real time in the Rviz environment, including the overall simulation execution and the preview of single-point targets. The monitoring synchronization function refers to synchronizing the real robot data collected in real time to the simulation module to render the same posture; The robot modeling method uses a URDF file to describe the robot structure model, including the parameter configuration of multiple links and joints; the URDF model is verified by a simulation tool, and a control strategy is generated based on the model to realize robot motion control.
2. A robot control, monitoring and simulation system according to claim 1, characterized in that: The control module uses the Rviz visualization tool for 3D interaction, offering both joint control and Cartesian control. Joint control achieves the target pose by adjusting the angles of each joint, while Cartesian control operates by directly setting the spatial position and pose of the end effector. Users set the target pose through a QT-based interface, which combines a planning control algorithm with a velocity interpolation algorithm to generate a smooth motion trajectory. Ultimately, users can select a control method, preview the motion process in real time through the Rviz visualization tool, or send commands to the actuators through the ROS communication middleware to complete the actual operation.
3. A robot control, monitoring and simulation system according to claim 1, characterized in that: The data monitoring module, based on QT and the ROS robot operating system, utilizes the ROS communication middleware to acquire data collected by the data acquisition module in the actuator. This data includes joint values, joint velocities, the robotic arm's Cartesian coordinates, joint power-on status, and brake status. The system displays this information to the user through a user interface (UI) to facilitate inspection and abnormality recovery. Simultaneously, the collected data is synchronized in real time to the Rviz visualization tool, which renders the robot's three-dimensional model posture based on joint angle information, thereby intuitively displaying the robot's motion state and spatial position. This process enables efficient monitoring of the robot's trajectory, posture changes, and joint status. The ROS middleware relies on wireless communication technologies (such as Wi-Fi), so it is necessary to ensure that the human-machine interaction host computer and the actuator are within the same local area network.
4. A robot control, monitoring and simulation system according to claim 1, characterized in that: The point table module includes the following functions: point table compilation, import and export, and point table execution. The point table consists of multiple hierarchically nested point sets, each of which contains the following parameters: sequence number, description, joint angle, motion type, speed, acceleration, and dwell time.
5. A robot control, monitoring and simulation system according to claim 4, characterized in that: The point table arrangement is used to systematically add and reorganize the point table. Its characteristics include three methods: users obtain the robot posture in three-dimensional interaction in the simulation module, manually input the robot posture, and monitor the real machine posture to assist users in determining the content of the point table, thereby reducing the difficulty of point table arrangement.
6. A robot control, monitoring and simulation system according to claim 5, characterized in that: The point table control function includes: a loop execution function, which loops and executes each point in the point table according to the hierarchical nesting relationship of the point table; a single-point execution mode, which allows the individual execution of any point in the point table. The point table control mode includes: an online control mode, which performs real-time control and operation of the robot based on the point table; and an offline control mode, which simulates the control process of the point table in a simulation environment. The point table arrangement module realizes user interface interaction through QT, and realizes motion simulation and execution preview of the robot model through Rviz.
7. A robot control, monitoring and simulation system according to claim 6, characterized in that: The USB serial port communication module is designed to realize data transmission between the actuator and the embedded device (robot), and can perform data transmission efficiently and stably. The module is connected to the embedded device through a standardized USB interface, acting as a bridge for data interaction. The data acquisition module is used to obtain the operating status data of the actuator, including but not limited to joint position, speed, acceleration, current and other information. The module collects data from the embedded device (robot) through the USB serial port communication module.
8. A robot control, monitoring and simulation system according to claim 7, characterized in that: The embedded control module is used to accurately control the movement of the robot joints and can drive the embedded device to perform precise positioning, speed and torque control according to the control instructions sent by the USB serial communication module.