Teleoperation data processing method and system based on virtual simulation
By employing a teleoperation data processing method based on virtual simulation, and utilizing message queue protocols and standardized mapping processing, the problem of low data acquisition efficiency was solved, data acquisition and processing were separated, and the efficiency and quality of data acquisition were improved.
Patent Information
- Application Number
- CN202511592517.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-03
- Publication Date
- 2026-02-24
AI Technical Summary
Existing data processing methods rely on real robot platforms to collect data in the field, resulting in low data collection efficiency. Furthermore, the data collection and processing modules are tightly coupled, requiring a pause in data collection to wait for processing when the data volume surges, which greatly reduces data collection efficiency.
A teleoperation data processing method based on virtual simulation is adopted. The raw operation data of heterogeneous input devices is acquired and standardized and mapped. The data acquisition and processing are separated by a message queue protocol. Timestamp alignment is performed using a sliding window and linear interpolation algorithm. The data is then standardized and compressed to form a teleoperation dataset.
It improves the efficiency of data acquisition, avoids the data acquisition process being affected by the data processing process, ensures the real-time nature and quality of the data, reduces the time from data acquisition to subsequent control of the robot model to collect data, and improves the overall data processing efficiency.
Smart Images
Figure CN121560448A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of data processing technology, and in particular relates to a teleoperation data processing method and system based on virtual simulation. Background Technology
[0002] Embodied intelligence, as an important branch of artificial intelligence, aims to enable intelligent agents such as robots to achieve intelligent behavior through interaction and manipulation of the environment. Training high-performance embodied intelligent agents relies on large-scale, high-quality teleoperation data.
[0003] Existing data processing methods rely on real robot platforms to collect data through on-site operations. However, each device can only collect data for one task at a time, and the amount of data collected per unit time is limited, which greatly reduces the efficiency of data collection. Furthermore, in existing data processing methods, the data acquisition module and the data processing module are usually tightly coupled. When the amount of data collected by the data acquisition module surges, the data processing module has limited processing speed and limited space for temporary data storage. As a result, the data acquisition module needs to pause data acquisition to wait for the data processing module to finish processing, which greatly reduces the efficiency of data collection. Summary of the Invention
[0004] The present invention aims to provide a teleoperation data acquisition method and system based on virtual simulation to solve the above-mentioned technical problems and improve the efficiency of data acquisition.
[0005] To address the aforementioned technical problems, this invention provides a teleoperation data processing method based on virtual simulation, comprising the following steps:
[0006] The system acquires raw operation data from various heterogeneous input devices and performs standardized mapping processing based on the raw operation data to obtain control commands in the same spatial coordinate system.
[0007] Based on the control commands, robot model control and state acquisition processing are performed in a preset virtual simulation environment to obtain robot state data and environmental perception data.
[0008] Based on the control commands, the robot state data, and the environmental perception data, topic channel publishing processing is performed under the message queue to obtain multi-source data streams;
[0009] Based on the multi-source data stream, timestamp alignment is performed under a preset sliding window and linear interpolation algorithm to obtain a multi-source aligned dataset.
[0010] The teleoperation dataset is obtained by standardizing the multi-source aligned dataset.
[0011] The above solution allows for the immediate commencement of the next round of data acquisition after the data collected in the virtual simulation environment is directly sent to the message queue, without waiting for subsequent data processing steps to be completed. In other words, this solution separates the data acquisition process from the data processing process through the message queue protocol, ensuring that the efficiency of data acquisition is not affected by the subsequent data processing process, thus greatly improving the efficiency of data acquisition.
[0012] Furthermore, the step of acquiring raw operation data from multiple heterogeneous input devices and performing standardized mapping processing based on the raw operation data to obtain control commands in the same spatial coordinate system includes: acquiring spatial tracking data from a VR device and performing conversion processing based on the spatial tracking data to obtain standard pose data; acquiring screen coordinate data and key event data from a keyboard and mouse device and performing mapping processing based on the screen coordinate data and the key event data to obtain three-dimensional spatial control commands; acquiring joystick input data and button input data from a gamepad device and performing conversion processing based on the joystick input data and the button input data to obtain motion control parameters; and performing standardized mapping processing based on the standard pose data, the three-dimensional spatial control commands, and the motion control parameters to obtain control commands in the same spatial coordinate system.
[0013] The standardized mapping process described above enables the spatial coordinate system of standard pose data, 3D spatial control commands, and motion control parameters to be unified, and also normalizes each data point. This ensures that the raw operational data input from different devices have consistent dimensions and ranges within the same reference frame, avoiding the problem of frequent general protocol conversions in existing technologies that greatly reduce the efficiency of converting operational data into control commands. This solution improves the efficiency of converting operational data into control commands, thereby reducing the time spent from acquiring operational data from various devices to subsequent data acquisition from the robot model, and thus improving data acquisition efficiency.
[0014] Furthermore, the step of performing topic channel publishing processing under the message queue based on the control commands, the robot state data, and the environmental perception data to obtain a multi-source data stream includes: performing buffering processing in a preset circular buffer based on the control commands, the robot state data, and the environmental perception data to obtain a buffered data stream; and performing topic channel publishing processing under the message queue based on the buffered data stream to obtain a multi-source data stream distributed by topic.
[0015] In the above scheme, control commands, robot status data, and environmental perception data are written into a circular buffer. The circular buffer in this scheme adopts a first-in-first-out overflow strategy, which can ensure the real-time performance of the obtained data and prevent memory overflow. Furthermore, under the message queue, this scheme divides the buffered data stream into different types of data and publishes them to the corresponding topic channels, resulting in a multi-source data stream distributed by topic. Subsequent data processing is based on this multi-source data stream, making data acquisition and data processing independent processes. Therefore, after the current round of data acquisition, this scheme does not need to wait for the data processing to be completed before the next round of data acquisition, thus improving the efficiency of data acquisition.
[0016] Furthermore, the step of aligning timestamps based on the multi-source data stream under a preset sliding window and a linear interpolation algorithm to obtain a multi-source aligned dataset includes: obtaining an initial multi-source dataset based on the multi-source data stream under a preset sliding window; and performing timestamp matching based on the initial multi-source dataset under a linear interpolation algorithm to obtain a multi-source aligned dataset.
[0017] The above scheme uses a sliding window to align the time of multi-source data streams, and solves the problem of inconsistent data sampling rates from different data sources by using a linear interpolation algorithm and timestamp matching, thereby improving the data quality of the resulting multi-source aligned dataset.
[0018] Furthermore, the step of standardizing the multi-source aligned dataset to obtain a teleoperation dataset includes: standardizing the multi-source aligned dataset to obtain a standardized multi-source dataset; and packaging and compressing the standardized multi-source dataset to obtain a teleoperation dataset.
[0019] The above scheme is based on the standardization of multi-source aligned datasets, and the standardized multi-source datasets are compressed and packaged for transmission, which ensures low-latency data transmission and improves data transmission efficiency.
[0020] Further, the standardization process based on the multi-source aligned dataset to obtain a standardized multi-source dataset includes: cleaning the multi-source aligned dataset to obtain a cleaned multi-source aligned dataset; and standardizing the cleaned multi-source aligned dataset to obtain a standardized multi-source dataset.
[0021] The above-mentioned scheme cleans and standardizes the multi-source aligned dataset, which can improve the quality of the obtained data; and this scheme transforms the multi-source aligned dataset into a standardized multi-source dataset, which facilitates the use of the data in the future.
[0022] This invention also provides a teleoperation data processing system based on virtual simulation, comprising: a heterogeneous device operation data conversion module, used to acquire raw operation data from multiple heterogeneous input devices, and perform standardized mapping processing based on the raw operation data to obtain control commands in the same spatial coordinate system; a simulation data acquisition module, used to perform robot model control and state acquisition processing in a preset virtual simulation environment based on the control commands obtained by the heterogeneous device operation data conversion module, to acquire robot state data and environmental perception data; a multi-source data acquisition module, used to perform topic channel publishing processing in a message queue based on the control commands obtained by the heterogeneous device operation data conversion module, the robot state data obtained by the simulation data acquisition module, and the environmental perception data obtained by the simulation data acquisition module, to obtain a multi-source data stream; a time alignment module, used to perform timestamp matching based on the multi-source data stream obtained by the multi-source data acquisition module under a preset sliding window and linear interpolation algorithm to obtain a multi-source aligned dataset; and a standardization module, used to perform standardization processing on the multi-source aligned dataset obtained by the time alignment module to obtain a teleoperation dataset.
[0023] Furthermore, the heterogeneous device operation data conversion module is used to acquire raw operation data from various heterogeneous input devices, and perform standardized mapping processing based on the raw operation data to obtain control commands under the same spatial coordinate system. This includes: acquiring spatial tracking data from a VR device and performing conversion processing based on the spatial tracking data to obtain standard pose data; acquiring screen coordinate data and key event data from a keyboard and mouse device, and performing mapping processing based on the screen coordinate data and the key event data to obtain three-dimensional spatial control commands; acquiring joystick input data and button input data from a gamepad device, and performing conversion processing based on the joystick input data and the button input data to obtain motion control parameters; and performing standardized mapping processing based on the standard pose data, the three-dimensional spatial control commands, and the motion control parameters to obtain control commands under the same spatial coordinate system.
[0024] Furthermore, the multi-source data acquisition module is used to perform topic channel publishing processing under a message queue based on the control commands obtained by the heterogeneous device operation data conversion module, the robot status data obtained by the simulation data acquisition module, and the environmental perception data obtained by the simulation data acquisition module, to obtain a multi-source data stream. This includes: buffering the control commands obtained by the heterogeneous device operation data conversion module, the robot status data obtained by the simulation data acquisition module, and the environmental perception data obtained by the simulation data acquisition module in a preset circular buffer to obtain a buffered data stream; and performing topic channel publishing processing under a message queue based on the buffered data stream to obtain a multi-source data stream distributed by topic.
[0025] Furthermore, the time alignment module is used to perform timestamp matching based on the multi-source data stream obtained by the multi-source data acquisition module under a preset sliding window and a linear interpolation algorithm to obtain a multi-source aligned dataset, including: obtaining an initial multi-source dataset based on the multi-source data stream obtained by the multi-source data acquisition module under a preset sliding window; and performing timestamp matching based on the initial multi-source dataset under a linear interpolation algorithm to obtain a multi-source aligned dataset.
[0026] The above solution separates the data acquisition process from the data processing process through a message queue protocol, ensuring that the efficiency of data acquisition is not affected by the subsequent data processing process. Therefore, after the acquired data is sent to the message queue, the next round of data acquisition can be carried out directly, which greatly improves the efficiency of data acquisition. Attached Figure Description
[0027] Figure 1 A flowchart illustrating the technical implementation of a teleoperation data processing method based on virtual simulation, as provided in an embodiment of the present invention;
[0028] Figure 2 This is a schematic diagram of a teleoperation data processing system architecture based on virtual simulation, provided as an embodiment of the present invention. Detailed Implementation
[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0030] Please see Figure 1 This embodiment provides a teleoperation data processing method based on virtual simulation, including the following steps:
[0031] Step S1: Obtain raw operation data from various heterogeneous input devices, and perform standardized mapping processing based on the raw operation data to obtain control commands in the same spatial coordinate system;
[0032] Step S2: Based on the control commands, perform robot model control and state acquisition processing in a preset virtual simulation environment to obtain robot state data and environmental perception data;
[0033] Step S3: Based on the control commands, the robot state data, and the environmental perception data, perform topic channel publishing processing under the message queue to obtain a multi-source data stream;
[0034] Step S4: Based on the multi-source data stream, perform timestamp alignment under a preset sliding window and linear interpolation algorithm to obtain a multi-source aligned dataset;
[0035] Step S5: Perform standardization processing on the multi-source aligned dataset to obtain the teleoperation dataset.
[0036] The above embodiment allows for the immediate commencement of the next round of data collection after the data collected in the virtual simulation environment is directly sent to the message queue, without waiting for subsequent data processing steps to be completed. In other words, this embodiment separates the data collection process from the data processing process through the message queue protocol, ensuring that the efficiency of data collection is not affected by the subsequent data processing process, thus greatly improving the efficiency of data collection.
[0037] It should be noted that in this embodiment, control commands, robot status data, and environmental perception data are aligned using timestamps, which ensures the accurate correspondence of various types of data in the time dimension.
[0038] In one embodiment, the control commands include position vectors, attitude quaternions, action commands, velocity parameters, and precise timestamps. The robot model executes corresponding actions according to the control commands, completes the specified embodied operation task in the virtual simulation environment, and simultaneously collects robot state data and environmental perception data during the robot's actions. The robot state data includes a series of real-time operating data such as joint angles, position coordinates, velocity, acceleration, sensor readings, and environmental feedback. The environmental perception data includes the states of environmental elements, obstacles, and various interactive objects.
[0039] In one embodiment, control commands and robot state data are obtained from high-frequency sampled data to ensure the accuracy of motion control; environmental perception data is obtained from low-frequency sampled data to meet the needs of scene understanding. Furthermore, in this embodiment, data from different data sources have their own independent acquisition threads to avoid blocking data transmission from other data sources when the data volume from one data source surges.
[0040] In one embodiment, the process of controlling the robot model in a preset virtual simulation environment based on control commands is as follows: the virtual robot performs corresponding actions in the virtual simulation environment according to the received control commands; the physics engine calculates collision detection and dynamic response in real time; and the rendering engine presents the execution results to the operator in real time, forming intuitive visual feedback. After each control cycle is completed, the system determines whether the current task has been completed (e.g., whether the target object has been successfully grasped, whether the designated position has been reached, etc.). If the task has not been completed, the system continues to execute in a loop. If the task has been completed or the operator actively ends the operation, the system exits the operation loop.
[0041] Furthermore, the step of acquiring raw operation data from multiple heterogeneous input devices and performing standardized mapping processing based on the raw operation data to obtain control commands in the same spatial coordinate system includes: acquiring spatial tracking data from a VR device and performing conversion processing based on the spatial tracking data to obtain standard pose data; acquiring screen coordinate data and key event data from a keyboard and mouse device and performing mapping processing based on the screen coordinate data and the key event data to obtain three-dimensional spatial control commands; acquiring joystick input data and button input data from a gamepad device and performing conversion processing based on the joystick input data and the button input data to obtain motion control parameters; and performing standardized mapping processing based on the standard pose data, the three-dimensional spatial control commands, and the motion control parameters to obtain control commands in the same spatial coordinate system.
[0042] The standardized mapping process described in the above embodiments enables the spatial coordinate system of standard pose data, three-dimensional spatial control commands, and motion control parameters to be unified, and also normalizes each data. This ensures that the original operation data input from different devices can have consistent dimensions and ranges within the same reference frame, avoiding the problem of frequent general protocol conversions in existing technologies, which greatly reduces the efficiency of converting operation data into control commands. This embodiment improves the efficiency of converting operation data into control commands, thereby reducing the time spent from acquiring operation data from each device to subsequent data acquisition from the robot model, and thus improving the efficiency of data acquisition.
[0043] In one embodiment, a VR data parser (OpenVR driver) converts the 6-DOF spatial tracking data of the VR device into standard pose data; a keyboard and mouse mapper (HID driver) maps the 2D screen coordinates and key event data of the keyboard and mouse device into 3D spatial control commands; and a gamepad parser (XInput driver) converts the joystick input data and button input data of the gamepad device into motion control parameters.
[0044] Furthermore, the step of performing topic channel publishing processing under the message queue based on the control commands, the robot state data, and the environmental perception data to obtain a multi-source data stream includes: performing buffering processing in a preset circular buffer based on the control commands, the robot state data, and the environmental perception data to obtain a buffered data stream; and performing topic channel publishing processing under the message queue based on the buffered data stream to obtain a multi-source data stream distributed by topic.
[0045] In the above embodiments, control commands, robot state data, and environmental perception data are written into a circular buffer. The circular buffer in this embodiment adopts a first-in-first-out (FIFO) overflow strategy, which can ensure the real-time performance of the obtained data and prevent memory overflow. In addition, under the message queue, the buffered data stream is divided into different types of data and published to the corresponding topic channels to obtain a multi-source data stream distributed by topic. Subsequent data processing is based on this multi-source data stream, making data acquisition and data processing independent processes. Therefore, after the current round of data acquisition, it is not necessary to wait for the data processing to be completed before the next round of data acquisition, which improves the efficiency of data acquisition.
[0046] In one embodiment, the message queue is built based on the MQTT protocol, and the circular buffer is based on a lock-free design.
[0047] Furthermore, the step of aligning timestamps based on the multi-source data stream under a preset sliding window and a linear interpolation algorithm to obtain a multi-source aligned dataset includes: obtaining an initial multi-source dataset based on the multi-source data stream under a preset sliding window; and performing timestamp matching based on the initial multi-source dataset under a linear interpolation algorithm to obtain a multi-source aligned dataset.
[0048] The above embodiments use a sliding window to perform time alignment of multi-source data streams, and solve the problem of inconsistent data sampling rates from different data sources by using linear interpolation algorithms and timestamp matching, thereby improving the data quality of the resulting multi-source aligned dataset.
[0049] Furthermore, the step of standardizing the multi-source aligned dataset to obtain a teleoperation dataset includes: standardizing the multi-source aligned dataset to obtain a standardized multi-source dataset; and packaging and compressing the standardized multi-source dataset to obtain a teleoperation dataset.
[0050] The above embodiments are based on the standardization of multi-source aligned datasets, and the standardized multi-source datasets are compressed and packaged for transmission, which ensures low-latency data transmission and improves data transmission efficiency.
[0051] In one embodiment, the packaging and compression process is implemented using the LZ4 algorithm.
[0052] Further, the standardization process based on the multi-source aligned dataset to obtain a standardized multi-source dataset includes: cleaning the multi-source aligned dataset to obtain a cleaned multi-source aligned dataset; and standardizing the cleaned multi-source aligned dataset to obtain a standardized multi-source dataset.
[0053] The above embodiments clean and standardize the multi-source aligned dataset, which can improve the quality of the obtained data; and this embodiment transforms the multi-source aligned dataset into a standardized multi-source dataset, which facilitates the use of the data in the future.
[0054] In one embodiment, the cleaning process includes data cleaning, format conversion, and quality inspection.
[0055] In one embodiment, a persistent storage layer is also used to store the obtained teleoperation dataset. The persistent storage layer described in this embodiment adopts a combination strategy of multiple storage engines: InfluxDB is dedicated to storing high-frequency time-series data and supports efficient time range queries and aggregation analysis; MongoDB stores structured task execution records and metadata; and MinIO object storage is used to save large-capacity raw data files and video recordings.
[0056] In one embodiment, the construction process of the virtual simulation environment includes: building a complete environment framework (such as room structure, factory layout) on a 3D modeling platform such as Blender or 3ds Max; adding detailed environmental elements (furniture, equipment, decorations, etc.) to the environment framework and creating various obstacles and interactive objects; after completing the modeling, importing the scene model file into a virtual simulation engine such as Unity or Unreal Engine; integrating the PhysX or Bullet physics engine into the virtual simulation engine, and configuring the physical simulation parameters under the physics engine, i.e., setting specific physical simulation parameters and the gravitational acceleration value (usually 9.8 m / s²). 2 The system determines the friction coefficient, collision detection accuracy, and response mode of objects of different materials, and adjusts the physical properties of objects such as mass and density to ensure that the physical behavior in the virtual simulation environment is consistent with the real world. In the scene with the above-configured physical properties, the system determines the position coordinates and initial state of the target object, arranges the spatial distribution and geometry of obstacles, sets the physical properties and behavior rules of interactive objects, and assigns a unique identifier to all task objects to accurately place task-related objects for easy subsequent use.
[0057] In one embodiment, the construction of the robot model in the virtual simulation environment includes: establishing the robot's kinematic model (joint structure, link parameters) and configuring the dynamic model (mass distribution, inertial parameters) in a robot dynamics simulation platform such as Gazebo or MuJoCo; defining the motion constraints and limits of each joint; setting the robot's initial position coordinates (x, y, z) and attitude angles (roll, pitch, yaw) in the virtual environment; and integrating the completed robot model into the Unity or Unreal Engine virtual environment.
[0058] Please see Figure 2 This embodiment also provides a teleoperation data processing system based on virtual simulation, including: a heterogeneous device operation data conversion module, used to acquire raw operation data from multiple heterogeneous input devices, and perform standardized mapping processing based on the raw operation data to obtain control commands in the same spatial coordinate system; a simulation data acquisition module, used to perform robot model control and state acquisition processing in a preset virtual simulation environment based on the control commands obtained by the heterogeneous device operation data conversion module, to acquire robot state data and environmental perception data; a multi-source data acquisition module, used to perform topic channel publishing processing in a message queue based on the control commands obtained by the heterogeneous device operation data conversion module, the robot state data obtained by the simulation data acquisition module, and the environmental perception data obtained by the simulation data acquisition module, to obtain a multi-source data stream; a time alignment module, used to perform timestamp matching based on the multi-source data stream obtained by the multi-source data acquisition module under a preset sliding window and linear interpolation algorithm to obtain a multi-source aligned dataset; and a standardization module, used to perform standardization processing on the multi-source aligned dataset obtained by the time alignment module to obtain a teleoperation dataset.
[0059] Furthermore, the heterogeneous device operation data conversion module is used to acquire raw operation data from various heterogeneous input devices, and perform standardized mapping processing based on the raw operation data to obtain control commands under the same spatial coordinate system. This includes: acquiring spatial tracking data from a VR device and performing conversion processing based on the spatial tracking data to obtain standard pose data; acquiring screen coordinate data and key event data from a keyboard and mouse device, and performing mapping processing based on the screen coordinate data and the key event data to obtain three-dimensional spatial control commands; acquiring joystick input data and button input data from a gamepad device, and performing conversion processing based on the joystick input data and the button input data to obtain motion control parameters; and performing standardized mapping processing based on the standard pose data, the three-dimensional spatial control commands, and the motion control parameters to obtain control commands under the same spatial coordinate system.
[0060] Furthermore, the multi-source data acquisition module is used to perform topic channel publishing processing under a message queue based on the control commands obtained by the heterogeneous device operation data conversion module, the robot status data obtained by the simulation data acquisition module, and the environmental perception data obtained by the simulation data acquisition module, to obtain a multi-source data stream. This includes: buffering the control commands obtained by the heterogeneous device operation data conversion module, the robot status data obtained by the simulation data acquisition module, and the environmental perception data obtained by the simulation data acquisition module in a preset circular buffer to obtain a buffered data stream; and performing topic channel publishing processing under a message queue based on the buffered data stream to obtain a multi-source data stream distributed by topic.
[0061] Furthermore, the time alignment module is used to perform timestamp matching based on the multi-source data stream obtained by the multi-source data acquisition module under a preset sliding window and a linear interpolation algorithm to obtain a multi-source aligned dataset, including: obtaining an initial multi-source dataset based on the multi-source data stream obtained by the multi-source data acquisition module under a preset sliding window; and performing timestamp matching based on the initial multi-source dataset under a linear interpolation algorithm to obtain a multi-source aligned dataset.
[0062] The above embodiments separate the data acquisition process from the data processing process through the message queue protocol, ensuring that the efficiency of data acquisition is not affected by the subsequent data processing process. Therefore, after the acquired data is sent to the message queue, the next round of data acquisition can be carried out directly, which greatly improves the efficiency of data acquisition.
[0063] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.
Claims
1. A teleoperation data processing method based on virtual simulation, characterized in that, Includes the following steps: The system acquires raw operation data from various heterogeneous input devices and performs standardized mapping processing based on the raw operation data to obtain control commands in the same spatial coordinate system. Based on the control commands, robot model control and state acquisition processing are performed in a preset virtual simulation environment to obtain robot state data and environmental perception data. Based on the control commands, the robot state data, and the environmental perception data, topic channel publishing processing is performed under the message queue to obtain multi-source data streams; Based on the multi-source data stream, timestamp alignment is performed under a preset sliding window and linear interpolation algorithm to obtain a multi-source aligned dataset. The teleoperation dataset is obtained by standardizing the multi-source aligned dataset.
2. The teleoperation data processing method based on virtual simulation according to claim 1, characterized in that, The process of acquiring raw operation data from multiple heterogeneous input devices and performing standardized mapping processing based on the raw operation data to obtain control commands in the same spatial coordinate system includes: Acquire spatial tracking data from the VR device, and perform conversion processing based on the spatial tracking data to obtain standard pose data; The screen coordinate data and key event data from the keyboard and mouse device are acquired, and a mapping process is performed based on the screen coordinate data and key event data to obtain three-dimensional spatial control commands. Acquire joystick input data and button input data from the gamepad device, and perform conversion processing based on the joystick input data and button input data to obtain motion control parameters; Based on the standard pose data, the three-dimensional spatial control commands, and the motion control parameters, a standardized mapping process is performed to obtain control commands in the same spatial coordinate system.
3. The teleoperation data processing method based on virtual simulation according to claim 1, characterized in that, Based on the control commands, the robot state data, and the environmental perception data, topic channel publishing processing is performed under the message queue to obtain a multi-source data stream, including: Based on the control commands, the robot state data, and the environmental perception data, buffering is performed in a preset circular buffer to obtain a buffered data stream. Based on the buffered data stream, topic channel publishing processing is performed under the message queue to obtain a multi-source data stream distributed by topic.
4. The teleoperation data processing method based on virtual simulation according to claim 1, characterized in that, The process of aligning timestamps based on the multi-source data streams using a preset sliding window and linear interpolation algorithm to obtain a multi-source aligned dataset includes: Based on the multi-source data stream, an initial multi-source dataset is obtained within a preset sliding window; Based on the initial multi-source dataset, timestamp matching is performed using a linear interpolation algorithm to obtain a multi-source aligned dataset.
5. The teleoperation data processing method based on virtual simulation according to claim 1, characterized in that, The standardization process based on the multi-source aligned dataset to obtain the teleoperation dataset includes: The multi-source aligned dataset is standardized to obtain a standardized multi-source dataset. Based on the standardized multi-source dataset, a teleoperation dataset is obtained by packaging and compressing it.
6. The teleoperation data processing method based on virtual simulation according to claim 5, characterized in that, The standardization process based on the multi-source aligned dataset to obtain a standardized multi-source dataset includes: The multi-source aligned dataset is cleaned to obtain a cleaned multi-source aligned dataset. The cleaned multi-source aligned dataset is then standardized to obtain a standardized multi-source dataset.
7. A teleoperation data processing system based on virtual simulation, characterized in that, A teleoperation data processing method based on virtual simulation as described in any one of claims 1 to 6, comprising: The heterogeneous device operation data conversion module is used to acquire raw operation data from various heterogeneous input devices, and perform standardized mapping processing based on the raw operation data to obtain control commands in the same spatial coordinate system. The simulation data acquisition module is used to perform robot model control and state acquisition processing in a preset virtual simulation environment based on the control commands obtained by the heterogeneous device operation data conversion module, and to acquire robot state data and environmental perception data. The multi-source data acquisition module is used to perform topic channel publishing processing under the message queue based on the control commands obtained by the heterogeneous device operation data conversion module, the robot status data obtained by the simulation data acquisition module, and the environmental perception data obtained by the simulation data acquisition module, to obtain a multi-source data stream; The time alignment module is used to perform timestamp alignment based on the multi-source data stream obtained by the multi-source data acquisition module, under a preset sliding window and linear interpolation algorithm, to obtain a multi-source aligned dataset. The standardization module is used to standardize the multi-source aligned dataset obtained by the time alignment module to obtain a teleoperation dataset.
8. The teleoperation data processing system based on virtual simulation according to claim 7, characterized in that, The heterogeneous device operation data conversion module is used to acquire raw operation data from various heterogeneous input devices, and based on the raw operation data, perform standardized mapping processing to obtain control commands in the same spatial coordinate system, including: Acquire spatial tracking data from the VR device, and perform conversion processing based on the spatial tracking data to obtain standard pose data; The screen coordinate data and key event data from the keyboard and mouse device are acquired, and a mapping process is performed based on the screen coordinate data and key event data to obtain three-dimensional spatial control commands. Acquire joystick input data and button input data from the gamepad device, and perform conversion processing based on the joystick input data and button input data to obtain motion control parameters; Based on the standard pose data, the three-dimensional spatial control commands, and the motion control parameters, a standardized mapping process is performed to obtain control commands in the same spatial coordinate system.
9. A teleoperation data processing system based on virtual simulation according to claim 7, characterized in that, The multi-source data acquisition module is used to perform topic channel publishing processing under a message queue based on the control commands obtained by the heterogeneous device operation data conversion module, the robot state data obtained by the simulation data acquisition module, and the environmental perception data obtained by the simulation data acquisition module, to obtain a multi-source data stream, including: Based on the control commands obtained by the heterogeneous device operation data conversion module, the robot status data obtained by the simulation data acquisition module, and the environmental perception data obtained by the simulation data acquisition module, the data is buffered in a preset circular buffer to obtain a buffered data stream. Based on the buffered data stream, topic channel publishing processing is performed under the message queue to obtain a multi-source data stream distributed by topic.
10. A teleoperation data processing system based on virtual simulation according to claim 7, characterized in that, The time alignment module is used to perform timestamp alignment based on the multi-source data stream obtained by the multi-source data acquisition module, under a preset sliding window and linear interpolation algorithm, to obtain a multi-source aligned dataset, including: Based on the multi-source data stream obtained by the multi-source data acquisition module, an initial multi-source dataset is obtained under a preset sliding window; Based on the initial multi-source dataset, timestamp matching is performed using a linear interpolation algorithm to obtain a multi-source aligned dataset.