Teleoperation MTP Latency Measurement Using Time-Stamped Motion Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for measuring motion-to-photon latency in teleoperation systems with remote vision-through mixed reality (MR) are subjective, imprecise, and lack real-time monitoring capabilities, making it difficult to assess and improve the Quality of Experience (QoE) in applications like tele-driving and teleoperation of robots and drones.
Innovation Solution
A method utilizing motion tracking sensors on MR devices and robots, synchronized with time stamps, to measure and monitor end-to-end motion-to-photon latency by comparing time information across multiple transactions in the teleoperation system, enabling precise and real-time latency assessment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-speed camera measurement is used, then MTP latency can be measured, but measurement precision is poor due to human perception dependency
Solution Approach 1:
The patent replaces the mechanical/optical measurement system (high-speed cameras and human perception) with an electronic timing system. Motion tracking data with time stamps is automatically generated by sensors and processors, eliminating the need for visual recording and human analysis. This substitution of mechanical measurement with electronic timing directly improves measurement precision while reducing device complexity.
Solution Approach 2:
The system enables self-service measurement by automatically generating motion tracking data with embedded time stamps at the teleoperation client and comparing it with image data time stamps at the teleoperation object. The measurement process is autonomous, requiring no external high-speed cameras or human operators, thus improving precision and simplifying the measurement system.
2Loss of time
If high-speed camera measurement is used, then MTP latency can be measured, but real-time monitoring capability is lost
Solution Approach 1:
The patent implements continuous motion tracking that generates time-stamped data in real-time during teleoperation. Unlike discrete high-speed camera snapshots, the system continuously monitors motion and compares it with image data timestamps, enabling ongoing real-time MTP latency measurement and monitoring throughout the operation.
Solution Approach 2:
By replacing the frame-based mechanical measurement approach of high-speed cameras with continuous electronic timing and data comparison, the system achieves real-time measurement capability. The electronic timing system processes motion and image data continuously, enabling real-time monitoring without the limitations of discrete photographic frames.
3Ease of operation
If traditional high-speed camera method is used, then measurement can be conducted, but labor intensity is high
Solution Approach 1:
The system performs self-service measurement by automatically generating motion tracking data with time stamps, receiving image data with time stamps, and computing MTP latency without human intervention. The automated comparison and calculation eliminate the labor-intensive processes of manual picture taking, storage, and analysis required by traditional high-speed camera methods.
Solution Approach 2:
The patent replaces the manual mechanical measurement process with an automated electronic system. Sensors and processors automatically track motion, timestamp data, receive image data, and calculate latency, substituting human labor with automated electronic operations that are easier to execute and more consistent.
4Reliability
If MTP latency is not shortened, then system operation continues, but Quality of Experience deteriorates
Solution Approach 1:
The patent implements feedback by continuously measuring MTP latency through time stamp comparison and providing this measurement data for QoE assessment. This feedback mechanism enables monitoring of latency impacts on service quality, allowing system operators to identify and address QoE deterioration caused by excessive MTP latency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method, means, a teleoperation client and a teleoperation object for measuring a motionto-photon, MTP, latency in a teleoperation system are provided, comprising: obtaining motion tracking data for a motion of a first object at a teleoperation client, assigning first time information to the motion tracking data, transmitting the motion tracking data and the first time information from the teleoperation client to a teleoperation object, controlling a motion of a second object using the motion tracking data at the teleoperation object, obtaining image data for the motion of the second object based on the motion tracking data at the teleoperation object, obtaining second time information associated with the image data, and measuring an MTP latency based on the first time information and the second time information.