Teleoperation method for unstructured environment based on 5G + AR

Abstract

The invention discloses a teleoperation method for an unstructured environment based on 5G + AR. The method comprises the following steps: firstly, carrying out rapid three-dimensional reconstructionon an unstructured environment where a robot is located, and mapping a reconstructed scene into a virtual reality space; and then, capturing a gesture action of a user, establishing a virtual model ofthe hand of the user, and mapping the gesture action into the virtual reality space, thereby realizing interaction between the user and the virtual object; finally, introducing an auxiliary technology to help the user finish teleoperation work more accurately and quickly.

Description

technical field [0001] The invention belongs to the field of teleoperation, in particular to a 5G+AR-based teleoperation method for an unstructured environment. Background technique [0002] In the field of traditional teleoperation (U. Keller, H. J.A. Hedel, et al. ChARMin: The First Actuated Exoskeleton Robot for Pediatric Arm Rehabilitation, IEEE / ASME Transactions on Mechatronics, vol. 21, no. 5, pp. 2201 - 2213, 2016. SehoonOh, Hanseung Woo, Kyoungchul Kong, “Frequency-Shaped Impedance Control for Safe Human–Robot Interaction in Reference Tracking Application,” IEEE / ASME Transactions on Mechatronics, vol. 19, no. 6, pp. 1907-1916, 2014. H.Ishida , et al., “Development of robot remote controller 'Armrest Joystick’ —Study of bilateral remote control”, IEEE International Conference on Robotics and Biomimetics IEEE, pp. 2413-2418, 2016.), the operator usually uses some kind of controller Such as joystick, mouse, keyboard, etc. to operate the robot, and observe the operation...

AI Technical Summary

Problems solved by technology

This increases the learning curve for operators who are not familiar with these controls

At the same time, determining the task execution status by observing the screen also requires the user to continuously observe the display while operating, so that the user's attention needs to be continuously shifted between the display and the controller, reducing the efficiency of interaction

[0003] In order to improve the traditional interaction methods, natural interaction methods such as speech and gestures are proposed (Yanan Li, Ge, S.S., “Human–Robot Collaboration Based on Motion Intention Estimation, IEEE / ASME Transactions on Mechatronics,” vol. 19, no. 3, pp. 1007-1014, 2014. X. Xu, A. Song, D. Ni, H. Li, P. Xiong and C. Zhu, "Visual-Haptic AidTeleoperation Based on 3-D Environment Modeling and Updating," in IEEETransactions on Industrial Electronics, vol. 63, no. 10, pp. 6419-6428, Oct.2016.), but for some complex operations, the operator cannot accurately describe with voice and gestures, so the The tasks that can be completed are limited

Furthermore, scholars (B. Browatzki, et al., "Active In-HandObject Recognition on a Humanoid Robot", IEEE Transactions on Robotics, vol.30, no. 5, pp. 1260-1269, 2014.) hope to use The marker captures the operator's actions to operate the remote robot. However, once the marker is blocked during the operation, the operator's actions cannot be captured

Images