High-altitude operation remote operation method and system based on stereo vision and gesture control

Abstract

The invention discloses a high-altitude operation remote operation method and system based on stereo vision and gesture control. A PC end receives high-altitude operation area scene information acquired by a binocular stereo camera and displays the information on a wearable stereo display; according to the stereo scene work condition displayed by the wearable stereo display, hand operation is performed and movement information of hands and fingers is tracked in real time, the tracked movement information of hands and fingers is sent to the PC end and processed to be converted into a movement instruction of a bionic manipulator; and after the bionic manipulator receives the processed and converted movement instruction to follow correspondingly to complete remote operation of high-altitude operation. By use of the stereo vision and gesture control combined remote operation method and system, the personal safety of a high-altitude operator can be guaranteed, three-dimensional perception of the operator for a site scene during remote operation is improved, and the interaction nature and immersion of the remote operation are enhanced.

Description

technical field [0001] The invention belongs to the technical field of high-altitude operation teleoperation, and in particular relates to a high-altitude operation teleoperation method and system based on stereo vision and gesture control. Background technique [0002] In recent years, with the rapid development of economy, there are more and more high-rise buildings in our country. At present, high-altitude operations involve fields such as power maintenance, elevated construction, high-rise equipment installation, and cleaning. Operators are required to climb or stand on very high tower cranes for manual operations, which is very dangerous, and accidents may occur if they are not careful. , It will cause irreparable losses to society and enterprises, so it is especially necessary to replace spiders with robots to complete extremely dangerous high-altitude operations. [0003] Most of the current high-altitude operation robots can only complete some simple and repetitive ...

AI Technical Summary

Problems solved by technology

Most of the existing teleoperation modes are for the operator to understand and judge the working conditions of the high-altitude operation area through the two-dimensional monitoring screen at the operation end. It is difficult for the operator to accurately judge the three-dimensional relationship between the various components at high altitude; Two-dimensional working condition information uses a joystick or a joystick to control the manipulator to complete the job. This interactive mode is blunt, the immersion is poor, and there is a certain rate of misoperation, which is difficult for untrained operators.

[0004] Therefore, there is an urgent need for a high-altitude operation teleoperation system and method with a strong sense of immersion and natural interaction to solve the problems of lack of three-dimensional information in traditional teleoperation methods, difficulty in using interactive devices, and weak immersion.

Images