Real-time 3D remote man-machine interaction system

Abstract

The invention discloses a real-time 3D remote man-machine interaction system. The system at least comprises a service operation end and a client interaction end, wherein the service operation end at least comprises a computer, a depth camera, an image processing module, a data transmitting/receiving module, a service operation end command processing module and a 3D image displaying module; and the client interaction end at least comprises a computer, a leap motion module, a 3D image displaying module, a command processing module, an image processing module, a data transmitting/receiving module and a hand information processing module. Information of operators is acquired by the low-cost depth camera, and is transmitted to a client side by a local area network, and real-time transmission is realized. In the aspect of interaction, human hand information is acquired by Leap Motion so as to control a simulated mouse, and human and 3D projection interaction is realized. The real-time 3D remote man-machine interaction system is convenient and simple to operate.

Description

technical field [0001] The present invention relates to the field of human-computer interaction and real-time remote 3D video communication, in particular to the technical background of a real-time 3D remote human-computer interaction system. Background technique [0002] Nowadays, the rapid development of binocular cameras, such as ZED, KINECT and other binocular cameras, makes it more and more convenient to obtain depth image information, with low cost and high precision. Especially with the KINECT binocular camera, users can easily obtain and use 3D point cloud data. Applying KINCKET to a real-time 3D remote human-computer interaction system can save manufacturing costs with a high precision. [0003] With the emergence of PCL (Point Cloud Library), it is easier for people to process 3D images, and the results are more stable. A real-time 3D remote AR interactive system uses PCL to process the collected 3D point cloud data, which has great advantages. [0004] With the...

AI Technical Summary

Problems solved by technology

[0006] 1. Chinese open patent: CN 103793060 A A user interaction system and method: the virtual object of the interaction interface moves following the 3D position information of the extracted human hand. This invention realizes the 3D interaction system, but the invention is only one-way interaction, and the user must Wearing a head-mounted 3D display, etc., the user's operation is cumbersome

[0007] 2. Chinese open patent: CN 105389006 A A Kinect-based naked-eye 3D interactive improvement method: use Kinect to collect human hand joint information for interaction, but before the interaction, Kinect must be adjusted to a suitable position and angle. Due to the complicated installation, it is limited its scope of use

[0008] 3. Chinese open patent: CN 103269423 A Scalable three-dimensional display remote video communication method: use RGB-D camera to collect images of people, process them and send them to the second user, and the second user receives and displays the task image information and Play sound, the invention only realizes information extraction and transmission, and there is no two-way interactive process

Images