Augmented reality remote guidance method and system in combination with 3D camera

Abstract

The invention discloses an augmented reality remote guidance method and system in combination with a 3D camera. The method includes the steps that a video stream and corresponding augmented reality information sent by an operation guidance subsystem are received, wherein the video stream is obtained by carrying out optical collection on a target, and the augmented reality information is obtained by registering a virtual vidicon; a 3D model of the target is established according to the augmented reality information; the video stream is displayed through a 3D viewport, and the 3D model of the target is loaded; limb motion information of a guider is collected through the 3D camera; a limb 3D model and 3D motion information are generated according to the limb motion information and are loaded and displayed in the 3D viewport; 3D guidance information is generated according to the limb 3D model and the 3D motion information; the 3D guidance information is sent to the operation guidance subsystem so that the operation guidance subsystem can generate and display a guidance scene according to the video stream, the augmented reality information and the 3D guidance information. By means of the method, accurate and visible limb operation guidance is provided for field operation staff.

Description

technical field [0001] The present application relates to the technical field of remote guidance, in particular to an augmented reality remote guidance method and system combined with a 3D camera. Background technique [0002] The existing technical solutions for remotely guiding on-site operations mainly include two parts, one is the on-site end and the other is the remote end. The operator at the on-site end wears or holds a smart terminal with an optical acquisition device to collect the video of the on-site environment and After sending it to the remote end, the instructor at the remote end can mark the received on-site live video, and send it back to the display device of the operator at the on-site end in real time, so as to provide more intuitive guidance for the operation of the operator. [0003] For guidance annotations related to limbs, the traditional method is that the instructor at the remote end uses a mouse and keyboard to operate a preset model (such as a ha...

AI Technical Summary

Problems solved by technology

[0003] For guidance annotations related to limbs, the traditional method is that the instructor at the remote end uses a mouse and keyboard to operate a preset model (such as a hand) to perform some simulated operations. It is often difficult to accurately express the actual operation that the simulated operation wants to achieve, so it is difficult to provide accurate and intuitive physical operation guidance for on-site operators

Images