Remote guidance method and apparatus based on augmented reality, terminal and storage medium

Abstract

Embodiments of the invention disclose a remote guidance method and apparatus based on augmented reality, a terminal and a storage medium. The method includes steps: obtaining a two-dimensional video of a target scene, and transmitting the two-dimensional video to a remote terminal; obtaining a two-dimensional pixel coordinate corresponding to a marking point in a marking image frame in the two-dimensional video by the remote terminal if the guidance mode of remote guidance is a marking mode; determining a current camera coordinate corresponding to the marking point according to a preset three-dimensional coordinate estimation rule and the two-dimensional pixel coordinate, wherein the current camera coordinate is a current three-dimensional space coordinate corresponding to the marking point of a camera coordinate system; and rendering a three-dimensional virtual model corresponding to the marking point according to a presenting mode and the current camera coordinate to display the three-dimensional model in the target scene. According to the technical scheme, the marking point can be tracked and rendered in real time so that the remote guidance accuracy and the guidance efficiencyare improved.

Description

technical field [0001] Embodiments of the present invention relate to computer vision technology, and in particular to an augmented reality-based remote guidance method, device, terminal and storage medium. Background technique [0002] In the field of computer vision research, augmented reality (Augmented Reality, AR) can enable smart devices to understand the scene they are in, and render objects that do not exist in the real scene, so that users can get more from the real scene. Accurate information, or better entertainment. In the remote guidance based on augmented reality, it is necessary to render the 3D virtual model corresponding to the marked point on a certain 2D image frame of the instructor at the remote end, so that the operator at the field end can see the 3D virtual model for accurate operation. [0003] Generally, there are two ways of existing remote guidance. The first method is plane-to-plane interaction, specifically: the field end sends the two-dimensi...

AI Technical Summary

Problems solved by technology

[0004] However, corresponding to the first remote guidance method, the on-site operator can only infer the meaning of the remote-side instructor’s mark by viewing the image, which makes the guidance not intuitive enough

Corresponding to the second remote guidance method, since a depth camera needs to be added to the field end, the hardware cost is greatly increased

In addition, when the operator at the field end wants to observe the marking point from another angle, since there is no tracking algorithm in this method, the marking point will be marked at the wrong position, so the operator at the field end needs to complete the marking within the marking time. , the augmented reality glasses that remain fully worn have no spatial movement

It can be seen that the practical limitations of this remote guidance method are relatively large

Images