Remote operation guiding system and method based on mixed reality

A remote operation and mixed reality technology, applied in the field of remote operation guidance system based on mixed reality, can solve problems such as single interaction mode, inability to accurately and intuitively express complex operation guidance intentions, and low interaction dimension, so as to relieve fatigue and reduce information Communication barriers, the effect of improving efficiency

Pending Publication Date: 2020-08-11
NANJING SOARING CYBER REALITY INNOVATION CENT CO LTD
View PDF11 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art, and provide a remote operation guidance system a

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Remote operation guiding system and method based on mixed reality
  • Remote operation guiding system and method based on mixed reality
  • Remote operation guiding system and method based on mixed reality

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0034] Step 1: Before the task starts, the local operator on the AR client and the remote expert on the VR client need to enter the mixed reality remote collaboration system. Then, the local operator starts the AR client, and the AR client automatically turns on the HMD camera to capture the working scene in real time At the same time, the system automatically turns on the HoloLens to obtain the local operator's gaze point and render it in the local operator's field of vision in real time, and then uses the built-in camera of HoloLens to collect on-site video and use the depth sensor to collect on-site point cloud data and gesture information;

[0035] Step 2: The remote expert starts the VR client, and at the same time, the system automatically starts the aGlass-DK2 eye tracker, LeapMotion, and HTC VIVE helmet to collect the remote expert's eye movement information and gesture information;

[0036] Step 3: Turn on the central server and connect to the network to test that the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a remote operation guiding system and method based on mixed reality. The system comprises an AR client, a VR client and a central server. The AR client transmits the collectedon-site video to the VR client through a network for rendering a video stream to an interactive virtual plane in a three-dimensional space. The VR client transmits annotation contents of experts on avirtual plane to the central server through a network for fusion of virtual annotation and an on-site three-dimensional reconstruction model, and the central server transmits fusion data to the AR client through the network for realizing fusion display with a physical space. According to the method, information blind spots in remote operation guidance can be eliminated, and the problem that complex operation guidance intention cannot be accurately and visually expressed due to a single interaction mode and low interaction dimension in the remote operation guidance is solved.

Description

technical field [0001] The invention relates to a remote operation guidance method for mechanical assembly or maintenance, in particular to a remote operation guidance system and method based on mixed reality. Background technique [0002] With the development of advanced manufacturing industry, the mechanical structure has become more and more sophisticated, so that the mechanical assembly and maintenance process frequently requires the technical support of experts from different departments such as process designers and structural designers. The physical space is isolated. When on-site technical workers encounter problems and need expert support, they can only communicate by phone, and even need experts to arrive at the site to demonstrate and guide in person. However, telephone communication can only convey information through voice, which is not conducive to solving assembly and maintenance problems involving complex operations. The time cost of local experts frequently ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L29/06H04N13/204H04N13/194H04N13/275H04N13/383H04N13/398G06F3/01G06Q10/00G06Q50/04
CPCH04N13/204H04N13/194H04N13/275H04N13/383H04N13/398G06F3/011G06Q10/20G06Q50/04H04L67/131Y02P90/30Y04S10/50
Inventor 王鹏白晓亮邵立常壮
Owner NANJING SOARING CYBER REALITY INNOVATION CENT CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap