Screen-oriented augmented reality interaction method and device and storage medium

An augmented reality and interactive method technology, applied in the input/output process of data processing, instruments, electrical digital data processing, etc., can solve problems such as inability to interact, and achieve the effect of enhanced display, close-range display convenience, and strong versatility

Pending Publication Date: 2022-01-21
XI AN JIAOTONG UNIV
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to provide a screen-oriented augmented reality interaction method, device and storage medium, to solve the problem that the augmented reality environment cannot interact with traditional physical devices with screens as display terminals, and to provide a more powerful virtual reality Combined content display interaction function simplifies user operation and improves user experience

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Screen-oriented augmented reality interaction method and device and storage medium
  • Screen-oriented augmented reality interaction method and device and storage medium
  • Screen-oriented augmented reality interaction method and device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0057] An implementation process of the present invention is as figure 2 shown.

[0058] Process ① is to set the virtual operation surface to coincide with the real screen display area. According to the shape of the screen, a virtual operation surface is generated in the virtual-real mixed space presented by the augmented reality device. If the real screen is flat, a virtual operating plane is generated; if the real screen is curved, a virtual operating surface corresponding to the curvature is generated. Through translation, rotation, and zooming, the virtual operation surface can completely coincide with the entire display area of ​​the real screen. Such as figure 2 As shown in FIG. 1 , the user first moves the position of the virtual operation surface to coincide with the real screen, and adjusts the positions of other virtual operation surfaces in space. The virtual operation surface is set to be invisible to the naked eye in the virtual-real mixed space, and the sta...

Embodiment 2

[0078] see image 3 , the screen-oriented augmented reality interaction method of the embodiment of the present invention includes in webpage information retrieval:

[0079] Step S1, writing the program, burning the program into the augmented reality device, naming it the GazeAnnotation application, and granting permission to the application that can use eye-tracking.

[0080] Step S2, wear the augmented reality device, open the GazeAnnotation application, and you can see the curtain of the "Webpage Auxiliary Information Prompt System", and the virtual operation surface is attached to the curtain.

[0081] Step S3, through the gesture interaction module, move the black virtual operation surface to the position of the real screen, and perform operations such as zooming and rotation, so that the virtual operation surface and the real computer screen overlap as much as possible. Such as figure 2 As shown in the process ①, in this way, the user can use the augmented reality dev...

Embodiment 3

[0092] The steps of using the screen-oriented augmented reality interaction method on the computer screen in the embodiment of the present invention include:

[0093] While using a computer screen, the user wears an augmented reality device such as figure 2 As shown in process ①, in the virtual-real mixed space under the perspective of the augmented reality device, create a virtual operation surface, adjust its shape, size, position and angle, so that it completely overlaps with the display area of ​​the real computer screen; set the virtual operation surface as virtual and real Invisible to the naked eye in hybrid spaces. Create one or more auxiliary virtual screens and place them around the real computer screen to assist the real computer screen to display other related content.

[0094] Such as figure 2 As shown in process ②, the user selects the page to be viewed. Such as figure 2 As shown in the process ③④⑤⑥, when the user touches the real computer screen control w...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a screen-oriented augmented reality interaction method and device and a storage medium, and the method comprises the steps: creating a virtual operation surface in a virtual-real mixed space, and adjusting the shape, size, position and angle of the virtual operation surface, so as to enable the virtual operation surface to coincide with a real screen display region; creating one or more auxiliary screens to assist the real screen in displaying other related contents; through space coordinate calculation, establishing a mathematical corresponding relation between space coordinates of all points on a virtual operation surface in the virtual-real mixed space and pixel points on a real screen. In this way, interaction modes such as gestures and eye movement can be used for triggering an interaction event in the operating system to respond to the original operating system, and augmented reality response can be carried out in the virtual-real mixed space according to the concerned content; and the virtual screen around the real screen or the virtual operation surface of the real screen is touched to realize multi-screen switching and information display. According to the invention, a virtuality and reality combined content display interaction function can be provided based on a real screen, and long-distance interaction operation of a large screen is also supported.

Description

technical field [0001] The invention belongs to the technical field of augmented reality interaction, and in particular relates to a screen-oriented augmented reality interaction method, device and storage medium. Background technique [0002] Augmented reality devices can realize the superposition of virtual objects and real objects, creating a user experience combining virtual and real. At the same time, augmented reality devices provide a variety of interaction methods, including voice interaction, gesture interaction and eye movement interaction, so that users can interact naturally with virtual objects under the perspective of the device. [0003] However, in the augmented reality environment, it is difficult for users to interact with traditional physical devices (such as computer monitors, mobile phones, tablet computers, etc.) that use screens as display terminals, and cannot be integrated with the original computer interaction system in the device. When interacting...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/0481G06F3/0487G06F3/04883
CPCG06F3/0481G06F3/0487G06F3/04883
Inventor 蔡忠闽金瑞达
Owner XI AN JIAOTONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap