Three-dimensional interaction method based on intelligent mobile phone and system thereof

Abstract

A three-dimensional interaction method based on an intelligent mobile phone and a system thereof are disclosed. The system comprises a client, a network layer and a server. The client is connected with the server through the network layer, wherein the client and the server comprise an interaction interface layer and an application logic layer. After receiving a space attitude and a touch screen state, the client interaction interface layer sends the space attitude and the touch screen state to the client application logic layer so as to calculate a motion state and carry out coding. Then, coded data is sent to the server through the network layer. The server application logic layer decodes the received data and restores a two-dimensional or three-dimensional interaction mapping state. And the data is displayed through the server interaction interface layer. According to the invention, the nature man-machine interaction based on mobile phone motion tracking is supported. It is innovatively brought forward that the space attitude of the mobile phone is calculated based on a sensor and the interaction of the finger motion based on the touch screen is calculated too. Touch screen input is combined and a real-time and accurate interaction process can be achieved. Power consumption is small and endurance of interaction equipment is enhanced.

Description

technical field [0001] The invention belongs to the field of human-computer interaction, and in particular relates to a smart phone-based three-dimensional interaction method and system. Background technique [0002] In recent years, with the development of computer software, hardware and information network, the source of people's information acquisition is no longer limited to the traditional desk. Large-size, high-resolution electronic screens that can present more content are increasingly appearing in non-public places (family-style large-size LCD TVs, home projectors) and public places (meeting rooms, lecture halls, airports, train stations, etc.) , shopping malls, museums) etc. Although electronic large screens can present a large amount of rich content, most of them are not interactive due to the lack of suitable interactive technologies and interactive devices, making it difficult for users to obtain interesting content or influence and change the content of the lar...

AI Technical Summary

Problems solved by technology

The first interaction method maps the functions of the physical buttons and the operations on the large screen; due to the limitation of the number of physical buttons on the mobile phone, this interaction method can only achieve a few relatively simple operations. To achieve complex or more operations, You need to use combination keys, which will inevitably increase the user's memory burden

The second interactive method sends and displays the content on the large screen to the touch screen of the mobile phone through the wireless network, and the operation on the touch screen is directly mapped to the operation on the large screen; Real-time transmission and interaction process are prone to delays, and for some three-dimensional application scenarios, the two-dimensional touch operation on the touch screen cannot correctly reflect the user's interaction intention

The third interaction method uses the mobile phone as a pointing device through vision-based motion tracking. However, due to the high complexity of most camera-based attitude calculation algorithms, the interaction process has a large delay, and turning on the camera consumes a lot of power. , will cause the battery life of the interactive device to decline, thereby affecting the user experience; and most camera-based algorithms can only achieve the interaction of the two degrees of freedom in the horizontal and vertical directions. quasi-device screen or background environment)

[0004] To sum up, no matter what kind of interaction mode is used, it has defects in varying degrees in terms of timely information transmission, convenient and fast operation, 3D scene reproduction and energy consumption.

Images