Keyboard tracking method and tracking system

Abstract

The application provides a keyboard tracking method and a tracking system. The keyboard tracking method includes the following steps: acquiring an image sequence including images of a keyboard and a user's hand; using a deep learning algorithm to process the image sequence including the keyboard images to obtain a 2d key of the keyboard The coordinates of the point, the mask of the keyboard and the mask of the hand; the optimization algorithm based on feature points is used to process the coordinates of the 2d key points of the keyboard and the mask of the keyboard, and the 3d estimated pose of the keyboard is obtained; The optimization algorithm optimizes the 3D estimated pose of the keyboard to obtain the optimized 3D pose of the keyboard; obtains the model information of the keyboard, renders the 3D pose and hand mask of the keyboard of the obtained model, and completes the tracking of the keyboard. The present application enables the user to use the keyboard for input naturally, which is convenient to operate and improves the user experience.

Description

technical field [0001] The present application belongs to the technical field of image processing, and in particular relates to a keyboard tracking method and a tracking system. Background technique [0002] Virtual reality technology is an important branch of simulation technology. It uses computer technology and various software and hardware tools to generate real-time dynamic, three-dimensional and realistic images and scenes, and can imitate various human perceptions and communicate with users. interact. Since the germination of virtual reality technology in 1963, the theory of virtual reality technology has been relatively complete, and in recent years, virtual reality technology has been widely and deeply researched and applied in many industries such as military simulation, entertainment games, medical care, and construction. [0003] In the human-computer interaction process of the existing virtual reality device, the following two ways are usually used to realize t...

AI Technical Summary

Problems solved by technology

One is to use handles, gestures, etc. for text input. This method has problems such as slow input speed and poor user input experience.

The other is to manually calibrate the virtual keyboard and the real keyboard, and then use SLAM to track. In this way, the keyboard needs to be re-calibrated every time it moves, which is inconvenient to operate and the user input experience is poor.

Images