Enhanced assembly teaching system based on fingertip characteristics and control method thereof

Abstract

The invention provides an enhanced assembly teaching system based on fingertip characteristics and a control method thereof. The enhanced assembly teaching system comprises an image acquisition module, an image preprocessing module, a hand area segmentation module, a fingertip detection and tracking module and a virtual component model space registration module. The method includes: collecting images of the finger and the interaction plane; preprocessing the acquired image; carrying out segmentation and edge extraction on the hand area; carrying out finger tip detection based on curvature operation and least square fitting, and tracking the finger tip through a method based on combination of Kalman filtering and particle filtering; performing calibration, computer rendering and virtual component model space registration on the image acquisition equipment; and enabling the fingertips to interact with the virtual component to complete plug-in mounting. According to the method, fingertipsare used as new computer input to complete interaction with the virtual object, inconvenience brought by an materialized handheld identifier is abandoned, and when motion is nonlinear, Kalman filtering and particle filtering are combined to improve the positioning accuracy and real-time performance of the target object.

Description

technical field [0001] The invention belongs to the field of enhanced assembly of electronic components, and in particular relates to an enhanced assembly teaching system based on fingertip features and a control method thereof. Background technique [0002] Augmented assembly is the application of augmented reality technology in the field of mechanical and electronic assembly. Enhanced assembly combines the advantages of traditional assembly and virtual assembly, which not only saves the cost of making a large number of parts, but also brings a sense of real experience to user operations. Users operate virtual objects in a real environment, which greatly improves security. guarantee. As an active part of the assembly process, the importance of the human hand is the first. Also in the enhanced assembly, the detection and positioning of the human hand play a decisive role in the subsequent calibration and fusion of virtual and real. In traditional augmented reality interact...

AI Technical Summary

Problems solved by technology

This method cannot realize the direct interaction between the human hand and the virtual environment in a natural state, and the user experience of the user is limited.

Images