Hand key point space coordinate acquisition method based on binocular vision

Abstract

The invention provides a binocular vision-based hand key point space coordinate acquisition method, which comprises the following steps of: firstly, performing three-dimensional calibration on binocular cameras, and establishing coordinate system conversion models, namely acquiring internal and external parameters and distortion coefficients of the cameras and a rotation and translation matrix between the two cameras; secondly, preprocessing the video shot by the binocular camera, wherein the preprocessing comprises cutting, distortion correction and the like; processing the video frame by frame by using a machine learning pipeline, and obtaining pixel coordinates of 21 key points of the hand; and finally, calculating the real coordinates of the hand key points in the three-dimensional space by adopting a least square method based on an optical axis convergence model. According to the method, binocular vision information is utilized, the three-dimensional space coordinates of the 21 hand key points containing all the joint points are accurately positioned and recovered by simulating the human eye structure, hand form reconstruction is more accurate, and accurate technical support is provided for application research of hand key point positioning in human-computer interaction.

Description

technical field [0001] The present invention relates to the field of computer vision, in particular to a method for acquiring three-dimensional coordinates of human key points based on binocular vision, and at the same time relates to technical fields such as digital image processing, spatial three-dimensional information acquisition, and human-computer interaction. Background technique [0002] Binocular stereo vision is an important branch of computer vision. It simulates the human visual system, uses the principle of parallax to sense the three-dimensional spatial information of objects, and reconstructs the shape and position of the object scene. Among them, accurately detecting and locating the spatial position of the hand from the video has always been a hot issue in this field, and it has high application value in the fields of virtual reality, augmented reality, somatosensory games, human-computer interaction, and three-dimensional measurement. [0003] However, the ...

AI Technical Summary

Problems solved by technology

[0003] However, the acquisition of the current three-dimensional coordinates of the hand is limited to individual key points, such as fingertips and palms, which is not enough to reconstruct the motion posture of the entire hand and its positioning in space, and most of them are based on skin color or edge The scheme of detecting and then performing fingertip search on the boundary is very susceptible to the influence of background and ambient light, which reduces the robustness of the algorithm

[0004] At the same time, most of the feature extraction schemes used in the existing spatial point 3D coordinate recovery technology are SIFT algorithms. Although this algorithm has good stability and invariance, it is not enough for feature extraction of smooth edge objects, and it needs to be processed later. The effect of rough matching and fine matching is not good, and it is not the best solution for the extraction of hand key points

[0005] Although there have been many researches on full-hand pose estimation algorithms based on depth images in recent years, due to the small fingertip area and fast movement, the quality of the finger part of the depth map generated by it is poor, resulting in low detection accuracy. The obtained three-dimensional space coordinate error is also larger

Images