Fingertip tracking method based on deep learning and K-curvature method

Abstract

The invention discloses a fingertip tracking method based on deep learning and a K-curvature method, and the method comprises the steps: training a preprocessed data set through a YOLOv3 network model, and obtaining a fingertip detection model; acquiring a video stream by using a camera, inputting the video stream into a detection model, detecting detection frame information, and initializing a Kalman filter; then obtaining a prediction frame by using a Kalman filter, calculating IOU of the detection frame and the prediction frame of the frame, setting an IOU threshold value, judging whether the IOU is greater than the IOU threshold value or not, and if the IOU is greater than the IOU threshold value, updating the Kalman filter to obtain a fingertip tracking frame; otherwise, correcting the fingertip position by using a K-curvature method, and updating the Kalman filter; and finally, setting a time threshold T-max, and stopping tracking if tracking information is not detected in a time threshold frame. The influence of a complex environment on the detection accuracy is weakened, the detection speed is increased, and the accuracy and robustness are improved.

Description

technical field [0001] The invention belongs to target detection and tracking technology, in particular to a fingertip tracking method based on deep learning and K-curvature method. Background technique [0002] Hand posture detection and tracking is a popular direction in the field of human-computer interaction and computer vision. Its sub-direction fingertip detection and tracking technology is an important part of human hand posture detection and tracking technology. By detecting and tracking fingertips, it can Human-computer interaction behaviors such as handwriting, clicking on virtual screens in the air, gesture recognition, and smart teaching provide a good foundation. [0003] In the fingertip detection and tracking technology, the correctness of fingertip detection and the speed and accuracy of tracking are very important; at present, the algorithms based on target detection are mainly divided into traditional detection algorithms and detection algorithms based on d...

AI Technical Summary

Problems solved by technology

[0003] In the fingertip detection and tracking technology, the correctness of fingertip detection and the speed and accuracy of tracking are very important; at present, the algorithms based on target detection are mainly divided into traditional detection algorithms and detection algorithms based on deep learning. Detection algorithms mainly include DPM (Deformable Parts Model), selective search, etc. These methods have shortcomings such as high time complexity, poor robustness and poor generalization due to complex environments in practical applications. Occluded fingertips are difficult to detect and track effectively

[0004] The fingertip detection and tracking method based on deep learning can increase the accuracy and robustness very well. The mainstream deep learning target detection algorithm is mainly divided into two-stage detection algorithm and single-stage detection algorithm. Among them, YOLO in the single-stage detection algorithm The series takes both speed and accuracy into consideration; tracking algorithms are widely used at present, such as Kalman filter and particle filter based on filter theory, but pure filter-based tracking algorithms in practical applications are not suitable for temporarily occluded indicators. The missed detection rate of the tip is high, and there is also a lot of room for improvement in real-time performance

Images