Gesture and pose prediction method and system based on multi-view images

A prediction method and multi-view technology, applied in the field of image processing, can solve problems such as being easily affected by sunlight, poor outdoor experience, self-occlusion of gestures, etc., and achieve the effects of reducing complexity, simple structure, and improving accuracy

Active Publication Date: 2022-07-05
QILU UNIV OF TECH
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Since human gestures have many degrees of freedom, gestures can make a lot of gestures in three-dimensional space, which also leads to the problem of self-occlusion of gestures. In addition, it is difficult for computers to obtain three-dimensional objects only through two-dimensional images. information, making it difficult for the computer to directly predict the attitude angle using only the two-dimensional image of the hand gesture
[0005] The inventors found that at present, in the related technologies of hand gesture and attitude prediction, most of the depth images collected by the depth camera are used to estimate the gesture and attitude. The advantage is that the collected images contain the depth information of the object, but most depth cameras are sensitive to light. It has the disadvantage of being easily affected by sunlight, and is often used indoors, and the outdoor experience is poor

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Gesture and pose prediction method and system based on multi-view images
  • Gesture and pose prediction method and system based on multi-view images
  • Gesture and pose prediction method and system based on multi-view images

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0054] Embodiment 1 of the present disclosure provides a gesture and attitude prediction method based on a multi-view camera, which is different from the previous method for gesture and gesture prediction based on a depth image containing depth information. A training set that enables prediction of arbitrary gestures.

[0055] In this embodiment, a binocular camera is preferably used for gesture prediction, such as figure 1 As shown in the figure, a convolutional neural network is used as a base learner for regression prediction of preset gestures, and a deep integrated gesture and pose prediction model is established by combining with an ensemble learning algorithm. The method mainly includes two stages of model training and prediction.

[0056] The present embodiment provides a detailed description of the proposed gesture and attitude prediction system and method for gestures in the right-hand Cartesian coordinate system. Specifically, it mainly includes the following techni...

Embodiment 2

[0103] Embodiment 2 of the present disclosure provides a wearable gesture and posture prediction system based on a dual-view camera, which solves the problem of difficulty in predicting the spatial posture of a computer only through two-dimensional gesture images, and can realize the prediction of any gesture in space. Prediction of poses from different angles.

[0104] This embodiment takes a typical gesture called a right-hand Cartesian coordinate gesture as an example, such as Image 6 shown, that is, the vertical posture of the thumb, index finger and middle finger of the right hand, and the angle data of the thumb, index finger and middle finger of the gesture in space can be predicted The gesture posture prediction system and prediction method are further described for this hand shape.

[0105] Figure 7 is the module diagram of the wearable gesture and posture prediction system, Figure 4 and Figure 5 The schematic diagrams of the two stages of training and predic...

Embodiment 3

[0111] Embodiment 3 of the present disclosure provides a gesture and posture prediction system based on multi-view images, including:

[0112] a data acquisition module, configured to: acquire gesture image data from at least two viewing angles;

[0113] The posture prediction module is configured to: obtain each gesture image prediction angle according to the acquired gesture images from each perspective and the preset posture prediction model corresponding to each perspective;

[0114] an attitude determination module, configured to: take the average value of the predicted angles of each gesture image as the final gesture angle;

[0115] Among them, in the training of the preset attitude prediction model, the maximum error of the kth convolutional neural network on the training set, the relative error of each sample, the regression error rate of the kth convolutional neural network and the weak learner are calculated. Weight coefficient, update the sample weight according t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present disclosure provides a gesture and posture prediction method and system based on multi-view images, which acquires gesture image data from at least two perspectives; and obtains each gesture image according to the acquired gesture images from each perspective and a preset posture prediction model corresponding to each perspective. Prediction angle; the average of the prediction angles of each gesture image is the final gesture angle; wherein, in the training of the preset posture prediction model, the maximum error of the kth convolutional neural network on the training set and the relative error of each sample are calculated , the regression error rate of the kth convolutional neural network and the weight coefficient of the weak learner, and update the sample weight according to the obtained maximum error, relative error, regression error rate and the weight coefficient of the weak learner; A prediction model is established for each gesture image, and the average of the prediction results is taken as the final prediction result, which improves the accuracy of gesture pose prediction.

Description

technical field [0001] The present disclosure relates to the technical field of image processing, and in particular, to a method and system for gesture and posture prediction based on multi-view images. Background technique [0002] The statements in this section merely provide background related to the present disclosure and do not necessarily constitute prior art. [0003] Gestures are a silent language. Gestures have a strong function of information expression and transmission. People use gestures to express various emotions in life. With the continuous development of artificial intelligence, people can also use gestures to interact with computers. Human-computer interaction is performed with the computer by means of gestures, so that the computer can execute the set commands. The use of gestures for human-computer interaction has many advantages, such as convenient operation, separation from physical contact, remote control, and richer and more natural interactive ope...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & AuthorityPatents(China)
IPC IPC(8): G06V40/20G06V10/774G06V10/26G06V10/44G06V10/82G06K9/62G06N3/04G06N3/08
CPCG06N3/08G06N3/045G06F18/2148
Inventor纪鹏王先建张芳芳马凤英张慧曹茂永
OwnerQILU UNIV OF TECH