55 results about "Hand skeleton" patented technology

The invention discloses a television program host interaction system based on Kinect. The television program host interaction system is characterized in that the system comprises a digital camera, Kinect equipment, a subtitle machine and synchronous display equipment, wherein the digital camera is used for shooting a television program; the Kinect equipment is used for obtaining an RGB (Red, Green and Blue) image and human body skeleton information of a host in the television program; the subtitle machine is used for carrying out image registration on the RGB image and a video image, obtaining hand skeleton information in the human body skeleton information and generating a semantic responding command and a control responding command; and the host can adjust the position of a hand in real time according to the synchronous display equipment. The invention provides a novel non-contact type hoisting way for television hoisting, so that the host can more freely and flexibly control a program playing process on a space; and meanwhile, the investment of labor and equipment cost is reduced.

The invention discloses a virtual hand grasp simulating method based on motion capture. The method includes the following steps of building a correct virtual hand skeleton model according to physiological anatomical properties of a real human hand; building a skeleton kinematic model via the virtual hand skeleton model; importing the skeleton kinematic model into a test platform, carrying out impact test between the virtual hand and an object according to a classification method expressed by a 3D (three-dimensional) model and a level bounding box ; obtaining a torsion spring model and a drawing force spring model of a nonlinear spring model according to a linear spring model, and calculating five grasping forces of each finger; obtaining a resultant force of each finger by combining the five grasping forces respectively for each finger, and providing a rendered result of the resultant force; judging whether the rendered result of the resultant force satisfies an ergonomic test or not, ending the steps on yes; and adjusting a corresponding product on no until the product satisfies the ergonomic test. The virtual hand grasp simulating method based on motion capture is capable of shortening the production cycle and reducing the product cost.

The invention provides a hand position marking method and apparatus. The method comprises the steps of obtaining a to-be-identified image, wherein the to-be-identified image at least comprises a handshape; and according to the to-be-identified image and a pre-built hand position identification model, determining a hand position and a hand skeleton key point position in the to-be-identified image.According to the hand position marking method and apparatus provided by the invention, the hand position identification accuracy can be improved and the identification process is not influenced by environmental factors.

The invention provides an AR-oriented navigation type interactive paradigm system. The system comprises an input and perception module, a fusion module and an application expression module; the inputand sensing module obtains hand skeleton depth information through kinect to obtain a gesture depth map, and gesture recognition is completed; t experimental keywords are classified through voice keyword recognition and extraction, andthe similarity probability is obtained through similarity calculation to complete voice recognition; the fusion module classifies user intentions by calculating signal perception of different states in different modes according to gesture recognition, voice recognition and force feedback expression, determines multi-mode intentions and sets an intention expert knowledge base; and the expression module is used for carrying out voice navigation and visual presentation on the virtual chemical experiment and carrying out perceptual force feedback expression. Gesture and voice multi-modes are fused by adopting a natural interaction technology, and a natural interaction mode is explored, so that the user load is reduced, and different user experiences are brought.

The invention discloses a stereoscopic hand 3D skeleton model real-time reconstruction method based on a binocular color imaging system. The binocular color imaging system is used for shooting a handto obtain two color images of the same hand in different viewing angles. The method includes: S1, detecting the position of the hand; S2, detecting plane key points of the hand; S3, recovering three-dimensional coordinates according to the plane positions of the key points in a binocular image; and S4, performing fitting to form a three-dimensional hand skeleton by employing the three-dimensionalcoordinates of the key points. According to the method, 3D reconstruction of the hand is performed only by employing the binocular color image without any extra special equipment. The invention also provides an apparatus based on the method.

The invention relates to a hand physiological information detection method, which comprises the following specific steps of: detecting hand integrity; detecting wrist flexibility; finger flexibility detection. The invention provides a hand physiological information processing method based on a skeleton key point dynamic identification technology. A hand in motion is scanned in real time, and computer analysis is carried out through a unique algorithm and a model, so that hand information can be analyzed and processed quickly and accurately, data analysis is carried out on related information such as whether hand skeleton key points, hand wrist and upper limb functions are perfect and hand integrity and hand coordination, and accurate integrity and flexibility detection results are given.

The embodiment of the invention provides a bone age detection method. The bone age detection method comprises the following steps: acquiring basic information of a target user provided by a client; acquiring hand skeleton image information of a target user; inputting the hand bone image information into a first bone age evaluation model so as to output a first probability corresponding to each first bone age stage through the first bone age evaluation model; determining the first bone age stage with the highest first probability as preliminary bone age information of the target user; inputtingthe basic information of the target user and the preliminary bone age information into a second bone age evaluation model so as to output a second probability corresponding to each second bone age stage through the second bone age evaluation model; and determining the second bone age stage with the highest second probability as the final bone age information of the target user, generating an electronic diagnosis form according to the final bone age information, and sending the electronic diagnosis form to the client. According to the embodiment of the invention, bone age detection efficiencyand detection accuracy are effectively improved.

The invention relates to a gesture tracking and accurate fingertip positioning method based on Kinect. According to the method, the tracking quality of hand disordered motion and the accuracy of fingertip positioning are improved. Taking Kinect as an example, the method comprises: reading depth information from Kinect, and completing shielding repair of hand skeleton information through assistanceof skeleton data and forward kinematics; after the optimized bone joint point information is obtained, conducting smooth processing on the tracked palm motion information through a Kalman filtering algorithm; then, extracting a hand region from the depth image by using the real-time palm coordinate information and adopting a connected region analysis algorithm of firstly performing coarse extraction and then performing fine extraction; and finally, obtaining a convex hull of the hand contour, filtering out fingertip approximate points by adopting a method of analyzing a geometrical relationship between the convex hull and a palm center point, and calculating a curvature value to obtain accurate fingertip points. Therefore, real-time tracking of the hand is realized, and the accuracy of fingertip positioning is improved according to experimental result analysis.

The invention provides a gesture recognition and control method and device and virtual reality device.The gesture recognition method comprises the steps of acquiring a hand image, collected by each camera in binocular cameras, of a user; recognizing a first group of hand skeleton points in the obtained hand image through a pre-constructed recognition model, and obtaining a hand skeleton point image in which the recognized first group of hand skeleton points are marked on a hand region of the hand image; according to the obtained hand skeleton point image, obtaining a two-dimensional position relationship and a three-dimensional position relationship between skeleton points in a second group of hand skeleton points as hand posture data of the user; and performing gesture recognition of the user according to the hand posture data.

The invention provides a gesture recognition system and method based on VR virtual office, and the system comprises: a camera which is disposed above a user operation region, and collects a depth image and an infrared image of the user operation region according to frames; the model construction part that is used for constructing a hand detection model and a hand skeleton point recognition model;the on-line detection tracking part that is used for inputting the images acquired by the camera into the hand detection model and the hand skeleton point identification model to obtain coordinates ofa plurality of skeleton points of the hand under camera coordinates; the first calibration part that is used for transferring the skeleton points of the hand under the camera coordinate system of theskeleton point identification module to the skeleton points of the hand under the helmet-mounted display coordinate system; and the rendering part that is used for rendering the three-dimensional model of the hand corresponding to the current frame in the virtual scene of the helmet-mounted display. According to the gesture recognition system and method, the gesture position information of the hand in the virtual scene is restored with high precision and high stability.

The invention provides a hand motion reconstruction method and device, and the method comprises the steps: obtaining a hand depth data set, wherein each hand depth data in the hand depth data set comprises hand depth picture information and a corresponding hand skeleton coordinate; controlling a preset hand model to adjust the posture according to each hand depth data in the hand depth data set, and obtaining posture parameters of the fitting hand model after posture adjustment; and according to the attitude parameter Gaussian distribution function of the fitting hand model corresponding to each hand depth data, carrying out hand motion reconstruction according to the Gaussian distribution function. the technical problems that motion of hands is flexible and complex, serious shielding often occurs in the process of interacting with an object in the prior art can be solved, a more accurate three-dimensional reconstruction result can be obtained through isomorphic single-viewpoint RGB-Ddata, and a better object and hand interaction reconstruction result can be obtained under a simpler hardware condition.

Provided is an artificial upper limb where a labor tool can be installed. A tool base connected with the labor tool is arranged at the palm part of a hand skeleton and provided with a locking device matched with a tool handle. A thumb in fingers is connected with a linkage mechanism. The linkage mechanism comprises a front arm receiving cavity, an upper arm clamping belt, a hollow bolt, a pull rod, a rotating plate, a shaft rod, a hinge arm and a power rod, wherein one end of the rotating plate is fixedly connected with the thumb, the other end of the rotating plate is connected with one end of the pull rod through a tension spring in a hanging mode, the other end of the pull rod is movably connected with a shifting fork at one end of the power rod in a hanging mode, the other end of the power rod is hinged to one end of the hinge arm, and the other end of the hinge arm is fixedly connected with the upper arm clamping belt. The power rod is composed of a pull arm and an adjusting arm which are connected through threads. A rubber cushion is laid between the hand skeleton and the arm receiving cavity. A baffle is arranged on the hand skeleton. Various labor tools can be installed on the artificial upper limb, and physically-challenged people can conveniently live a daily life and participate in labor.

Example implementations described herein are directed to systems and methods for skill assessment, such as hand washing compliance in hospitals, or assembling products in factories. Example implementations involve body part tracking (e.g., hands), skeleton tracking and deep neural networks to detect and recognize sub-tasks and to assess the skill on each sub-task. Furthermore, the order of the sub-tasks is checked for correctness. Beyond monitoring individual users, example implementations can be used for analyzing and improving workflow designs with multiple sub-tasks.

The invention discloses an anti-shielding hand key node tracking method. The method comprises: a panoramic camera is constructed and calibrated, a large number of hand attitude image samples are obtained, the image samples are inputted into a convolutional neural network for image pooling convolutional processing, so that the convolutional neural network is trained and is iteratively optimized; and a common camera captures real-time image information of shielded hand key nodes, real-time image identification is carried out on the trained convolutional neural network after iterative optimization, and dotted line images of hand key nodes and hand skeleton are outputted. According to the invention, the shielded images are captured in real time by the common camera and the shielded hand key nodes are identified and tracked by using the convolutional neural network after iterative optimization. Therefore, a defect that the shielded hand key nodes can not be identified and tracked in the prior art is overcome; and thus the shielded hand key nodes can be identified and tracked in a shielded state.

The invention discloses a hand geometric motion reconstruction method and device based on a multi-voxel block. The method comprises a step of acquiring a human hand skeleton according to depth data ofa first frame and initializing the multi-voxel block, a step of performing motion estimation on a hand through a reconstructed hand model in the multi-voxel block to obtain new depth data, and a stepof fusing the new depth data into the multi-voxel block and updating a geometric model of the hand so as to simultaneously reconstruct the motion and geometric models of the hand. According to the method, the geometric model and motion information of the human hand can be reconstructed at the same time, the dependence on a parametric model in a traditional human hand motion reconstruction processis avoided, the robustness of the human hand reconstruction technology is enhanced and the application range of the human hand reconstruction technology is expanded, and the development of the humanhand reconstruction technology is promoted.

The making process of handicraft hand includes the following steps: making hand mold; making hand and wrist skeleton with metal wire; preparing silicone rubber material for injection mold; setting the hand skeleton inside some mold, injecting silicone rubber, vibration to eliminate bubble, curing and demolding, trimming; preparing nail painting silicone rubber material and painting nail; and bending fingers to form different hand shapes. Thus handicraft hand has vivid shape capable of being altered at will.

In a telepresence scenario with remote users discussing a document or a slide, it can be difficult to follow which parts of the document are being discussed. One way to address this problem is to provide feedback by showing where the user's hand is pointing at on the document, which also enables more expressive gestural communication than a simple remote cursor. An important practical problem is how to transmit this remote feedback efficiently with high resolution document images. This is not possible with standard videoconferencing systems which have insufficient resolution. We propose a method based on using hand skeletons to provide the feedback. The skeleton can be captured using a depth camera or a webcam (with a deep network algorithm), and the small data can be transmitted at a high frame rate (without a video codec).

The invention provides a deformable grabbing interaction method under a virtual reality environment. The method specifically includes the steps of creating and adopting two layers of skin for bindinga hand skeleton to form a virtual 3D hand model, creating a smooth directional distance field of virtual 3D objects through a smoothing filter operator, and then adopting the distance field for driving a three-dimensional virtual hand contact surface to be deformed. The defects that interaction of a virtual reality hand grip is unnatural and hands which are not identified by force feedback glovesare likely to penetrate through the objects are overcome, and by using the two layers of skin for binding the virtual 3D hand skeleton and adopting equidistant surfaces for creating the virtual 3D objects, interaction is achieved; moreover, the smoothing operator filters out high-frequency contact burr to make the hand model always attached to the surface of a grabbed object, and therefore the natural interaction experience is improved.

The invention discloses a hand epiphysis development grade determination method and related equipment. The method comprises the steps of obtaining a feature map of a first hand skeleton image; inputting the feature map into a preset hand epiphysis development grade determination model to obtain a position probability map of hand epiphysis points and a development grade probability map of the handepiphysis points output by the hand epiphysis development grade determination model; and determining the development level of the hand epiphysis point according to the position probability graph of the hand epiphysis point and the development level probability graph of the hand epiphysis point. According to the invention, the probability that each position in the hand bone image is an epiphysis point position and the probability that the development level of each position is determined are determined through the hand epiphysis development level determination model, and the development level ofeach hand epiphysis point in the hand skeleton image is determined; according to the technical means, the technical problem that an existing bone age prediction method is low in reliability is solved, and then the technical effect of scientifically determining the bone age is achieved.

The embodiment of the invention relates to the technical field of robots, in particular to a robot internal skeleton structure and a robot. The robot internal skeleton structure comprises a head skeleton, a thoracic cavity skeleton, a left hand skeleton, a right hand skeleton, a lower body skeleton and a chassis skeleton, wherein the upper end of the thoracic cavity skeleton is rotatably connectedwith the head skeleton, the left hand skeleton is rotatably connected with one side of the thoracic cavity skeleton, the right hand skeleton is rotatably connected with the other side of the thoraciccavity skeleton, the upper end of the lower body skeleton is rotatably connected with the lower end of the thoracic cavity skeleton, and the chassis skeleton is rotatably connected with the lower endof the lower body skeleton. In this way, modular design of a robot internal skeleton can be achieved, and the robot internal skeleton is conveniently demounted and mounted.

The invention discloses an intelligent bionic manipulator. The intelligent bionic manipulator comprises hand bone modules having the same sizes and shapes with real human hand skeletons, joints connected through bionic ligaments, pneumatic artificial muscles having the same layouts with real human hand muscles, and an air supply control system connected with each pneumatic artificial muscle; the pneumatic artificial muscles comprise rubber pipes, and fiber woven sleeves sleeving the rubber pipes; the rubber pipes are connected with air pipes of the air supply control system; the pneumatic artificial muscles are divided into internal muscles and external muscles; and the outer diameters of the internal muscles are smaller than the outer diameters of the external muscles. When the output force is guaranteed, the size of the bionic manipulator is reduced, the flexibility of the bionic manipulator is improved, and the smoothness and the stability are improved. Compared with a traditional bionic manipulator, the bionic manipulator is high in shape and size similarity with human hands, and has the characteristics of low mass, flexibility and smoothness.

The invention provides a Parkinson's disease fist-clenching task evaluation method and system, a storage medium and a terminal. The Parkinson's disease fist-clenching task evaluation method comprises the following steps: acquiring video information including the fist-clenching action of a Parkinson's disease patient; extracting hand skeleton sequence data from the video information, and preprocessing the hand skeleton sequence data; extracting hand joint point position features and hand skeleton vector features based on the preprocessed hand skeleton sequence; inputting the hand joint point position features and the hand skeleton vector features into a double-flow adaptive space-time diagram convolutional neural network; and inputting the fist-clenching action features output by the double-flow adaptive space-time diagram convolutional neural network into a classifier to obtain an evaluation score of the Parkinson's disease fist-clenching task. According to the evaluation method for the Parkinson's disease fist-clenching task, reliability evaluation of the Parkinson's disease fist-clenching task is achieved based on the tree pooling module and the space-time diagram convolutional network.

The invention provides a method for estimating the proportion of virtual hand bones in a pow transformation simulation VR system which analyzes the skeletal structure of the hand by kinematics, and simplifies the human hand skeleton model. Based on real-time attitude angle data of inertial sensors, combined with the statistical value of hand skeletal length in anatomy as the skeletal length of virtual hand, two special postures are calibrated respectively, and the skeleton ratio of human hand is calculated. According to the different structure of user's hand, the skeleton ratio of virtual handis adaptively changed to realize the fine reappearance of virtual hand to user's hand movement, and the virtual hand is driven to realize the training of substation simulation, and the virtual hand in the substation simulation virtual environment is driven to complete the corresponding action. The method realizes the initial deviation correction of the sensor data and reduces the error in the process of hand fine motion tracking caused by the influence of the individual hand proportion difference of the user.

The invention relates to a determination method and device for human hand skeleton parameters, electronic equipment and a storage medium. The method comprises the steps of collecting a target hand type; determining a coordinate C # ij of a jth generation child node on each finger in the target hand shape in a world coordinate system; and calculating a rotation matrix R'ij of the local coordinate system CS'ij relative to the local coordinate system CS'i (j-1) in the human hand skeleton according to the incidence relation between the coordinate C'ij and the coordinate C # ij of the jth generation child node in the rotated target hand shape in the local coordinate system CS'ij. A target hand shape of a rotation matrix is determined so that the target hand shape does not need to be manually drawn; instead, the embodiment can be directly adopted to construct the association relationship between the coordinate Cij and the coordinate C'ij, and the rotation matrix of each node in the target hand type relative to the father node is automatically determined according to the association relationship, so that the reduction of manual participation in the process of determining the rotation matrix is facilitated, and the cost and time consumption are reduced.

The embodiment of the invention provides a gesture recognition method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining a to-be-recognized image which comprises a hand region and a background region; performing extraction processing on the to-be-identified image through a pre-trained hand extraction model to obtain a hand region image corresponding to the hand region, the hand region image not including any region in the background region; processing the hand region image through a pre-trained hand skeleton construction model to obtain a handGaussian heat map; and recognizing the hand Gaussian heat map through a pre-trained gesture classification model to obtain a gesture classification result. According to the embodiment of the invention, the background interference signal in the to-be-recognized image is removed through the hand extraction model, so as to obtain the interference-free hand region image, thereby improving the precision of subsequent gesture recognition.

The invention discloses an action recognition method and device based on a first person view angle. The method comprises the following steps: acquiring a to-be-processed RGB video frame, wherein the to-be-processed RGB video frame comprises hand action image information based on a first person view angle; inputting all the RGB video frames to be processed into a pre-trained HOPE-Net deep neural network to obtain corresponding hand joint point position information; selecting a predetermined number of target RGB video frames from all the to-be-processed RGB video frames, and inputting the selected target RGB video frames into the I3D model for identification to obtain corresponding video frame features; inputting the hand joint point position information into an AGCN model to obtain corresponding position information features; and fusing the video frame features and the position information features in a one-to-one correspondence manner to obtain the probability of identifying the action instruction. The video frame is sequentially subjected to hand skeleton joint extraction and RGB and skeleton action feature extraction, and finally feature fusion is performed to obtain the action instruction probability, so that dependence on external hardware equipment is eliminated, and the method has strong robustness for illumination and scene change.

The invention discloses a visual gesture recognition system and method. A palm detection module is designed by utilizing rigid characteristics of a palm to detect whether a hand exists in a current image or not; after it is detected that the hand exists in the current image, 21 3D hand skeleton key point coordinates are positioned in the detected hand area; and identifying a specific gesture according to the obtained position relationship between the key points. Compared with a touch key mode, non-contact man-machine interaction can be achieved, and a new scheme is provided for man-machine interaction.

The invention discloses a rigidity-variable skeleton of a rigid-flexible coupling dexterous hand. The dexterous hand comprises a palm skeleton and single-finger skeletons rotationally connected with the palm skeleton. The single-finger skeletons include a thumb skeleton; the thumb skeleton comprises a fingertip and a tail knuckle; each of the other single finger skeletons comprises a fingertip, a middle knuckle and a tail knuckle, and the middle knuckle and the fingertip are movably connected through a joint, the middle knuckle and the tail knuckle are movably connected through a joint, and the fingertip and the tail knuckle are movably connected through a joint; each joint comprises a shell, an air bag arranged in the shell and a set of rotating bosses which penetrate through the wall of the shell and are coaxially arranged; each middle knuckle and the corresponding fingertip which are rotationally connected with each other through the corresponding rotating bosses, each middle knuckle and the corresponding tail knuckle are rotationally connected with each other through the corresponding rotating bosses, and each fingertip and the corresponding tail knuckle are rotationally connected with each other through the corresponding rotating bosses; and when each air bag expands, the outer wall of the air bag abuts against one ends of the corresponding rotating bosses to drive the rotating bosses to move towards the two sides along the axis, and the other ends of the corresponding rotating boss abut against the corresponding fingertip or the corresponding middle knuckle or the corresponding tail knuckle. According to the rigidity-variable skeleton of the rigid-flexible coupling dexterous hand, the rigidity of the finger joints can be quickly changed, so that the whole hand is better attached to an object, finer objects are grabbed, and the safety of human beings is guaranteed.

The invention relates to a human body hand skeleton detection method, which comprises the following steps: firstly, processing a to-be-recognized picture, obtaining an initial posture of a target human body hand skeleton, then obtaining feature maps output by different decoding layers in the process of processing the to-be-recognized picture, and then processing the feature maps to obtain feature map data; extracting position data corresponding to an initial posture from the feature graph data as input data, and finally inputting the initial posture and the input data to a trained graph convolutional neural network to obtain a final posture of the target hand skeleton, wherein the matrix representation of the graph convolutional neural network is determined according to the constraint relation of the human hand skeleton structure. According to the technical scheme, the positions of the shielded key points can be accurately adjusted by combining the basic constraint information between the human body joint structures and the related data of the shielded key points contained in the mined feature map, so that the detection accuracy of the human body hand skeleton is relatively high.

Embodiments of the invention provide a sample data generation method and device and electronic equipment. When sample data used for training a deep learning model is constructed, a hand kinematics model can be constructed, and the hand kinematics model is used for describing the influence of a hand posture and a hand skeleton length on the position of a hand key point, generating a plurality of groups of sample data according to hand posture information of a standard gesture predefined by a user and the hand kinematics model; Through the method, a large amount of sample data can be quickly and conveniently obtained, a user does not need to manually label hand key points, the sample data obtaining difficulty is reduced, the constructed sample data can be more diversified and higher in accuracy, and therefore the prediction precision of the trained deep learning model can be improved.