Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

237 results about "Hand joint" patented technology

Multimodal three-dimensional hand grid reconstruction method based on diffusion model

The invention belongs to the technical field of human body posture recognition, and discloses a multi-modal three-dimensional hand grid reconstruction method based on a diffusion model, and the method comprises the steps: collecting a depth map and point cloud data through a feature extraction model, and extracting hand joint features according to the depth map and the point cloud data; the initialized standard Gaussian distribution noise and the extracted hand joint features are input into a diffusion model together for iterative denoising, and the hand joint positions are recovered from the noise step by step through multiple iterations; and reconstructing a three-dimensional hand grid through a grid reconstruction module according to the hand joint position. According to the method, based on the color image, the depth map and the three-dimensional point cloud data are fused, so that high-precision three-dimensional hand grid reconstruction is realized, and the speed and accuracy of hand posture reconstruction are remarkably improved.
Owner:JIANGXI AGRICULTURAL UNIVERSITY

Skeleton sign language recognition method of double-flow space-time dynamic graph convolutional network fused with residual learning

The invention discloses a skeleton sign language recognition method of a double-flow space-time dynamic graph convolutional network fused with residual learning, and belongs to the technical field of artificial intelligence and gesture recognition. According to the method, an input gesture skeleton sequence relative to a face is divided into two data streams, namely a hand form data stream and a wrist track data stream through double-reference-system differential homeomorphic mapping; the method comprises the following steps: firstly, processing hand posture data, and capturing a hand joint spatial topological relation by combining a spatial-temporal dynamic graph convolutional network (STDGCNN) with a residual convolutional block; meanwhile, a Finsler trajectory dynamics encoder (FTDE) is adopted to carry out differential geometric modeling on the wrist trajectory, and the direction sensitivity characteristic of the trajectory is captured through a multi-scale causal convolutional network. Then, mutual enhancement of double-flow features is realized through a bidirectional cross feature enhancer (BCFE), and the problem of geometric inconsistency of a heterogeneous feature space is solved through a geometric-driven optimal transmission fusion device (Geometric-OT). The method solves the challenge that a traditional sign language recognition method processes complex space-time correlation of gesture forms and motion tracks at the same time, the technical problems of insufficient relation between hands and faces, insufficient feature expression ability and low space-time feature extraction efficiency, and the problems of geometric inconsistency, single reference system and the like. And the identification accuracy and the real-time performance are obviously improved. Experiments show that the accuracy rate of the method in complex hundreds of sign language vocabulary recognition tasks reaches 95% or above on average, the reasoning speed is only 17ms on average, high-precision real-time sign language recognition is achieved, and the method has higher robustness in complex environments such as noise and shielding.
Owner:刘良锦

Augmented reality device for acquiring three-dimensional position information about hand joints, and method for operating same

An augmented reality device for obtaining three-dimensional (3D) position information of a plurality of hand joints of a user includes a plurality of cameras configured to photograph the user's hand and obtain images; memory storing instructions; and at least one processor; the instructions, when executed, may cause the device to recognize the plurality of hand joints from the images; obtain two-dimensional (2D) joint coordinate values for feature points corresponding to the hand joints; retrieve, from a look up table (LUT), 3D position coordinate values corresponding to distortion model parameters of the cameras, a positional relationship between the cameras, and the 2D joint coordinate values; and output the 3D position information of the plurality of hand joints based on the 3D position coordinate values.
Owner:SAMSUNG ELECTRONICS CO LTD

Biomechanical modeling and normalized training data generation method and device for human hand posture estimation, equipment and medium

The invention relates to a biomechanical modeling and normalized training data generation method and device for human hand posture estimation, equipment and a medium. The method comprises the following steps: acquiring target three-dimensional hand posture data of a target object; for each target object, acquiring reference static biomechanical characteristics and dynamic biomechanical characteristics corresponding to each target three-dimensional hand posture of the target object; for each target three-dimensional hand posture, determining a three-dimensional joint coordinate of the target three-dimensional hand posture and a two-dimensional joint coordinate corresponding to the three-dimensional joint coordinate based on the reference static biomechanical characteristics and the dynamic biomechanical characteristics; wherein the three-dimensional joint coordinates represent the spatial positions of the hand joint points; and determining hand posture training data based on the three-dimensional joint coordinates and the corresponding two-dimensional joint coordinates. By adopting the method, the individual biomechanical structure difference can be eliminated, and the number of training data can be expanded more accurately and reasonably.
Owner:CHINA SOUTHERN POWER GRID ARTIFICIAL INTELLIGENCE TECHNOLOGY CO LTD

Hand pose recognition method and apparatus, device, storage medium, and program product

A hand pose recognition method is performed by a computer device, including: acquiring a current frame of a multi-lens video of a target object; performing hand detection on a first view of the current frame to obtain a first lens detection result; performing hand estimation on a second view of the current frame to obtain a second lens estimation result; removing, from the hand detection boxes in the first view and the hand estimation boxes in the second view, redundant boxes corresponding to redundant hands, and then performing hand joint point recognition on remaining boxes to obtain two-dimensional joint points; converting the two-dimensional joint points into three-dimensional joint points in a three-dimensional hand coordinate system; and converting the three-dimensional joint points in the three-dimensional hand coordinate system into three-dimensional joint points of the current frame in a world coordinate system according to pose estimation parameters corresponding to the current frame.
Owner:TENCENT TECHNOLOGY (SHENZHEN) CO LTD

Self-adaptive exoskeleton robot

The invention discloses a self-adaptive exoskeleton robot which comprises a hand exoskeleton device used for capturing the degree of freedom of hand joints of 20 persons at the same time and conducting force feedback. The perception and force feedback system is used for data acquisition and master control driving and is matched with IO resources of a master control chip; the self-adaptive wearing module is used for sewing the finger exoskeleton to a light, thin and elastic fiber glove, the wearing mode is reinforced through a restraint ring, and meanwhile, an exoskeleton palm structure is designed to be a variable structure capable of being attached to the back of the hand in a self-adaptive mode; the man-machine interaction module is used for man-machine interaction; and the master-slave control system realizes master-slave control. According to the invention, the bionic flexible muscle optimization design, the force feedback technology and the touch force sensor integrated with force feedback are adopted, so that the force of a user can be acquired, and a foundation is laid for force closed-loop control.
Owner:KUNSHAN ZHIBO MICRO INTELLIGENT TECHNOLOGY CO LTD

Stroke hand rehabilitation training method based on multi-modal data fusion

The invention discloses a stroke hand rehabilitation training method based on multi-modal data fusion, particularly relates to the field of hand rehabilitation training, and is used for solving the problems that in the existing rehabilitation process, the motion solidification risk cannot be recognized, and the training scheme is lack of individualized dynamic adjustment. According to the method, hand joint motion features are obtained by constructing a standardized hand function micro-evaluation task, and a personal motion mode baseline is established in combination with dynamic time warping and multi-scale permutation entropy analysis; extracting a neural motion curing index based on the difference change between the current motion mode and the optimal motion mode, and analyzing and generating a function recovery momentum feature by combining the rehabilitation progress gradient and the acceleration inflection point; and fusing the electromyographic signal coupling coordination index to generate a rehabilitation risk score, and when the risk exceeds a dynamic threshold value, triggering adaptive recombination of a training strategy to realize accurate early warning and personalized training regulation and control. Rehabilitation monitoring precision and training pertinence can be effectively improved, and high-quality recovery of stroke hand functions is promoted.
Owner:FUJIAN UNIV OF TECH

Control device for substrate transfer robot and control method for joint motor

A control device controls a substrate transfer robot including a hand, a joint, and a joint motor. The control device modifies a target position of the hand when it places substrate at transfer destination, based on positional misalignment information indicating misalignment of substrate. To change a position at which the hand places the substrate to target position after modification, the control device performs first velocity control and second velocity control. In first velocity control, velocity of hand that is approaching target position before modification with the velocity decreasing is made constant by correcting. In second velocity control, velocity of hand decreases by slope equal to the slope of velocity of the hand immediately before first velocity control and hand is stopped at target position after modification. Time during which the first velocity control is performed increases as a magnitude of change in the target position due to the modification increases.
Owner:KAWASAKI JUKOGYO KK

Multi-source heterogeneous data synchronous mechanical arm bionic control method and device and electronic equipment

The invention provides a multi-source heterogeneous data synchronous mechanical arm bionic control method and device and electronic equipment, and relates to the technical field of robot control, and the method comprises the following steps: decoding hand joint angle data and a synchronization timestamp thereof, wrist pose data and a synchronization timestamp thereof, and arm pose data and a synchronization timestamp thereof, and obtaining a decoding result; the decoded hand joint angle data, the decoded wrist pose data and the decoded arm pose data are transmitted to a mechanical arm controller through a low-delay network; and the mechanical arm controller executes dynamic remapping based on the decoded hand joint angle data, the decoded wrist pose data and the decoded arm pose data, and simulation control of the mechanical arm is achieved. According to the method, the arm pose data participates in dynamic correction of the base coordinate system, space positioning deviation is eliminated, and the mechanical arm reproduces actions; timing sequence block optimization considers fine grabbing and long track actions, and action restoration authenticity is improved.
Owner:SHANGHAI TARS ROBOTICS CO LTD

Teleoperation method, system and equipment based on smart type library and storage medium

The invention discloses a smart type library-based teleoperation method, system and device, and a storage medium. The method comprises the following steps of: constructing a smart operation type library; analyzing the task instruction through a preset multi-modal large language model MLLM to obtain a corresponding operation step, and matching a dexterous operation type corresponding to the current step from a dexterous operation type library; intuitive accurate control between the hand and the manipulator is realized through an interpolation mapping strategy. The interpolation mapping strategy specifically comprises the following steps: capturing a fingertip position of a human hand in real time; calculating the normalized projection proportion of the current posture of the human hand relative to the stretching state and the contraction state; and according to the projection proportion, linear interpolation is carried out on the extension and contraction joint angles of the manipulator, and a manipulator joint angle instruction is generated. According to the method, the adaptability and stability of teleoperation are provided by establishing a systematized smart type library and combining a type automatic retrieval module assisted by a multi-modal large language model and an interpolation mapping control strategy.
Owner:SUN YAT SEN UNIV

Gesture interaction method and system, readable storage medium and computer

The invention provides a gesture interaction method and system, a readable storage medium and a computer. The method comprises the following steps: acquiring a sight focusing area of a user for a virtual marker; inputting the hand image into a convolutional neural network model for convolution operation to obtain a convolutional feature image; performing pixel processing based on the hand coding image of the hand image to obtain a corresponding pixel feature image; importing the multi-scale fusion unit into a multi-resolution network model for model optimization to obtain a network optimization model; and inputting the convolutional feature image and the pixel feature image into a network optimization model to output a hand posture estimation result of the user, and performing collision judgment according to the hand posture estimation result and the sight focusing area to realize gesture interaction of the user. According to the method, attitude estimation is carried out on the convolution feature image and the pixel feature image by using the network optimization model, and the influence of network parameter quantity on hand attitude estimation is reduced, so that the detection error of hand joint points is reduced.
Owner:HUIZHI WORLD (HANGZHOU) TECHNOLOGY CO LTD

Virtual-real combined mechanical arm-dexterous hand system mapping operation and sample collection method

The invention relates to the technical field of intelligent robots, in particular to a virtual-real combined mechanical arm-dexterous hand system mapping operation and sample collection method. Comprises: configuring a mapping operation system; collecting the tail end position and posture information of the tail end of the forearm of the operator in real time based on the teleoperation device; acquiring three-dimensional information of key points of hands of an operator in real time based on a camera; a mechanical arm tail end mapping value is calculated according to the tail end position and the posture information; calculating a joint mapping value of the dexterous hand according to the three-dimensional information of the key points; and the mechanical arm and the dexterous hand are driven to move in real time according to the mechanical arm tail end mapping value and the dexterous hand joint mapping value, real-time data of all joints in the mapping operation process and observation images in the operation process are recorded, and sample collection is completed. The embodiment of the invention provides a virtual and real combined mechanical arm-dexterous hand system mapping operation and sample collection method, which can be used for carrying out mechanical arm-dexterous hand system mapping operation and sample collection.
Owner:BEIJING INST OF CONTROL ENG

Method for detecting hand joints in glasses equipment, storage medium, electronic equipment and product

The invention relates to the technical field of visual inspection, and particularly provides a method for detecting a hand joint in glasses equipment, a storage medium, electronic equipment and a product, and the method can comprise the steps: obtaining the confidence coefficient of a hand region corresponding to a current frame image; under the condition that the confidence coefficient is determined to be greater than a preset threshold value, processing the hand region to obtain a to-be-detected hand region image; and inputting the to-be-detected hand region image into a target hand joint detection model to obtain a hand joint detection result. According to some embodiments of the invention, the accuracy of hand joint detection can be improved.
Owner:HANGZHOU QIUGUOJIHUA TECHNOLOGY CO LTD

Robot control method and system based on virtual reality and hand motion capture

The embodiment of the invention provides a robot control method and system based on virtual reality and hand motion capture, and the method comprises the steps: collecting the hand posture data of a user, obtaining the first hand posture data of the user through a virtual reality device, the first hand posture data comprises the hand end posture information, and obtaining the first hand posture data of the user through a virtual reality device; obtaining second hand posture data of the user through wearable hand motion capture equipment, wherein the second hand posture data comprises hand joint fine motion information; standardizing the hand position and posture data, and mapping the hand position and posture data into robot target motion data based on a preset calibration matrix, the robot target motion data including first robot target motion data mapped based on the first hand position and posture data and second robot target motion data mapped based on the second hand position and posture data; robot motion control data is generated based on the robot target motion data, and robot parts are controlled to move. According to the robot control method and system based on virtual reality and hand motion capture, the operation precision of the robot can be greatly improved, the operation real-time performance and smoothness are enhanced, and the motion state of the robot can be monitored in real time.
Owner:YISHENG TECHNOLOGY (SHENZHEN) CO LTD

Hand posture reconstruction method, system and equipment based on sEMG and medium

The invention discloses a hand posture reconstruction method, system and device based on sEMG and a medium. The method comprises the following steps that a forearm sEMG signal to be estimated is collected; performing denoising processing on the forearm sEMG signal based on a continuous denoising network, and segmenting the forearm sEMG signal after denoising processing into a plurality of continuous signals; performing feature extraction on the continuous signals based on a mixed attention mechanism to obtain continuous signal features; hand posture prediction based on posture correction processing is carried out on the continuous signal features based on a dual-decoder prediction architecture, and hand continuous posture signals are obtained; calculating a hand joint angular acceleration based on the hand continuous attitude signal, and converting a hand attitude reconstruction image according to the hand joint angular acceleration; according to the method, the sEMG features are extracted more effectively by fusing sparse attention and channel attention mechanisms, the attitude correction capability is remarkably enhanced by using a double-decoder structure, and the real-time accuracy of hand attitude estimation is improved.
Owner:JIANGSU IND INNOVATION CENT OF INTELLIGENT EQUIP CO LTD

Rehabilitation training control method and system based on flexible hand joint rehabilitation glove

The invention relates to the technical field of data processing, in particular to a rehabilitation training control method and system based on a flexible hand joint rehabilitation glove, and the method comprises the steps: obtaining an individualized physiological parameter baseline of a patient; real-time data of a patient in the rehabilitation training process are collected through a multi-modal sensor array on the flexible glove, the active participation degree, the joint movement smoothness and the muscle fatigue degree of the patient are calculated based on the real-time data, the active movement intention of the patient is recognized, and a real-time rehabilitation state evaluation result of the patient is obtained; generating a control instruction through an adaptive control engine according to a preset personalized rehabilitation scheme and the real-time rehabilitation state evaluation result; the generated control instruction is sent to a driving control circuit so as to control a driving unit to execute corresponding bending or stretching actions and assist the patient in completing target training actions; according to the invention, personalized, self-adaptive and quantitative evaluation of rehabilitation training can be realized, and the efficiency and effect of hand rehabilitation training are improved.
Owner:佛山市康复医院有限公司 +2

Endocavity ultrasonic probe

A hand pose recognition method is performed by a computer device, including: acquiring a current frame of a multi-lens video of a target object; performing hand detection on a first view of the current frame to obtain a first lens detection result; performing hand estimation on a second view of the current frame to obtain a second lens estimation result; removing, from the hand detection boxes in the first view and the hand estimation boxes in the second view, redundant boxes corresponding to redundant hands, and then performing hand joint point recognition on remaining boxes to obtain two-dimensional joint points; converting the two-dimensional joint points into three-dimensional joint points in a three-dimensional hand coordinate system; and converting the three-dimensional joint points in the three-dimensional hand coordinate system into three-dimensional joint points of the current frame in a world coordinate system according to pose estimation parameters corresponding to the current frame.
Owner:SHENZHEN MINDRAY BIO MEDICAL ELECTRONICS CO LTD

Surgical operating room minimally invasive instrument control system and method

The invention relates to the technical field of minimally invasive instrument control, in particular to a minimally invasive instrument control system and method for a surgical operating room. The system comprises a main hand operation module, an operation position monitoring module and a tactile feedback module. The touch feedback module controls the touch operation arm to make contact with the focus to obtain the contact feedback, the contact feedback is sent to the master hand operation module, the vibration instrument arranged in the master hand operation module is controlled to feed back vibration, and the focus area is determined through the vibration mode matched with the reference focus area. A hand joint encoder held by a doctor in hand is fed back to the hand in real time, the hardness of a focus area currently in contact with a touch operation arm is determined, in cooperation with an intraoperative patient focus image displayed by a 3D image real-time display module in real time, the state of a focus operation point is predicted, touch feedback is provided for the remote operation doctor, the operation precision is further improved, and the operation efficiency is improved. Doctors are assisted in surgical operation.
Owner:ZHUZHOU CENT HOSPITAL

Hand posture recognition method, head-mounted display device and storage medium

The invention discloses a hand posture recognition method, a head-mounted display device and a storage medium, and relates to the field of man-machine interaction. In the method, projection correction is carried out on an image of a first hand region in a current frame image by iterating a projection direction for multiple times; the present invention corrects an image captured due to inclination of an observation line of sight into an image in which the line of sight is perpendicular to the center. In this way, the hand image sent into model reasoning overcomes the distortion influence, and the positions and the relative distances of the hand joint points obtained through model reasoning are more accurate. According to the invention, the problem that the coordinates of the predicted 3D hand articulation points are inaccurate due to image distortion is solved.
Owner:HANGZHOU QIUGUOJIHUA TECHNOLOGY CO LTD

Dexterous hand motion algorithm control method, system and device, storage medium and program product

The invention relates to robot motion control, in particular to a dexterous hand motion algorithm control method, system and device, a storage medium and a program product. The dexterous hand motion algorithm control method comprises the following steps: visual detection: acquiring a hand image through a camera, and identifying three-dimensional coordinates of key points of hand joints by adopting a visual detection algorithm; kinematics calculation: based on the coordinates of the joint key points, obtaining target execution angles of a plurality of motors through kinematics forward and inverse calculation; communication distribution: receiving the target execution angle through a master control unit, and distributing the target execution angle to a plurality of slave control units according to a preset distribution rule; and motor control: each slave control unit generates a driving signal by adopting a PID control algorithm based on the received target execution angle, and controls the corresponding motor to move. The dexterous hand motion algorithm control method provided by the invention is high in response speed, high in control precision, stable in communication and accurate in kinematics solution.
Owner:FUTURE BEAT INTELLIGENT TECHNOLOGY (ZHEJIANG) CO LTD

Hand external rehabilitation nursing frame

The utility model relates to the technical field of nursing frames, and discloses an external hand rehabilitation nursing frame which comprises a frame body assembly. The frame body assembly comprises a first plate body and a second plate body, wherein one end of the first plate body is provided with a second plate body; the third plate body is arranged on the second plate body; the elastic band is arranged on the third plate body; the fourth plate body is in threaded connection with the outer side of the second plate body; an adjusting assembly is arranged on the second plate body. After the fourth plate body is pushed and the position of the fourth plate body is adjusted, the fingers are fixed through the elastic bands, or after the fourth plate body is detached before use and the fingers are fixed again, finger joints move, the fourth plate body rotates on the rod body, the torsional spring is pulled, and rehabilitation training is conducted on the fingers. The rehabilitation nursing frame has the advantages of being simple in structure, free of influence on hand joint movement of a patient and capable of adjusting the hand fixing position, and the problems that an existing rehabilitation nursing frame for finger trauma and fracture is single in function and small in application range are solved.
Owner:中国人民解放军总医院第八医学中心

Driving method of 3D gesture model and electronic equipment

The invention provides a driving method of a 3D gesture model and electronic equipment. The driving method is used for improving the driving accuracy of the 3D gesture model. Comprising the following steps: obtaining predicted attitude data of each hand joint point in a current frame of hand image based on attitude data of each hand joint point in a specified frame number of hand images before the current frame of hand image, and obtaining a hand time sequence diagram; obtaining intermediate attitude data of each hand joint point by using a double-time-sequence model based on a time sequence feature graph obtained by performing feature extraction on a target region in the hand time sequence graph and a hand feature graph obtained by performing feature extraction on the current frame of hand image; if the intermediate posture data of the specified hand joint point meets a specified condition, inputting the current frame of hand image and a hand time sequence diagram of a target standard hand posture obtained by matching the intermediate posture data with the posture data of each standard hand posture into a double-time-sequence model to obtain target posture data of each hand joint point, and driving the 3D gesture model by using the target attitude data.
Owner:HISENSE ELECTRONICS TECH SHENZHEN CO LTD

Gesture recognition method fusing continuous frame segmentation and spatial features

The invention discloses a gesture recognition method fusing continuous frame segmentation and spatial features. The gesture recognition method comprises the steps that three-dimensional coordinates of hand joint points are obtained through Mediape; performing dual normalization processing to eliminate size and position differences; extracting point cloud convex hull features, joint point distance features and specific triangularization area combination features; fusing a plurality of types of features and then training a Transform or GCN model; carrying out automatic segmentation on the continuous gesture sequence based on double cosine similarity thresholds; and inputting the segmented stable paragraphs into the model to realize real-time identification. The method provided by the invention has high precision and strong robustness in a complex scene, and is suitable for the fields of man-machine interaction, intelligent control and the like.
Owner:GUILIN UNIV OF ELECTRONIC TECH

Hand pose estimation for machine learning based gesture recognition

The technology disclosed performs hand pose estimation on a so-called “joint-by-joint” basis. So, when a plurality of estimates for the 28 hand joints are received from a plurality of expert networks (and from master experts in some high-confidence scenarios), the estimates are analyzed at a joint level and a final location for each joint is calculated based on the plurality of estimates for a particular joint. This is a novel solution discovered by the technology disclosed because nothing in the field of art determines hand pose estimates at such granularity and precision. Regarding granularity and precision, because hand pose estimates are computed on a joint-by-joint basis, this allows the technology disclosed to detect in real time even the minutest and most subtle hand movements, such a bend / yaw / tilt / roll of a segment of a finger or a tilt an occluded finger, as demonstrated supra in the Experimental Results section of this application.
Owner:HAND TRACKING IP PROTECTION LLC

Hand posture estimation method and device and head-mounted display equipment

The invention provides a hand posture estimation method and device and head-mounted display equipment, and relates to the technical field of man-machine interaction, and the method comprises the steps: determining the pixel position and the relative distance of each hand joint point, and the bending degree of each finger according to a hand binocular image; according to the bending degree of each finger, an optimization variable of the hand posture is determined; according to the optimization variable, the pixel position and the relative distance of each hand joint point and the bending degree of each finger, an optimization function of the hand posture is generated; and performing iterative optimization on the hand posture according to the optimization function to obtain a hand posture estimation result. By combining the pixel position and the relative distance of each hand joint point and the bending degree of the finger, the hand posture is iteratively optimized, so that the estimation accuracy of the hand posture is improved.
Owner:HANGZHOU QIUGUOJIHUA TECHNOLOGY CO LTD

Hand posture estimation method based on dual aggregation network enhancement

A hand posture estimation method based on dual aggregation network enhancement comprises the following steps: generating a hand point cloud from a hand depth image, and inputting the hand depth image and the point cloud into a local coding fusion module to generate fused image features and point cloud features; inputting the fused 3D point cloud features into an initial state generator to initialize a hidden state; inputting the hidden state into a regression module to obtain initial estimation of a joint point; initially estimating and fusing the image features and the point cloud features, and inputting the image features and the point cloud features into a point cloud image consistency aggregation module to generate enhanced point cloud features; the enhanced point cloud feature and initial estimation input resampling module outputs high-dimensional hand joint point features; the enhanced point cloud features, the high-dimensional hand articulation point features and the hidden state are jointly input into a dynamic graph enhanced aggregation module to obtain enhanced high-dimensional articulation point features; and inputting the enhanced high-dimensional joint point features into a regression module to obtain a final hand joint point coordinate position. According to the method, gesture posture estimation under the shielding condition can be improved, and hand joint output is enhanced.
Owner:CHINA UNIV OF MINING & TECH

Intelligent interaction system and method based on gesture recognition

The invention relates to the technical field of gesture recognition interaction, and discloses an intelligent interaction system and method based on gesture recognition. The method comprises the following steps: when an environment meets a condition, synchronously acquiring an image sequence of a gesture and depth distance field data; a hand joint point two-dimensional motion track is extracted from an image, surface deformation fluctuation information is separated from depth data, the two-dimensional motion track and the surface deformation fluctuation information are subjected to space-time registration fusion, a continuous motion track curved surface in a three-dimensional space is constructed, and then geometric topology features and dynamic change features of the continuous motion track curved surface are extracted. And inputting the fusion features into a neural network classifier subjected to incremental learning training, outputting corresponding semantic tags and confidence evaluation values, and mapping to generate a control instruction after verification. According to the method, an accurate three-dimensional dynamic model is constructed through deep fusion of multi-source data, and an incremental learning mechanism is utilized to enable the system to have online adaptive capability, so that the recognition precision of complex gestures and the long-term applicability of the system are improved.
Owner:BEIJING LINGBAN WORKSHOP TECHNOLOGY CO LTD

Hand movement mapping method based on data glove

The invention discloses a hand motion mapping method based on a data glove, which comprises the following steps of: judging the type of a task to be performed according to data acquired by the data glove, and selecting different mapping methods. If the current gesture is judged to be the gesture representation, applying a gesture recognition model, if the current gesture exists in a gesture library, applying a standard gesture in the library, and if the current gesture does not exist, applying direct joint angle mapping to four fingers except the thumb, and applying Cartesian space mapping to the thumb. And if the task is judged to be the operation task, the data of the data glove is directly processed by a pre-trained neural network and is directly mapped into a joint angle corresponding to the target manipulator. The training thought of the neural network is that the human hand joint angle represented by the data glove and each joint angle of the target manipulator under the same gesture action are obtained at the same time, a neural network model is constructed, the angle of the data glove serves as input, the corresponding angle of the manipulator serves as output, and training is conducted. Finally, the motion mapping process from the hand to the manipulator is achieved.
Owner:ZHEJIANG UNIV OF TECH

Hand multi-modal information sensing device and robot humanoid skill learning method

The invention provides a hand multi-mode information sensing device and a robot humanoid skill learning method, and the device comprises a multi-mode touch sensor module which is used for sensing a hand multi-mode touch signal; the flexible strain sensor module is used for sensing a hand joint bending signal; the multi-modal motion capture module is used for sensing hand motion and posture signals, and the hand motion and posture signals comprise hand motion signals and / or hand posture signals; at least one of the hand multi-mode touch signals, the hand joint bending signals and the hand movement and posture signals forms hand multi-mode information representing gesture actions, hand movement and hand touch. According to the invention, sensing of hand multi-modal information in a hand operation process can be realized based on the hand multi-modal information sensing device, establishment of a sensing strategy and a control strategy is facilitated, cross-object and cross-task robust skill migration is realized, and self-adaptive operation capability and task success rate of a robot in an actual complex environment are remarkably improved.
Owner:TSINGHUA UNIVERSITY

A minimally invasive instrument control system and method for surgical operating room

The present invention relates to the technical field of minimally invasive instrument control, and in particular, to a minimally invasive instrument control system and method for surgical operating rooms. The system comprises a master hand operation module, an operation position monitoring module and a tactile feedback module. The present invention controls the contact between a tactile operating arm and a lesion through a tactile feedback module, obtains contact feedback, and sends the contact feedback to the master hand operation module, controls the vibration feedback of a vibration instrument built into the master hand operation module, and feeds back vibrations in real time to the hand through a hand joint encoder held by a doctor by referring to the vibration mode matched with the lesion area, determines the hardness of the lesion area currently contacted by the tactile operating arm, and cooperates with the intraoperative patient lesion image displayed in real time by a 3D image real-time display module to predict the state of the lesion operation point, provide tactile feedback for doctors performing remote surgery, further improve the accuracy of the surgery, and assist doctors in performing surgical operations.
Owner:ZHUZHOU CENT HOSPITAL