161 results about "Hand joint" patented technology

The invention discloses a skeleton sign language recognition method of a double-flow space-time dynamic graph convolutional network fused with residual learning, and belongs to the technical field of artificial intelligence and gesture recognition. According to the method, an input gesture skeleton sequence relative to a face is divided into two data streams, namely a hand form data stream and a wrist track data stream through double-reference-system differential homeomorphic mapping; the method comprises the following steps: firstly, processing hand posture data, and capturing a hand joint spatial topological relation by combining a spatial-temporal dynamic graph convolutional network (STDGCNN) with a residual convolutional block; meanwhile, a Finsler trajectory dynamics encoder (FTDE) is adopted to carry out differential geometric modeling on the wrist trajectory, and the direction sensitivity characteristic of the trajectory is captured through a multi-scale causal convolutional network. Then, mutual enhancement of double-flow features is realized through a bidirectional cross feature enhancer (BCFE), and the problem of geometric inconsistency of a heterogeneous feature space is solved through a geometric-driven optimal transmission fusion device (Geometric-OT). The method solves the challenge that a traditional sign language recognition method processes complex space-time correlation of gesture forms and motion tracks at the same time, the technical problems of insufficient relation between hands and faces, insufficient feature expression ability and low space-time feature extraction efficiency, and the problems of geometric inconsistency, single reference system and the like. And the identification accuracy and the real-time performance are obviously improved. Experiments show that the accuracy rate of the method in complex hundreds of sign language vocabulary recognition tasks reaches 95% or above on average, the reasoning speed is only 17ms on average, high-precision real-time sign language recognition is achieved, and the method has higher robustness in complex environments such as noise and shielding.

An augmented reality device for obtaining three-dimensional (3D) position information of a plurality of hand joints of a user includes a plurality of cameras configured to photograph the user's hand and obtain images; memory storing instructions; and at least one processor; the instructions, when executed, may cause the device to recognize the plurality of hand joints from the images; obtain two-dimensional (2D) joint coordinate values for feature points corresponding to the hand joints; retrieve, from a look up table (LUT), 3D position coordinate values corresponding to distortion model parameters of the cameras, a positional relationship between the cameras, and the 2D joint coordinate values; and output the 3D position information of the plurality of hand joints based on the 3D position coordinate values.

The invention relates to a biomechanical modeling and normalized training data generation method and device for human hand posture estimation, equipment and a medium. The method comprises the following steps: acquiring target three-dimensional hand posture data of a target object; for each target object, acquiring reference static biomechanical characteristics and dynamic biomechanical characteristics corresponding to each target three-dimensional hand posture of the target object; for each target three-dimensional hand posture, determining a three-dimensional joint coordinate of the target three-dimensional hand posture and a two-dimensional joint coordinate corresponding to the three-dimensional joint coordinate based on the reference static biomechanical characteristics and the dynamic biomechanical characteristics; wherein the three-dimensional joint coordinates represent the spatial positions of the hand joint points; and determining hand posture training data based on the three-dimensional joint coordinates and the corresponding two-dimensional joint coordinates. By adopting the method, the individual biomechanical structure difference can be eliminated, and the number of training data can be expanded more accurately and reasonably.

A hand pose recognition method is performed by a computer device, including: acquiring a current frame of a multi-lens video of a target object; performing hand detection on a first view of the current frame to obtain a first lens detection result; performing hand estimation on a second view of the current frame to obtain a second lens estimation result; removing, from the hand detection boxes in the first view and the hand estimation boxes in the second view, redundant boxes corresponding to redundant hands, and then performing hand joint point recognition on remaining boxes to obtain two-dimensional joint points; converting the two-dimensional joint points into three-dimensional joint points in a three-dimensional hand coordinate system; and converting the three-dimensional joint points in the three-dimensional hand coordinate system into three-dimensional joint points of the current frame in a world coordinate system according to pose estimation parameters corresponding to the current frame.

The invention discloses a stroke hand rehabilitation training method based on multi-modal data fusion, particularly relates to the field of hand rehabilitation training, and is used for solving the problems that in the existing rehabilitation process, the motion solidification risk cannot be recognized, and the training scheme is lack of individualized dynamic adjustment. According to the method, hand joint motion features are obtained by constructing a standardized hand function micro-evaluation task, and a personal motion mode baseline is established in combination with dynamic time warping and multi-scale permutation entropy analysis; extracting a neural motion curing index based on the difference change between the current motion mode and the optimal motion mode, and analyzing and generating a function recovery momentum feature by combining the rehabilitation progress gradient and the acceleration inflection point; and fusing the electromyographic signal coupling coordination index to generate a rehabilitation risk score, and when the risk exceeds a dynamic threshold value, triggering adaptive recombination of a training strategy to realize accurate early warning and personalized training regulation and control. Rehabilitation monitoring precision and training pertinence can be effectively improved, and high-quality recovery of stroke hand functions is promoted.

The invention provides a multi-source heterogeneous data synchronous mechanical arm bionic control method and device and electronic equipment, and relates to the technical field of robot control, and the method comprises the following steps: decoding hand joint angle data and a synchronization timestamp thereof, wrist pose data and a synchronization timestamp thereof, and arm pose data and a synchronization timestamp thereof, and obtaining a decoding result; the decoded hand joint angle data, the decoded wrist pose data and the decoded arm pose data are transmitted to a mechanical arm controller through a low-delay network; and the mechanical arm controller executes dynamic remapping based on the decoded hand joint angle data, the decoded wrist pose data and the decoded arm pose data, and simulation control of the mechanical arm is achieved. According to the method, the arm pose data participates in dynamic correction of the base coordinate system, space positioning deviation is eliminated, and the mechanical arm reproduces actions; timing sequence block optimization considers fine grabbing and long track actions, and action restoration authenticity is improved.

The invention discloses a smart type library-based teleoperation method, system and device, and a storage medium. The method comprises the following steps of: constructing a smart operation type library; analyzing the task instruction through a preset multi-modal large language model MLLM to obtain a corresponding operation step, and matching a dexterous operation type corresponding to the current step from a dexterous operation type library; intuitive accurate control between the hand and the manipulator is realized through an interpolation mapping strategy. The interpolation mapping strategy specifically comprises the following steps: capturing a fingertip position of a human hand in real time; calculating the normalized projection proportion of the current posture of the human hand relative to the stretching state and the contraction state; and according to the projection proportion, linear interpolation is carried out on the extension and contraction joint angles of the manipulator, and a manipulator joint angle instruction is generated. According to the method, the adaptability and stability of teleoperation are provided by establishing a systematized smart type library and combining a type automatic retrieval module assisted by a multi-modal large language model and an interpolation mapping control strategy.

The embodiment of the invention provides a robot control method and system based on virtual reality and hand motion capture, and the method comprises the steps: collecting the hand posture data of a user, obtaining the first hand posture data of the user through a virtual reality device, the first hand posture data comprises the hand end posture information, and obtaining the first hand posture data of the user through a virtual reality device; obtaining second hand posture data of the user through wearable hand motion capture equipment, wherein the second hand posture data comprises hand joint fine motion information; standardizing the hand position and posture data, and mapping the hand position and posture data into robot target motion data based on a preset calibration matrix, the robot target motion data including first robot target motion data mapped based on the first hand position and posture data and second robot target motion data mapped based on the second hand position and posture data; robot motion control data is generated based on the robot target motion data, and robot parts are controlled to move. According to the robot control method and system based on virtual reality and hand motion capture, the operation precision of the robot can be greatly improved, the operation real-time performance and smoothness are enhanced, and the motion state of the robot can be monitored in real time.

The invention relates to the technical field of data processing, in particular to a rehabilitation training control method and system based on a flexible hand joint rehabilitation glove, and the method comprises the steps: obtaining an individualized physiological parameter baseline of a patient; real-time data of a patient in the rehabilitation training process are collected through a multi-modal sensor array on the flexible glove, the active participation degree, the joint movement smoothness and the muscle fatigue degree of the patient are calculated based on the real-time data, the active movement intention of the patient is recognized, and a real-time rehabilitation state evaluation result of the patient is obtained; generating a control instruction through an adaptive control engine according to a preset personalized rehabilitation scheme and the real-time rehabilitation state evaluation result; the generated control instruction is sent to a driving control circuit so as to control a driving unit to execute corresponding bending or stretching actions and assist the patient in completing target training actions; according to the invention, personalized, self-adaptive and quantitative evaluation of rehabilitation training can be realized, and the efficiency and effect of hand rehabilitation training are improved.

A hand pose recognition method is performed by a computer device, including: acquiring a current frame of a multi-lens video of a target object; performing hand detection on a first view of the current frame to obtain a first lens detection result; performing hand estimation on a second view of the current frame to obtain a second lens estimation result; removing, from the hand detection boxes in the first view and the hand estimation boxes in the second view, redundant boxes corresponding to redundant hands, and then performing hand joint point recognition on remaining boxes to obtain two-dimensional joint points; converting the two-dimensional joint points into three-dimensional joint points in a three-dimensional hand coordinate system; and converting the three-dimensional joint points in the three-dimensional hand coordinate system into three-dimensional joint points of the current frame in a world coordinate system according to pose estimation parameters corresponding to the current frame.

The invention discloses a hand posture recognition method, a head-mounted display device and a storage medium, and relates to the field of man-machine interaction. In the method, projection correction is carried out on an image of a first hand region in a current frame image by iterating a projection direction for multiple times; the present invention corrects an image captured due to inclination of an observation line of sight into an image in which the line of sight is perpendicular to the center. In this way, the hand image sent into model reasoning overcomes the distortion influence, and the positions and the relative distances of the hand joint points obtained through model reasoning are more accurate. According to the invention, the problem that the coordinates of the predicted 3D hand articulation points are inaccurate due to image distortion is solved.

The invention relates to robot motion control, in particular to a dexterous hand motion algorithm control method, system and device, a storage medium and a program product. The dexterous hand motion algorithm control method comprises the following steps: visual detection: acquiring a hand image through a camera, and identifying three-dimensional coordinates of key points of hand joints by adopting a visual detection algorithm; kinematics calculation: based on the coordinates of the joint key points, obtaining target execution angles of a plurality of motors through kinematics forward and inverse calculation; communication distribution: receiving the target execution angle through a master control unit, and distributing the target execution angle to a plurality of slave control units according to a preset distribution rule; and motor control: each slave control unit generates a driving signal by adopting a PID control algorithm based on the received target execution angle, and controls the corresponding motor to move. The dexterous hand motion algorithm control method provided by the invention is high in response speed, high in control precision, stable in communication and accurate in kinematics solution.

The invention provides a driving method of a 3D gesture model and electronic equipment. The driving method is used for improving the driving accuracy of the 3D gesture model. Comprising the following steps: obtaining predicted attitude data of each hand joint point in a current frame of hand image based on attitude data of each hand joint point in a specified frame number of hand images before the current frame of hand image, and obtaining a hand time sequence diagram; obtaining intermediate attitude data of each hand joint point by using a double-time-sequence model based on a time sequence feature graph obtained by performing feature extraction on a target region in the hand time sequence graph and a hand feature graph obtained by performing feature extraction on the current frame of hand image; if the intermediate posture data of the specified hand joint point meets a specified condition, inputting the current frame of hand image and a hand time sequence diagram of a target standard hand posture obtained by matching the intermediate posture data with the posture data of each standard hand posture into a double-time-sequence model to obtain target posture data of each hand joint point, and driving the 3D gesture model by using the target attitude data.

The invention discloses a gesture recognition method fusing continuous frame segmentation and spatial features. The gesture recognition method comprises the steps that three-dimensional coordinates of hand joint points are obtained through Mediape; performing dual normalization processing to eliminate size and position differences; extracting point cloud convex hull features, joint point distance features and specific triangularization area combination features; fusing a plurality of types of features and then training a Transform or GCN model; carrying out automatic segmentation on the continuous gesture sequence based on double cosine similarity thresholds; and inputting the segmented stable paragraphs into the model to realize real-time identification. The method provided by the invention has high precision and strong robustness in a complex scene, and is suitable for the fields of man-machine interaction, intelligent control and the like.

The invention provides a hand posture estimation method and device and head-mounted display equipment, and relates to the technical field of man-machine interaction, and the method comprises the steps: determining the pixel position and the relative distance of each hand joint point, and the bending degree of each finger according to a hand binocular image; according to the bending degree of each finger, an optimization variable of the hand posture is determined; according to the optimization variable, the pixel position and the relative distance of each hand joint point and the bending degree of each finger, an optimization function of the hand posture is generated; and performing iterative optimization on the hand posture according to the optimization function to obtain a hand posture estimation result. By combining the pixel position and the relative distance of each hand joint point and the bending degree of the finger, the hand posture is iteratively optimized, so that the estimation accuracy of the hand posture is improved.

The invention relates to the technical field of gesture recognition interaction, and discloses an intelligent interaction system and method based on gesture recognition. The method comprises the following steps: when an environment meets a condition, synchronously acquiring an image sequence of a gesture and depth distance field data; a hand joint point two-dimensional motion track is extracted from an image, surface deformation fluctuation information is separated from depth data, the two-dimensional motion track and the surface deformation fluctuation information are subjected to space-time registration fusion, a continuous motion track curved surface in a three-dimensional space is constructed, and then geometric topology features and dynamic change features of the continuous motion track curved surface are extracted. And inputting the fusion features into a neural network classifier subjected to incremental learning training, outputting corresponding semantic tags and confidence evaluation values, and mapping to generate a control instruction after verification. According to the method, an accurate three-dimensional dynamic model is constructed through deep fusion of multi-source data, and an incremental learning mechanism is utilized to enable the system to have online adaptive capability, so that the recognition precision of complex gestures and the long-term applicability of the system are improved.

The invention discloses a hand motion mapping method based on a data glove, which comprises the following steps of: judging the type of a task to be performed according to data acquired by the data glove, and selecting different mapping methods. If the current gesture is judged to be the gesture representation, applying a gesture recognition model, if the current gesture exists in a gesture library, applying a standard gesture in the library, and if the current gesture does not exist, applying direct joint angle mapping to four fingers except the thumb, and applying Cartesian space mapping to the thumb. And if the task is judged to be the operation task, the data of the data glove is directly processed by a pre-trained neural network and is directly mapped into a joint angle corresponding to the target manipulator. The training thought of the neural network is that the human hand joint angle represented by the data glove and each joint angle of the target manipulator under the same gesture action are obtained at the same time, a neural network model is constructed, the angle of the data glove serves as input, the corresponding angle of the manipulator serves as output, and training is conducted. Finally, the motion mapping process from the hand to the manipulator is achieved.

The invention provides a hand multi-mode information sensing device and a robot humanoid skill learning method, and the device comprises a multi-mode touch sensor module which is used for sensing a hand multi-mode touch signal; the flexible strain sensor module is used for sensing a hand joint bending signal; the multi-modal motion capture module is used for sensing hand motion and posture signals, and the hand motion and posture signals comprise hand motion signals and / or hand posture signals; at least one of the hand multi-mode touch signals, the hand joint bending signals and the hand movement and posture signals forms hand multi-mode information representing gesture actions, hand movement and hand touch. According to the invention, sensing of hand multi-modal information in a hand operation process can be realized based on the hand multi-modal information sensing device, establishment of a sensing strategy and a control strategy is facilitated, cross-object and cross-task robust skill migration is realized, and self-adaptive operation capability and task success rate of a robot in an actual complex environment are remarkably improved.

1. The name of the design product: adjustable eating fork. 2. The use of the design product: designed for the general silver-haired population or the elderly, the disabled, and people whose hand joint flexion function is limited, whose grip function is not in place, whose grip strength is insufficient, and whose hands are shaky. 3. The design points of the design product: in shape. 4. The picture or photo that best indicates the design points: perspective view.

The invention discloses a human motion classification method, device and equipment based on RFID and a medium, and the method comprises the steps: collecting human skeleton phase data, and carrying out the preprocessing of the data, and forming RFID skeleton activity data; constructing a human body action diagram, and connecting hand joint nodes and foot joint nodes in the human body action diagram to form an initial human body skeleton structure diagram; and performing graph convolution processing on the initial human body skeleton structure graph to generate a human body action prediction classification label. By adding an improved graph structure formed by connection between the hands and the feet in the human body structure graph, the topological distance between the nodes is reduced, the nodes can be aggregated to more neighbor node information, the aggregation capability of joint features is improved, and the joint feature aggregation efficiency is improved. And during action recognition of cooperation of both hands and hands and feet, richer action feature information can be captured through aggregation features among non-physically connected joints, and the recognition precision of the model on complex postures is improved.

The embodiment of the invention provides a dexterous hand joint angle determination method and equipment. The method comprises the steps that power output parameters of a driving motor of a target dexterous hand are obtained, the driving displacement amount of the target dexterous hand is determined according to the power output parameters, the joint angle of the target dexterous hand is determined according to the driving displacement amount and a first corresponding relation, and the first corresponding relation is determined based on the geometric structure of the target dexterous hand. The first corresponding relation is used for representing the incidence relation between the driving displacement and the joint angle of the target dexterous hand. According to the method, the driving displacement is determined, and the joint angle of the target dexterous hand is determined based on the first corresponding relation between the driving displacement constructed through the geometric structure of the target dexterous hand and the joint angle of the target dexterous hand, so that the determination precision of the joint angle is improved, additional sensors do not need to be installed at the joints, and the accuracy of the joint angle determination is improved. The space is saved.

The embodiment of the invention provides a mechanical finger joint miniature encoder mounting structure for improving control precision, and belongs to the technical field of medical instrument robot dexterous hands and high-precision sensing. The structure comprises a hollow annular magnetic encoder, a customized mounting base and an error compensation mechanism. The MPT-8-20 series encoder is directly integrated on a rotating shaft of a finger joint through the customized mounting base; the mounting base is provided with a positioning interface which is precisely matched with an encoder shell and a joint part, so that the concentricity and a constant air gap of an encoder sensing unit and a rotating part are ensured; the error compensation mechanism is integrated in the base and is used for passively inhibiting installation errors and environmental interference. According to the invention, the measurement precision of less than 0.01 degree in an extremely limited space is realized based on a commercial encoder, high anti-interference performance and reliability are realized, the integration problem of high-precision and high-reliability pose feedback of the miniature dexterous hand joint is effectively solved, and the miniature dexterous hand joint is suitable for miniature dexterous hands.

The utility model discloses a finger assembly and mechanical hand relates to mechanical hand technical field, finger assembly includes: finger body, mounting body, the mounting body is equipped with rotary seat and swing seat, the rotary seat rotatablely connected in the mounting body around rotary axis, swing seat rotatablely connected in the mounting body around swing axis, the finger body rotatablely connected in the swing seat around rotary axis, the finger body with rotary seat forms spherical hinge cooperation, and rotary axis with swing axis cross distribution. The finger assembly of the utility model, through setting rotary seat and swing seat, make the finger body can carry out the bending and swing of human hand joint, and simple structure, the volume is smaller.

The invention relates to the field of carbon fiber composite materials, in particular to a carbon fiber composite material and a pat, and the carbon fiber composite material is composed of carbon fiber cloth and an epoxy resin system. The epoxy resin system is composed of bisphenol A epoxy resin, polyfunctional novolac epoxy resin, long-chain organosilicon epoxy resin and a curing agent, the prepared carbon fiber composite material has excellent mechanical properties and good damping performance, and when the carbon fiber composite material serves as a Pick racket, the elasticity and shock absorption performance are good during ball feeding, and the service life of the Pick racket is prolonged. The hand feeling is better, and hand joints can be protected.

The application belongs to the field of geotechnical engineering monitoring, and discloses an intelligent sensing type geotechnic drainage plate structure and a monitoring system thereof, which comprises a drainage plate core, a high-permeability filter membrane, a flexible film sensor, a positioning sensor and a multifunctional hand joint. The high-permeability filter membrane is fixed on the surface of the drainage plate core; the flexible film sensor and the positioning sensor are combined and fixed on the surface of the high-permeability filter membrane; the multifunctional hand joint comprises a low-power battery module, a sensing signal acquisition device, a positioning data acquisition device and a wireless data transmission device; the multifunctional hand joint is installed on the drainage plate with the sensor system to form the intelligent sensing type geotechnic drainage plate structure, which can be arranged horizontally and vertically, and can monitor the shape and spatial position of the drainage plate. The application has the advantages of high monitoring data accuracy, low manufacturing cost, simple structure and fast layout.

The invention discloses a finger joint point spatial position estimation method, a finger joint point spatial position estimation device and head-mounted display equipment. The method comprises the following steps: performing hand recognition on an image acquired by a depth camera through a palm region detection model, and if the image has a hand region, marking the hand region from the image to obtain a hand region image; correcting the hand region image to obtain a distortionless hand region image; identifying the pixel coordinates of the articulation points in the distortionless hand region image and the relative distance of each articulation point through an articulation point detection model; and according to the pixel coordinates of the joint points and the relative distance, calculating through a preset objective function to obtain the absolute distance of the joint points. According to the invention, the calculation amount in the 3D hand joint point estimation process can be reduced, and the technical effects of meeting the high real-time requirement in AR / VR application are achieved.

This application discloses a virtual avatar driving method, apparatus, device, and readable storage medium. The method includes: capturing images; acquiring a joint prediction network and a limb prediction network; inputting the captured images into the joint prediction network to obtain position data and joint confidence scores of multiple human joints output by the joint prediction network; inputting the position data into the limb prediction network to obtain posture data corresponding to each torso joint output by the limb prediction network; if the position data includes hand joint position data and the joint confidence score is greater than a preset threshold, acquiring a hand prediction network; acquiring a hand capture image; inputting the hand capture image into the hand prediction network to obtain posture data corresponding to multiple hand joints output by the hand prediction network; and driving the virtual avatar based on the position data and posture data of each human joint. Therefore, this application can achieve accurate synchronization between the virtual avatar and human movements.

A system for hand movements recognition and analysis comprises an image capture module, a recognition module, and a movement analysis module. The image capture module is installed in an operational area and is configured to capture a continuous image of the at least one hand movement of an operator during an operation period. The recognition module is coupled with the image capture module and comprises a 2D joint recognition model, a joint 3D coordinate recognition model and a hand skeletal joint position global optimization model to recognize a hand joint position of the operator based on the continuous image and generate a hand joint position coordinate. The movement analysis module comprises a hand movement analysis model for receiving the hand joint position coordinate, analyzing and comparing the hand joint position coordinate of the operator with a standard hand movement parameter set, and generating an analysis and comparison result to evaluate the correctness of the at least one hand movement of the operator in the operating area during the operation period.

The present invention relates to the technical field of intelligent robots, and particularly relates to a mapping operation and sample acquisition method for a virtual-real combined robotic arm-dexterous hand system. The method includes: configuring a mapping operation system; based on a teleoperation device, real-time collecting the end position and attitude information of the end of the operator's forearm; based on a camera, real-time collecting the three-dimensional information of the key points of the operator's hand; calculating the mapping value of the end of the robotic arm according to the end position and attitude information; calculating the mapping value of the dexterous hand joints according to the three-dimensional information of the key points; driving the robotic arm and the dexterous hand to move in real time according to the mapping value of the end of the robotic arm and the mapping value of the dexterous hand joints, and recording the real-time data of each joint and the observed images during the operation process of the mapping operation to complete sample acquisition. The embodiment of the present invention provides a mapping operation and sample acquisition method for a virtual-real combined robotic arm-dexterous hand system, which can perform the mapping operation and sample acquisition of the robotic arm-dexterous hand system.