127 results about "Hand joint" patented technology

An augmented reality device for obtaining three-dimensional (3D) position information of a plurality of hand joints of a user includes a plurality of cameras configured to photograph the user's hand and obtain images; memory storing instructions; and at least one processor; the instructions, when executed, may cause the device to recognize the plurality of hand joints from the images; obtain two-dimensional (2D) joint coordinate values for feature points corresponding to the hand joints; retrieve, from a look up table (LUT), 3D position coordinate values corresponding to distortion model parameters of the cameras, a positional relationship between the cameras, and the 2D joint coordinate values; and output the 3D position information of the plurality of hand joints based on the 3D position coordinate values.

The invention relates to a biomechanical modeling and normalized training data generation method and device for human hand posture estimation, equipment and a medium. The method comprises the following steps: acquiring target three-dimensional hand posture data of a target object; for each target object, acquiring reference static biomechanical characteristics and dynamic biomechanical characteristics corresponding to each target three-dimensional hand posture of the target object; for each target three-dimensional hand posture, determining a three-dimensional joint coordinate of the target three-dimensional hand posture and a two-dimensional joint coordinate corresponding to the three-dimensional joint coordinate based on the reference static biomechanical characteristics and the dynamic biomechanical characteristics; wherein the three-dimensional joint coordinates represent the spatial positions of the hand joint points; and determining hand posture training data based on the three-dimensional joint coordinates and the corresponding two-dimensional joint coordinates. By adopting the method, the individual biomechanical structure difference can be eliminated, and the number of training data can be expanded more accurately and reasonably.

The invention discloses a stroke hand rehabilitation training method based on multi-modal data fusion, particularly relates to the field of hand rehabilitation training, and is used for solving the problems that in the existing rehabilitation process, the motion solidification risk cannot be recognized, and the training scheme is lack of individualized dynamic adjustment. According to the method, hand joint motion features are obtained by constructing a standardized hand function micro-evaluation task, and a personal motion mode baseline is established in combination with dynamic time warping and multi-scale permutation entropy analysis; extracting a neural motion curing index based on the difference change between the current motion mode and the optimal motion mode, and analyzing and generating a function recovery momentum feature by combining the rehabilitation progress gradient and the acceleration inflection point; and fusing the electromyographic signal coupling coordination index to generate a rehabilitation risk score, and when the risk exceeds a dynamic threshold value, triggering adaptive recombination of a training strategy to realize accurate early warning and personalized training regulation and control. Rehabilitation monitoring precision and training pertinence can be effectively improved, and high-quality recovery of stroke hand functions is promoted.

The invention provides a multi-source heterogeneous data synchronous mechanical arm bionic control method and device and electronic equipment, and relates to the technical field of robot control, and the method comprises the following steps: decoding hand joint angle data and a synchronization timestamp thereof, wrist pose data and a synchronization timestamp thereof, and arm pose data and a synchronization timestamp thereof, and obtaining a decoding result; the decoded hand joint angle data, the decoded wrist pose data and the decoded arm pose data are transmitted to a mechanical arm controller through a low-delay network; and the mechanical arm controller executes dynamic remapping based on the decoded hand joint angle data, the decoded wrist pose data and the decoded arm pose data, and simulation control of the mechanical arm is achieved. According to the method, the arm pose data participates in dynamic correction of the base coordinate system, space positioning deviation is eliminated, and the mechanical arm reproduces actions; timing sequence block optimization considers fine grabbing and long track actions, and action restoration authenticity is improved.

The embodiment of the invention provides a robot control method and system based on virtual reality and hand motion capture, and the method comprises the steps: collecting the hand posture data of a user, obtaining the first hand posture data of the user through a virtual reality device, the first hand posture data comprises the hand end posture information, and obtaining the first hand posture data of the user through a virtual reality device; obtaining second hand posture data of the user through wearable hand motion capture equipment, wherein the second hand posture data comprises hand joint fine motion information; standardizing the hand position and posture data, and mapping the hand position and posture data into robot target motion data based on a preset calibration matrix, the robot target motion data including first robot target motion data mapped based on the first hand position and posture data and second robot target motion data mapped based on the second hand position and posture data; robot motion control data is generated based on the robot target motion data, and robot parts are controlled to move. According to the robot control method and system based on virtual reality and hand motion capture, the operation precision of the robot can be greatly improved, the operation real-time performance and smoothness are enhanced, and the motion state of the robot can be monitored in real time.

The invention relates to the technical field of data processing, in particular to a rehabilitation training control method and system based on a flexible hand joint rehabilitation glove, and the method comprises the steps: obtaining an individualized physiological parameter baseline of a patient; real-time data of a patient in the rehabilitation training process are collected through a multi-modal sensor array on the flexible glove, the active participation degree, the joint movement smoothness and the muscle fatigue degree of the patient are calculated based on the real-time data, the active movement intention of the patient is recognized, and a real-time rehabilitation state evaluation result of the patient is obtained; generating a control instruction through an adaptive control engine according to a preset personalized rehabilitation scheme and the real-time rehabilitation state evaluation result; the generated control instruction is sent to a driving control circuit so as to control a driving unit to execute corresponding bending or stretching actions and assist the patient in completing target training actions; according to the invention, personalized, self-adaptive and quantitative evaluation of rehabilitation training can be realized, and the efficiency and effect of hand rehabilitation training are improved.

The invention relates to robot motion control, in particular to a dexterous hand motion algorithm control method, system and device, a storage medium and a program product. The dexterous hand motion algorithm control method comprises the following steps: visual detection: acquiring a hand image through a camera, and identifying three-dimensional coordinates of key points of hand joints by adopting a visual detection algorithm; kinematics calculation: based on the coordinates of the joint key points, obtaining target execution angles of a plurality of motors through kinematics forward and inverse calculation; communication distribution: receiving the target execution angle through a master control unit, and distributing the target execution angle to a plurality of slave control units according to a preset distribution rule; and motor control: each slave control unit generates a driving signal by adopting a PID control algorithm based on the received target execution angle, and controls the corresponding motor to move. The dexterous hand motion algorithm control method provided by the invention is high in response speed, high in control precision, stable in communication and accurate in kinematics solution.

The invention discloses a gesture recognition method fusing continuous frame segmentation and spatial features. The gesture recognition method comprises the steps that three-dimensional coordinates of hand joint points are obtained through Mediape; performing dual normalization processing to eliminate size and position differences; extracting point cloud convex hull features, joint point distance features and specific triangularization area combination features; fusing a plurality of types of features and then training a Transform or GCN model; carrying out automatic segmentation on the continuous gesture sequence based on double cosine similarity thresholds; and inputting the segmented stable paragraphs into the model to realize real-time identification. The method provided by the invention has high precision and strong robustness in a complex scene, and is suitable for the fields of man-machine interaction, intelligent control and the like.

The invention provides a hand posture estimation method and device and head-mounted display equipment, and relates to the technical field of man-machine interaction, and the method comprises the steps: determining the pixel position and the relative distance of each hand joint point, and the bending degree of each finger according to a hand binocular image; according to the bending degree of each finger, an optimization variable of the hand posture is determined; according to the optimization variable, the pixel position and the relative distance of each hand joint point and the bending degree of each finger, an optimization function of the hand posture is generated; and performing iterative optimization on the hand posture according to the optimization function to obtain a hand posture estimation result. By combining the pixel position and the relative distance of each hand joint point and the bending degree of the finger, the hand posture is iteratively optimized, so that the estimation accuracy of the hand posture is improved.

The invention relates to the technical field of gesture recognition interaction, and discloses an intelligent interaction system and method based on gesture recognition. The method comprises the following steps: when an environment meets a condition, synchronously acquiring an image sequence of a gesture and depth distance field data; a hand joint point two-dimensional motion track is extracted from an image, surface deformation fluctuation information is separated from depth data, the two-dimensional motion track and the surface deformation fluctuation information are subjected to space-time registration fusion, a continuous motion track curved surface in a three-dimensional space is constructed, and then geometric topology features and dynamic change features of the continuous motion track curved surface are extracted. And inputting the fusion features into a neural network classifier subjected to incremental learning training, outputting corresponding semantic tags and confidence evaluation values, and mapping to generate a control instruction after verification. According to the method, an accurate three-dimensional dynamic model is constructed through deep fusion of multi-source data, and an incremental learning mechanism is utilized to enable the system to have online adaptive capability, so that the recognition precision of complex gestures and the long-term applicability of the system are improved.

The invention discloses a hand motion mapping method based on a data glove, which comprises the following steps of: judging the type of a task to be performed according to data acquired by the data glove, and selecting different mapping methods. If the current gesture is judged to be the gesture representation, applying a gesture recognition model, if the current gesture exists in a gesture library, applying a standard gesture in the library, and if the current gesture does not exist, applying direct joint angle mapping to four fingers except the thumb, and applying Cartesian space mapping to the thumb. And if the task is judged to be the operation task, the data of the data glove is directly processed by a pre-trained neural network and is directly mapped into a joint angle corresponding to the target manipulator. The training thought of the neural network is that the human hand joint angle represented by the data glove and each joint angle of the target manipulator under the same gesture action are obtained at the same time, a neural network model is constructed, the angle of the data glove serves as input, the corresponding angle of the manipulator serves as output, and training is conducted. Finally, the motion mapping process from the hand to the manipulator is achieved.

The invention provides a hand multi-mode information sensing device and a robot humanoid skill learning method, and the device comprises a multi-mode touch sensor module which is used for sensing a hand multi-mode touch signal; the flexible strain sensor module is used for sensing a hand joint bending signal; the multi-modal motion capture module is used for sensing hand motion and posture signals, and the hand motion and posture signals comprise hand motion signals and / or hand posture signals; at least one of the hand multi-mode touch signals, the hand joint bending signals and the hand movement and posture signals forms hand multi-mode information representing gesture actions, hand movement and hand touch. According to the invention, sensing of hand multi-modal information in a hand operation process can be realized based on the hand multi-modal information sensing device, establishment of a sensing strategy and a control strategy is facilitated, cross-object and cross-task robust skill migration is realized, and self-adaptive operation capability and task success rate of a robot in an actual complex environment are remarkably improved.

1. The name of the design product: adjustable eating fork. 2. The use of the design product: designed for the general silver-haired population or the elderly, the disabled, and people whose hand joint flexion function is limited, whose grip function is not in place, whose grip strength is insufficient, and whose hands are shaky. 3. The design points of the design product: in shape. 4. The picture or photo that best indicates the design points: perspective view.

The embodiment of the invention provides a dexterous hand joint angle determination method and equipment. The method comprises the steps that power output parameters of a driving motor of a target dexterous hand are obtained, the driving displacement amount of the target dexterous hand is determined according to the power output parameters, the joint angle of the target dexterous hand is determined according to the driving displacement amount and a first corresponding relation, and the first corresponding relation is determined based on the geometric structure of the target dexterous hand. The first corresponding relation is used for representing the incidence relation between the driving displacement and the joint angle of the target dexterous hand. According to the method, the driving displacement is determined, and the joint angle of the target dexterous hand is determined based on the first corresponding relation between the driving displacement constructed through the geometric structure of the target dexterous hand and the joint angle of the target dexterous hand, so that the determination precision of the joint angle is improved, additional sensors do not need to be installed at the joints, and the accuracy of the joint angle determination is improved. The space is saved.

The embodiment of the invention provides a mechanical finger joint miniature encoder mounting structure for improving control precision, and belongs to the technical field of medical instrument robot dexterous hands and high-precision sensing. The structure comprises a hollow annular magnetic encoder, a customized mounting base and an error compensation mechanism. The MPT-8-20 series encoder is directly integrated on a rotating shaft of a finger joint through the customized mounting base; the mounting base is provided with a positioning interface which is precisely matched with an encoder shell and a joint part, so that the concentricity and a constant air gap of an encoder sensing unit and a rotating part are ensured; the error compensation mechanism is integrated in the base and is used for passively inhibiting installation errors and environmental interference. According to the invention, the measurement precision of less than 0.01 degree in an extremely limited space is realized based on a commercial encoder, high anti-interference performance and reliability are realized, the integration problem of high-precision and high-reliability pose feedback of the miniature dexterous hand joint is effectively solved, and the miniature dexterous hand joint is suitable for miniature dexterous hands.

The utility model discloses a finger assembly and mechanical hand relates to mechanical hand technical field, finger assembly includes: finger body, mounting body, the mounting body is equipped with rotary seat and swing seat, the rotary seat rotatablely connected in the mounting body around rotary axis, swing seat rotatablely connected in the mounting body around swing axis, the finger body rotatablely connected in the swing seat around rotary axis, the finger body with rotary seat forms spherical hinge cooperation, and rotary axis with swing axis cross distribution. The finger assembly of the utility model, through setting rotary seat and swing seat, make the finger body can carry out the bending and swing of human hand joint, and simple structure, the volume is smaller.

The invention relates to the field of carbon fiber composite materials, in particular to a carbon fiber composite material and a pat, and the carbon fiber composite material is composed of carbon fiber cloth and an epoxy resin system. The epoxy resin system is composed of bisphenol A epoxy resin, polyfunctional novolac epoxy resin, long-chain organosilicon epoxy resin and a curing agent, the prepared carbon fiber composite material has excellent mechanical properties and good damping performance, and when the carbon fiber composite material serves as a Pick racket, the elasticity and shock absorption performance are good during ball feeding, and the service life of the Pick racket is prolonged. The hand feeling is better, and hand joints can be protected.

The application belongs to the field of geotechnical engineering monitoring, and discloses an intelligent sensing type geotechnic drainage plate structure and a monitoring system thereof, which comprises a drainage plate core, a high-permeability filter membrane, a flexible film sensor, a positioning sensor and a multifunctional hand joint. The high-permeability filter membrane is fixed on the surface of the drainage plate core; the flexible film sensor and the positioning sensor are combined and fixed on the surface of the high-permeability filter membrane; the multifunctional hand joint comprises a low-power battery module, a sensing signal acquisition device, a positioning data acquisition device and a wireless data transmission device; the multifunctional hand joint is installed on the drainage plate with the sensor system to form the intelligent sensing type geotechnic drainage plate structure, which can be arranged horizontally and vertically, and can monitor the shape and spatial position of the drainage plate. The application has the advantages of high monitoring data accuracy, low manufacturing cost, simple structure and fast layout.

The invention discloses a finger joint point spatial position estimation method, a finger joint point spatial position estimation device and head-mounted display equipment. The method comprises the following steps: performing hand recognition on an image acquired by a depth camera through a palm region detection model, and if the image has a hand region, marking the hand region from the image to obtain a hand region image; correcting the hand region image to obtain a distortionless hand region image; identifying the pixel coordinates of the articulation points in the distortionless hand region image and the relative distance of each articulation point through an articulation point detection model; and according to the pixel coordinates of the joint points and the relative distance, calculating through a preset objective function to obtain the absolute distance of the joint points. According to the invention, the calculation amount in the 3D hand joint point estimation process can be reduced, and the technical effects of meeting the high real-time requirement in AR / VR application are achieved.

This application discloses a virtual avatar driving method, apparatus, device, and readable storage medium. The method includes: capturing images; acquiring a joint prediction network and a limb prediction network; inputting the captured images into the joint prediction network to obtain position data and joint confidence scores of multiple human joints output by the joint prediction network; inputting the position data into the limb prediction network to obtain posture data corresponding to each torso joint output by the limb prediction network; if the position data includes hand joint position data and the joint confidence score is greater than a preset threshold, acquiring a hand prediction network; acquiring a hand capture image; inputting the hand capture image into the hand prediction network to obtain posture data corresponding to multiple hand joints output by the hand prediction network; and driving the virtual avatar based on the position data and posture data of each human joint. Therefore, this application can achieve accurate synchronization between the virtual avatar and human movements.

The invention discloses a multi-modal fusion self-adaptive sign language digital character generation method and system. By extracting three-dimensional coordinates of hand joint points, contour masks and multi-modal features of an optical flow field, an interpolation strategy is dynamically selected by using a reinforcement learning model according to states such as the number of missing frames and gesture complexity, and hierarchical cross-modal attention fusion and biomechanical constraints of a hybrid neural network are combined, so that high-precision gesture completion is realized. And the system applies skeleton dynamics and muscle-skin coupling constraint to the complemented joint points to ensure that the actions conform to the human anatomy law, and finally drives the digital figure to generate smooth sign language actions. According to the method, joint point errors are effectively reduced in a shielding scene, and the precision and efficiency of sign language digitization can be improved.

The invention discloses a gesture recognition method and system, a head-mounted display device and a medium, and relates to the technical field of image processing, and the disclosed gesture recognition method comprises the steps: obtaining hand images collected by each camera, setting the hand images corresponding to the same collection moment as a group, and obtaining each to-be-processed combination; inputting the to-be-processed combinations into a preset prediction model to obtain hand joint point coordinates corresponding to the to-be-processed combinations; and obtaining a target hand joint point coordinate according to each hand joint point coordinate, and performing gesture recognition based on the target hand joint point coordinate to obtain a gesture recognition result. The technical scheme of the invention aims to solve the technical problem of how to improve the accuracy of gesture recognition.

This invention discloses a human-machine collaborative safety monitoring method that integrates temporal features and multi-task risk assessment, belonging to the field of industrial human-machine collaborative safety protection. The method first constructs a static 3D point cloud template of the machine tool offline as a danger boundary, simultaneously acquires RGB-D images and performs pixel-level registration and fusion, and extracts initial features using a deep backbone network. It then uses a ConvLSTM convolutional long short-term memory network to model spatiotemporal features, suppresses instantaneous occlusion, and regresses to obtain a smooth and continuous 3D coordinate sequence of hand joints. Based on the coordinates, kinematic vectors are calculated and a hand gesture code of the work intention is generated. The multi-dimensional features are input into a multi-gate hybrid expert network (MMoE), and the weights are dynamically adjusted to output a comprehensive risk score. The system executes normal operation, linear deceleration, and hardware emergency braking according to the score, linking the machine tool PLC or robot controller. This enables high-precision hand positioning and occlusion tracking, predicts movement trends to reduce lag, balances safety and production efficiency, has strong industrial applicability, and is suitable for complex human-machine collaborative safety protection.

This invention provides a 3D gesture recognition method, system, electronic device, storage medium, and program product. The method includes: in a near-field interaction mode, acquiring a first 3D image containing a hand captured by a 3D camera; obtaining first 3D position information of the hand based on the first 3D image; obtaining a first 2D image based on the first 3D image; obtaining hand joint information based on the first 2D image; and determining 3D position information of the hand joints based on the first 3D position information and the hand joint information to determine gesture information; in a far-field interaction mode, acquiring a second 3D image containing a hand captured by a 3D camera; acquiring a second 2D image containing a hand captured by a 2D camera; obtaining second 3D position information of the hand based on the second 3D image; obtaining hand joint information based on the second 2D image; and performing coordinate fusion on the second 3D position information and the hand joint information to determine the 3D position information of the hand joints to determine gesture information.

The utility model belongs to the technical field of gloves, and relates to a wear-resistant heat insulation type rope descending glove. Comprising a glove body, and the glove body comprises a thumb part, a four-finger part, a palm part, a hand back part and a wrist part; a special-shaped wear-resistant reinforcing layer is arranged on the palm part; the wrist part is provided with an adjusting loop; the special-shaped wear-resistant reinforcing layer comprises a near-finger-end palm region reinforcing layer and a near-wrist-end palm region reinforcing layer; and a thumb wear-resistant reinforcing layer is arranged at the thumb part. Materials with good tensile strength and toughness are adopted, so that impact force can be absorbed and dispersed, and harm to hands is reduced; in the rope descending process, the hand can be vibrated by a rope, equipment and the environment. The shock absorption design of the palm center can absorb and disperse the shock, impact on hand joints and muscles is reduced, and therefore fatigue and discomfort are reduced.

The application discloses a kind of head-mounted gesture interaction device design methods for 3D large screen.The interaction device includes positioning module, gesture recognition module, 3D glasses and large screen terminal display module.Positioning module and gesture recognition module are fixed on 3D glasses, and positioning module is used to determine the view point coordinates of user in interactive space.Large screen terminal display module receives the view point coordinates and hand joint data provided by positioning module and gesture recognition module, and the 3D virtual reality scene content required by user view point is shown through screen.The 3D glasses are used to watch the content shown on screen.The interaction device binds positioning module, gesture recognition module and user's view point, and realizes synchronous hand tracking under dynamic view point.The pose of gesture recognition module is determined by the design method, and under the premise of not affecting effective interactive space, the factors affecting recognition accuracy, such as gesture occlusion, are weakened to the greatest extent, the flexibility and recognition accuracy are improved, and the cost is reduced.

The application provides an image recognition method, device and electronic equipment. The method comprises: acquiring a hand point cloud of a depth image; inputting the hand point cloud into a preset neural network model to obtain a hand joint position output by the preset neural network model; determining a hand grid of the depth image based on the hand joint position; determining a target gesture matched with the hand joint position; and determining a hand recognition result of the depth image based on the hand grid and the target gesture. The application can improve the accuracy of the hand recognition result.

The invention discloses a vertical orthogonal transmission conjugate planetary speed reduction structure for a dexterous hand joint, and relates to the technical field of robot dexterous hands, the vertical orthogonal transmission conjugate planetary speed reduction structure comprises a power source, a steering transmission unit, a first-stage speed reduction unit and a second speed reduction unit; one end of the steering transmission unit is connected with the output end of the power source, and the power output by the power source can be vertically steered and then output; the first-stage speed reduction unit and the second-stage speed reduction unit are coaxially arranged and located at the end, away from a power source, of the steering transmission unit, and the input end of the first-stage speed reduction unit is in transmission connection with the end, away from the power source, of the steering transmission unit. The output end of the first-stage speed reduction unit is in transmission connection with the input end of the second-stage speed reduction unit, and the output end of the second-stage speed reduction unit is externally connected with transmission equipment. According to the vertical orthogonal transmission conjugate planetary speed reduction structure for the dexterous hand joint, large-torque and large-reduction-ratio vertical orthogonal transmission can be achieved in a compact space.

The invention discloses a robot dexterous operation synthetic data generation method and system based on reinforcement learning, and the method comprises the steps: carrying out the preprocessing of motion capture data containing hand joint angles and object poses, and converting the posture parameters of a hand parameterized model into a unified control space; generating an action instruction through an accumulated residual action control strategy; performing strategy training on the action instruction by utilizing a reinforcement learning strategy, wherein an observation state space comprises a hand joint state, relative position information and an object pose; in the strategy execution process, a hand object contact event is detected, and contact force information containing multi-coordinate system representation is generated; operation track data is generated based on the original motion capture data, synthetic operation data containing complete contact force information is generated through a hybrid strategy, and data quality screening is executed; the invention provides a technical scheme capable of efficiently generating robot flexible operation synthetic data containing contact force information with high quality.