80 results about "Hand joint" patented technology

The embodiment of the invention provides a robot control method and system based on virtual reality and hand motion capture, and the method comprises the steps: collecting the hand posture data of a user, obtaining the first hand posture data of the user through a virtual reality device, the first hand posture data comprises the hand end posture information, and obtaining the first hand posture data of the user through a virtual reality device; obtaining second hand posture data of the user through wearable hand motion capture equipment, wherein the second hand posture data comprises hand joint fine motion information; standardizing the hand position and posture data, and mapping the hand position and posture data into robot target motion data based on a preset calibration matrix, the robot target motion data including first robot target motion data mapped based on the first hand position and posture data and second robot target motion data mapped based on the second hand position and posture data; robot motion control data is generated based on the robot target motion data, and robot parts are controlled to move. According to the robot control method and system based on virtual reality and hand motion capture, the operation precision of the robot can be greatly improved, the operation real-time performance and smoothness are enhanced, and the motion state of the robot can be monitored in real time.

The invention relates to the technical field of data processing, in particular to a rehabilitation training control method and system based on a flexible hand joint rehabilitation glove, and the method comprises the steps: obtaining an individualized physiological parameter baseline of a patient; real-time data of a patient in the rehabilitation training process are collected through a multi-modal sensor array on the flexible glove, the active participation degree, the joint movement smoothness and the muscle fatigue degree of the patient are calculated based on the real-time data, the active movement intention of the patient is recognized, and a real-time rehabilitation state evaluation result of the patient is obtained; generating a control instruction through an adaptive control engine according to a preset personalized rehabilitation scheme and the real-time rehabilitation state evaluation result; the generated control instruction is sent to a driving control circuit so as to control a driving unit to execute corresponding bending or stretching actions and assist the patient in completing target training actions; according to the invention, personalized, self-adaptive and quantitative evaluation of rehabilitation training can be realized, and the efficiency and effect of hand rehabilitation training are improved.

The invention discloses a gesture recognition method fusing continuous frame segmentation and spatial features. The gesture recognition method comprises the steps that three-dimensional coordinates of hand joint points are obtained through Mediape; performing dual normalization processing to eliminate size and position differences; extracting point cloud convex hull features, joint point distance features and specific triangularization area combination features; fusing a plurality of types of features and then training a Transform or GCN model; carrying out automatic segmentation on the continuous gesture sequence based on double cosine similarity thresholds; and inputting the segmented stable paragraphs into the model to realize real-time identification. The method provided by the invention has high precision and strong robustness in a complex scene, and is suitable for the fields of man-machine interaction, intelligent control and the like.

The invention relates to the technical field of gesture recognition interaction, and discloses an intelligent interaction system and method based on gesture recognition. The method comprises the following steps: when an environment meets a condition, synchronously acquiring an image sequence of a gesture and depth distance field data; a hand joint point two-dimensional motion track is extracted from an image, surface deformation fluctuation information is separated from depth data, the two-dimensional motion track and the surface deformation fluctuation information are subjected to space-time registration fusion, a continuous motion track curved surface in a three-dimensional space is constructed, and then geometric topology features and dynamic change features of the continuous motion track curved surface are extracted. And inputting the fusion features into a neural network classifier subjected to incremental learning training, outputting corresponding semantic tags and confidence evaluation values, and mapping to generate a control instruction after verification. According to the method, an accurate three-dimensional dynamic model is constructed through deep fusion of multi-source data, and an incremental learning mechanism is utilized to enable the system to have online adaptive capability, so that the recognition precision of complex gestures and the long-term applicability of the system are improved.

The invention discloses a hand motion mapping method based on a data glove, which comprises the following steps of: judging the type of a task to be performed according to data acquired by the data glove, and selecting different mapping methods. If the current gesture is judged to be the gesture representation, applying a gesture recognition model, if the current gesture exists in a gesture library, applying a standard gesture in the library, and if the current gesture does not exist, applying direct joint angle mapping to four fingers except the thumb, and applying Cartesian space mapping to the thumb. And if the task is judged to be the operation task, the data of the data glove is directly processed by a pre-trained neural network and is directly mapped into a joint angle corresponding to the target manipulator. The training thought of the neural network is that the human hand joint angle represented by the data glove and each joint angle of the target manipulator under the same gesture action are obtained at the same time, a neural network model is constructed, the angle of the data glove serves as input, the corresponding angle of the manipulator serves as output, and training is conducted. Finally, the motion mapping process from the hand to the manipulator is achieved.

The invention provides a hand multi-mode information sensing device and a robot humanoid skill learning method, and the device comprises a multi-mode touch sensor module which is used for sensing a hand multi-mode touch signal; the flexible strain sensor module is used for sensing a hand joint bending signal; the multi-modal motion capture module is used for sensing hand motion and posture signals, and the hand motion and posture signals comprise hand motion signals and / or hand posture signals; at least one of the hand multi-mode touch signals, the hand joint bending signals and the hand movement and posture signals forms hand multi-mode information representing gesture actions, hand movement and hand touch. According to the invention, sensing of hand multi-modal information in a hand operation process can be realized based on the hand multi-modal information sensing device, establishment of a sensing strategy and a control strategy is facilitated, cross-object and cross-task robust skill migration is realized, and self-adaptive operation capability and task success rate of a robot in an actual complex environment are remarkably improved.

The embodiment of the invention provides a mechanical finger joint miniature encoder mounting structure for improving control precision, and belongs to the technical field of medical instrument robot dexterous hands and high-precision sensing. The structure comprises a hollow annular magnetic encoder, a customized mounting base and an error compensation mechanism. The MPT-8-20 series encoder is directly integrated on a rotating shaft of a finger joint through the customized mounting base; the mounting base is provided with a positioning interface which is precisely matched with an encoder shell and a joint part, so that the concentricity and a constant air gap of an encoder sensing unit and a rotating part are ensured; the error compensation mechanism is integrated in the base and is used for passively inhibiting installation errors and environmental interference. According to the invention, the measurement precision of less than 0.01 degree in an extremely limited space is realized based on a commercial encoder, high anti-interference performance and reliability are realized, the integration problem of high-precision and high-reliability pose feedback of the miniature dexterous hand joint is effectively solved, and the miniature dexterous hand joint is suitable for miniature dexterous hands.

The utility model discloses a finger assembly and mechanical hand relates to mechanical hand technical field, finger assembly includes: finger body, mounting body, the mounting body is equipped with rotary seat and swing seat, the rotary seat rotatablely connected in the mounting body around rotary axis, swing seat rotatablely connected in the mounting body around swing axis, the finger body rotatablely connected in the swing seat around rotary axis, the finger body with rotary seat forms spherical hinge cooperation, and rotary axis with swing axis cross distribution. The finger assembly of the utility model, through setting rotary seat and swing seat, make the finger body can carry out the bending and swing of human hand joint, and simple structure, the volume is smaller.

The invention relates to the field of carbon fiber composite materials, in particular to a carbon fiber composite material and a pat, and the carbon fiber composite material is composed of carbon fiber cloth and an epoxy resin system. The epoxy resin system is composed of bisphenol A epoxy resin, polyfunctional novolac epoxy resin, long-chain organosilicon epoxy resin and a curing agent, the prepared carbon fiber composite material has excellent mechanical properties and good damping performance, and when the carbon fiber composite material serves as a Pick racket, the elasticity and shock absorption performance are good during ball feeding, and the service life of the Pick racket is prolonged. The hand feeling is better, and hand joints can be protected.

The application belongs to the field of geotechnical engineering monitoring, and discloses an intelligent sensing type geotechnic drainage plate structure and a monitoring system thereof, which comprises a drainage plate core, a high-permeability filter membrane, a flexible film sensor, a positioning sensor and a multifunctional hand joint. The high-permeability filter membrane is fixed on the surface of the drainage plate core; the flexible film sensor and the positioning sensor are combined and fixed on the surface of the high-permeability filter membrane; the multifunctional hand joint comprises a low-power battery module, a sensing signal acquisition device, a positioning data acquisition device and a wireless data transmission device; the multifunctional hand joint is installed on the drainage plate with the sensor system to form the intelligent sensing type geotechnic drainage plate structure, which can be arranged horizontally and vertically, and can monitor the shape and spatial position of the drainage plate. The application has the advantages of high monitoring data accuracy, low manufacturing cost, simple structure and fast layout.

The invention discloses a gesture recognition method and system, a head-mounted display device and a medium, and relates to the technical field of image processing, and the disclosed gesture recognition method comprises the steps: obtaining hand images collected by each camera, setting the hand images corresponding to the same collection moment as a group, and obtaining each to-be-processed combination; inputting the to-be-processed combinations into a preset prediction model to obtain hand joint point coordinates corresponding to the to-be-processed combinations; and obtaining a target hand joint point coordinate according to each hand joint point coordinate, and performing gesture recognition based on the target hand joint point coordinate to obtain a gesture recognition result. The technical scheme of the invention aims to solve the technical problem of how to improve the accuracy of gesture recognition.

This invention discloses a human-machine collaborative safety monitoring method that integrates temporal features and multi-task risk assessment, belonging to the field of industrial human-machine collaborative safety protection. The method first constructs a static 3D point cloud template of the machine tool offline as a danger boundary, simultaneously acquires RGB-D images and performs pixel-level registration and fusion, and extracts initial features using a deep backbone network. It then uses a ConvLSTM convolutional long short-term memory network to model spatiotemporal features, suppresses instantaneous occlusion, and regresses to obtain a smooth and continuous 3D coordinate sequence of hand joints. Based on the coordinates, kinematic vectors are calculated and a hand gesture code of the work intention is generated. The multi-dimensional features are input into a multi-gate hybrid expert network (MMoE), and the weights are dynamically adjusted to output a comprehensive risk score. The system executes normal operation, linear deceleration, and hardware emergency braking according to the score, linking the machine tool PLC or robot controller. This enables high-precision hand positioning and occlusion tracking, predicts movement trends to reduce lag, balances safety and production efficiency, has strong industrial applicability, and is suitable for complex human-machine collaborative safety protection.

This invention provides a 3D gesture recognition method, system, electronic device, storage medium, and program product. The method includes: in a near-field interaction mode, acquiring a first 3D image containing a hand captured by a 3D camera; obtaining first 3D position information of the hand based on the first 3D image; obtaining a first 2D image based on the first 3D image; obtaining hand joint information based on the first 2D image; and determining 3D position information of the hand joints based on the first 3D position information and the hand joint information to determine gesture information; in a far-field interaction mode, acquiring a second 3D image containing a hand captured by a 3D camera; acquiring a second 2D image containing a hand captured by a 2D camera; obtaining second 3D position information of the hand based on the second 3D image; obtaining hand joint information based on the second 2D image; and performing coordinate fusion on the second 3D position information and the hand joint information to determine the 3D position information of the hand joints to determine gesture information.

The application provides an image recognition method, device and electronic equipment. The method comprises: acquiring a hand point cloud of a depth image; inputting the hand point cloud into a preset neural network model to obtain a hand joint position output by the preset neural network model; determining a hand grid of the depth image based on the hand joint position; determining a target gesture matched with the hand joint position; and determining a hand recognition result of the depth image based on the hand grid and the target gesture. The application can improve the accuracy of the hand recognition result.

The invention discloses a vertical orthogonal transmission conjugate planetary speed reduction structure for a dexterous hand joint, and relates to the technical field of robot dexterous hands, the vertical orthogonal transmission conjugate planetary speed reduction structure comprises a power source, a steering transmission unit, a first-stage speed reduction unit and a second speed reduction unit; one end of the steering transmission unit is connected with the output end of the power source, and the power output by the power source can be vertically steered and then output; the first-stage speed reduction unit and the second-stage speed reduction unit are coaxially arranged and located at the end, away from a power source, of the steering transmission unit, and the input end of the first-stage speed reduction unit is in transmission connection with the end, away from the power source, of the steering transmission unit. The output end of the first-stage speed reduction unit is in transmission connection with the input end of the second-stage speed reduction unit, and the output end of the second-stage speed reduction unit is externally connected with transmission equipment. According to the vertical orthogonal transmission conjugate planetary speed reduction structure for the dexterous hand joint, large-torque and large-reduction-ratio vertical orthogonal transmission can be achieved in a compact space.

The invention discloses a robot dexterous operation synthetic data generation method and system based on reinforcement learning, and the method comprises the steps: carrying out the preprocessing of motion capture data containing hand joint angles and object poses, and converting the posture parameters of a hand parameterized model into a unified control space; generating an action instruction through an accumulated residual action control strategy; performing strategy training on the action instruction by utilizing a reinforcement learning strategy, wherein an observation state space comprises a hand joint state, relative position information and an object pose; in the strategy execution process, a hand object contact event is detected, and contact force information containing multi-coordinate system representation is generated; operation track data is generated based on the original motion capture data, synthetic operation data containing complete contact force information is generated through a hybrid strategy, and data quality screening is executed; the invention provides a technical scheme capable of efficiently generating robot flexible operation synthetic data containing contact force information with high quality.

The invention relates to a method, a device and a system for preventing fingertip pinching mold penetration in VR gesture tracking and application. The method comprises the following steps: acquiring real-time three-dimensional space coordinates of a thumb fingertip joint point and an index finger fingertip joint point, and calculating a distance d between the thumb fingertip joint point and the index finger fingertip joint point; establishing a multi-level threshold group including an approach threshold and a contact threshold, establishing an anti-mold-penetrating preparation mode and an anti-mold-penetrating constraint mode, and triggering a corresponding mode based on the relationship between d and the multi-level threshold group; after the anti-mold-penetration preparation mode is triggered, the three-dimensional space coordinate positions of the thumb fingertip joint point and the index finger fingertip joint point are corrected; driving a virtual hand model to render based on the corrected fingertip position and other uncorrected hand joint point data; the device and the system are implemented based on the method, and are applied to the correction of kneading mold penetration in a VR man-machine gesture interaction process. According to the method, the interaction naturalness and immersion are remarkably improved, stable system performance is ensured, the problem of physical authenticity of micro-interaction is solved, and coherence, smoothness and naturalness of the whole hand posture are ensured.

The utility model discloses a finished product tobacco case correcting device, include: fixed release station and correction mechanism, and fixed release station is by several electric roller horizontal equal interval setting and is composed, and the correction mechanism includes the front frame and rear frame, and the front movable baffle and rear movable baffle and elastic component, and one end of elastic component is connected with the frame, and the other end is connected with movable baffle, and movable baffle can be under the action of elastic component and carry out elastic expansion. The present application will be the former rear frame widen, plus finished product tobacco case correcting device inclination certain angle and can elastic expansion movable frame, ensure that mechanical hand release not in the middle tobacco case to release station, be forcibly corrected back to the position in the middle, avoid the damage of tobacco case and mechanical hand joint, and greatly improve the finished product cigarette's delivery efficiency, also alleviate the labor intensity of operation maintenance personnel.

The application discloses a hand posture recognition method, a hand posture recognition model training method and device, a computer device and a storage medium, and belongs to the technical field of artificial intelligence. The application processes a first video frame of a video based on a two-dimensional posture recognition model to determine a plurality of two-dimensional hand joint nodes describing a two-dimensional hand posture; determines a plurality of three-dimensional hand joint nodes of the hand based on a three-dimensional posture recognition model matched with a chiral feature of the hand and the plurality of two-dimensional hand joint nodes; and processes the plurality of three-dimensional hand joint nodes by a constraint loss function and a second video frame before the first video frame. Through the above technical solution, when the hand posture of the video is recognized, the gap between the three-dimensional hand joint nodes of the plurality of video frames is constrained, the relevance between the hands in the plurality of video frames is referred to, the continuity and fluency of the hand posture determined for the video are ensured, and the stability of the hand posture recognition result is greatly improved.

The application discloses a hand 3D joint detection method and system based on a 2D heat map, and belongs to the technical field of computer vision. The method comprises the following steps: a convolution network model is used to extract features of a hand picture to obtain a feature map; regression processing is performed on feature points in the feature map to obtain a depth value z of a hand joint in a wrist coordinate system and a depth value z0 of a wrist joint in a camera coordinate system; regression processing is performed on the feature points in the feature map to obtain a 2D heat map of the hand joint, and 2D coordinates of the hand joint are determined; and the 3D position of the hand joint is determined according to the depth value z' of the hand joint in the wrist coordinate system, the depth value z0 of the wrist joint in the camera coordinate system and the 2D coordinates. The application obtains the depth value and the 2D heat map of the hand joint respectively, and then obtains the position of the hand 3D joint, so that the data amount of processing is reduced, and the operation speed and accuracy are improved.

The utility model provides a harmonic speed reduction module for a dexterous hand joint of a humanoid robot, and relates to the technical field of humanoid finger joints, the harmonic speed reduction module comprises an axial magnetic flux motor and a double-rigid-wheel harmonic speed reducer, the axial magnetic flux motor comprises a motor shell, an input shaft, a permanent magnet rotor and two PCB stators; the double-rigid-gear harmonic speed reducer comprises a wave generator, a flexible bearing, a flexible gear, a first rigid gear, a second rigid gear and a supporting bearing, the input shaft is fixedly connected with the wave generator, the first rigid gear serves as an inner ring of the supporting bearing and is detachably and fixedly connected with the motor shell, and a connecting plate is arranged between the first rigid gear and the motor shell; the rigid gear II is positioned on one side of the rigid gear I far away from the motor shell as a support bearing outer ring; the flexible bearings comprise two first flexible bearings which are arranged in parallel in the axial direction of the input shaft, and a limiting protrusion located between outer rings of the two first bearings is arranged in the middle of the inner wall of the flexible gear; an end cover is arranged on the side, away from the first rigid wheel, of the second rigid wheel. The axial length of the joint module is shortened, and meanwhile the load capacity and the transmission precision of the joint module are improved.

The application discloses an augmented reality environment-based auxiliary assembly human factor ergonomics evaluation method and system based on hand trajectory, converts three-dimensional coordinates of hand joints collected from an operator into a coordinate system with the head of the operator as an origin to obtain hand centroid coordinates, divides the hand movement trajectory into a plurality of discrete action segments based on the motion speed and the motion direction change of the hand centroid at each time; for each action segment, hand movement characteristic indexes are calculated, including an eye-hand unit vector, an actual extension distance, a motion amplitude and smoothness; based on the hand movement characteristic indexes, the operation visibility, the reachability and the posture load score of each action segment of the operator and the overall posture load of the task are comprehensively calculated to perform real-time human factor ergonomics quantitative evaluation. The application can provide technical support for real-time objective quantitative evaluation of human factor ergonomics of assembly operation, action characteristic analysis and process flow optimization.

The invention discloses a GNN-based hand tremor amplitude identification method. The method comprises the following steps of obtaining videos of a hand of a detected object in two states; the acquired video data are preprocessed; generating a node graph by taking hand joints in the video as nodes; constructing a hand tremor amplitude grade evaluation model; and evaluating a to-be-tested video by using the trained model to generate an evaluation result. As a classification method, the hand joint tremor amplitude recognition method provided by the invention can quickly recognize the hand tremor level with high precision.

The invention provides a hand data set generation method and electronic equipment, which are used for improving the precision and efficiency of a hand data set. Comprising the following steps: receiving the motion trail of a head-mounted display and the motion trail of each hand joint point sent by a motion capture system; transforming the motion trail of the head-mounted display by using each first transformation relation to obtain the motion trail of each camera mounted on the head-mounted display; driving the three-dimensional hand model by using the motion trail of the three-dimensional hand model obtained according to the motion trail of each hand joint point, and driving the virtual camera to photograph the three-dimensional hand model by using the motion trail of the virtual camera obtained according to the motion trail of each camera to obtain a plurality of hand green screen images; synthesizing the plurality of hand green screen images and the plurality of background images to obtain a plurality of target hand images; and obtaining a hand data set based on the plurality of target hand images, the hand articulation points, the 3D position coordinates corresponding to the plurality of target hand images and the image position coordinates of the hand articulation points in the plurality of target hand images.

The present disclosure relates to an information processing device and an information processing method that make it possible to improve the accuracy of estimating the shape of a person's hand estimated from a captured image of the person who is a subject. In the present invention, a mesh is estimated from an image of the person who is the subject, the mesh is used to set a hand joint parameter to 0 and determine the size from the wrist to the tip of the middle finger when the palm is opened as the size of a hand region, the skeleton of the person is estimated from the image, a center position of the hand region is determined from the joint group corresponding to the skeleton of the person, and the hand shape is estimated on the basis of a hand image obtained by cutting out, with the center position of the hand region as a reference, the size range obtained by multiplying the size of the hand region by a prescribed coefficient. The present disclosure can be applied to virtual viewpoint image reproduction apparatuses.

The application discloses a kind of based on deep learning's electromyographic signal data generation method, belong to deep learning field;First, build including deep angle sequence encoder, context encoder, electromyographic signal generator and discriminator's confrontation model framework, and place on user hand joint, generate electromyographic signal data;Among them, deep angle sequence encoder is used to encode hand joint angle sequence control signal, and input context encoder generates the latent information of electromyographic signal, generates corresponding electromyographic signal using electromyographic signal generator;Electromyographic signal discriminator discriminates whether each time step conforms to real electromyographic signal mode;Then, electromyographic signal data is used to train electromyographic signal classification network based on convolutional neural network, and is deployed to actual embedded device or mobile terminal, carries out real-time gesture recognition.The application effectively expands existing electromyographic signal dataset, and simultaneously uses sequence information as control signal, effectively improves the quality of data generation.

The application discloses a dynamic hand gesture reconstruction method and device based on a millimeter wave radar signal, which comprises the following steps: preprocessing hand ADC data received by a radar, then feeding the data into a learnable beamforming layer, and outputting a spatial feature vector containing the response intensity of all preset azimuth and pitch angle directions; performing dimension rearrangement and stacking on the spatial feature vector, and constructing a 4D millimeter wave cube; inputting a 3D convolutional neural network for feature extraction, including an initial main trunk layer and a plurality of feature extraction stages, using a path aggregation feature pyramid network to aggregate features of different scales after multiple attention enhancement, and adjusting the output features into a frame-level spatial feature sequence; simultaneously capturing short-term and long-term motion patterns of the frame-level spatial feature sequence through two parallel branches to realize hybrid time sequence dynamic modeling, performing multi-strategy feature aggregation on the output, and outputting final 3D hand joint point coordinates through a gesture regression head.

The embodiment of the application provides a dexterous hand joint angle determination method and device. The method comprises the following steps: acquiring a driving motor power output parameter of a target dexterous hand; determining a driving displacement amount of the target dexterous hand according to the driving motor power output parameter; determining a joint angle of the target dexterous hand according to the driving displacement amount and a first corresponding relationship. The first corresponding relationship is determined based on the geometric structure of the target dexterous hand, and is used to represent the correlation between the driving displacement amount and the joint angle of the target dexterous hand. The method of the embodiment determines the driving displacement amount, and determines the joint angle of the target dexterous hand based on the first corresponding relationship between the driving displacement amount and the joint angle of the target dexterous hand constructed through the geometric structure of the target dexterous hand, thereby improving the determination accuracy of the joint angle, and saving space without the need of installing additional sensors at the joint.

The application discloses a hand action recognition method based on 3D gesture sequence analysis. For the 3D skeleton action sequence of the hand action, the speed, angle and relative position information of the hand joint are extracted frame by frame, the joint rate of each skeleton joint is calculated, the minimum value position of the joint rate waveform diagram is taken as the gesture action segmentation point, a plurality of action frame sequences are obtained, then the finger state is matched and coded frame by frame, and the coding result corresponding to the gesture action is obtained. After adding the pose information, the plurality of gesture actions are combined into an action matrix, and the matching of the hand action is completed according to the action matrix. The application uses the joint rate to replace the segmentation method of the fixed-length time window, so that the segmentation result can contain the complete gesture action and meet the actual situation. For each gesture action, the matching and coding are performed frame by frame, so that the interference frame or invalid information frame can be effectively removed, the matching difficulty and search time are reduced, and the recognition accuracy is improved.