37 results about "Hand grasp" patented technology

The invention discloses a dexterous hand perception control method and device, equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the steps that under the condition that a target multi-fingered dexterous hand grabs a target object is recognized through a pressure sensor array, pressure collection data are obtained from a pressure sensor array, and current joint data of all finger joints of the target multi-fingered dexterous hand are obtained from an encoder; according to the pressure collection data and the current joint data, the current grabbing state of the target multi-fingered dexterous hand is determined; when it is recognized that the current grabbing state does not meet the safe grabbing condition, dexterous hand pose data of the target multi-fingered dexterous hand are obtained from an inertial sensor, and a target grabbing mode of the target multi-fingered dexterous hand is determined according to the current space pose data, the pressure collection data, the current joint data and the dexterous hand pose data; and the target multi-fingered dexterous hand is controlled to grab in a target grabbing mode. According to the scheme, the sensing grabbing efficiency and accuracy of the dexterous hand are improved.

The invention discloses a dexterous hand adaptive force control system based on an improved iTransform neural network. The dexterous hand adaptive force control system comprises a sensing module, a prediction module and a control module. The sensing module collects contact data in real time through a multi-channel capacitive sensor, and the contact data is input into the prediction module after high-dimensional time sequence processing. And the prediction module analyzes the stress time sequence characteristics by using an improved iTransform network, and generates a contact force prediction model. And the control module combines task planning, parameter optimization and real-time feedback through an efficient architecture of a microcontroller and a processor, and dynamically calibrates the grabbing force. High-dimensional time sequence processing is carried out on multi-channel sensor data, efficient processing and high-speed transmission are carried out on the data through a brain-like architecture of a system microcontroller unit and a high-performance processor, and a dexterous hand is operated in real time to grab a task.

The invention discloses an intelligent learning method for grabbing of a three-finger dexterous hand for multiple types of workpieces, and belongs to the technical field of intelligent grabbing application of manipulators. A pre-defined rule and simulation expert double-layer mode is provided, a high-quality demonstration data set is automatically generated, a simulation grabbing scene of a mechanical arm and a three-fingered dexterous hand is constructed, recognition and classification of multiple types of workpieces are achieved by analyzing workpiece contour features, grabbing hand shapes are matched according to the recognition and classification, strategy pre-training is carried out through simulation learning, and the recognition and classification efficiency is improved. And optimizing the strategy based on a PPO algorithm and a composite reward function. Aiming at the problem of lack of a standardized three-fingered dexterous hand parameter model, a modeling method based on parameterized URDF and automatic script generation is used to realize efficient configuration and optimization of the three-fingered dexterous hand in simulation. Through performance evaluation, stable and high-quality grabbing of multiple types of workpieces can be achieved.

The invention belongs to the technical field of manipulators, and relates to a dexterous hand grabbing pose prediction method and device, and the method comprises the steps: obtaining an RGBD image of a target object; segmenting the RGBD image according to the semantic information of the RGBD image to obtain an image mask corresponding to the grabbable part of the target object; acquiring a depth point cloud of the target object; performing calculation processing on the depth point cloud according to the image mask to obtain a grabbable point cloud corresponding to a part of or all the image masks; and based on the grabbable point cloud, using a pose prediction model to predict and obtain the grabbing pose of the dexterous hand. According to the technical scheme, the semantic information of the RGBD image can be fully utilized in the process of predicting the grabbing pose of the dexterous hand, and it is ensured that the pose prediction model only uses part or all of the point clouds of the grabbable part of the target object as input to predict the grabbing pose of the dexterous hand; the technical problem that according to an existing dexterous hand grabbing pose prediction method, due to weak semantics of the point cloud, the predicted grabbing pose is wrong is solved.

The utility model discloses a comfortable lifting handle with two hand grasping parts capable of being combined for use, which comprises a lifting handle with a lantern ring structure, the lifting handle is integrally formed by injection molding and is provided with the two hand grasping parts, and the comfortable lifting handle is characterized in that each hand grasping part consists of a barrel-shaped body and a connecting belt for connecting two ends of the barrel-shaped body with a lantern ring; an opening is formed in the axial side face of the barrel-shaped body, and the minimum width of the opening is smaller than the inner diameter of the barrel-shaped body. The lifting handle solves the problem that when an existing lifting handle with two hand grabbing parts is used, the two hand grabbing parts cannot be used in a combined mode. In actual use, the two hand grabbing parts can be clamped in from the opening of the other barrel-shaped body through any barrel-shaped body and wrapped to form a combined state, and on the contrary, the two hand grabbing parts can be separated to form an independent state as long as the two hand grabbing parts are pulled open in the opposite direction.

The invention belongs to the technical field of robots, and particularly relates to a dexterous grabbing planning method based on conditional diffusion generation and accessibility perception, which comprises the following steps of: initializing a grabbing posture, and acquiring a multi-fingered dexterous hand grabbing simulation data set, the simulation data set comprises scene point cloud data, stably grabbed palm posture data and finger joint angle data; the multi-fingered dexterous hand grabbing simulation data set is input into a conditional diffusion generation model, a complete grabbing posture is obtained, the conditional diffusion generation model is used for extracting local features in a scene point cloud, and the complete grabbing posture is obtained according to the local features; according to the reachability graph of the mechanical arm, training a stacking auto-encoder, and obtaining a grabbing reachability evaluator; the complete grabbing posture is input into the grabbing accessibility evaluator, and the accessibility probability is obtained; and on the basis of the reachable probability, in combination with the grabbing stability probability, a target grabbing posture is obtained, and grabbing is conducted according to the target grabbing posture.

The invention discloses a universal robot dexterous hand grabbing teaching device and a grabbing strategy learning method. The method comprises the following steps: acquiring multi-modal data; establishing an SLAM model, carrying out real-time pose calculation on the multi-modal data, and returning the multi-modal data in the SLAM model in real time in combination with the action of the dexterous hand; the method comprises the following steps: constructing a learning network comprising a visual encoder ViT, a multi-head cross attention mechanism, a force perception encoder MLP, a splicing module and a Diffusion Policy; training a learning model; generating an action conforming to a target; obtaining delay compensation data; obtaining an updated robot action generation strategy; and predicting a subsequent action sequence in real time. The device comprises an IMU camera, a paper mirror, a fixing device, a touch sensor and a computer program product. According to the teaching device and the grabbing strategy learning method, the teaching device and the grabbing strategy learning method can be suitable for various different robot platforms, and execution of various complex tasks such as operation tasks in the fields of industrial automation, family service, medical care and the like is achieved through imitation learning.

The invention discloses an under-driven dexterous hand grabbing pose generation method based on deep learning, and belongs to the technical field of robot operation and intelligent grabbing. According to the method, the prior knowledge of a large language model LLM to real capture is introduced into a data set, and a reasonable capture pose is realized by using object semantic-geometric joint features of a point cloud encoder Point Transform and the LLM. According to the method, three types of grabbing strategy classification models for the under-actuated five-finger dexterous hand are constructed, including two-finger kneading, three-finger kneading and five-finger grabbing, and are used for restraining and guiding generation of subsequent fine grabbing postures. According to the selected grabbing strategy, a network is generated to predict six-dimensional poses and angles of all joints of the dexterous hand, and a grabbing posture matched with the shape of the object is obtained. According to the method, two constraint loss items are additionally added to realize grabbing quality optimization. The method can be suitable for intelligent grabbing tasks of industrial robots, service robots and precision operation equipment under the condition that negative grabbing of samples and complex post-processing optimization are not needed.

This invention discloses a method and system for generating grasping postures of a multi-fingered dexterous hand based on CVAE and Ball Query algorithms, belonging to the field of robot grasping control technology. The method includes: (1) a data sampling step; (2) a data preprocessing step: standardizing the collected data to generate enhanced point cloud data and constructing a training dataset; (3) a model training step: using a multi-scale feature extraction module combined with global semantics and local geometric features extracted by the Ball Query algorithm, generating grasping postures using a conditional variational autoencoder, and optimizing model parameters through reconstruction loss and KL divergence; (4) a real-time deployment step: integrating the trained model into a physical platform, filtering candidate grasping postures based on parallel collision detection, and realizing real-time grasping control by combining inverse kinematics verification. This invention effectively solves the problem of insufficient generalization ability of traditional methods in grasping complex objects, and achieves adaptive grasping of unknown objects while ensuring grasping stability.

A method and system for dexterous robot manipulation based on visual keypoint guidance is disclosed. The method includes: S1, establishing a base coordinate system, solving for initial keypoints based on the base coordinate system, and locating the dexterous hand based on the initial keypoints to determine the end-effector posture when grasping a tool; S2, the dexterous hand grasps the tool using a pre-established coarse gesture library and based on the end-effector posture when grasping the tool; S3, using the dexterous hand to move the tool, identifying new keypoints after the shift, and calculating the transformation matrix between the tool head coordinate system and the base coordinate system, as well as the transformation matrix between the target object coordinate system and the base coordinate system, based on the new keypoints; S4, calculating the end-effector position of the tool head in operation on the target object, until the dexterous hand completes the operation on the target object. This invention transforms the interaction process between the dexterous hand and the target object into the alignment of keypoints between the object and the hand, conforming to the rules of dexterous hand grasping tools and exhibiting strong generalization ability.

This invention discloses a five-finger dexterity hand grasping detection method based on soft-mask region representation and multi-task learning. This method decomposes high-dimensional continuous grasping parameters into three sub-tasks: grasping quality prediction, grasping width regression, and grasping gesture classification. It constructs a soft-mask multi-color grasping region representation to generate pixel-level grasping quality, width, and gesture labels. A grasping-oriented channel-space-geometric attention mechanism is designed to construct a lightweight multi-task generative grasping detection network. Taking RGB-D images as input, it outputs grasping quality maps, width maps, and gesture maps in parallel. An adaptive weighted loss function based on effective region constraints is used for training to suppress background interference and dynamically balance multi-task learning. During inference, the grasping center is located by searching for peaks in the quality map, and the corresponding width and gesture are read to achieve single-target or multi-target grasping detection. This invention improves the accuracy, real-time performance, and robustness of grasping detection in multi-object scenes while reducing model complexity.

The invention discloses a robot double-dexterous-hand cooperative grasping method based on demonstration data, and belongs to the technical field of robot control, and the method comprises the steps: driving and obtaining multi-mode demonstration data of a robot through a man-machine interaction mode, marking a key frame and a point cloud of a target object, and constructing a structured data set; constructing and training to obtain a grasping posture generation model; inputting the point cloud feature vector of the object to be grasped into a grasping posture generation model to generate a candidate grasping posture vector; and the candidate grabbing posture vector is converted into a grabbing action executed by the robot, the grabbing posture is dynamically adjusted in real time, and cooperative grabbing of the double dexterous hands of the robot is achieved. According to the method, stable and executable double-dexterous-hand grabbing actions are generated by the robot in a real scene through multi-mode demonstration data learning, force feedback adjustment is conducted in combination with touch perception, the operation feasibility and grabbing stability of large-size objects are effectively improved, and wide application of the double dexterous hands in practical application is promoted.

The invention provides a three-dimensional object double-dexterous-hand grabbing generation method based on a deep diffusion model, relates to the technical field of robot grabbing, and automatically realizes generation of double-hand grabbing gestures facing dexterous hands by utilizing a deep learning technology. The method specifically comprises the steps of object three-dimensional point cloud input, depth VAE generation model two-hand grabbing parameter prediction, VAE feature division decoding, diffusion model feature denoising training, high feature quality two-hand grabbing gesture acquisition and the like, and can automatically generate two-hand grabbing gestures based on three-dimensional object point cloud data and a deep learning model. And the purposes of precision, intelligence and high efficiency of the grabbing process of the double dexterous hands are achieved.

The application belongs to the technical field of dexterous hand grasping model, and in particular to a reinforcement learning and multi-specialist hybrid model, and the establishment of the reinforcement learning and multi-specialist hybrid model comprises the following steps: step 1: preparation of a diversified data set: an object data set containing various morphological and geometric characteristics is constructed, and a simulation model of a dexterous hand is combined for training and evaluation; step 2: training of an expert strategy using a reward function enhanced PPO: a generalist-specialist hybrid model for dexterous hand operation based on reinforcement learning is constructed. The application realizes efficient generalization of dexterous hand grasping through a phased learning strategy; first, a basic expert strategy for dexterous operation is trained using reinforcement learning, and a high-performance expert model targeted at different objects and operation modes is obtained; subsequently, under the Generalist-Specialist Learning framework, multiple expert strategies are gradually distilled into a more compact generalist strategy; unlike the traditional direct distillation method.

The invention relates to the technical field of three-dimensional data processing, particularly provides a hand grasping posture data enhancement method and system based on a contact graph, and aims to solve the problems that errors generally exist between hand postures and posture distribution of a model in an existing data enhancement method; and the generalization ability of the model and the generation effect of the hand grasping posture are poor. In order to achieve the purpose, the method comprises the steps that a sampling data set on a target model is obtained on the basis of the data type of the target model, and the sampling data set at least comprises a plurality of sampling face data sets or a plurality of sampling point data; generating a contact graph based on the sampling data set; and generating a data enhanced contact model based on the data type of the target model and the contact graph. According to the method, the data enhancement of the sampled data is realized, the hand posture corresponding to the original data of the target model is ensured not to have any mismatch, and the data enhancement of the model in the grasping posture data is realized by utilizing the contact graph technology design.

The invention discloses a dexterous hand grabbing posture generation method based on object shapes. Firstly, a grabbing posture data set is constructed; then, constructing a de-noising diffusion network, and introducing a grabbing posture prompt text to train the de-noising diffusion network; the grabbing posture prompt text at least comprises an article type, a shape and grabbing posture description, and inputting the grabbing posture prompt text into the large language model to obtain grabbing posture text description; performing text coding on the grabbed posture text description to obtain grabbed posture text features; cascading the grabbed posture text features with the object point cloud features to obtain point cloud text fusion features; inputting the point cloud text fusion feature and the noise capture posture into a denoising diffusion network, and predicting noise; and finally, inputting the object point cloud features, the noise grabbing posture and the random time step into the trained de-noising diffusion network to generate a final grabbing posture. The grabbing posture better conforms to the shape of an object, and the grabbing stability and success rate of the dexterous hand are improved.

1. The name of the design product: hand grab (intelligent robot). 2. The use of the design product: the design product is used for the hand grab of intelligent robot. 3. The design points of the design product: in shape. 4. The picture or photo that best indicates the design points: perspective view.

In some embodiments, a computing system receives a representation of an object from a client device. The computing system generates a contact representation for hand-object interaction based on the representation of the object. The object-centric contact representation includes a contact map indicating contact points on the representation of the object, a hand part map indicating hand parts contacting the object, and a direction map comprising contact directions of the hand parts contacting the object. The computing system generates a hand grasp representation with respect to the object based on the contact representation using a model-based optimization algorithm. The computing system provides the hand grasp representation to the client device.

The present invention discloses a hand grasping force evaluation device, a motion-mechanics coupling evaluation system and method. Without changing the physical state of hand grasping, the present invention detects the hand grasping force, restoring the real grasping conditions and process; meanwhile, the hand grasping force evaluation device allows the setting of grasping conditions including object shape, roughness and weight on the outer surface of the target object to be grasped, enriching the scope of grasping tests and evaluations; for hand grasping evaluation, feature evaluations under motion-mechanics coupling conditions are proposed, specifically including a hand motion trajectory area-time coupling index and a hand grasping pressure-time coupling index, which can more accurately represent the fluency and smoothness of the target hand motion under the task of hand grasping actions; the present invention can be applied to the task evaluation of hand grasping actions of both natural hands and prosthetic hands, and is particularly applicable to the task evaluation of hand grasping actions of prosthetic hands with tactile feedback.

The invention discloses a dexterous grabbing method based on a conditional diffusion model, and belongs to the field of dexterous hand grabbing in robotics. According to the method, a diffusion mechanism for directly generating the grabbing posture in the native joint space of the dexterous hand is designed, and a hand sensing module and a multi-physical constraint loss function (penetration, self-collision and force sealing) are introduced, so that the posture is corrected in real time in the denoising generation process, and the hand is effectively prevented from penetrating through the mold. And finally, the generative model is combined with an adaptive module during three-stage testing based on a numerical method, and rapid screening, local fine tuning and fine trimming and high-precision physical evaluation are performed on candidate grabbing postures, so that efficient, safe and physically feasible high-quality flexible grabbing is realized.

The invention provides a robot control method and device and a robot, and is applied to the technical field of robots. Performing target detection on the visual image to obtain a candidate image area, and screening masks in the candidate image area; determining three-dimensional observation information in the candidate image area based on the screened mask, and aligning the three-dimensional observation information with a CAD model matched with the target object to obtain pose information of the target object; according to the pose information and the identifier of the CAD model, determining a hand grabbing pose of the robot from a preset hand pose database; according to the method, the robot is subjected to hand posture control according to the hand grabbing posture, object posture estimation is conducted by combining the three-dimensional observation information and the CAD model, then the hand grabbing posture is determined, hand posture control is conducted on the robot, the precision and efficiency of the robot for executing body grabbing are improved, and then the task execution effect is improved.

The invention relates to a closed-loop brain-computer interface rehabilitation training method and device, equipment and a storage medium. The method comprises the steps that the hand fine movement ability of a trainer is evaluated, and a training target strength interval value is set according to an evaluation result; the method comprises the following steps: collecting an electroencephalogram signal when a trainee grasps a hand in an actual motion / motion imagination, and collecting an electromyographic signal when the trainee grasps the hand through an electromyographic signal collection module; acquiring a motion intention strength value of the trainer as an actual strength value; the difference value between the actual force value and the training target force value is calculated, and a control instruction is generated according to the difference value to control rehabilitation training equipment; and providing tactile feedback force as large as the difference value for the trainer through rehabilitation training equipment, so that the trainer adjusts the motor imagery / actual grasping force and generates training actions for rehabilitation training. According to the method, the brain-computer interface technology and the 3D game are fused, so that the interactivity and interestingness of trainees are enhanced, and the training enthusiasm and initiative are improved.

Images from a camera (208) on, for example, a VR HMD (200) of a hand grasping a computer game controller (212) are identified. The images can be cropped (306) to a region containing the controller and the hand to simplify processing, and a virtual representation of the hand is presented on a display (202) such as the HMD's display, where the virtual representation is generated based on both image analysis of the region and touch signals (302) from the controller.

The application provides a dexterous hand grasping action control method, electronic equipment and storage medium, and belongs to the technical field of robot dexterous hands. The method comprises the following steps: receiving a grasping task, and acquiring image information of a target object in the grasping task; determining a target diameter corresponding to the target object by analyzing the image information; comparing the target diameter with a plurality of preset grasping diameters of the dexterous hand, and determining a target grasping mode according to a comparison result, wherein the target grasping mode comprises a fingertip grasping mode, a fingertip and second joint cooperative grasping mode and a full hand grasping mode; and controlling the dexterous hand to execute the grasping task in the target grasping mode, wherein in the grasping process, three-dimensional force data collected by three-dimensional force sensors arranged at different positions of the dexterous hand is monitored to determine a real-time grasping state, and a corresponding grasping adjustment strategy is executed according to the real-time grasping state. The application can improve the grasping success rate and operation adaptability of the dexterous hand when facing objects of different sizes.

The invention provides a grabbing control method, system and device for a dexterous hand, and the method comprises the steps: controlling the dexterous hand to move to a target object, and obtaining the pressure information between the dexterous hand and the target object; acquiring forward pressure data based on the pressure information; the forward pressure data is compared with preset grabbing force data, and the stress change state of the dexterous hand when the target object is grabbed is determined; the stress change state of the dexterous hand comprises a deformation state, an offset state and a sliding state; and a control instruction is generated based on the stress change state of the dexterous hand, and the grabbing force of the dexterous hand on the target object is adjusted according to the control instruction. The pressure sensing precision and the response speed can be improved, and the dexterous hand can more accurately sense the contact force and make corresponding adjustment.

The invention discloses a MediaPipe-based hand grabbing motion key frame extraction system and method, and belongs to the technical field of computer vision and robots, and the method comprises the steps: obtaining a hand motion track through a MediaPipe hand key point detector according to the analysis of the characteristics of hand grabbing motion, constructing a hand motion speed time sequence curve, and obtaining a key frame of the hand grabbing motion; gaussian filtering and a local extremum detection strategy are used for identifying, capturing and placing speed trough frames related to operation, and robustness is improved through a dual-threshold filtering mechanism. According to the method, the defect that key operation frames are easy to lose due to uniform sampling is overcome, a visual context with higher information density is provided for a visual language model, and the accuracy of instruction generation in mechanical arm imitation learning is remarkably improved.

The invention discloses a two-degree-of-freedom manipulator for grabbing and in-palm fine operation, which comprises a simulation finger, a movable platform and a static platform, the height of the movable platform can be adjusted, a main gear and an auxiliary gear which are meshed with each other are arranged in the movable platform, and a first rack and a second rack are respectively meshed with two sides of the main gear; the number of the simulation fingers is at least three, one simulation finger is arranged on the top side of the first rack, the other two simulation fingers are arranged on the top side of the second rack, each simulation finger comprises an upper finger, a lower finger and a connecting rod, and the lower finger and the connecting rod are rotationally connected to the bottom of the upper finger. Each connecting rod is rotationally connected with the upper end face of the rack on the same side. The mechanical hand is a novel flexible mechanical hand capable of achieving grabbing and in-palm operation, can be composed of at least three fingers, simulates a thumb, an index finger and a middle finger respectively, has two degrees of freedom, and can achieve most of hand grabbing actions such as enveloping, pinching and opening and in-palm operation of screwing and twisting without wrist compensation.

The invention discloses a robot dexterous operation method and system based on visual key point guidance, and the method comprises the steps: S1, building a base coordinate system, solving an initial key point according to the base coordinate system, positioning a dexterous hand based on the initial key point, and solving the posture of the tail end of a mechanical arm when a tool is grabbed; s2, the dexterous hand grabs the tool through a pre-established coarse gesture library according to the posture of the tail end of the mechanical arm when the tool is grabbed; s3, using the dexterous hand to move the tool, identifying new key points after displacement again, and calculating a transformation matrix between the tool head coordinate system and the base coordinate system and a transformation matrix between the target object coordinate system and the base coordinate system according to the new key points; and S4, calculating the tail end position of the operation of the tool head on the target object until the dexterous hand holds the tool to complete the operation on the target object. According to the method, the interaction process of the dexterous hand and the target object is converted into alignment of the object and the key point of the hand, the rule when the dexterous hand grabs a tool is met, and extremely high generalization is achieved.

The present application belongs to the field of point cloud processing, and relates to a single-hand saliency map and a few double-hand annotation guided double-hand saliency analysis method. The present application first generates a double-hand saliency map by using a single-hand grasping saliency map and single-double-hand grasping correlation. First, the point cloud of an object is input into a single-hand saliency map prediction model to obtain a single-hand saliency map, which is used as an initial value of a double-hand saliency map. Second, a double-hand grasping contact point dataset is constructed, and a corresponding relationship between left and right hand grasping annotation points and saliency is established. Based on the corresponding relationship and the initial value, a double-hand grasping saliency map learning network named BSPN is designed to generate an object double-hand saliency map. An iterative training strategy is further proposed to update the initial value of the double-hand saliency map. Finally, the point cloud is input into a contact point prediction network trained with the assistance of BSPN to obtain double-hand grasping contact points. The present application is evaluated by using an existing grasping saliency dataset, and a human-like double-hand grasping gesture is successfully generated.

A system includes: a hand module to, based on a demonstration of a human hand grasping an object, determine first and second vectors that are normal to and parallel to a palm of the human hand, respectively, and a position of the human hand; a gripper module to determine third and fourth vectors that are normal to and parallel to a palm of a gripper of a robot, respectively, and a present position of the gripper; and an actuation module to: move the gripper when open such that the present position of the gripper is at the position of the human hand, the third and first vectors are aligned, and the fourth and second vectors are aligned; close fingers of the gripper based on minimizing a first loss; and actuate the fingers of the gripper to minimize a second loss determined based on the first loss and a third loss.