25 results about "Hand grasp" patented technology

The invention discloses a dexterous hand perception control method and device, equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the steps that under the condition that a target multi-fingered dexterous hand grabs a target object is recognized through a pressure sensor array, pressure collection data are obtained from a pressure sensor array, and current joint data of all finger joints of the target multi-fingered dexterous hand are obtained from an encoder; according to the pressure collection data and the current joint data, the current grabbing state of the target multi-fingered dexterous hand is determined; when it is recognized that the current grabbing state does not meet the safe grabbing condition, dexterous hand pose data of the target multi-fingered dexterous hand are obtained from an inertial sensor, and a target grabbing mode of the target multi-fingered dexterous hand is determined according to the current space pose data, the pressure collection data, the current joint data and the dexterous hand pose data; and the target multi-fingered dexterous hand is controlled to grab in a target grabbing mode. According to the scheme, the sensing grabbing efficiency and accuracy of the dexterous hand are improved.

The invention discloses an intelligent learning method for grabbing of a three-finger dexterous hand for multiple types of workpieces, and belongs to the technical field of intelligent grabbing application of manipulators. A pre-defined rule and simulation expert double-layer mode is provided, a high-quality demonstration data set is automatically generated, a simulation grabbing scene of a mechanical arm and a three-fingered dexterous hand is constructed, recognition and classification of multiple types of workpieces are achieved by analyzing workpiece contour features, grabbing hand shapes are matched according to the recognition and classification, strategy pre-training is carried out through simulation learning, and the recognition and classification efficiency is improved. And optimizing the strategy based on a PPO algorithm and a composite reward function. Aiming at the problem of lack of a standardized three-fingered dexterous hand parameter model, a modeling method based on parameterized URDF and automatic script generation is used to realize efficient configuration and optimization of the three-fingered dexterous hand in simulation. Through performance evaluation, stable and high-quality grabbing of multiple types of workpieces can be achieved.

The invention belongs to the technical field of robots, and particularly relates to a dexterous grabbing planning method based on conditional diffusion generation and accessibility perception, which comprises the following steps of: initializing a grabbing posture, and acquiring a multi-fingered dexterous hand grabbing simulation data set, the simulation data set comprises scene point cloud data, stably grabbed palm posture data and finger joint angle data; the multi-fingered dexterous hand grabbing simulation data set is input into a conditional diffusion generation model, a complete grabbing posture is obtained, the conditional diffusion generation model is used for extracting local features in a scene point cloud, and the complete grabbing posture is obtained according to the local features; according to the reachability graph of the mechanical arm, training a stacking auto-encoder, and obtaining a grabbing reachability evaluator; the complete grabbing posture is input into the grabbing accessibility evaluator, and the accessibility probability is obtained; and on the basis of the reachable probability, in combination with the grabbing stability probability, a target grabbing posture is obtained, and grabbing is conducted according to the target grabbing posture.

The invention discloses an under-driven dexterous hand grabbing pose generation method based on deep learning, and belongs to the technical field of robot operation and intelligent grabbing. According to the method, the prior knowledge of a large language model LLM to real capture is introduced into a data set, and a reasonable capture pose is realized by using object semantic-geometric joint features of a point cloud encoder Point Transform and the LLM. According to the method, three types of grabbing strategy classification models for the under-actuated five-finger dexterous hand are constructed, including two-finger kneading, three-finger kneading and five-finger grabbing, and are used for restraining and guiding generation of subsequent fine grabbing postures. According to the selected grabbing strategy, a network is generated to predict six-dimensional poses and angles of all joints of the dexterous hand, and a grabbing posture matched with the shape of the object is obtained. According to the method, two constraint loss items are additionally added to realize grabbing quality optimization. The method can be suitable for intelligent grabbing tasks of industrial robots, service robots and precision operation equipment under the condition that negative grabbing of samples and complex post-processing optimization are not needed.

This invention discloses a method and system for generating grasping postures of a multi-fingered dexterous hand based on CVAE and Ball Query algorithms, belonging to the field of robot grasping control technology. The method includes: (1) a data sampling step; (2) a data preprocessing step: standardizing the collected data to generate enhanced point cloud data and constructing a training dataset; (3) a model training step: using a multi-scale feature extraction module combined with global semantics and local geometric features extracted by the Ball Query algorithm, generating grasping postures using a conditional variational autoencoder, and optimizing model parameters through reconstruction loss and KL divergence; (4) a real-time deployment step: integrating the trained model into a physical platform, filtering candidate grasping postures based on parallel collision detection, and realizing real-time grasping control by combining inverse kinematics verification. This invention effectively solves the problem of insufficient generalization ability of traditional methods in grasping complex objects, and achieves adaptive grasping of unknown objects while ensuring grasping stability.

A method and system for dexterous robot manipulation based on visual keypoint guidance is disclosed. The method includes: S1, establishing a base coordinate system, solving for initial keypoints based on the base coordinate system, and locating the dexterous hand based on the initial keypoints to determine the end-effector posture when grasping a tool; S2, the dexterous hand grasps the tool using a pre-established coarse gesture library and based on the end-effector posture when grasping the tool; S3, using the dexterous hand to move the tool, identifying new keypoints after the shift, and calculating the transformation matrix between the tool head coordinate system and the base coordinate system, as well as the transformation matrix between the target object coordinate system and the base coordinate system, based on the new keypoints; S4, calculating the end-effector position of the tool head in operation on the target object, until the dexterous hand completes the operation on the target object. This invention transforms the interaction process between the dexterous hand and the target object into the alignment of keypoints between the object and the hand, conforming to the rules of dexterous hand grasping tools and exhibiting strong generalization ability.

This invention discloses a five-finger dexterity hand grasping detection method based on soft-mask region representation and multi-task learning. This method decomposes high-dimensional continuous grasping parameters into three sub-tasks: grasping quality prediction, grasping width regression, and grasping gesture classification. It constructs a soft-mask multi-color grasping region representation to generate pixel-level grasping quality, width, and gesture labels. A grasping-oriented channel-space-geometric attention mechanism is designed to construct a lightweight multi-task generative grasping detection network. Taking RGB-D images as input, it outputs grasping quality maps, width maps, and gesture maps in parallel. An adaptive weighted loss function based on effective region constraints is used for training to suppress background interference and dynamically balance multi-task learning. During inference, the grasping center is located by searching for peaks in the quality map, and the corresponding width and gesture are read to achieve single-target or multi-target grasping detection. This invention improves the accuracy, real-time performance, and robustness of grasping detection in multi-object scenes while reducing model complexity.

The invention discloses a robot double-dexterous-hand cooperative grasping method based on demonstration data, and belongs to the technical field of robot control, and the method comprises the steps: driving and obtaining multi-mode demonstration data of a robot through a man-machine interaction mode, marking a key frame and a point cloud of a target object, and constructing a structured data set; constructing and training to obtain a grasping posture generation model; inputting the point cloud feature vector of the object to be grasped into a grasping posture generation model to generate a candidate grasping posture vector; and the candidate grabbing posture vector is converted into a grabbing action executed by the robot, the grabbing posture is dynamically adjusted in real time, and cooperative grabbing of the double dexterous hands of the robot is achieved. According to the method, stable and executable double-dexterous-hand grabbing actions are generated by the robot in a real scene through multi-mode demonstration data learning, force feedback adjustment is conducted in combination with touch perception, the operation feasibility and grabbing stability of large-size objects are effectively improved, and wide application of the double dexterous hands in practical application is promoted.

The invention provides a three-dimensional object double-dexterous-hand grabbing generation method based on a deep diffusion model, relates to the technical field of robot grabbing, and automatically realizes generation of double-hand grabbing gestures facing dexterous hands by utilizing a deep learning technology. The method specifically comprises the steps of object three-dimensional point cloud input, depth VAE generation model two-hand grabbing parameter prediction, VAE feature division decoding, diffusion model feature denoising training, high feature quality two-hand grabbing gesture acquisition and the like, and can automatically generate two-hand grabbing gestures based on three-dimensional object point cloud data and a deep learning model. And the purposes of precision, intelligence and high efficiency of the grabbing process of the double dexterous hands are achieved.

The application belongs to the technical field of dexterous hand grasping model, and in particular to a reinforcement learning and multi-specialist hybrid model, and the establishment of the reinforcement learning and multi-specialist hybrid model comprises the following steps: step 1: preparation of a diversified data set: an object data set containing various morphological and geometric characteristics is constructed, and a simulation model of a dexterous hand is combined for training and evaluation; step 2: training of an expert strategy using a reward function enhanced PPO: a generalist-specialist hybrid model for dexterous hand operation based on reinforcement learning is constructed. The application realizes efficient generalization of dexterous hand grasping through a phased learning strategy; first, a basic expert strategy for dexterous operation is trained using reinforcement learning, and a high-performance expert model targeted at different objects and operation modes is obtained; subsequently, under the Generalist-Specialist Learning framework, multiple expert strategies are gradually distilled into a more compact generalist strategy; unlike the traditional direct distillation method.

The invention relates to the technical field of three-dimensional data processing, particularly provides a hand grasping posture data enhancement method and system based on a contact graph, and aims to solve the problems that errors generally exist between hand postures and posture distribution of a model in an existing data enhancement method; and the generalization ability of the model and the generation effect of the hand grasping posture are poor. In order to achieve the purpose, the method comprises the steps that a sampling data set on a target model is obtained on the basis of the data type of the target model, and the sampling data set at least comprises a plurality of sampling face data sets or a plurality of sampling point data; generating a contact graph based on the sampling data set; and generating a data enhanced contact model based on the data type of the target model and the contact graph. According to the method, the data enhancement of the sampled data is realized, the hand posture corresponding to the original data of the target model is ensured not to have any mismatch, and the data enhancement of the model in the grasping posture data is realized by utilizing the contact graph technology design.

1. The name of the design product: hand grab (intelligent robot). 2. The use of the design product: the design product is used for the hand grab of intelligent robot. 3. The design points of the design product: in shape. 4. The picture or photo that best indicates the design points: perspective view.

The invention discloses a dexterous grabbing method based on a conditional diffusion model, and belongs to the field of dexterous hand grabbing in robotics. According to the method, a diffusion mechanism for directly generating the grabbing posture in the native joint space of the dexterous hand is designed, and a hand sensing module and a multi-physical constraint loss function (penetration, self-collision and force sealing) are introduced, so that the posture is corrected in real time in the denoising generation process, and the hand is effectively prevented from penetrating through the mold. And finally, the generative model is combined with an adaptive module during three-stage testing based on a numerical method, and rapid screening, local fine tuning and fine trimming and high-precision physical evaluation are performed on candidate grabbing postures, so that efficient, safe and physically feasible high-quality flexible grabbing is realized.

The invention provides a robot control method and device and a robot, and is applied to the technical field of robots. Performing target detection on the visual image to obtain a candidate image area, and screening masks in the candidate image area; determining three-dimensional observation information in the candidate image area based on the screened mask, and aligning the three-dimensional observation information with a CAD model matched with the target object to obtain pose information of the target object; according to the pose information and the identifier of the CAD model, determining a hand grabbing pose of the robot from a preset hand pose database; according to the method, the robot is subjected to hand posture control according to the hand grabbing posture, object posture estimation is conducted by combining the three-dimensional observation information and the CAD model, then the hand grabbing posture is determined, hand posture control is conducted on the robot, the precision and efficiency of the robot for executing body grabbing are improved, and then the task execution effect is improved.

The invention relates to a closed-loop brain-computer interface rehabilitation training method and device, equipment and a storage medium. The method comprises the steps that the hand fine movement ability of a trainer is evaluated, and a training target strength interval value is set according to an evaluation result; the method comprises the following steps: collecting an electroencephalogram signal when a trainee grasps a hand in an actual motion / motion imagination, and collecting an electromyographic signal when the trainee grasps the hand through an electromyographic signal collection module; acquiring a motion intention strength value of the trainer as an actual strength value; the difference value between the actual force value and the training target force value is calculated, and a control instruction is generated according to the difference value to control rehabilitation training equipment; and providing tactile feedback force as large as the difference value for the trainer through rehabilitation training equipment, so that the trainer adjusts the motor imagery / actual grasping force and generates training actions for rehabilitation training. According to the method, the brain-computer interface technology and the 3D game are fused, so that the interactivity and interestingness of trainees are enhanced, and the training enthusiasm and initiative are improved.

Images from a camera (208) on, for example, a VR HMD (200) of a hand grasping a computer game controller (212) are identified. The images can be cropped (306) to a region containing the controller and the hand to simplify processing, and a virtual representation of the hand is presented on a display (202) such as the HMD's display, where the virtual representation is generated based on both image analysis of the region and touch signals (302) from the controller.

The application provides a dexterous hand grasping action control method, electronic equipment and storage medium, and belongs to the technical field of robot dexterous hands. The method comprises the following steps: receiving a grasping task, and acquiring image information of a target object in the grasping task; determining a target diameter corresponding to the target object by analyzing the image information; comparing the target diameter with a plurality of preset grasping diameters of the dexterous hand, and determining a target grasping mode according to a comparison result, wherein the target grasping mode comprises a fingertip grasping mode, a fingertip and second joint cooperative grasping mode and a full hand grasping mode; and controlling the dexterous hand to execute the grasping task in the target grasping mode, wherein in the grasping process, three-dimensional force data collected by three-dimensional force sensors arranged at different positions of the dexterous hand is monitored to determine a real-time grasping state, and a corresponding grasping adjustment strategy is executed according to the real-time grasping state. The application can improve the grasping success rate and operation adaptability of the dexterous hand when facing objects of different sizes.

The invention provides a grabbing control method, system and device for a dexterous hand, and the method comprises the steps: controlling the dexterous hand to move to a target object, and obtaining the pressure information between the dexterous hand and the target object; acquiring forward pressure data based on the pressure information; the forward pressure data is compared with preset grabbing force data, and the stress change state of the dexterous hand when the target object is grabbed is determined; the stress change state of the dexterous hand comprises a deformation state, an offset state and a sliding state; and a control instruction is generated based on the stress change state of the dexterous hand, and the grabbing force of the dexterous hand on the target object is adjusted according to the control instruction. The pressure sensing precision and the response speed can be improved, and the dexterous hand can more accurately sense the contact force and make corresponding adjustment.

The invention discloses a MediaPipe-based hand grabbing motion key frame extraction system and method, and belongs to the technical field of computer vision and robots, and the method comprises the steps: obtaining a hand motion track through a MediaPipe hand key point detector according to the analysis of the characteristics of hand grabbing motion, constructing a hand motion speed time sequence curve, and obtaining a key frame of the hand grabbing motion; gaussian filtering and a local extremum detection strategy are used for identifying, capturing and placing speed trough frames related to operation, and robustness is improved through a dual-threshold filtering mechanism. According to the method, the defect that key operation frames are easy to lose due to uniform sampling is overcome, a visual context with higher information density is provided for a visual language model, and the accuracy of instruction generation in mechanical arm imitation learning is remarkably improved.

The present application belongs to the field of point cloud processing, and relates to a single-hand saliency map and a few double-hand annotation guided double-hand saliency analysis method. The present application first generates a double-hand saliency map by using a single-hand grasping saliency map and single-double-hand grasping correlation. First, the point cloud of an object is input into a single-hand saliency map prediction model to obtain a single-hand saliency map, which is used as an initial value of a double-hand saliency map. Second, a double-hand grasping contact point dataset is constructed, and a corresponding relationship between left and right hand grasping annotation points and saliency is established. Based on the corresponding relationship and the initial value, a double-hand grasping saliency map learning network named BSPN is designed to generate an object double-hand saliency map. An iterative training strategy is further proposed to update the initial value of the double-hand saliency map. Finally, the point cloud is input into a contact point prediction network trained with the assistance of BSPN to obtain double-hand grasping contact points. The present application is evaluated by using an existing grasping saliency dataset, and a human-like double-hand grasping gesture is successfully generated.

A system includes: a hand module to, based on a demonstration of a human hand grasping an object, determine first and second vectors that are normal to and parallel to a palm of the human hand, respectively, and a position of the human hand; a gripper module to determine third and fourth vectors that are normal to and parallel to a palm of a gripper of a robot, respectively, and a present position of the gripper; and an actuation module to: move the gripper when open such that the present position of the gripper is at the position of the human hand, the third and first vectors are aligned, and the fourth and second vectors are aligned; close fingers of the gripper based on minimizing a first loss; and actuate the fingers of the gripper to minimize a second loss determined based on the first loss and a third loss.

The application discloses a five-fingered dexterous hand grabbing information generation method based on a self-expanding database, and belongs to the technical field of robot five-fingered dexterous hand grabbing. The method comprises the following steps: constructing an automatic gesture recognition model, an automatic region division model and a grabbing posture recognition model; training the automatic gesture recognition model; training the automatic region division model according to a grabbing task to be performed by a five-fingered dexterous hand and a grabbed object; based on the three models, constructing a grabbing information database and performing self-expansion; and based on the grabbing information database, entering an online state to find grabbing information. Through the application, the three models can quickly recognize grabbing gestures, grabbing regions and grabbing postures in a picture data set, and store the same in a grabbing information database in a complete operation instruction unit; when facing a new grabbing task and a grabbed object, grabbing information can be efficiently and accurately found, and a large amount of picture labeling and training work is avoided.

The present application relates to the technical field of robot learning, and provides a five-fingered hand grasping control method and system with teaching learning capability. The method comprises the following steps: under a three-dimensional motion capture system, a world coordinate system is established; under the condition of wearing a data glove, hand posture information data when a hand grips different objects is collected; based on the hand posture information data, a mapping relationship between the hand and the five-fingered hand is established; based on the hand posture information data, a generative adversarial network is used to analyze the hand posture, a coordination matrix is constructed, and a pre-grasping posture is generated according to different grasped objects; through the motion capture system, the object to be gripped is identified, the closest preset posture is matched for the current pre-grasping posture of the five-fingered hand based on the mapping relationship between the hand and the five-fingered hand, and a quadratic programming-based inverse dynamics control method is used to control the five-fingered hand to grip.

The invention particularly provides a parameterized hand posture data collection method based on virtual reality. The method comprises the following steps: acquiring a grasping posture original data set, and generating an original hand model based on the grasping posture original data set; dynamically optimizing joint point positions of the original hand model to obtain a joint point remapped initial hand model; substituting a to-be-grabbed target into the preliminary hand model to obtain a grabbed hand model, and performing collision optimization on the grabbed hand model to obtain a hand posture model after the grabbed posture is optimized; and substituting the hand posture model after the grasping posture optimization into a joint point regression network module to obtain hand posture combination parameters, and generating a hand grasping data set based on the hand posture combination parameters. Precise and real-time processing of hand data mapping according to virtual reality is realized, the accuracy of the hand grasping data set is improved, and the data coverage of the hand grasping data is perfected.

1. Name of the design product: children's stroller toy. 2. Use of the design product: mainly used to exercise children's hand grasping ability, crawling ability, and cultivate children's spatial cognitive ability. 3. Design points of the design product: in shape. 4. Picture or photo that best indicates the design points: perspective view.