A robot grasping control method

Abstract

This invention discloses a robot grasping control method, comprising: acquiring a training dataset, the training dataset containing tactile data collected during the grasping process of the robot's multi-fingered dexterous hand, the finger joint angles of the robot's multi-fingered dexterous hand, the grasping pattern of the robot's multi-fingered dexterous hand, and pre-labeled stable grasping discrimination information; training an initial stable grasping discrimination model using the tactile data, grasping pattern, and stable grasping discrimination information in the training dataset to obtain a target stable grasping discrimination model; training an initial grasping adjustment model using the tactile data, finger joint angles, and grasping pattern in the training dataset to obtain a target grasping adjustment model; generating a grasping controller based on the target stable grasping discrimination model and the target grasping adjustment model; and inputting the tactile data, the finger joint angles of the robot's multi-fingered dexterous hand, and the grasping pattern into the grasping controller to achieve grasping control of the robot.

Description

[Technical Field]

[0001] This invention relates to a robot grasping control method, belonging to the field of robot grasping. [Background Technology]

[0002] Traditional robotic grasping methods often rely on predefined grasping paradigms and cannot adaptively adjust to target attributes, making precise and flexible grasping difficult. Some methods apply deep learning techniques to robotic grasping, aiming to improve robot performance in unstructured scenarios. However, existing methods mainly focus on two-finger robotic grasping, lacking the flexibility and versatility of human hand grasping and unable to perform fine grasping. Because robotic multi-finger dexterous hands have more fingers, by controlling and coordinating the movements of multiple fingers, robots can perform more complex, precise, and diverse operations.

[0003] Existing robot grasping control methods primarily rely on visual perception to detect the grasping point. However, depending solely on the grasping point to control a robot's multi-fingered dexterous hand lacks precise control over finger angles, easily leading to insufficient stability and accuracy during the grasping process. Furthermore, these methods cannot utilize contact information during grasping to determine the current grasping state and make timely adjustments. Moreover, existing methods typically only consider a single grasping mode, failing to fully leverage the diverse grasping modes of a robot's multi-fingered dexterous hand. Therefore, proposing a robot grasping control method is of significant research importance. [Summary of the Invention]

[0004] In view of this, the present invention proposes a robot grasping control method for robot grasping operations.

[0005] This invention provides a robot grasping control method, comprising:

[0006] Obtain a training dataset, which includes tactile data collected during the robot's multi-finger dexterous hand grasping process, finger joint angles of the robot's multi-finger dexterous hand, grasping patterns of the robot's multi-finger dexterous hand, and pre-labeled stable grasping discrimination information.

[0007] Using the tactile data, the grasping pattern, and the stable grasping discrimination information in the training dataset, the initial stable grasping discrimination model is trained to obtain the target stable grasping discrimination model;

[0008] Using the tactile data, finger joint angles, and grasping patterns in the training dataset, the initial grasping adjustment model is trained to obtain the target grasping adjustment model;

[0009] A grasping controller is generated based on the target stable grasping discrimination model and the target grasping adjustment model;

[0010] The tactile data, the finger joint angles of the robot's multi-fingered dexterous hand, and the grasping pattern are input into the grasping controller to achieve grasping control of the robot.

[0011] In the above method, obtaining the training dataset includes:

[0012] Use tactile sensors to collect tactile data during the robot's multi-fingered dexterous hand grasping process;

[0013] The robot operating system platform is used to obtain the finger joint angles and grasping patterns during the robot's multi-finger dexterity grasping process; the grasping patterns include three modes: basic grasping, wide-finger grasping, and syn-finger grasping.

[0014] In the above method, obtaining the training dataset includes:

[0015] Select several objects of different shapes and hardnesses. For each object, at the beginning of each data collection cycle, place the object in a different orientation at the center of a horizontal plane and control the robot to perform the following operations:

[0016] When an object is placed in a designated position and the robot reaches a designated height above the object, the robot's multi-fingered dexterous hand performs a grasping operation on the object.

[0017] After the robot successfully grasps the object, it rises vertically to a specified height. During the robot's lifting process, tactile sensors are used to collect tactile data, and the robot's operating system platform is used to collect the finger joint angles and grasping patterns of the robot's multi-fingered dexterous hand.

[0018] After the robot reaches the designated posture, it remains still for a specified time before returning to the initial position, where it releases the object.

[0019] In the above method, the step of training an initial stable grasping discrimination model using the tactile data, the grasping pattern, and the stable grasping discrimination information in the training dataset to obtain a target stable grasping discrimination model includes...

[0020] The tactile data is input into the GCN feature extraction sub-model of the initial stable grasping discrimination model to obtain the graph feature information of the tactile data;

[0021] The grasping pattern of the robot's multi-fingered dexterous hand is encoded as One-Hot vector information;

[0022] The graphical feature information and One-Hot vector information of the tactile data are input into the feature fusion unit of the initial stable grasping discrimination model to obtain stable grasping discrimination information;

[0023] Based on the stable grasping discrimination information and the pre-labeled stable grasping discrimination information, it is determined whether the initial stable grasping discrimination model has reached the preset convergence condition. If yes, the current initial stable grasping discrimination model is used as the target stable grasping discrimination model; if no, the above training process continues until the initial stable grasping discrimination model reaches the preset convergence condition.

[0024] In the above method, the GCN feature extraction sub-model includes: multiple tactile array sensors;

[0025] Each tactile array sensor consists of 16 tactile units, and each tactile unit is composed of a three-dimensional force vector. The basic matrix of a single joint tactile sensing unit is... Construct a static haptic graph G = (V, E) with fixed edge connections, where V represents each haptic unit of the haptic array sensor, and E represents the edge connections between haptic units. Define the GCN feature extraction sub-model as f, and the stability classification of grasping stability is represented as c ∈ (0, 1), where the values ​​0 and 1 represent the stable state and the sliding state, respectively. For the predicted value, the GCN feature extraction sub-module learns the mapping function f through the static haptic map G, the fundamental matrix F, and the stability classification, as follows:

[0026]

[0027] Multi-scale convolutional layers are used to extract feature information, as shown below:

[0028]

[0029] in, W is the feature vector of haptic unit v in the (l+1)th layer, N(v) is the set of neighboring haptic units of haptic unit v, and W is the feature vector of haptic unit v in the (l+1)th layer. (l) Let be the learnable weight matrix of the l-th layer. Let c be the feature vector of the haptic unit u in the l-th layer. vu The normalization constant is σ is the ReLU activation function, m represents the m-th convolutional layer, and n represents the number of convolutional layers.

[0030] In the above method, the feature fusion unit of the initial stable grasping discrimination model includes:

[0031] The feature vectors extracted by the GCN feature extraction sub-model are concatenated with the One-Hot vector information, and the concatenated result is input into the fully connected layer, as shown below:

[0032]

[0033] Z = contact(O1, O2, ,O3,m)

[0034] Among them, H (l) This represents the haptic unit feature matrix of the l-th layer of haptic units. This represents the sum of the adjacency matrix A and the identity matrix I. express The degree matrix, σ is the ReLU activation function, W (l) This represents the haptic unit weight matrix of the l-th layer of haptic units, O i represents the graph feature information of a single joint, m represents the One-Hot vector information of the grasping mode, and Z represents the feature vector obtained after concatenation.

[0035] In the above method, the pre-labeled stable grasping discrimination information includes the tactile data and the robot grasping state corresponding to the robot's multi-finger dexterous hand grasping mode. The robot grasping state includes a stable state and a sliding state.

[0036] In the above method, the step of training an initial grasping adjustment model using the tactile data, the finger joint angles, and the grasping pattern from the training dataset to obtain a target grasping adjustment model includes:

[0037] The graphical feature information of the tactile data and the grasping pattern of the robot's multi-fingered dexterous hand are input into the initial grasping adjustment model based on MLP to obtain the predicted finger joint angles of the robot's multi-fingered dexterous hand.

[0038] Based on the deviation between the predicted finger joint angles of the robot's multi-fingered dexterous hand and the actual finger joint angles of the robot's multi-fingered dexterous hand, it is determined whether the initial grasping adjustment model has reached the preset convergence condition. If yes, the current initial grasping adjustment model is used as the target grasping adjustment model; if no, the above training process continues until the initial stable grasping adjustment model reaches the preset convergence condition.

[0039] The initial crawling and adjustment model in the above method includes:

[0040] Establish an MLP-based model K s The graph feature information G, which maps the tactile data of each joint of the robot's multi-fingered dexterous hand, is used to map the movement distance J of each joint of the robot's multi-fingered dexterous hand.

[0041] J = K s (G,m;W,b)

[0042] Where J = [θ1, θ2, θ3] T It is the joint angle vector, G = [G1, G2, G3] THere, m represents the image feature information of each finger, w represents the grasping mode, and b represents the weight matrix and bias vector of the model, respectively.

[0043] Define a loss function L to evaluate the deviation value, as follows:

[0044]

[0045] In the above method, the step of inputting tactile data, the finger joint angles of the robot's multi-fingered dexterous hand, and the grasping pattern into the grasping controller to achieve grasping control of the robot includes...

[0046] The grasping control of the grasping controller is divided into a position control stage and a grasping adjustment stage. The transition between the two stages is achieved through a piecewise linear function of the vector norm of the joint angles of the robot's multi-fingered dexterous hand, as shown below:

[0047]

[0048] Where J represents the current finger angle of the multi-finger hand. The finger angle is represented within the grasp parameters, and state represents the result of the target stable grasp discrimination model, with 0 indicating a stable state and 1 indicating a sliding state.

[0049] Position control phase: Given a predefined target multi-fingered dexterous hand joint angle, the robot's multi-fingered dexterous hand gradually closes its fingers. During the closing process, tactile data and the robot's multi-fingered dexterous hand grasping pattern are input into the target stable grasping discrimination model to obtain stable grasping discrimination information. When the predefined multi-fingered dexterous hand joint angle is reached and the discrimination information of the target stable grasping discrimination model is stable, the robot achieves stable grasping; if the discrimination result of the target stable grasping discrimination model is sliding, the robot stops the position control phase and enters the grasping adjustment phase.

[0050] Grasping and Adjustment Phase: The current tactile data's graphical feature information, predefined tactile data's graphical feature information, and the grasping pattern are input into the target grasping and adjustment model. The target grasping and adjustment model obtains the adjustment angles required for the finger joints of the multi-fingered dexterous hand. The tactile data and the robot's multi-fingered dexterous hand grasping pattern are input into the target stable grasping discrimination model. The target stable grasping discrimination model obtains the current grasping state. The robot continuously adjusts the angles of the finger joints of the multi-fingered dexterous hand until the discrimination result of the target stable grasping discrimination model is stable.

[0051] The grab controller is represented as:

[0052]

[0053] Among them, K p Here, J represents the gain matrix for the position control phase, and J is the current angle of the multi-finger dexterous hand joint. To predefine the joint angles of multi-finger dexterous hands, K s To capture and adjust the model, G is the predefined graph feature vector of tactile data, and G is the actual graph feature vector.

[0054] As can be seen from the above technical solutions, the embodiments of the present invention have the following beneficial effects:

[0055] This invention proposes to construct a stable grasping discrimination model using tactile data and the grasping patterns of a robot's multi-fingered dexterous hand, and to construct a grasping adjustment model using tactile data, finger angles, and grasping patterns of the robot's multi-fingered dexterous hand. By combining the stable grasping discrimination model and the grasping adjustment model, a robot grasping controller is constructed to achieve stable grasping control of the robot. [Attached Image Description]

[0056] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort or labor.

[0057] Figure 1 This is a flowchart illustrating the robot grasping control method provided in this embodiment of the invention;

[0058] Figure 2 This is a schematic diagram of the stable crawling discrimination model provided in the embodiments of the present invention.

Detailed Implementation Methods

[0059] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0060] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0061] This invention provides a robot grasping control method, please refer to the following embodiments. Figure 1 This is a flowchart illustrating the robot grasping control method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes the following steps:

[0062] Step 101: Obtain the training dataset. The training dataset includes tactile data collected during the robot's multi-finger dexterous hand grasping process, finger joint angles of the robot's multi-finger dexterous hand, grasping patterns of the robot's multi-finger dexterous hand, and pre-labeled stable grasping discrimination information.

[0063] Use tactile sensors to collect tactile data during the robot's multi-fingered dexterous hand grasping process;

[0064] The robot operating system platform is used to obtain the finger joint angles and grasping patterns during the robot's multi-finger dexterous hand grasping process; the grasping patterns include three modes: basic grasping, wide-finger grasping, and syn-finger grasping.

[0065] Select several objects of different shapes and hardnesses. For each object, at the beginning of each data collection cycle, place the object in a different orientation at the center of a horizontal plane and control the robot to perform the following operations:

[0066] When an object is placed in a designated position and the robot reaches a designated height above the object, the robot's multi-fingered dexterous hand performs a grasping operation on the object.

[0067] After the robot successfully grasps the object, it rises vertically to a specified height. During the robot's lifting process, tactile sensors are used to collect tactile data, and the robot's operating system platform is used to collect the finger joint angles and grasping patterns of the robot's multi-fingered dexterous hand.

[0068] After the robot reaches the designated posture, it remains still for a specified time before returning to the initial position, where it releases the object.

[0069] Step 102: Use the tactile data, grasping patterns and stable grasping discrimination information in the training dataset to train the initial stable grasping discrimination model to obtain the target stable grasping discrimination model.

[0070] Please refer to Figure 2 This is a schematic diagram of the stable grasping discrimination model in the embodiments of the present invention.

[0071] The tactile data is input into the GCN feature extraction sub-model of the initial stable grasping discrimination model to obtain the graph feature information of the tactile data;

[0072] Encode the grasping pattern of the robot's multi-fingered dexterous hand into One-Hot vector information;

[0073] The graphical feature information and One-Hot vector information of the tactile data are input into the feature fusion unit of the initial stable grasping discrimination model to obtain stable grasping discrimination information;

[0074] Based on the stable crawling discrimination information and the pre-labeled stable crawling discrimination information, it is determined whether the initial stable crawling discrimination model has reached the preset convergence condition. If yes, the current initial stable crawling discrimination model is used as the target stable crawling discrimination model; if no, the above training process continues until the initial stable crawling discrimination model reaches the preset convergence condition.

[0075] GCN feature extraction sub-model, including: multiple tactile array sensors;

[0076] Each tactile array sensor consists of 16 tactile units, and each tactile unit is composed of a three-dimensional force vector. The basic matrix of a single joint tactile sensing unit is... Construct a static haptic graph G = (V, E) with fixed edge connections, where V represents each haptic unit of the haptic array sensor, and E represents the edge connections between haptic units. Define the GCN feature extraction sub-model as f, and the stability classification of grasping stability is represented as c ∈ (0, 1), where the values ​​0 and 1 represent the stable state and the sliding state, respectively. For the predicted value, the GCN feature extraction sub-module learns the mapping function f through the static haptic map G, the fundamental matrix F, and the stability classification c, as follows:

[0077]

[0078] Multi-scale convolutional layers are used to extract feature information, as shown below:

[0079]

[0080] in, W is the feature vector of haptic unit v in the (l+1)th layer, N(v) is the set of neighboring haptic units of haptic unit v, and W is the feature vector of haptic unit v in the (l+1)th layer. (l) Let be the learnable weight matrix of the l-th layer. Let c be the feature vector of the haptic unit u in the l-th layer. vu The normalization constant is σ is the ReLU activation function, m represents the m-th convolutional layer, and n represents the number of convolutional layers.

[0081] The feature fusion unit of the initial stable grasping and discrimination model concatenates the feature vector extracted by the GCN feature extraction sub-model with the One-Hot vector information, and inputs the concatenation result into the fully connected layer, as shown below:

[0082]

[0083] Z = contact(O1, O 2, ,O3,m)

[0084] Among them, H(l) This represents the haptic unit feature matrix of the l-th layer of haptic units. This represents the sum of the adjacency matrix A and the identity matrix I. express The degree matrix, σ is the ReLU activation function, W (l) This represents the haptic unit weight matrix of the l-th layer of haptic units, O i represents the graph feature information of a single joint, m represents the One-Hot vector information of the grasping mode, and Z represents the feature vector obtained after concatenation.

[0085] The pre-labeled stable grasping discrimination information includes the tactile data and the robot grasping state corresponding to the robot's multi-finger dexterous hand grasping mode. The robot grasping state includes a stable state and a sliding state.

[0086] Step 103: Use the tactile data, finger joint angles and grasping patterns in the training dataset to train the initial grasping adjustment model to obtain the target grasping adjustment model.

[0087] The graphical feature information of the tactile data and the grasping pattern of the robot's multi-fingered dexterous hand are input into the initial grasping adjustment model based on MLP to obtain the predicted finger joint angles of the robot's multi-fingered dexterous hand.

[0088] Based on the deviation between the predicted finger joint angles of the robot's multi-fingered dexterous hand and the actual finger joint angles of the robot's multi-fingered dexterous hand, it is determined whether the initial grasping adjustment model has reached the preset convergence condition. If yes, the current initial grasping adjustment model is used as the target grasping adjustment model; if no, the above training process continues until the initial stable grasping adjustment model reaches the preset convergence condition.

[0089] Establish an MLP-based model K s The graph feature information G, which maps the tactile data of each joint of the robot's multi-fingered dexterous hand, is used to map the movement distance J of each joint of the robot's multi-fingered dexterous hand.

[0090] J = K s (G,m;W,b)

[0091] Where J = [θ1, θ2, θ3] T It is the joint angle vector, G = [G1, G2, G3] T Here, m represents the image feature information of each finger, w represents the grasping mode, and b represents the weight matrix and bias vector of the model, respectively.

[0092] Define a loss function L to evaluate the deviation value, as follows:

[0093]

[0094] Step 104: Generate a grasping controller based on the target stable grasping discrimination model and the target grasping adjustment model.

[0095] Step 105: Input the tactile data, the finger joint angles of the robot's multi-finger dexterous hand, and the grasping mode into the grasping controller to achieve grasping control of the robot.

[0096] The grasping control of the grasping controller is divided into a position control stage and a grasping adjustment stage. The transition between the two stages is achieved through a piecewise linear function of the vector norm of the joint angles of the robot's multi-fingered dexterous hand, as shown below:

[0097]

[0098] Where J represents the current finger angle of the multi-finger hand. The finger angle is represented within the grasp parameters, and state represents the result of the target stable grasp discrimination model, with 0 indicating a stable state and 1 indicating a sliding state.

[0099] Position control phase: Given a predefined target multi-fingered dexterous hand joint angle, the robot's multi-fingered dexterous hand gradually closes its fingers. During the closing process, tactile data and the robot's multi-fingered dexterous hand grasping pattern are input into the target stable grasping discrimination model to obtain stable grasping discrimination information. When the predefined multi-fingered dexterous hand joint angle is reached and the discrimination information of the target stable grasping discrimination model is stable, the robot achieves stable grasping; if the discrimination result of the target stable grasping discrimination model is sliding, the robot stops the position control phase and enters the grasping adjustment phase.

[0100] Grasping and Adjustment Phase: The current tactile data's graphical feature information, predefined tactile data's graphical feature information, and the grasping pattern are input into the target grasping and adjustment model. The target grasping and adjustment model obtains the adjustment angles required for the finger joints of the multi-fingered dexterous hand. The tactile data and the robot's multi-fingered dexterous hand grasping pattern are input into the target stable grasping discrimination model. The target stable grasping discrimination model obtains the current grasping state. The robot continuously adjusts the angles of the finger joints of the multi-fingered dexterous hand until the discrimination result of the target stable grasping discrimination model is stable.

[0101] The grab controller is represented as:

[0102]

[0103] Among them, K p Here, J represents the gain matrix for the position control phase, and J is the current angle of the multi-finger dexterous hand joint. To predefine the joint angles of multi-finger dexterous hands, K s To capture and adjust the model, G is the predefined graph feature vector of tactile data, and G is the actual graph feature vector.

[0104] The technical solutions of the embodiments of the present invention have the following beneficial effects:

[0105] This invention constructs a stable grasping discrimination model using tactile data and the grasping patterns of a robot's multi-fingered dexterous hand, and constructs a grasping adjustment model using tactile data, finger angles, and grasping patterns of the robot's multi-fingered dexterous hand. By combining the stable grasping discrimination model and the grasping adjustment model, a robot multi-fingered dexterous hand grasping controller is constructed to achieve stable grasping control of the robot.

[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0107] The contents not described in detail in this specification are common knowledge to those skilled in the art.

Claims

1. A robot grasping control method, characterized in that, The method includes: Obtain a training dataset, which includes tactile data collected during the robot's multi-finger dexterous hand grasping process, finger joint angles of the robot's multi-finger dexterous hand, grasping patterns of the robot's multi-finger dexterous hand, and pre-labeled stable grasping discrimination information. Using the tactile data, the grasping pattern, and the stable grasping discrimination information in the training dataset, the initial stable grasping discrimination model is trained to obtain the target stable grasping discrimination model; Using the tactile data, finger joint angles, and grasping patterns from the training dataset, an initial grasping adjustment model is trained to obtain a target grasping adjustment model, including: The tactile data is input into the GCN feature extraction sub-model of the initial stable grasping discrimination model to obtain the graph feature information of the tactile data; The grasping pattern of the robot's multi-fingered dexterous hand is encoded as One-Hot vector information; The graphical feature information and One-Hot vector information of the tactile data are input into the feature fusion unit of the initial stable grasping discrimination model to obtain stable grasping discrimination information; Based on the stable grasping discrimination information and the pre-labeled stable grasping discrimination information, it is determined whether the initial stable grasping discrimination model has reached the preset convergence condition. If yes, the current initial stable grasping discrimination model is used as the target stable grasping discrimination model; if no, the above training process continues until the initial stable grasping discrimination model reaches the preset convergence condition. The GCN feature extraction sub-model includes: multiple tactile array sensors; Each tactile array sensor consists of 16 tactile units, and each tactile unit is composed of a three-dimensional force vector. The basic matrix of a single joint tactile sensing unit is... Construct a static haptic graph with fixed edge connections. ,in This represents the individual tactile units of the tactile array sensor. The edge connections between each haptic unit are represented, and the GCN feature extraction sub-model is defined as follows: The stability classification of crawling stability is represented as follows: The values ​​0 and 1 represent the steady state and the sliding state, respectively. For the predicted value, the GCN feature extraction sub-model uses a static tactile image. Basic matrix Classification of stability Learn mapping functions , means as follows: Multi-scale convolutional layers are used to extract feature information, as shown below: in, It is the first In-layer haptic unit eigenvectors, For haptic unit The set of adjacent haptic units, For the first The learnable weight matrix of the layer, For the first In-layer haptic unit eigenvectors, The normalization constant is , for Activation function Indicates the first One convolutional layer, Indicates the number of convolutional layers; A grasping controller is generated based on the target stable grasping discrimination model and the target grasping adjustment model; The tactile data, the finger joint angles of the robot's multi-fingered dexterous hand, and the grasping pattern are input into the grasping controller to achieve grasping control of the robot.

2. The method according to claim 1, characterized in that, Obtain the training dataset, including: Use tactile sensors to collect tactile data during the robot's multi-fingered dexterous hand grasping process; The robot operating system platform is used to obtain the finger joint angles and grasping patterns during the robot's multi-finger dexterity grasping process; the grasping patterns include three modes: basic grasping, wide-finger grasping, and syn-finger grasping.

3. The method according to claim 1 or 2, characterized in that, The acquisition of the training dataset includes: Select several objects of different shapes and hardnesses. For each object, at the beginning of each data collection cycle, place the object in a different orientation at the center of a horizontal plane and control the robot to perform the following operations: When an object is placed in a designated position and the robot reaches a designated height above the object, the robot's multi-fingered dexterous hand performs a grasping operation on the object. After the robot successfully grasps the object, it rises vertically to a specified height. During the robot's lifting process, tactile sensors are used to collect tactile data, and the robot's operating system platform is used to collect the finger joint angles and grasping patterns of the robot's multi-fingered dexterous hand. After the robot reaches the designated posture, it remains still for a specified time before returning to the initial position, where it releases the object.

4. The method according to claim 1, characterized in that, The feature fusion unit of the initial stable grasping and discrimination model includes: The feature vectors extracted by the GCN feature extraction sub-model are concatenated with the One-Hot vector information, and the concatenated result is input into the fully connected layer, as shown below: in, Indicates the first The tactile unit feature matrix of the layered tactile unit, Representing the adjacency matrix With the identity matrix The sum of, express The degree matrix, for Activation function Indicates the first The haptic unit weight matrix of the layer haptic unit. Graph feature information representing a single joint. One-Hot vector information representing the capture mode. This represents the feature vector obtained after concatenation.

5. The method according to claim 1, characterized in that, The pre-labeled stable grasping discrimination information includes the tactile data and the robot grasping state corresponding to the robot's multi-finger dexterous hand grasping mode. The robot grasping state includes a stable state and a sliding state.

6. The method according to claim 1, characterized in that, The process of training an initial grasping adjustment model using the tactile data, finger joint angles, and grasping patterns from the training dataset to obtain a target grasping adjustment model includes: The graphical feature information of the tactile data and the grasping pattern of the robot's multi-fingered dexterous hand are input into the initial grasping adjustment model based on MLP to obtain the predicted finger joint angles of the robot's multi-fingered dexterous hand. Based on the deviation between the predicted finger joint angles of the robot's multi-fingered dexterous hand and the actual finger joint angles of the robot's multi-fingered dexterous hand, it is determined whether the initial grasping adjustment model has reached the preset convergence condition. If yes, the current initial grasping adjustment model is used as the target grasping adjustment model; if no, the above training process continues until the initial grasping adjustment model reaches the preset convergence condition.

7. The method according to claim 6, characterized in that, The initial crawling and adjustment model includes: Building an MLP-based model Graph feature information that maps tactile data of each joint of the robot's multi-fingered dexterous hand. The distance of movement of each joint in the robot's multi-fingered dexterous hand : in, It is a joint angle vector. For the image feature information of each finger, For crawling mode, and These represent the weight matrix and bias vector of the model, respectively. Define loss function The deviation value is evaluated as follows: 。 8. The method according to claim 1, characterized in that, The process of inputting tactile data, the finger joint angles of the robot's multi-fingered dexterous hand, and the grasping pattern into the grasping controller to achieve grasping control of the robot includes: The grasping control of the grasping controller is divided into a position control stage and a grasping adjustment stage. The transition between the two stages is achieved through a piecewise linear function of the vector norm of the joint angles of the robot's multi-fingered dexterous hand, as shown below: in, Indicates the current finger angle in a multi-fingered hand. The finger angle is represented within the grasp parameters, and state represents the result of the target stable grasp discrimination model, with 0 indicating a stable state and 1 indicating a sliding state. Position control phase: Given a predefined target multi-fingered dexterous hand joint angle, the robot's multi-fingered dexterous hand gradually closes its fingers. During the closing process, tactile data and the robot's multi-fingered dexterous hand grasping pattern are input into the target stable grasping discrimination model to obtain stable grasping discrimination information. When the predefined multi-fingered dexterous hand joint angle is reached and the discrimination information of the target stable grasping discrimination model is stable, the robot achieves stable grasping; if the discrimination result of the target stable grasping discrimination model is sliding, the robot stops the position control phase and enters the grasping adjustment phase. Grasping and Adjustment Phase: The current tactile data's graphical feature information, predefined tactile data's graphical feature information, and the grasping pattern are input into the target grasping and adjustment model. The target grasping and adjustment model obtains the adjustment angles required for the finger joints of the multi-fingered dexterous hand. The tactile data and the robot's multi-fingered dexterous hand grasping pattern are input into the target stable grasping discrimination model. The target stable grasping discrimination model obtains the current grasping state. The robot continuously adjusts the angles of the finger joints of the multi-fingered dexterous hand until the discrimination result of the target stable grasping discrimination model is stable.

9. The method according to claim 1, characterized in that, The grab controller is represented as: in, This is the gain matrix for the position control stage. For the current multi-finger dexterity hand joint angles, To predefine the joint angles of multi-finger dexterous hands To capture and adjust the model, For predefined tactile data, graphical feature vectors These are the actual graph feature vectors.

Images