Gripping method, robotic arm, and robot

Abstract

A gripping method, a robotic arm, and a robot. The gripping method is applied to a controller (1401) of a robotic arm (1), and comprises: during the process of the robotic arm (1) gripping a target object, acquiring real-time contact information generated when the robotic arm (1) grips the target object; and on the basis of the real-time contact information, adjusting current gripping postures of at least two finger mechanisms (1302) of the robotic arm (1), such that contact information generated by the at least two finger mechanisms (1302) after the postures are adjusted matches predicted contact information, wherein the real-time contact information is collected by means of at least one of tactile sensors (28) provided on the at least two finger mechanisms (1302) and a tactile sensor (28) provided on a palm portion (1303) of the robotic arm (1). In this way, the current gripping postures of the at least two finger mechanisms (1302) can be adjusted by means of the predicted contact information and the real-time contact information, such that the robotic arm (1) grips the target object by using a suitable gripping posture, thereby improving the accuracy of the robotic arm (1) in terms of gripping an object.

Description

A grasping method, robotic arm and robot

[0001] Cross-reference to related applications

[0002] This application is based on and claims priority to Chinese Patent Application No. 202411204331.7, filed on August 29, 2024, entitled "Grasping Method, Manipulator and Robot", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of robotics, and more particularly to a grasping method, a robotic arm, and a robot. Background Technology

[0004] With the increasing level of intelligence, people are using robotic arms on robots to grasp objects. In related technologies, when a robotic arm needs to grasp an object, it is often necessary to first determine the object's dimensions and then control the robotic arm to grasp it based on those dimensions. However, the problem with this approach is that if the object's dimensions are not accurately determined, the robotic arm's grasping posture will not match the object itself, resulting in low accuracy in grasping the object. Summary of the Invention

[0005] In view of this, embodiments of this application provide at least one grasping method, a robotic arm, and a robot.

[0006] The technical solution of this application embodiment is implemented as follows:

[0007] In a first aspect, embodiments of this application provide a grasping method, the grasping method being applied to a controller of a robotic arm; the method includes:

[0008] During the process of the robotic arm grasping the target object, real-time contact information generated when the robotic arm grasps the target object is acquired;

[0009] Based on the real-time contact information, the current grasping posture of at least two fingers of the robotic hand is adjusted so that the contact information generated by the adjusted at least two fingers matches the predicted contact information.

[0010] The real-time contact information is collected by at least one of the tactile sensors located on the at least two finger mechanisms and the tactile sensors located on the palm of the robotic hand.

[0011] Secondly, embodiments of this application provide a grasping method, the grasping method being applied to a controller of a robot including a robotic arm; the method includes:

[0012] Obtain the size information of the target object;

[0013] The robot arm's posture information is determined based on the size information of the target object, and then sent to the robot arm's controller.

[0014] The controller of the robotic hand is used to acquire real-time contact information generated when the robotic hand grasps the target object based on the robotic hand's posture information; based on the real-time contact information, adjust the current grasping posture of at least two finger mechanisms of the robotic hand so that the contact information generated by the at least two finger mechanisms after posture adjustment matches the predicted contact information; the real-time contact information is collected by at least one of the tactile sensors disposed on the at least two finger mechanisms and the tactile sensors disposed on the palm of the robotic hand.

[0015] Thirdly, embodiments of this application provide a robotic hand, including a controller, at least two finger mechanisms, and a palm portion; wherein...

[0016] The controller of the robotic arm is used to acquire real-time contact information generated when the robotic arm grasps the target object during the process of the robotic arm grasping the target object;

[0017] The controller of the robotic hand is used to adjust the current grasping posture of at least two finger mechanisms of the robotic hand based on the real-time contact information, so that the contact information generated by the at least two finger mechanisms after posture adjustment matches the predicted contact information.

[0018] The real-time contact information is collected by at least one of the tactile sensors located on the at least two finger mechanisms and the tactile sensors located on the palm of the robotic hand.

[0019] Fourthly, embodiments of this application provide a robot, including a controller and a robotic arm;

[0020] The robot's controller is used to acquire the size information of the target object;

[0021] The robot's controller is used to determine the robot hand's posture information based on the size information of the target object, and send the robot hand's posture information to the robot hand's controller;

[0022] The controller of the robotic hand is used to acquire real-time contact information generated when the robotic hand grasps the target object based on the robotic hand's posture information; based on the real-time contact information, adjust the current grasping posture of at least two finger mechanisms of the robotic hand so that the contact information generated by the at least two finger mechanisms after posture adjustment matches the predicted contact information; the real-time contact information is collected by at least one of the tactile sensors disposed on the at least two finger mechanisms and the tactile sensors disposed on the palm of the robotic hand.

[0023] Fifthly, embodiments of this application provide a gripping device applied to the controller of a robotic arm; the gripping device includes:

[0024] The first acquisition unit is used to acquire real-time contact information generated when the robotic arm grasps the target object during the process of the robotic arm grasping the target object.

[0025] An adjustment unit is used to adjust the current grasping posture of at least two fingers of the robotic hand based on the real-time contact information, so that the contact information generated by the at least two fingers after posture adjustment matches the predicted contact information.

[0026] The real-time contact information is collected by at least one of the tactile sensors located on the at least two finger mechanisms and the tactile sensors located on the palm of the robotic hand.

[0027] Sixthly, embodiments of this application provide a gripping device applied to the controller of a robot including a robotic arm; the gripping device includes:

[0028] The second acquisition unit is used to acquire the size information of the target object;

[0029] A determining unit is used to determine the robot arm's posture information based on the size information of the target object;

[0030] A sending unit is used to send the robot arm's posture information to the robot arm's controller;

[0031] The controller of the robotic hand is used to acquire real-time contact information generated when the robotic hand grasps the target object based on the robotic hand's posture information; based on the real-time contact information, adjust the current grasping posture of at least two finger mechanisms of the robotic hand so that the contact information generated by the at least two finger mechanisms after posture adjustment matches the predicted contact information; the real-time contact information is collected by at least one of the tactile sensors disposed on the at least two finger mechanisms and the tactile sensors disposed on the palm of the robotic hand.

[0032] In a seventh aspect, embodiments of this application provide a robot, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the program to implement some or all of the steps in the above-described method.

[0033] Eighthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements some or all of the steps in the above-described method.

[0034] Ninthly, embodiments of this application provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement some or all of the steps in the above-described method.

[0035] In this embodiment, during the process of the robotic arm grasping a target object, the controller of the robotic arm can acquire real-time contact information generated when grasping the target object; and adjust the current grasping posture of at least two fingers of the robotic arm based on the real-time contact information, so that the contact information generated by the at least two fingers after posture adjustment matches the predicted contact information. In this way, by adjusting the current grasping posture of at least two fingers based on the predicted contact information, the at least two fingers after posture adjustment can ultimately match the contact information generated when grasping the object with the predicted contact information, thereby enabling the robotic arm to adopt a suitable grasping posture to grasp the target object, thus improving the accuracy of the robotic arm in grasping the object.

[0036] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this application. Attached Figure Description

[0037] Figure 1 is a schematic diagram of the implementation process of a crawling method provided in an embodiment of this application;

[0038] Figure 2a is a schematic diagram of the structure of the robotic arm provided in an embodiment of this application;

[0039] Figure 2b is a schematic diagram of the structure of the robotic arm provided in an embodiment of this application;

[0040] Figure 2c is a schematic diagram of the implementation process of a crawling method provided in an embodiment of this application;

[0041] Figure 3 is a schematic diagram of the finger mechanism provided in an embodiment of this application;

[0042] Figure 4 is a schematic diagram of the structure of the finger mechanism provided in the embodiment of this application;

[0043] Figure 5 is a schematic diagram of the finger mechanism provided in the embodiment of this application.

[0044] Figure 6 is a schematic diagram of the finger mechanism provided in an embodiment of this application;

[0045] Figure 7 is a schematic diagram of the finger mechanism provided in the embodiment of this application;

[0046] Figure 8 is a schematic diagram of the implementation process of a crawling method provided in an embodiment of this application;

[0047] Figure 9 is a schematic diagram of the implementation process of a crawling method provided in an embodiment of this application;

[0048] Figure 10 is a schematic diagram of the implementation process of a crawling method provided in an embodiment of this application;

[0049] Figure 11 is a schematic diagram of the implementation process of a crawling method provided in an embodiment of this application;

[0050] Figure 12 is a schematic diagram of the implementation process of a crawling method provided in an embodiment of this application;

[0051] Figure 13 is a schematic diagram of the structure of the robotic arm provided in an embodiment of this application;

[0052] Figure 14 is a schematic diagram of the structure of the robot provided in the embodiment of this application.

[0053] Reference numerals: 1-Manipulator; 11-Body; 12-Palpognathic mechanism; 121-Palpognathic drive assembly; 1211-Connecting seat; 1212-Power push rod assembly; 12121-First power push rod; 12122-Second power push rod; 122-Palpognathic linkage assembly; 1221-First link; 1222-Second link; 1223-Third link; 1224-Fourth link; 123-Palpognathic phalanx assembly; 1231-Proximal phalanx of the palm; 1232-Middle phalanx of the first palm; 1233-Middle phalanx of the second palm; 1234-Palpognathic distal phalanx; 124-Fixing plate; 13-Thumb mechanism; 131-Thumb drive assembly; 132-Thumb phalanx assembly; 1321-Proximal phalanx of the thumb; 1322-Middle phalanx of the first thumb; 1 323 - Second middle phalanx of the thumb; 1324 - Thumb distal phalanx; 14 - First pivot; 15 - Second pivot; 16 - Third pivot; 17 - Fourth pivot; 18 - Fifth pivot; 19 - Sixth pivot; 20 - Seventh pivot; 21 - Eighth pivot; 22 - Ninth pivot; 23 - Tenth pivot; 24 - Eleventh pivot; 25 - Twelfth pivot; 261 - Distance acquisition device; 271 - Image acquisition device; 272 - Complementary lighting; 28 - Tactile sensor; 1302 - Finger mechanism; 1303 - Palm part; 1400 - Robot; 1401 - Controller. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0056] The terms “first / second / third” are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that “first / second / third” may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this application.

[0058] To address the technical problem of low accuracy in object grasping in related technologies, this application provides a grasping method that can be applied to the controller of a robotic arm. As shown in Figure 1, the method includes steps S101 and S102:

[0059] Step S101: During the process of the robotic arm grasping the target object, real-time contact information generated when the robotic arm grasps the target object is acquired.

[0060] Here, the target object is the object to be grasped by the robotic arm. In some embodiments, the target object can be determined based on environmental image information surrounding the robotic arm. For example, after acquiring the environmental image information, the robotic arm's controller can determine the size of each object if it detects the presence of objects in the image, and then determine the target object based on the size of each object. It is understood that due to the size limitations of the robotic arm itself, it can grasp objects corresponding to its own size, thus enabling the determination of the target object based on the size of each object. In other embodiments, the entity determining the target object can be the robot's controller, which includes the aforementioned robotic arm. Typically, a robot has a robotic arm, and the robotic arm is deployed at the end of the robotic arm. Of course, the robotic arm can also be deployed on the robot in other ways, which are not limited here.

[0061] In some embodiments, the robotic hand can grasp a target object using a preset grasping posture. This preset grasping posture can be a posture pre-set in the controller of the robotic hand or the controller of the robot, or a grasping posture that can be obtained from storage areas such as other terminals, servers, or the cloud via data transmission methods such as networks. It is understood that the controller of the robotic hand or the controller of the robot stores multiple historical grasping postures, and the preset grasping posture can be the one with the highest frequency of use among these historical postures or the one with the highest success rate in similar scenarios. In other embodiments, the robotic hand can also grasp the target object using a real-time grasping posture. This real-time grasping posture can be determined in real-time by the controller of the robotic hand or the controller of the robot based on the size information of the target object.

[0062] In this embodiment of the application, the robotic hand includes at least two finger mechanisms and a palm portion, wherein the at least two finger mechanisms are respectively connected to the palm portion, and each of the at least two finger mechanisms and the palm portion may be provided with a tactile sensor. Exemplarily, the tactile sensor may be at least one of the following: a contact sensor, a force-torque sensor, a pressure sensor, and a slip sensor.

[0063] In some embodiments, the real-time contact information generated when the robotic hand grasps a target object may be collected by tactile sensors disposed on at least two finger mechanisms, or by a combination of tactile sensors disposed on at least two finger mechanisms and tactile sensors disposed on the palm of the robotic hand, or by tactile sensors disposed on the palm of the robotic hand.

[0064] It is understandable that the tactile sensors that collect real-time contact information are related to the current grasping posture of the robotic hand when grasping the target object. If the current grasping posture indicates that the target object is grasped through at least two finger mechanisms, then the tactile sensors that collect real-time contact information are tactile sensors disposed on at least two finger mechanisms; if the current grasping posture indicates that the target object is grasped through at least two finger mechanisms and the palm, then the tactile sensors that collect real-time contact information are tactile sensors disposed on at least two finger mechanisms and tactile sensors disposed on the palm of the robotic hand; if the current grasping posture indicates that the target object is grasped through the palm, then the tactile sensors that collect real-time contact information are tactile sensors disposed on the palm of the robotic hand. It is understandable that when the palm includes at least two movable connected parts, the object can be grasped solely through the palm.

[0065] In some embodiments, if the current grasping posture represents grasping the target object through at least two finger mechanisms and the palm, but due to the low matching degree between the current grasping posture and the size information of the target object, or the mismatch between the position of the robot hand and the position of the target object, it is possible that when grasping the target object with the current grasping posture, only the finger mechanisms or only the palm touches the target object. In this case, even if the current grasping posture represents grasping the target object through at least two finger mechanisms and the palm, the tactile sensor that collects real-time contact information can be a tactile sensor set on at least two finger mechanisms, or a tactile sensor set on the palm of the robot hand.

[0066] Step S102: Based on the real-time contact information, adjust the current gripping posture of at least two finger mechanisms of the robotic hand so that the contact information generated by the at least two finger mechanisms after posture adjustment matches the predicted contact information.

[0067] Here, the predicted contact information can be obtained based on the image information and size information of the target object.

[0068] Understandably, size information characterizes the size of the target object, and the opening degree of at least two fingers of the robotic hand can differ for target objects of different sizes. Image information identifies the material of the target object, and for objects of the same size but different materials, the gripping force output by at least two fingers of the robotic hand will also differ. Both the gripping force and the opening degree of the robotic hand's at least two fingers can be reflected in the contact information generated when the robotic hand grasps the target object. Therefore, the predicted contact information obtained from the image and size information of the target object can correct the current gripping posture of the robotic hand's at least two fingers, enabling the robotic hand to adopt an appropriate gripping posture to grasp the target object.

[0069] The process of identifying the material of a target object through image information may include: inputting image information into a trained neural network model to obtain the material of the target object. The training process of the neural network model may include: constructing a sample set; this sample set includes image information of objects with different materials, and each image contains label information representing the material type of the object; training the neural network model using this sample set to obtain the trained neural network model.

[0070] In some embodiments, the deviation information between real-time contact information and predicted contact information can be determined first, and then the movement information of at least two finger mechanisms can be determined based on the deviation information. The at least two finger mechanisms are controlled to move according to the movement information. It is determined whether the contact information generated by the at least two finger mechanisms after the movement matches the predicted contact information. If they match, the adjustment ends. If they do not match, the at least two finger mechanisms are adjusted again according to the above method until the contact information generated by the at least two finger mechanisms after the posture adjustment matches the predicted contact information.

[0071] In some embodiments, determining the predicted contact information based on the image information and size information of the target object can be performed by the controller of the robotic hand or the controller of the robot. After the robot controller determines the predicted contact information, it can send the predicted contact information to the controller of the robotic hand, so that the controller of the robotic hand can adjust the current grasping posture of at least two fingers of the robotic hand based on the real-time contact information, so that the contact information generated by the at least two fingers after posture adjustment matches the predicted contact information.

[0072] In some embodiments, the predicted contact information includes contact information corresponding to different tactile sensors. After obtaining the predicted contact information, the controller of the robotic arm can continuously adjust the current grasping posture of at least two finger mechanisms of the robotic arm, so that the real-time contact information generated by tactile sensors distributed at different positions of the robotic arm matches the contact information corresponding to different tactile sensors in the predicted contact information.

[0073] In this embodiment, the controller of the robotic arm acquires real-time contact information generated during the robotic arm's grasping of a target object. Then, based on this real-time contact information, the controller adjusts the current grasping posture of at least two fingers of the robotic arm, ensuring that the contact information generated by the adjusted fingers matches the predicted contact information. In this way, by comparing the predicted and real-time contact information, the current grasping posture of at least two fingers can be adjusted, ultimately ensuring that the contact information generated by the adjusted fingers when grasping an object matches the predicted contact information. This allows the robotic arm to adopt a suitable grasping posture to grasp the target object, thereby improving the accuracy of object grasping.

[0074] As shown in Figure 2a, the robotic hand 1 includes a body 11 and at least two finger mechanisms; wherein, the at least two finger mechanisms include palmar finger mechanisms 12 and thumb mechanisms 13. Figure 2a shows that the at least two finger mechanisms include three palmar finger mechanisms 12 and one thumb mechanism 13. In other embodiments, the number of palmar finger mechanisms 12 may also be four. This application embodiment does not limit the number of palmar finger mechanisms. The body 11 includes a palm portion (not shown), and the at least two finger mechanisms are movably connected to the palm portion. Tactile sensors 28 are provided on the at least two finger mechanisms and the palm portion. When the finger mechanisms and / or the palm portion grasp a target object, contact information can be collected through the tactile sensors 28.

[0075] In some embodiments, each of the at least two finger mechanisms includes a finger driving component and a finger knuckle component. When the finger mechanism is a palmar-finger mechanism, the palmar-finger mechanism includes a palmar-finger driving component and a palmar-finger knuckle component; when the finger mechanism is a thumb mechanism, the thumb mechanism includes a thumb driving component and a thumb knuckle component. As shown in FIG2b, the palmar-finger mechanism 12 includes a palmar-finger driving component 121 and a palmar-finger knuckle component 123, and the thumb mechanism 13 includes a thumb driving component 131 and a thumb knuckle component 132.

[0076] In this embodiment of the application, as shown in FIG2c, based on FIG2a and FIG2b, the above-mentioned grabbing method can also be implemented by steps S201 to S203:

[0077] Step S201: Receive robot hand posture information sent by the robot's controller; the robot hand posture information is determined by the robot's controller based on the size information of the target object; the robot hand posture information includes joint rotation information of the finger joint components of each finger mechanism.

[0078] Here, the robot hand posture information refers to the initial grasping posture adopted by the robot hand when grasping the target object. This robot hand posture information is determined by the robot based on the size information of the target object.

[0079] Understandably, contact information indicates the gripping force and opening degree of the robotic arm when grasping a target object; in other words, contact information can determine the accuracy of the robotic arm's grasp of the target object. However, when the robotic arm approaches the target object but has not yet begun to grasp it, contact information cannot be obtained because the robotic arm has not yet made contact with the target object. At this time, it is necessary to determine the initial grasping posture adopted by the robotic arm when grasping the target object (i.e., the aforementioned robotic arm posture information). This grasping posture is not the final grasping posture of the robotic arm; it needs to be adjusted based on real-time contact information until the real-time contact information matches the predicted contact information.

[0080] In this embodiment of the application, the robot controller stores the correspondence between the size information of different objects and the posture information of different robotic arms. By using the size information of the target object, the posture information of the robotic arm corresponding to the size information of the target object can be determined from multiple correspondences.

[0081] In this embodiment, the robotic hand posture information includes the joint rotation information of the knuckle components of each finger mechanism. It can be understood that the robotic hand grasps objects by rotating the finger knuckle components to output gripping force. Therefore, the gripping posture of the robotic hand can be characterized by the joint rotation information of the finger knuckle components.

[0082] Step S202: For each of the finger mechanisms, determine the motion information of the finger drive component of the finger mechanism based on the joint rotation information of the finger knuckle component of the finger mechanism.

[0083] In this embodiment, the movement of the finger-driving component can drive the rotation of the finger joint component, and the controller of the robotic hand can control the movement of the finger-driving component. Therefore, after obtaining the joint rotation information of the finger joint component, the controller of the robotic hand needs to convert the joint rotation information of the finger joint component into the movement information of the finger-driving component.

[0084] In this embodiment, the joint rotation information of the finger knuckle assembly includes the joint rotation direction, joint rotation angle, and joint rotation speed. The motion information of the finger driving assembly includes the motion direction, motion distance, and motion speed. The joint rotation direction of the finger knuckle assembly corresponds to the motion direction of the finger driving assembly, the joint rotation angle of the finger knuckle assembly corresponds to the motion distance of the finger driving assembly, and the joint rotation speed of the finger knuckle assembly corresponds to the motion speed of the finger driving assembly.

[0085] For example, when the joint of the finger knuckle component rotates clockwise, the movement direction of the finger driving component can be towards the finger knuckle component; when the joint of the finger knuckle component rotates counterclockwise, the movement direction of the finger driving component can be away from the finger knuckle component.

[0086] Step S203: For each of the finger mechanisms, control the finger drive component of the finger mechanism to move according to the motion information, so as to drive the finger joint component of the finger mechanism to rotate according to the joint rotation information.

[0087] It is understandable that the motion information of the finger drive components of a finger mechanism includes the direction of movement, the distance of movement, and the speed of movement. Furthermore, the motion information of the finger drive components of different finger mechanisms can be the same or different. Therefore, the controller of the robotic hand can control the movement of different finger drive components according to their motion information, thereby driving the finger joint components of different finger mechanisms to rotate according to their own joint rotation information, thus positioning the robotic hand in a grasping posture for grasping a target object.

[0088] In this embodiment, the joint rotation information of the finger joint components of each finger mechanism, determined by the size information of the target object, can be converted to obtain the motion information of the finger driving component of each finger mechanism. Then, the finger driving component of each finger mechanism is controlled to move according to the motion information, thereby driving the finger joint components of each finger mechanism to rotate according to the joint rotation information. In this way, through data conversion, the joint rotation information of the finger joint components of each finger mechanism can be converted into motion information of the finger driving components that can be controlled by the controller of the robotic arm. Thus, the finger driving components can drive the finger joint components to rotate according to the joint rotation information, thereby improving the accuracy of controlling the robotic arm to grasp the target object.

[0089] Referring to Figures 2b, 3, and 4, the palm and finger drive assembly 121 includes a connecting seat 1211 and a power push rod assembly 1212. The connecting seat 1211 is fixed to the protective part on one side along the second direction. A sliding groove assembly is provided on the connecting seat 1211. The power push rod assembly 1212 is located in the sliding groove assembly and slides relative to the sliding groove assembly to drive the palm and finger joint assembly 123 to grasp the object to be grasped. The power push rod assembly 1212 includes a first power push rod 12121. The sliding groove assembly includes a first sliding groove extending along the first direction provided on the connecting seat 1211. The first power push rod 12121 is disposed in the first sliding groove. The palm and finger linkage assembly 122 includes a first connecting rod 1221 and a second connecting rod 12121. 22. The metacarpophalangeal joint assembly 123 includes a proximal metacarpophalangeal joint 1231. A first connecting rod 1221 is connected to a first power push rod 12121 via a first rotating shaft 14, and is connected to the proximal metacarpophalangeal joint 1231 on both sides along a third direction via a second rotating shaft 15. A second connecting rod 1222 is connected to the proximal metacarpophalangeal joint 1231 on both sides along a third direction via a third rotating shaft 16. The second connecting rod 1222 is connected to the connecting part via a fourth rotating shaft 17. The third direction has an angle with the first direction and the second direction. The axes of the first rotating shaft 14, the second rotating shaft 15 and the third rotating shaft 16 are parallel to the third direction. The axis of the fourth rotating shaft 17 is parallel to the direction away from the palm 111.

[0090] In this embodiment of the application, based on Figures 2b, 3 and 4, the step S203 above, "controlling the finger drive component of the finger mechanism to move according to the motion information", can be achieved through step S2031:

[0091] Step S2031: Control the two first power push rods to move simultaneously along the direction of movement close to the finger joint assembly, causing the proximal finger joint to rotate around the first rotation axis, so that the proximal finger joint bends towards the side closer to the palm.

[0092] For example, step S2031 is described below using a palm-finger mechanism. As shown in Figure 5, the two first power push rods 121121 of the palm-finger drive assembly of the palm-finger mechanism 12 are simultaneously pushed out at the same speed in the direction close to the palm-finger joint assembly (i.e., direction C in Figure 5). At this time, the proximal palm-finger joint 1231 rotates clockwise around the axis of the third rotating shaft 16, thereby causing the proximal palm-finger joint 1231 to bend towards the palm side. The third rotating shaft 16 is the first rotating shaft in step S2031.

[0093] It is understandable that step S2031 is the step in which the controller of the robotic arm controls the movement of a finger mechanism. Figure 5 shows the finger posture of one of the palmar finger mechanisms after being controlled by the controller of the robotic arm. If the thumb mechanism is included among the at least two finger mechanisms, since the thumb mechanism and the palmar finger mechanism have the same structure, the thumb mechanism can be controlled by referring to the steps for controlling the palmar finger mechanism. As shown in Figure 2b, the thumb mechanism 13 includes a thumb drive component 131 and a thumb knuckle component 132. The thumb knuckle component 132 includes a proximal thumb knuckle 1321, a first middle thumb knuckle 1322, a second middle thumb knuckle 1323, and a distal thumb knuckle 1324. The thumb drive component 131 can drive the proximal thumb knuckle 1321 to bend toward the palm and drive the proximal thumb knuckle 1321 to swing along the extension direction of the plane where the palm is located.

[0094] In this embodiment, the controller of the robotic hand controls two first power push rods to move simultaneously along the direction of movement close to the finger joint assembly, causing the proximal finger joint to rotate around a third axis, thus bending the proximal finger joint towards the side closer to the palm. In this way, by controlling the movement of the two first power push rods, the bending of the proximal finger joint can be achieved, enabling the hand to grasp a target object through the bent proximal finger joint and the palm.

[0095] Referring to Figures 2b, 3, and 4, the power push rod assembly 1212 includes a second power push rod 12122. The sliding groove assembly includes a second sliding groove extending in a first direction, symmetrically positioned between two first sliding grooves on the connecting seat 1211. Along a direction away from the palm portion 111, the first sliding groove is higher than the first sliding groove. The palm-finger linkage assembly 122 includes a third link 1223 and a fourth link 1224. The third link 1223 is connected to the second power push rod 12122 via a fifth rotating shaft 18. Link 1223 is connected to the second link 1222 via the sixth pivot 19. The outer contours of both the fifth pivot 18 and the sixth pivot 19 are spherical. Link 1223 is connected to the fourth link 1224 via the seventh pivot 20. The proximal metacarpophalangeal phalanx 1231 forms a first receiving cavity with an opening along a first direction. Links 1223 and 1224 are located within the first receiving cavity. The metacarpophalangeal phalanx assembly 123 includes a first mid-metacarpophalangeal phalanx 1232, a second mid-metacarpophalangeal phalanx 1233, and... The distal metacarpophalangeal phalanx, the first mid-phalanx 1232 and the proximal metacarpophalangeal phalanx 1231 are connected by an eighth pivot 21 along two sides in a third direction, and the first mid-phalanx 1232 and the distal metacarpophalangeal phalanx 1234 are connected by a ninth pivot 22 along two sides in a third direction. The first mid-phalanx 1232 forms a second receiving cavity with an opening in a first direction. At least a portion of the second mid-phalanx 1233 is located within the second receiving cavity. The second mid-phalanx 1233 and the proximal metacarpophalangeal phalanx... Section 1231 is connected to both sides of the third direction via the tenth pivot 23. The fourth link 1224 is connected to the middle segment 1233 of the second metacarpophalangeal joint via the eleventh pivot 24. The middle segment 1233 of the second metacarpophalangeal joint is connected to the end segment of the metacarpophalangeal joint via the twelfth pivot 25 along both sides of the third direction. The axial directions of the fifth pivot 18, sixth pivot 19, seventh pivot 20, eighth pivot 21, ninth pivot 22, tenth pivot 23, eleventh pivot 24 and twelfth pivot 25 are all the same as those of the third direction.

[0096] When the second power push rod 12122 slides along the second sliding groove, the distal phalanx 1234 of the palm and fingers rotates along the fifth rotating shaft 18, the sixth rotating shaft 19, the seventh rotating shaft 20, the eighth rotating shaft 21, the ninth rotating shaft 22, the tenth rotating shaft 23, the eleventh rotating shaft 24, and the second rotating shaft 15. This rotation causes the first middle phalanx 1232, the second middle phalanx 1233, and the distal phalanx 1234 to move clockwise or counterclockwise, thereby causing the distal phalanx 1234 to bend, similar to the bending of the second phalanx of a human finger. This achieves the degree of freedom for bending the distal phalanx 1234 in the palm and finger mechanism. In one possible implementation method provided in this application embodiment, the bending angle of the distal phalanx 1234 is from 0 degrees to a preset acute angle. For example, the preset acute angle can be 76.3 degrees.

[0097] In some embodiments, based on Figures 2b, 3, and 4, the step S203 above, "controlling the finger drive component of the finger mechanism to move according to the motion information," can also be achieved through step S2032:

[0098] Step S2032: Control the movement direction of the second power push rod away from the finger joint assembly, drive the first connecting member of the finger mechanism to move counterclockwise around the first rotation axis of the finger mechanism, so that the second connecting member of the finger mechanism moves away from the palm and finger joint assembly.

[0099] The second connector drives the middle phalanx of the first palm to rotate clockwise around the second rotation axis of the finger mechanism, and the middle phalanx of the second palm to rotate clockwise around the third rotation axis of the finger mechanism, thereby driving the distal phalanx of the palm to rotate clockwise around the fourth and fifth rotation axes of the finger mechanism, so as to bend the distal phalanx of the palm.

[0100] For example, step S2032 is described below using a palm and finger mechanism as an example. As shown in Figures 3, 4, and 6, when the second power push rod 12122 moves away from the palm and finger joint assembly 123, the third connecting rod 1223 moves away from the palm and finger joint assembly 123. At this time, the third connecting rod 1223 moves counterclockwise around the third rotating shaft 16. At this time, the fourth connecting rod 1224 moves away from the palm and finger joint assembly 123. The first palm and finger middle joint 1232 rotates clockwise around the axis of the eighth rotating shaft 21, and the second palm and finger middle joint 1233 rotates clockwise around the axis of the tenth rotating shaft 23. This, in turn, drives the palm and finger end joint 1234 to rotate clockwise around the axes of the ninth rotating shaft 22 and the twelfth rotating shaft 25, so as to achieve the bending of the palm and finger end joint 1234. In this step, the third link 1223 is the first connecting member in step S2032, the third rotating shaft 16 is the first rotating shaft in step S2032, the fourth link 1224 is the second connecting member in step S2032, the eighth rotating shaft 21 is the second rotating shaft in step S2032, the tenth rotating shaft 23 is the third rotating shaft in step S2032, the ninth rotating shaft 22 is the fourth rotating shaft in step S2032, and the twelfth rotating shaft 25 is the fifth rotating shaft in step S2032.

[0101] Referring to Figures 2b, 3, and 4, the power push rod assembly 1212 further includes two first power push rods 12121, and the sliding groove assembly includes first sliding grooves symmetrically arranged along a third direction on the connecting seat 1211. The two first power push rods 12121 are respectively disposed in the two first sliding grooves. The two first connecting rods 1221 are respectively connected to the two first power push rods 12121 through a first rotating shaft 14, and are respectively connected to the two sides of the proximal phalanx 1231 along a third direction through a second rotating shaft 15. The second connecting rod 1222 is connected to the two sides of the proximal phalanx along a third direction through a third rotating shaft 16. The second connecting rod 1222 is connected to the connecting fixing plate 124 through a fourth rotating shaft 17. The outer contours of the first rotating shaft 14 and the second rotating shaft 15 are both spherical.

[0102] Since the outer contours of the first rotating shaft 14 and the second rotating shaft 15 are both spherical, in other words, the two first connecting rods 1221 are spherically hinged to the two first power push rods 12121 and the proximal metacarpophalangeal joint 1231, respectively. Here, the first connecting rod 1221 can deflect and swing relative to the first power push rod 12121 along the axial direction of the first rotating shaft 14. Correspondingly, the first connecting rod 1221 can also deflect and swing relative to the proximal metacarpophalangeal joint 1231 along the axial direction of the second rotating shaft 15 to realize the degree of freedom of the proximal metacarpophalangeal joint 1231 in the metacarpophalangeal finger mechanism. When the two first power push rods 12121 are pushed out at the same speed and in opposite directions at the same time, the proximal metacarpophalangeal joint 1231 rotates relative to the fourth rotating shaft 17, and the rotation angle is ±27 degrees.

[0103] In some embodiments, based on Figures 2b, 3, and 4, the step S203 above, "controlling the finger drive component of the finger mechanism to move according to the motion information," can also be achieved through step S2033:

[0104] Step S2033: Control one of the two first power push rods to move in a direction close to the finger joint assembly, and the other first power push rod to move in a direction close to the finger joint assembly, so that the third connector connected to the first power push rod moving in the direction close to the finger joint assembly, and the third connector connected to the first power push rod moving in the direction away from the finger joint assembly, deflect and swing.

[0105] The deflection and swing of the third connecting member corresponding to the two first power push rods can drive the proximal phalanx of the finger to deflect and swing.

[0106] For example, step S2032 is described below using a palmar finger mechanism. As shown in Figure 7, when one of the first power push rods 12121 moves in a direction close to the proximal phalanx 1231 of the palmar finger, and the other power push rod moves in a direction away from the proximal phalanx 1231, the first connecting rod 1221 connected to the first power push rod 12121 moving in the direction close to the proximal phalanx 1231 deflects and swings towards the other first connecting rod 1221. The other first connecting rod 1221 also deflects and swings in the same direction. At this time, the proximal phalanx 1231 rotates axially relative to the fourth rotating shaft 17, thereby achieving its deflection. Here, it should be noted that the direction of the deflection swing is always towards the side where the first power push rod 12121 moves in the direction away from the proximal phalanx 1231. Among them, the first connecting rod 1221 is the third connecting member in step S2033.

[0107] In this embodiment of the application, by controlling two first power push rods, the proximal phalanges of the corresponding fingers can be deflected and swung, thereby enabling the target object to be clamped by at least two deflected and swung fingers.

[0108] It is understandable that when the target object is held by the two-finger mechanism, the first power push rods of the two-finger mechanism move in opposite directions, thus achieving opposite deflection directions of the two-finger mechanism.

[0109] In some embodiments, as shown in FIG8, the step S102 above, "adjusting the current gripping posture of at least two fingers of the robotic hand based on the real-time contact information," can also be achieved through steps S801 to S803:

[0110] Step S801: Send the real-time contact information to the robot's controller.

[0111] Step S802: Receive robot hand posture adjustment information sent by the robot's controller; the robot hand posture adjustment information is determined based on the real-time contact information and the predicted contact information; the predicted contact information is obtained by inputting the size information of the target object and the image information of the target object into the trained grasping model.

[0112] Understandably, the robot's controller is a low-powered control chip (such as an MCU) and cannot perform complex algorithms. Therefore, the robot's controller needs to send real-time contact information to the robot's controller, which then determines the robot's posture adjustment information based on the real-time contact information and predicted contact information.

[0113] In some embodiments, the trained grasping model can output predicted contact information matching the size and material of the target object based on the target object's size information and image information. In this embodiment, the trained grasping model can first determine the material of the target object using its image information, and then output predicted contact information matching the target object's size and material based on its size and material.

[0114] Step S803: Based on the robot hand posture adjustment information, adjust the current gripping posture of the at least two finger mechanisms.

[0115] In this embodiment of the application, the robot hand posture adjustment information includes the joint rotation adjustment information of the finger joint component of each finger mechanism. The robot hand controller first converts the joint rotation adjustment information of the finger joint component of each finger mechanism to obtain the motion adjustment information of the finger drive component of each finger mechanism. Then, by controlling the finger drive component of each finger mechanism to move according to the motion adjustment information, the controller drives the finger joint component of the finger mechanism to rotate according to the joint rotation adjustment information.

[0116] In this embodiment, relatively accurate predicted contact information can be obtained by using the size information and image information of the target object. Then, by combining the predicted contact information with the real-time contact information generated by the robotic hand grasping the target object, the robotic hand posture adjustment information is obtained, which requires posture adjustment. Finally, the current grasping posture of at least two finger mechanisms is adjusted according to the robotic hand posture adjustment information. Because the robotic hand posture adjustment information is determined based on the predicted contact information and the real-time contact information, the posture can be adjusted based on the current grasping posture of at least two finger mechanisms, thereby reducing the time required for the at least two finger mechanisms to move again and improving the efficiency of the robotic hand grasping the target object. Furthermore, since the predicted contact information is determined by the size information and image information of the target object, the robotic hand posture adjustment information can conform to the attribute information of the target object, thereby improving the accuracy of the robotic hand grasping the target object.

[0117] This application embodiment also provides a grasping method, which is applied to the controller of a robot, the robot including the robotic arm in the above embodiments. As shown in FIG9, the grasping method can be implemented through steps S901 and S902:

[0118] Step S901: Obtain the size information of the target object.

[0119] Step S902: Determine the robot hand posture information based on the size information of the target object, and send the robot hand posture information to the controller of the robot hand.

[0120] The controller of the robotic hand is used to acquire real-time contact information generated when the robotic hand grasps the target object based on the robotic hand's posture information; based on the real-time contact information, adjust the current grasping posture of at least two finger mechanisms of the robotic hand so that the contact information generated by the at least two finger mechanisms after posture adjustment matches the predicted contact information; the real-time contact information is collected by at least one of the tactile sensors disposed on the at least two finger mechanisms and the tactile sensors disposed on the palm of the robotic hand.

[0121] In this embodiment, the robot controller first determines the robot hand posture information for the robot hand to initially grasp the target object based on the size information of the target object, and then adjusts the robot hand posture information based on the real-time contact information generated when the robot hand grasps the target object, so that the robot hand can accurately grasp the object.

[0122] In some embodiments, as shown in FIG10, the above-described grasping method can also be implemented through steps S1001 to S1005:

[0123] Step S1001: Obtain environmental image information corresponding to the robotic arm; the environmental image information is acquired by an image acquisition device set on the back of the robotic arm.

[0124] Here, the environmental image information can be three-dimensional image information, which can be acquired by an image acquisition device installed on the back of the robotic arm. This image acquisition device can be a depth camera.

[0125] As shown in Figure 2b, the image acquisition device 271 is mounted on the back of the hand of the robotic arm 1 (not shown in the figure).

[0126] Step S1002: Based on the environmental image information, determine the target object and the motion trajectory between the robot arm and the target object.

[0127] In this embodiment, the robot's controller can perform semantic analysis on environmental image information to identify the target object. Then, based on the environmental image information, it performs 3D scene reconstruction and determines the 3D coordinates of the target object in the virtual 3D scene. Finally, based on the 3D coordinates of the robot arm and the target object, a trajectory generation algorithm is used to obtain the motion trajectory between the robot arm and the target object. Exemplarily, this trajectory generation algorithm can be based on optimization-based trajectory planning.

[0128] Step S1003: During the process of controlling the robotic arm to move along the motion trajectory, the motion trajectory is corrected based on the real-time image information acquired by the image acquisition device and the distance information between the target object and the robotic arm.

[0129] Here, the distance information is collected by a distance acquisition device located in the accommodating space of the robotic hand's palm. For example, this distance acquisition device can be a laser rangefinder or a Time of Flight (ToF) sensor.

[0130] As shown in Figure 2a, the robotic arm 1 includes a body 11, which forms a receiving space, and the distance acquisition device 261 is disposed in the receiving space.

[0131] Understandably, when the robotic arm is far from the target object, the image acquisition device mounted on the back of the robotic arm can capture real-time image information of the target object. Therefore, as the robotic arm moves along its trajectory, the trajectory can be corrected based on the real-time image information captured by the image acquisition device. This real-time image information includes the target object and its surrounding environment; thus, the trajectory can be corrected in real time during movement to avoid obstacles in the surrounding environment.

[0132] When the robotic arm approaches the target object, that is, when the distance between the robotic arm and the target object is relatively close, the image acquisition device is located on the back of the robotic arm, so it may not be able to acquire real-time image information including the target object. At this time, the distance acquisition device located in the palm can be used to acquire the distance information between the target object and the robotic arm, and the movement trajectory can be corrected again based on the distance information until the robotic arm is in a space where it can grasp the target object.

[0133] In this embodiment, the motion trajectory can be updated in real time using real-time image information acquired by an image acquisition device mounted on the back of the robotic arm and distance information between the target object and the robotic arm acquired by a distance acquisition device mounted in the accommodating space of the robotic arm's palm. This allows the robotic arm to avoid obstacles in the surrounding environment during movement, enabling it to successfully reach a space where it can grasp the target object.

[0134] In some embodiments, the step S1003 above, "correcting the motion trajectory based on the real-time image information acquired by the image acquisition device and the distance information between the target object and the robotic arm," can be achieved through steps S10031 and S10032:

[0135] Step S10031: Correct the motion trajectory based on the real-time image information acquired by the image acquisition device.

[0136] It is understandable that correcting the motion trajectory using real-time image information means that the distance between the robotic arm and the target object is relatively far, and the target object is included in the real-time image information acquired by the image acquisition device. Therefore, in this stage, the motion trajectory of the robotic arm can be corrected using the real-time image information acquired by the image acquisition device.

[0137] In some embodiments, as shown in FIG2b, the robotic arm 1 further includes a supplementary lighting element 272, which is disposed on the top of the image acquisition device 271. Exemplarily, the supplementary lighting element 272 can be an LED supplementary light. When the robotic arm grasps an object in a dimly lit environment, the supplementary lighting element can be turned on, thereby enabling the image acquisition device to acquire relatively clear real-time image information even in dimly lit environments.

[0138] Step S10032: If the target object cannot be identified based on the real-time image information, the motion trajectory is corrected again based on the distance information between the target object and the robotic arm.

[0139] Under normal circumstances, when a robotic arm grasps an object, its palm faces the object, while the image acquisition device is located on the back of the hand. As the robotic arm approaches the target object, the palm obstructs the image acquisition device, preventing it from capturing an image of the target object. Therefore, when the target object cannot be identified based on real-time image information, it indicates that the robotic arm has approached it. At this point, because the real-time image information acquired by the image acquisition device cannot identify the target object, it cannot continue to correct its trajectory based on real-time image information. Therefore, it is necessary to use a distance acquisition device located on the palm to acquire the distance information between the target object and the robotic arm. The robotic arm continues its trajectory based on this distance information until it is in a space where it can grasp the target object.

[0140] Step S1004: Determine whether the robotic arm is in a space where it can grasp the target object.

[0141] Step S1005: Control the robotic arm to grasp the target object.

[0142] In this embodiment, after correcting the motion trajectory, the robotic arm can be controlled to move along the corrected trajectory, and it can be determined in real time whether the robotic arm is within the space where it can grasp the target object. If the robotic arm is not within the space where it can grasp the target object, step S1003 is executed; if the robotic arm is within the space where it can grasp the target object, step S1005 is executed.

[0143] In this embodiment, the distance information between the target object and the robotic arm can be used to determine whether the robotic arm is within the space where it can grasp the target object. Specifically, when the distance information is less than or equal to a preset distance, it is determined that the robotic arm is within the space where it can grasp the target object; when the distance information is greater than the preset distance, it is determined that the robotic arm is not within the space where it can grasp the target object.

[0144] In this embodiment of the application, the robot's controller can send a preset grasping posture to the robot hand's controller, so that the robot hand's controller adopts the preset grasping posture and controls the robot hand's two finger mechanisms and palm to grasp the target object.

[0145] In this embodiment, during the initial stage of the robotic arm grasping the target object, real-time image information can be used to correct the motion trajectory. When the robotic arm approaches the target object, i.e., when the real-time image information can no longer identify the target object, the distance information between the target object and the robotic arm can be used to correct the motion trajectory again. In this way, different data can be used to correct the robotic arm's motion trajectory at different stages of grasping the target object, thereby improving the accuracy of the motion trajectory correction.

[0146] In some embodiments, as shown in FIG11, the above-described grasping method can also be implemented through steps S1101 to S1104:

[0147] Step S1101: Receive real-time contact information generated when the robot arm grasps the target object, sent by the controller of the robot arm.

[0148] Step S1102: Input the size information and image information of the target object into the trained grasping model to obtain the predicted contact information.

[0149] It's understandable that objects of different materials, even if the same size, will have different weights. Similarly, objects of different sizes, even if the same material, will have different weights. Therefore, for objects of different weights, the robotic arm needs to apply different contact pressures to grasp them.

[0150] Therefore, in this embodiment, a first sample set and a second sample set can be constructed. The image sample set includes multiple sample images, and each sample image contains a label representing the material of the object in the sample image. The image sample set can be used to train a grasping model, enabling the trained grasping model to determine the material of the target image based on the image information of the target object. The second sample set includes sample information of different materials and sizes, as well as contact information for each sample. The second sample set can be used to train a grasping model, enabling the trained grasping model to determine and predict contact information based on the size and material of the target object.

[0151] In this embodiment, the predicted contact information includes contact information corresponding to different contact positions. It is understood that when a robotic arm grasps an object, different parts of the robotic arm need to output contact pressure; therefore, the contact positions in the predicted contact information can correspond to the parts of the robotic arm that need to output contact pressure.

[0152] Step S1103: Compare the predicted contact information with the real-time contact information to obtain contact difference information.

[0153] In this embodiment, the real-time contact information also includes the contact pressure corresponding to different contact positions. When comparing the predicted contact information with the real-time contact information, it is necessary to compare the predicted contact information and the real-time contact information at the corresponding positions. That is, the contact difference information includes the contact difference information corresponding to different contact positions. For example, the predicted contact information includes a contact pressure of 2 Pa at position 1, a contact pressure of 3 Pa at position 2, and a contact pressure of 4 Pa ​​at position 3. The real-time contact information includes a contact pressure of 0.5 Pa at position 1 and a contact pressure of 2 Pa at position 3. Then the contact difference information can include: a contact difference of 1.5 Pa at position 1, a contact pressure of 3 Pa at position 2, and a contact pressure of 2 Pa at position 3.

[0154] Step S1104: Based on the contact difference information, generate the robot hand posture adjustment information and send the robot hand posture adjustment information to the robot hand's controller.

[0155] In this embodiment, the robot hand posture adjustment information includes joint rotation adjustment information of the finger knuckle components of the target finger mechanism in at least two finger mechanisms. The target finger mechanism refers to the finger mechanism requiring posture adjustment.

[0156] It is understood that the contact position in the contact difference information refers to the distribution position information of the tactile sensors on the robotic arm. Therefore, the target finger mechanism can be determined from at least two finger mechanisms by using the contact position in the contact difference information. For example, when the at least two finger mechanisms include finger mechanism 1, finger mechanism 2, finger mechanism 3, and finger mechanism 4, if the contact position in the contact difference information is position 1, position 2, and position 3, and if the tactile sensor corresponding to position 1 and position 2 is on finger mechanism 1, and the tactile sensor corresponding to position 3 is on finger mechanism 2, then finger mechanism 1 and finger mechanism 2 can be determined as the target finger mechanism.

[0157] Regarding joint rotation adjustment information, since different contact pressures correspond to the rotation information of the finger knuckle assembly, the rotation information of the finger knuckle assembly can be determined through the contact pressure in the contact difference information.

[0158] In this embodiment, a trained grasping model can determine predicted contact information based on the size and image information of the target object. This allows the trained grasping model to accurately determine predicted contact information that matches the material and size of the target object. Then, the contact difference information between the predicted and real-time contact information is determined. Finally, based on the contact difference information, robot hand posture adjustment information is generated. Thus, by transforming the contact difference information between the predicted and real-time contact information, the posture adjustment information required by the robot hand can be obtained, enabling accurate grasping of the target object.

[0159] In some embodiments, as shown in FIG12, the above-described grasping method can also be implemented through steps S1201 to S1214:

[0160] In step S1201, the controller of the dexterous hand acquires RGB images and depth image information of the surrounding environment of the dexterous hand through a depth camera.

[0161] In low-light environments, the LED fill light on the back of the hand is also turned on to compensate for image problems caused by insufficient light.

[0162] In step S1202, the dexterous hand's controller sends the RGB image and depth image information of the dexterous hand's surrounding environment to the robot's main controller.

[0163] The image information can be fed back to the robot's controller in real time via a high-speed USB data cable.

[0164] In step S1203, the robot body controller performs 3D reconstruction of the scene based on the RGB and depth image information of the surrounding environment, and performs semantic analysis on the RGB and depth image information of the surrounding environment to determine the grasping objects in the surrounding environment and identify the names of each object in the scene.

[0165] In step S1204, the robot body controller calculates the distance information from the object to the camera, and combines it with the coordinates in the 2D plane to calculate the three-dimensional coordinates of the corresponding pixel points of the grasped object features in the virtual three-dimensional space.

[0166] In step S1205, the robot body controller calculates the motion trajectory between the dexterous hand coordinate system and the object coordinate system based on the obtained three-dimensional coordinates.

[0167] In step S1206, during the movement along the trajectory, the robot body controller uses images acquired in real time by the depth camera to identify obstacles in the path, enabling the dexterous hand to avoid them and thus guiding the dexterous hand to the vicinity of the target object.

[0168] In step S1207, if no object is identified in the image acquired by the depth camera, the robot body controller sends an instruction to the dexterous hand controller to acquire object distance information and three-dimensional data of the object being measured.

[0169] In step S1208, the dexterous hand controller responds to the instruction, acquires the object distance information and the three-dimensional data of the measured object collected by the TOF sensor, and sends the object distance information and the three-dimensional data of the measured object to the robot body controller.

[0170] In step S1209, during the movement of the dexterous hand, the robot body controller continuously corrects the motion trajectory based on the object distance information obtained by the TOF sensor and the three-dimensional data of the object being measured, and adjusts the movement position and speed of the dexterous hand in real time so that the dexterous hand can approach the object.

[0171] In step S1210, during the process of aligning the grasping space of the dexterous hand with the shape space of the target object, the robot body controller sends a grasping posture corresponding to the size of the grasped object to the controller of the dexterous hand.

[0172] In step S1211, the dexterous hand controller initiates a closing motion of the dexterous hand based on the grasping posture corresponding to the size of the grasped object sent by the robot body controller, thereby enveloping the object within the palm.

[0173] Here, alignment of the dexterous hand's grasping space with the target object's shape space means that the dexterous hand's coordinates coincide with the object's coordinates, the dexterous hand's coordinates are at the endpoint of the motion trajectory, and the object is in contact with at least one of the following:

[0174] In step S1212, the controller of the dexterous hand acquires the contact pressure information collected by the tactile sensor and sends the contact pressure information to the robot body controller.

[0175] In step S1213, the robot body controller compares the contact pressure information with the grasping dataset to obtain the contact difference, and sends the contact difference to the dexterous hand controller.

[0176] Step S1213 can be achieved through the following steps:

[0177] In step S12131, the robot body controller inputs the object's size information and image information into the grasping large model to obtain a suitable contact force applied to each contact point.

[0178] In step S12132, the robot body controller compares the contact pressure information collected by the sensor with the contact force output by the grasping large model to obtain the contact difference, and sends the contact difference to the controller of the dexterous hand.

[0179] In step S1214, the controller of the dexterous hand controls the angle of each active joint of the dexterous hand based on the contact difference and adjusts the contact pressure of the tactile sensors in each part.

[0180] In this embodiment, the controller of the dexterous hand adjusts the contact pressure of the tactile sensors in various parts, which makes the dexterous hand's gripping force for objects of different materials and shapes reasonably controllable, ensuring a firm grip and safety.

[0181] In some embodiments, as shown in FIG13, this application embodiment also provides a robotic hand 1, which includes a controller 1301, at least two finger mechanisms 1302 (two finger mechanisms 1302 are shown in FIG13), and a palm portion 1303, wherein:

[0182] The controller 1301 of the robotic arm is used to acquire real-time contact information generated when the robotic arm grasps the target object during the process of the robotic arm grasping the target object.

[0183] The controller 1301 of the robotic hand is used to adjust the current grasping posture of at least two finger mechanisms of the robotic hand based on the real-time contact information, so that the contact information generated by the at least two finger mechanisms after posture adjustment matches the predicted contact information.

[0184] The real-time contact information is collected by tactile sensors mounted on the at least two finger mechanisms 1302, or the real-time contact information is collected jointly by tactile sensors mounted on the at least two finger mechanisms and tactile sensors mounted on the palm part 1303 of the robotic hand.

[0185] In some embodiments, the controller 1301 of the robotic hand is further configured to receive robotic hand posture information sent by the controller of the robot; the robotic hand posture information is determined by the controller of the robot based on the size information of the target object; the robotic hand posture information includes joint rotation information of the finger knuckle assembly of each finger mechanism; for each finger mechanism, motion information of the finger drive assembly of the finger mechanism is determined based on the joint rotation information of the finger knuckle assembly of the finger mechanism; for each finger mechanism, the finger drive assembly of the finger mechanism is controlled to move according to the motion information, so as to drive the finger knuckle assembly of the finger mechanism to rotate according to the joint rotation information.

[0186] In some embodiments, the finger drive assembly includes two first power push rods; the finger joint assembly includes a proximal phalanx of the finger; the motion information includes a motion direction close to the finger joint assembly; the controller 1301 of the robotic hand is further configured to control the two first power push rods to move simultaneously along the motion direction close to the finger joint assembly, thereby driving the proximal phalanx of the finger to rotate around a first rotation axis, causing the proximal phalanx of the finger to bend towards the side closer to the palm.

[0187] In some embodiments, the finger joint assembly includes a first metacarpophalangeal mid-phalangeal joint, a second metacarpophalangeal mid-phalangeal joint, and a distal phalangeal joint; the finger drive assembly includes a second power push rod; the motion information includes a motion direction away from the finger joint assembly; the controller 1301 of the robotic hand is further configured to control the second power push rod to move away from the finger joint assembly in the motion direction, thereby driving the first connector of the finger mechanism to move counterclockwise around the first rotation axis of the finger mechanism, causing the second connector of the finger mechanism to move away from the metacarpophalangeal joint assembly; wherein, the second connector drives the first metacarpophalangeal mid-phalangeal joint to rotate clockwise around the second rotation axis of the finger mechanism, and the second metacarpophalangeal mid-phalangeal joint to rotate clockwise around the third rotation axis of the finger mechanism, thereby driving the distal phalangeal joint to rotate clockwise around the fourth and fifth rotation axes of the finger mechanism, so as to bend the distal phalangeal joint.

[0188] In some embodiments, the motion information includes a direction away from the finger joint assembly; the controller 1301 of the robotic hand is further configured to control one of the two first power push rods to move in a direction close to the finger joint assembly, and the other first power push rod to move in a direction away from the finger joint assembly, such that the third connector connected to the first power push rod moving in the direction close to the finger joint assembly, and the third connector connected to the first power push rod moving in the direction away from the finger joint assembly, deflect and swing; wherein the deflection and swing of the third connectors corresponding to the two first power push rods can drive the proximal phalanx of the finger to deflect and swing.

[0189] In some embodiments, the controller 1301 of the robotic hand is further configured to send the real-time contact information to the controller of the robot; receive robotic hand posture adjustment information sent by the controller of the robot; the robotic hand posture adjustment information is determined based on the real-time contact information and the predicted contact information; the predicted contact information is obtained by inputting the size information of the target object and the image information of the target object into the trained grasping model; and adjust the current grasping posture of the at least two finger mechanisms based on the robotic hand posture adjustment information.

[0190] In some embodiments, as shown in FIG14, this application embodiment also provides a robot 1400, which includes a controller 1401 and a robotic arm 1, wherein:

[0191] The robot's controller 1401 is used to acquire the size information of the target object;

[0192] The robot's controller 1401 is used to determine the robot hand's posture information based on the size information of the target object, and send the robot hand's posture information to the robot hand's controller.

[0193] The controller of the robotic hand is used to acquire real-time contact information generated when the robotic hand grasps the target object based on the robotic hand's posture information; based on the real-time contact information, adjust the current grasping posture of at least two finger mechanisms of the robotic hand so that the contact information generated by the at least two finger mechanisms after posture adjustment matches the predicted contact information; the real-time contact information is collected by at least one of the tactile sensors disposed on the at least two finger mechanisms and the tactile sensors disposed on the palm of the robotic hand.

[0194] The robot's controller 1401 is also used to acquire environmental image information corresponding to the robotic hand; the environmental image information is acquired by an image acquisition device installed on the back of the robotic hand; based on the environmental image information, the target object and the motion trajectory between the robotic hand and the target object are determined; during the process of controlling the robotic hand to move along the motion trajectory, the motion trajectory is corrected based on the real-time image information acquired by the image acquisition device and the distance information between the target object and the robotic hand, until the robotic hand is in a space where it can grasp the target object; the distance information is acquired by a distance acquisition device installed in the accommodating space of the robotic hand's palm.

[0195] The robot's controller 1401 is also used to correct the motion trajectory based on the real-time image information acquired by the image acquisition device; and if the target object cannot be identified based on the real-time image information, to correct the motion trajectory again based on the distance information between the target object and the robot arm.

[0196] The robot's controller 1401 is further configured to receive real-time contact information generated when the robot grasps a target object, sent by the robot's controller; input the size information and image information of the target object into the trained grasping model to obtain predicted contact information; compare the predicted contact information with the real-time contact information to obtain contact difference information; generate robot posture adjustment information based on the contact difference information, and send the robot posture adjustment information to the robot's controller.

[0197] This application provides a gripping device, which is applied to the controller of a robotic arm; the gripping device includes:

[0198] The first acquisition unit is used to acquire real-time contact information generated when the robotic arm grasps the target object during the process of the robotic arm grasping the target object.

[0199] An adjustment unit is used to adjust the current grasping posture of at least two fingers of the robotic hand based on the real-time contact information, so that the contact information generated by the at least two fingers after posture adjustment matches the predicted contact information.

[0200] The real-time contact information is collected by at least one of the tactile sensors located on the at least two finger mechanisms and the tactile sensors located on the palm of the robotic hand.

[0201] This application provides a gripping device applied to the controller of a robot including a robotic arm; the gripping device includes:

[0202] The second acquisition unit is used to acquire the size information of the target object;

[0203] A determining unit is used to determine the robot arm's posture information based on the size information of the target object;

[0204] A sending unit is used to send the robot arm's posture information to the robot arm's controller;

[0205] The controller of the robotic hand is used to acquire real-time contact information generated when the robotic hand grasps the target object based on the robotic hand's posture information; based on the real-time contact information, adjust the current grasping posture of at least two finger mechanisms of the robotic hand so that the contact information generated by the at least two finger mechanisms after posture adjustment matches the predicted contact information; the real-time contact information is collected by at least one of the tactile sensors disposed on the at least two finger mechanisms and the tactile sensors disposed on the palm of the robotic hand.

[0206] The descriptions of the apparatus embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the apparatus provided in this disclosure can be used to perform the methods described in the method embodiments above. For technical details not disclosed in the apparatus embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0207] It should be noted that, in the embodiments of this application, if the above-described data processing method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0208] This application provides a robot including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above-described method.

[0209] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the above-described method. The computer-readable storage medium can be transient or non-transient.

[0210] This application provides a computer program including computer-readable code. When the computer-readable code is executed in a computer device, the processor in the computer device performs some or all of the steps for implementing the above-described method. The computer device can be a robot or a device deployed on a robot, and is not limited thereto.

[0211] This application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0212] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0213] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0214] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0215] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Industrial applicability

[0216] This application provides a grasping method, a robotic hand, and a robot. During the process of the robotic hand grasping a target object, the controller of the robotic hand can acquire real-time contact information generated during the grasping process. Based on this real-time contact information, the controller adjusts the current grasping posture of at least two fingers of the robotic hand, ensuring that the contact information generated by the adjusted fingers matches the predicted contact information. In this way, by adjusting the current grasping posture of at least two fingers using both predicted and real-time contact information, the adjusted fingers ultimately match the predicted contact information when grasping the object. This allows the robotic hand to adopt a suitable grasping posture to grasp the target object, thereby improving the accuracy of the robotic hand's object grasping.

Claims

1. A grasping method, the grasping method being applied to a controller of a robotic arm; the method comprising: During the process of the robotic arm grasping the target object, real-time contact information generated when the robotic arm grasps the target object is acquired; Based on the real-time contact information, the current grasping posture of at least two fingers of the robotic hand is adjusted so that the contact information generated by the adjusted at least two fingers matches the predicted contact information. The real-time contact information is collected by at least one of the tactile sensors located on the at least two finger mechanisms and the tactile sensors located on the palm of the robotic hand.

2. The grasping method according to claim 1, wherein, Each of the at least two finger mechanisms includes a finger driving component and a finger knuckle component; the method further includes: The robot receives robotic hand posture information sent by the robot's controller; the robotic hand posture information is determined by the robot's controller based on the size information of the target object; the robotic hand posture information includes joint rotation information of the finger knuckle components of each finger mechanism; For each of the finger mechanisms, the motion information of the finger drive component of the finger mechanism is determined based on the joint rotation information of the finger knuckle component of the finger mechanism; For each of the aforementioned finger mechanisms, the finger drive component of the finger mechanism is controlled to move according to the motion information, so as to drive the finger joint component of the finger mechanism to rotate according to the joint rotation information.

3. The grasping method according to claim 2, wherein, The finger actuation assembly includes at least one of the following: two first power push rods and a second power push rod; the finger joint assembly includes a proximal finger joint, and / or the finger joint assembly includes a first metacarpophalangeal mid-joint, a second metacarpophalangeal mid-joint, and a distal finger joint; The finger drive component that controls the finger mechanism moves according to the motion information, including at least one of the following: When the motion information includes a motion direction close to the finger joint assembly, the two first power push rods are controlled to move simultaneously along the motion direction close to the finger joint assembly, causing the proximal finger joint to rotate around the first rotation axis, so that the proximal finger joint bends towards the side closer to the palm. When the motion information includes a motion direction away from the finger joint assembly, the second power push rod is controlled to move away from the motion direction of the finger joint assembly, causing the first connecting member of the finger mechanism to move counterclockwise around the first rotation axis of the finger mechanism, causing the second connecting member of the finger mechanism to move away from the palm joint assembly; wherein, the second connecting member causes the first palm joint to rotate clockwise around the second rotation axis of the finger mechanism, and the second palm joint to rotate clockwise around the third rotation axis of the finger mechanism, thereby causing the distal palm joint to rotate clockwise around the fourth and fifth rotation axes of the finger mechanism, so as to bend the distal palm joint.

4. The grasping method according to claim 3, wherein, The motion information includes the direction away from the finger joint assembly; The finger drive component that controls the finger mechanism moves according to the motion information, including: Control one of the two first power push rods to move in a direction close to the finger joint assembly and the other first power push rod to move in a direction away from the finger joint assembly, so that the third connector connected to the first power push rod moving in the direction close to the finger joint assembly and the third connector connected to the first power push rod moving in the direction away from the finger joint assembly deflect and swing. The deflection and swing of the third connecting member corresponding to the two first power push rods can drive the proximal phalanx of the finger to deflect and swing.

5. The grasping method according to any one of claims 1 to 4, wherein, The step of adjusting the current gripping posture of at least two fingers of the robotic hand based on the real-time contact information includes: Send the real-time contact information to the robot's controller; The robot receives robotic arm posture adjustment information sent by the robot's controller; the robotic arm posture adjustment information is determined based on the real-time contact information and the predicted contact information; the predicted contact information is obtained by inputting the size information and image information of the target object into the trained grasping model; Based on the robotic hand posture adjustment information, the current gripping posture of the at least two finger mechanisms is adjusted.

6. A grasping method, the grasping method being applied to a controller of a robot including a robotic arm; the method comprising: Obtain the size information of the target object; The robot arm's posture information is determined based on the size information of the target object, and then sent to the robot arm's controller. The controller of the robotic hand is used to acquire real-time contact information generated when the robotic hand grasps the target object based on the robotic hand's posture information; based on the real-time contact information, adjust the current grasping posture of at least two finger mechanisms of the robotic hand so that the contact information generated by the at least two finger mechanisms after posture adjustment matches the predicted contact information; the real-time contact information is collected by at least one of the tactile sensors disposed on the at least two finger mechanisms and the tactile sensors disposed on the palm of the robotic hand.

7. The grasping method according to claim 6, wherein, The method further includes at least one of the following: The environmental image information corresponding to the robotic arm is acquired; the environmental image information is acquired by an image acquisition device installed on the back of the robotic arm; based on the environmental image information, the target object and the motion trajectory between the robotic arm and the target object are determined; During the process of controlling the robotic arm to move along the motion trajectory, the motion trajectory is corrected based on the real-time image information acquired by the image acquisition device and the distance information between the target object and the robotic arm, until the robotic arm is in a space where it can grasp the target object; the distance information is acquired by a distance acquisition device located in the accommodating space of the robotic arm's palm. The system receives real-time contact information generated when the robotic arm grasps a target object, sent by the controller of the robotic arm; inputs the size information and image information of the target object into the trained grasping model to obtain predicted contact information; compares the predicted contact information with the real-time contact information to obtain contact difference information; generates the robotic arm posture adjustment information based on the contact difference information, and sends the robotic arm posture adjustment information to the controller of the robotic arm.

8. The grasping method according to claim 7, wherein, The step of correcting the motion trajectory based on the real-time image information acquired by the image acquisition device and the distance information between the target object and the robotic arm includes: The motion trajectory is corrected based on the real-time image information acquired by the image acquisition device; If the target object cannot be identified based on the real-time image information, the motion trajectory is corrected again based on the distance information between the target object and the robotic arm.

9. A robotic hand, the robotic hand comprising a controller, at least two finger mechanisms, and a palm portion; The controller of the robotic arm is used to acquire real-time contact information generated when the robotic arm grasps the target object during the process of the robotic arm grasping the target object; The controller of the robotic hand is used to adjust the current grasping posture of at least two finger mechanisms of the robotic hand based on the real-time contact information, so that the contact information generated by the at least two finger mechanisms after posture adjustment matches the predicted contact information. in, The real-time contact information is collected by at least one of the tactile sensors located on the at least two finger mechanisms and the tactile sensors located on the palm of the robotic hand.

10. A robot, the robot comprising a controller and a robotic arm; The robot's controller is used to acquire the size information of the target object; The robot's controller is used to determine the robot hand's posture information based on the size information of the target object, and send the robot hand's posture information to the robot hand's controller; in, The controller of the robotic hand is used to acquire real-time contact information generated when the robotic hand grasps the target object based on the robotic hand's posture information; based on the real-time contact information, adjust the current grasping posture of at least two finger mechanisms of the robotic hand so that the contact information generated by the at least two finger mechanisms after posture adjustment matches the predicted contact information; the real-time contact information is collected by at least one of the tactile sensors disposed on the at least two finger mechanisms and the tactile sensors disposed on the palm of the robotic hand.

Images