Robot dexterous hand grabbing control system and grabbing method

Abstract

The invention discloses a robot dexterous hand grabbing control system and method. The grabbing control system comprises a controller, a dexterous hand, an image extraction device and a pressure sensor. Each finger of the dexterous hand is provided with a pressure sensor, and an image extraction device is installed on the robot and arranged at the position where the dexterous hand and a target object can be completely observed. The dexterous hand is connected with a dexterous hand action control device, and the dexterous hand action control device, the image extraction device and the pressure sensors are all in signal connection with the controller. According to the control system, the pressure sensor and the image extraction device are combined, the target object grabbing state of the dexterous hand is recognized and judged, and the problems that in existing robot grabbing related technologies, a grabbing control system is complex in structure, high in cost and the like are solved.

Description

Technical Field

[0001] The present application belongs to the field of robotics, specifically to the field of humanoid robot technology, and more specifically to a grasping control system and a grasping method of a dexterous robot hand. Background Art

[0002] Humanoid robots, due to their humanoid appearance and versatility, have demonstrated unique advantages in many fields. Humanoid robots have the advantages of strong adaptability, high flexibility, and multiple functions, and are being accepted by more and more fields.

[0003] Compared with traditional robots, the biggest advantage of humanoid robots is their grasping ability and grasping adaptability. Existing humanoid robots use a visual-tactile gripping solution for grasping dexterous hands. Generally, cameras and flexible contact bodies are installed on the fingertips of each dexterous finger of the dexterous hand. The camera takes a picture of the soft body and uses the soft body image obtained by the camera to identify the deformation of the flexible contact body to determine the size of the gripping force and determine whether to grasp or not. This gripping solution has the following two main shortcomings: (1) Because cameras are installed on the fingertips of each dexterous finger, the camera installation is difficult and the structure is complex; (2) All ten dexterous fingers need to be equipped with visual-tactile modules, which is expensive. Summary of the Invention

[0004] One of the purposes of this application is to provide a robot dexterous hand grasping control system, which uses a combination of pressure sensors and image extraction devices to identify and judge the state of the dexterous hand grasping the target object, thereby solving the defects of the existing robot grasping related technologies such as complex control system structure and high cost.

[0005] One of the purposes of this application is to provide a grasping method for a robot dexterous hand, in which the sliding characteristics of the target object after being grasped are obtained through a pressure sensor, and whether the target object is successfully grasped is accurately judged, or the grasping force is changed to ensure that the target object is effectively grasped.

[0006] The technical solution of the present application is a robot dexterous hand grasping control system, comprising a controller, a dexterous hand, an image acquisition device and a pressure sensor;

[0007] The pressure sensor is installed on each finger of the dexterous hand, and the image capture device is installed on the robot and placed in a position where the dexterous hand and the target object can be fully observed;

[0008] The dexterous hand is connected to a dexterous hand motion control device, and the dexterous hand motion control device, the image capture device and each of the pressure sensors are all connected to the controller signal.

[0009] As an optional implementation, the pressure sensor is a flexible pressure sensor, and each finger of the dexterous hand is covered with a flexible pressure sensor; when the dexterous hand grasps a target object, the target object applies pressure to the flexible pressure sensor.

[0010] A robot dexterous hand grasping method, which adopts the aforementioned robot dexterous hand grasping control system to grasp, comprises the following steps:

[0011] The dexterous hand first grasps and moves the target object. It then determines whether the pressure sensor receives a pressure signal, whether the target object moves, whether relative sliding occurs between the target object and the dexterous hand, and finally whether the contact surface between the target object and the dexterous hand deforms within a specified range.

[0012] Only when the contact surface between the target object and the dexterous hand deforms within a specified range is the grasping judged to be successful; otherwise, it is judged to be unsuccessful.

[0013] As an optional implementation, if the pressure sensor does not obtain a pressure signal, it is determined that the grasping is unsuccessful; if the pressure sensor obtains a pressure signal, it is determined whether the target object has been displaced.

[0014] As an optional implementation, if the target object does not move, it is determined that the grasping is unsuccessful; if the target object moves, it is determined whether relative sliding occurs between the target object and the dexterous hand;

[0015] As an optional implementation, if relative sliding occurs between the target object and the dexterous hand, the grasping force is changed and the grasping continues; if relative sliding does not occur between the target object and the dexterous hand, it is determined whether the contact surface between the target object and the dexterous hand is deformed within a specified range.

[0016] As an optional implementation, if the contact surface of the target object with the dexterous hand is deformed, and the deformation size is greater than the set maximum deformation value or the deformation size is less than the set minimum deformation value, it is determined that the grasping is not successful;

[0017] If the contact surface of the target object with the dexterous hand is deformed, and the deformation size is greater than the set minimum deformation value and less than the set maximum deformation value, it is judged that the grasping is successful.

[0018] As an optional implementation, after the dexterous hand grasps the target object, the pressure sensor obtains the pressure signal of the target object on the dexterous hand in real time. The pressure signal is an electrical signal. If the electrical signal of the pressure sensor shows a characteristic frequency of relative sliding, it is determined that relative sliding occurs between the target object and the dexterous hand. If the electrical signal of the pressure sensor does not show a characteristic frequency of relative sliding, it is determined that no relative sliding occurs between the target object and the dexterous hand.

[0019] As an optional embodiment, the robot dexterous hand grasping method includes the following steps:

[0020] Step S13: The dexterous hand grasps and moves the target object;

[0021] Step S14: Determine whether the pressure sensor senses a pressure signal; if the pressure sensor does not sense a pressure signal, return to step S13; if the pressure sensor senses a pressure signal, proceed to the following step S15;

[0022] Step S15. The image capture device obtains the position of the target object and determines whether the target object has moved; if the target object has moved, the process proceeds to the following step S16; if the target object has not moved, the process proceeds to the following step S19;

[0023] Step S16. The pressure sensor acquires in real time a pressure signal from the target object to the dexterous hand, the pressure signal being an electrical signal. If the electrical signal from the pressure sensor exhibits a characteristic frequency of relative sliding, it is determined that relative sliding has occurred between the target object and the dexterous hand, and the process proceeds to Step S24. If the electrical signal from the pressure sensor does not exhibit a characteristic frequency of relative sliding, it is determined that relative sliding has not occurred between the target object and the dexterous hand, and the process proceeds to Step S17.

[0024] Step S17. The image capture device acquires an image of the surface of the target object and, based on the acquired image of the surface of the target object, determines whether the contact surface of the target object with the dexterous hand has deformed within a specified range. If the contact surface of the target object with the dexterous hand has deformed within the specified range, the process proceeds to Step S18. If the contact surface of the target object with the dexterous hand has deformed outside the specified range, the process proceeds to Step S25.

[0025] Step S18: If the contact surface of the target object with the dexterous hand deforms, and the deformation is greater than the set minimum deformation value and less than the set maximum deformation value, then it is determined that the grasping is successful and the grasping is completed;

[0026] Step S19. The image capture device obtains the position of the dexterous hand in real time and determines whether the dexterous hand has moved based on the position of the dexterous hand; if the dexterous hand has moved, the process proceeds to the following step S20; if the dexterous hand has not moved, the process proceeds to the following step S22;

[0027] Step S20: If it is determined that the grasping force is less than the set grasping force or the dexterous hand fails to grasp the target object, proceed to step S21;

[0028] Step S21: increasing the gripping force according to the pressure signal characteristics obtained by the pressure sensor, and returning to step S13;

[0029] Step S22: If the moving force is less than the set minimum moving force, proceed to step S23;

[0030] Step S23. Increase the moving force and return to step S13;

[0031] Step S24: Increase the gripping force and return to step S13;

[0032] Step S25: If the contact surface of the target object with the dexterous hand is deformed and the deformation is greater than the set maximum deformation value, the grasping force is reduced and the process returns to step S13.

[0033] As an optional implementation, before the dexterous hand grasps and moves the target object, the following operations are also performed:

[0034] Step S10: Establish a database of the target object's shape, size, weight, volume, and surface roughness, and estimate the rated gripping force and rated moving force required for the target object corresponding to the information;

[0035] Step S11. Obtaining basic information of the target object, wherein the basic information of the target object includes at least the size, volume, weight and surface roughness of the target object;

[0036] Step S12: Estimate the rated grasping force and rated moving force required to grasp the target object based on the obtained basic information of the target object.

[0037] Compared with the existing technology, the beneficial effects of the robot dexterous hand grasping control system of the technical solution of this application are:

[0038] This control system uses a combination of pressure sensors and image capture devices to identify and judge the state of the dexterous hand grasping the target object. The judgment is accurate. The combined use of pressure sensors and image capture devices avoids the problem of installing cameras on each finger, reduces costs and simplifies control.

[0039] Compared with the prior art, the beneficial effects of the robot dexterous hand grasping method of the technical solution of this application are:

[0040] The grabbing process control is simple, the control steps are few, the grabbing efficiency is high, the sliding characteristics of the target object after being grabbed are obtained through the pressure sensor, it is accurately judged whether the target object is successfully grabbed or not, or the grabbing force is changed, and the target object is effectively grabbed. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0042] Figure 1 The system block diagram of the robot dexterous hand grabbing control system embodiment of the present application.

[0043] Figure 2 The flow chart of the robot dexterous hand grabbing method embodiment of the present application. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0045] In the present application, the terms "upper", "lower", "left", "right", "front", "back", "top", "bottom", "inner", "outer", "middle", "vertical", "horizontal", "lateral", "longitudinal" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. These terms are mainly used to better describe the present application and its embodiments, and are not used to limit the indicated devices, elements or components to have a specific orientation, or to be constructed and operated in a specific orientation.

[0046] In addition, in addition to being used to indicate the orientation or positional relationship, the above-mentioned part of the terms can also be used to indicate other meanings, for example, the term "upper" can also be used to indicate a certain dependent relationship or connection relationship in some cases. Those skilled in the art can understand the specific meaning of these terms in the present application according to the specific situation.

[0047] Furthermore, the terms "installed," "disposed," "provided with," "connected," and "connected" should be interpreted broadly. For example, they can refer to fixed connections, removable connections, or integral structures; mechanical connections or electrical connections; direct connections, indirect connections through an intermediary, or internal communication between two devices, elements, or components. Those skilled in the art will understand the specific meanings of these terms in this application based on the specific circumstances.

[0048] Furthermore, the terms "first," "second," etc., are primarily used to distinguish between different devices, elements, or components (which may or may not be of the same type and configuration), and are not intended to indicate or imply the relative importance or quantity of the devices, elements, or components indicated. Unless otherwise specified, "plurality" means two or more.

[0049] The technical solution of the present application will be further described below with reference to the embodiments and drawings.

[0050] See Figure 1 A robot dexterous hand grasping control system is disclosed. The control system includes a controller, a dexterous hand, an image acquisition device, and a pressure sensor.

[0051] Pressure sensors are installed on each finger of the dexterous hand. The image acquisition device is installed on the robot and placed in a position where it can fully observe the dexterous hand and the target object.

[0052] The dexterous hand is connected to a dexterous hand motion control device. The dexterous hand motion control device, the image acquisition device and each pressure sensor are all connected to the controller signal.

[0053] The target object is the object to be grasped or the object to be grasped.

[0054] After the dexterous hand grasps the target object, the target object will exert pressure on the pressure sensor; or after the dexterous hand motion control device controls the dexterous hand to perform the grasping action, the pressure sensor obtains the pressure signal exerted by the target object on the pressure sensor in real time; finally, the pressure sensor transmits the obtained pressure signal to the controller, and the controller judges the grasping situation or state of the dexterous hand on the target object, or the state of the target object based on the size of the pressure signal or the change of the pressure signal.

[0055] The robot's dexterous hand imitates ergonomics and generally has two hands, each with five fingers. An image acquisition device is installed on each finger, that is, it has pressure sensor one, pressure sensor two, and so on, up to pressure sensor ten, ten pressure sensors.

[0056] In the robot dexterous hand grasping control system of the present application, by setting pressure sensors on each finger of the dexterous hand and setting an image capture device at the same time, and the image capture device is installed on the robot and placed in a position where the dexterous hand and the target object can be fully observed, through the combination of pressure sensors and image capture devices, the grasping state and situation of the target object are judged by multiple parameters, and the judgment is accurate. After the target object is not grasped, or the target object is not effectively grasped, the grasping force or movement force can be adjusted in time to grasp the target object again, or continue to grasp the target object, to ensure that the target object is effectively grasped and that the target object is successfully grasped.

[0057] In the robot dexterous hand grasping control system of the present application, a pressure sensor is combined with an image extraction device to realize the judgment of the grasping state of the target object, and the judgment is accurate, avoiding the problem of installing cameras on each finger of the dexterous hand to obtain the image of the soft body on the dexterous hand to identify the deformation of the flexible contact body to judge the size of the clamping force. The control system is simple as a whole, low-cost, and the control is simple and accurate.

[0058] In some embodiments, the image capture device can be a camera, a camera, or a still camera. Preferably, the image capture device is a camera. In addition, because the image capture device is installed on the robot and placed in a position where it can fully observe the dexterous hand and the target object, it is generally installed on the head or shoulder of the robot, so that the image capture device can observe the robot's dexterous hand and the target object grasped by the dexterous hand. For the sake of convenience, the image capture device installed on the robot and placed in a position where it can fully observe the dexterous hand and the target object can be referred to as a remote image capture device. If the image capture device uses a camera, it is a remote camera.

[0059] In some embodiments, the pressure sensor is a flexible pressure sensor, and each finger of the dexterous hand is covered with a flexible pressure sensor; when the dexterous hand grasps a target object, the target object applies pressure to the flexible pressure sensor.

[0060] By using flexible pressure sensors and wrapping them around the fingers of the dexterous hand, the pressure exerted by the target object on the dexterous hand can be effectively and accurately obtained when the dexterous hand grasps the target object, and the sensing is accurate and flexible.

[0061] The flexible pressure sensor may be any one in the prior art that can meet the requirements of this application, such as the flexible pressure sensor disclosed in the existing patent application CN119618423A.

[0062] See Figure 2 , a process flow chart of a robot dexterous hand grasping method is disclosed.

[0063] The robot hand grasping method adopts the above-mentioned robot hand grasping control system to grasp, and includes the following steps:

[0064] The dexterous hand first grasps and moves the target object; then it determines whether the pressure sensor receives a pressure signal, whether the target object is displaced, and whether relative sliding occurs between the target object and the dexterous hand. Finally, it determines whether the contact surface between the target object and the dexterous hand deforms within a specified range. Only when the contact surface between the target object and the dexterous hand deforms within a specified range is the grasping determined to be successful; otherwise, it is determined to be unsuccessful.

[0065] In the robot dexterous hand grasping method of the present application, the dexterous hand grasps the target object, including grasping, fixing and moving, that is, the dexterous hand first grasps the target object, and then drags the target object to move.

[0066] In the robotic dexterous hand grasping method of this application, the dexterous hand refers to the dexterous hand of a humanoid robot. A typical humanoid robot has at least two dexterous hands, each typically having five dexterous fingers, and is considered an anthropomorphic robot. When grasping an object, the dexterous hand of a humanoid robot performs a grasping motion similar to that of a human hand.

[0067] In the robot dexterous hand grasping method of the present application, the target object refers to the object to be grasped or the object to be grasped.

[0068] In the robot dexterous hand grasping method of the present application, it is judged by whether the pressure sensor obtains a pressure signal, whether the target object is displaced, and whether relative sliding occurs between the target object and the dexterous hand. It is also judged whether the contact surface between the target object and the dexterous hand is deformed within a specified range. Only when the contact surface between the target object and the dexterous hand is deformed within a specified range, it is judged as a successful grasping, otherwise it is judged as an unsuccessful grasping. The judgment efficiency and accuracy are improved by combining multiple parameters. More importantly, by using these parameters for judgment, the need to obtain the deformation of the soft body on each dexterous finger of the humanoid robot is avoided, the difficulty of obtaining image information and the workload of judgment are reduced, the number of image acquisition devices is reduced, the cost is reduced, and the installation of image acquisition devices on the fingertips of each dexterous finger is avoided, reducing the difficulty of installation.

[0069] In this application, the dexterous hand first grasps and moves the target object, which means that the dexterous hand itself completes the grasping and moving action. It does not mean that the dexterous hand will definitely grasp the target object, nor does it mean that the dexterous hand will definitely grasp the target object and move the target object to an effective distance. Here, the dexterous hand first grasps and moves the target object, emphasizing the action of the dexterous hand itself. Of course, during the process of grasping and moving the dexterous hand itself, it may grasp the target object, or it may grasp the target object and move the target object to an effective distance.

[0070] Furthermore, after the dexterous hand grasps and moves the target object, if the pressure sensors installed on the fingers of the dexterous hand do not receive a pressure signal, it is determined that the grasping is unsuccessful. If the pressure sensors receive a pressure signal, it is determined whether the target object has moved.

[0071] The pressure sensor fails to receive a pressure signal, meaning that none of the dexterous hand's sensors receive a pressure signal. After the pressure sensor fails to receive a pressure signal, the dexterous hand motion control device controls the dexterous hand to re-grasp the object. Re-grasping here means re-adjusting the gripping force and angle to re-grasp the target object.

[0072] A pressure sensor receiving a pressure signal means that any one of the sensors on the dexterous hand can receive a pressure signal. This means that receiving a pressure signal from any one of the sensors on the dexterous hand does not necessarily mean that the object has been successfully grasped; further evaluation of other parameters is required.

[0073] Furthermore, before the dexterous hand performs the grasping operation, the image capture device (remote camera) first obtains (identifies) the initial position of the target object and transmits the target object initial position information to the controller. After the dexterous hand performs the grasping operation on the target object, the image capture device (remote camera) again obtains (identifies) the position of the target object, records it as the second position of the target object, and transmits the target object second position information to the controller. The controller determines whether the target object has been displaced after the dexterous hand grasping and moving operation based on the second position of the target object and the initial position of the target object. If the target object has not been displaced, it is determined that the grasping was not successful. If the target object has been displaced, it is determined whether relative sliding has occurred between the target object and the dexterous hand.

[0074] There are two situations in which the target object fails to move: one is that the dexterous hand fails to grasp the target object, and the other is that the dexterous hand grasps the target object, but the grasping force applied to the target object is too weak, making it unable to effectively drag the target object. In both cases, the dexterous hand is judged to have failed to grasp the target object.

[0075] At this point, if the grasping is judged to be unsuccessful, a second grasp and movement is required. Because the pressure sensor has already received the pressure signal, that is, it has been determined that the dexterous hand has grasped the target object, when grasping and moving again, the dexterous hand motion control device can control the dexterous hand to grasp again according to the original posture, or it can also change the original posture to grasp again. Preferably, grasping according to the original posture reduces the control of the dexterous hand motion control device and simplifies the control process.

[0076] After the dexterous hand grasps and moves the target object, if the target object moves, a determination is made as to whether relative displacement has occurred between the target object and the dexterous hand. If the target object moves after the dexterous hand grasps and moves, it cannot be confirmed that the dexterous hand has successfully grasped the target object. This may indicate an issue with excessive or insufficient grasping force, necessitating further extraction and determination of other parameters of the target object and the dexterous hand.

[0077] Furthermore, a determination is made as to whether relative sliding occurs between the target object and the dexterous hand. If so, the gripping force is adjusted to continue gripping. If no relative sliding occurs, a determination is made as to whether the contact surface between the target object and the dexterous hand has deformed within a specified range.

[0078] The pressure sensor (flexible pressure sensor) acquires the relative sliding information between the target object and the dexterous hand.

[0079] We've previously determined whether the pressure sensor receives a pressure signal and whether the target object has shifted. If these two parameters are positive, it can be determined that the dexterous hand has grasped the target object. After the dexterous hand grasps the target object, if the grasping force remains constant and there is no relative sliding between the target object and the dexterous hand, the real-time pressure signal sensed by the pressure sensor (flexible pressure sensor) on the dexterous hand remains unchanged. If the real-time pressure signal sensed by the pressure sensor (flexible pressure sensor) on the dexterous hand changes, it indicates that the target object has slipped relative to the dexterous hand.

[0080] Specifically, whether the pressure sensor (flexible pressure sensor) on the dexterous hand has a sliding characteristic can be determined by determining whether the electrical signal of the pressure sensor has a characteristic frequency of relative sliding.

[0081] In addition, it can also be judged from parameters such as pressure signal amplitude or phase.

[0082] In some embodiments, if any parameter of the pressure signal sensed by the flexible pressure sensor changes in frequency, amplitude or phase, it means that the flexible pressure sensor has a sliding feature, that is, the target object slides relative to the dexterous hand on the dexterous hand.

[0083] The frequency of a pressure signal refers to the number of periodic changes in the pressure signal per unit time. Frequency describes the speed of the periodic changes in the pressure signal. A high frequency indicates a fast pressure change, while a low frequency indicates a slow pressure change.

[0084] The amplitude of a pressure signal refers to its maximum or peak value. It can be the maximum range of pressure change or the average range of pressure change. A high amplitude indicates a large pressure change, while a low amplitude indicates a small pressure change.

[0085] The phase of a pressure signal refers to its position or offset in time. It describes the signal's starting point or time offset. Two pressure signals of the same frequency may have different phases, indicating a time offset.

[0086] After the dexterous hand grasps an object, the flexible pressure sensor captures the reaction force to the grasping force in real time, effectively capturing the pressure exerted by the object on the dexterous hand. Within the real-time pressure signals captured by the flexible pressure sensor, the frequency, amplitude, or phase of each pressure signal is recorded and analyzed to determine whether the flexible pressure sensor signal exhibits sliding characteristics.

[0087] The pressure sensor acquires the pressure signal of the target object on the dexterous hand in real time. The pressure signal is an electrical signal. If the pressure sensor's electrical signal exhibits a characteristic frequency of relative sliding, it is determined that relative sliding has occurred between the target object and the dexterous hand, indicating that the flexible pressure sensor signal has sliding characteristics. If the pressure sensor's electrical signal does not exhibit a characteristic frequency of relative sliding, it is determined that relative sliding has not occurred between the target object and the dexterous hand, indicating that the flexible pressure sensor signal has no sliding characteristics.

[0088] Furthermore, the system determines whether the contact surface between the target object and the dexterous hand has deformed within a specified range. If the contact surface between the target object and the dexterous hand deforms, and the deformation is greater than the set maximum deformation value or less than the set minimum deformation value, the grasp is determined to be unsuccessful.

[0089] If the contact surface of the target object with the dexterous hand is deformed, and the deformation size is greater than the set minimum deformation value and less than the set maximum deformation value, it is judged that the grasping is successful.

[0090] After the dexterous hand grasps and moves the target object, if the contact surface of the target object with the dexterous hand deforms, and the deformation is greater than the set maximum deformation value or less than the set minimum deformation value, the grasp is judged to be unsuccessful. If the contact surface of the target object with the dexterous hand deforms, and the deformation is greater than the set maximum deformation value, it means that the applied grasping force is too large, which may damage or destroy the target object, and a re-grasping operation is required. Generally, the grasping force is reduced and grasped again, or the dexterous hand slightly relaxes and withdraws some of the grasping force, and the deformation of the contact surface of the target object with the dexterous hand is judged again until the grasping force applied to the target object is within the set range, or the target object is damaged due to excessive grasping force, and the damaged target object is recycled to the recycling station.

[0091] Alternatively, if the contact surface of the target object with the dexterous hand deforms, and the deformation is less than the set minimum deformation value, this indicates that the grasping force applied by the dexterous hand on the target object is too small to effectively grasp the target object, or the target object cannot be moved after grasping, and re-grasping is also necessary. Re-grasping here generally involves increasing the grasping force to grasp again, or the dexterous hand continues to tighten its grip in its current state to increase the grasping force and ensure that the grasping force is appropriate. By determining whether the contact surface of the target object with the dexterous hand deforms within a specified range, the target object is grasped.

[0092] After the dexterous hand grasps and moves the target object, if the contact surface of the target object and the dexterous hand deforms, and the deformation size is greater than the set minimum deformation value and less than the set maximum deformation value, it is judged that the grasping is successful.

[0093] We have previously determined whether the target object has been displaced. If the target object has been displaced and there is no sliding between the dexterous hand and the dexterous hand, if the contact surface between the target object and the dexterous hand is deformed, and the deformation size is greater than the set minimum deformation value and less than the set maximum deformation value, then it is determined that the grasping is successful.

[0094] In this application, if the grasping is determined to be unsuccessful, a re-grasp is performed. Re-grasping generally does not change the originally set grasping posture of the dexterous hand, but only increases the grasping force to grasp the target object again. Alternatively, the dexterous hand, while currently grasping the target object, continues to tighten the dexterous hand to increase the grasping force to ensure that the grasping force is appropriate and grasp the target object. Alternatively, the dexterous hand motion control device may be controlled to change the grasping posture and grasping force of the dexterous hand to re-grasp the target object.

[0095] After determining that the dexterous hand has not successfully grasped the target object, it will grasp it again to ensure that the target object is grasped and the task is completed.

[0096] In some embodiments, a robotic dexterous hand grasping method comprises the following steps:

[0097] Step S13. The dexterous hand grasps and moves the target object. In step S13, the dexterous hand performs a preliminary grasp of the target object. During this grasping process, the dexterous hand may or may not grasp the target object, or even if it grasps the target object, the grasping force applied to the target object may be too great or too small. Whether the dexterous hand has effectively and appropriately grasped the target object will need to be further determined in subsequent steps.

[0098] Step S14: Determine whether the pressure sensor senses a pressure signal.

[0099] If the pressure sensor does not sense a pressure signal, the process returns to step S13. The pressure sensor acquires the pressure exerted by the target object on the dexterous hand in real time. If the pressure sensor does not sense a pressure signal, it means that the dexterous hand has not grasped the target object at all and needs to adjust its grasping posture and re-grasp.

[0100] If the pressure sensor senses a pressure signal, the process proceeds to step S15. The pressure sensor acquires the pressure exerted by the target object on the dexterous hand in real time. If the pressure sensor senses a pressure signal, it indicates that the dexterous hand has grasped the target object, or at least partially grasped the target object. The process proceeds to step S15 to further determine whether the dexterous hand has effectively grasped the target object.

[0101] Step S15. The image capture device acquires the target's position and determines whether the target has moved. If the target has moved, the process proceeds to Step S16 below. If the target has not moved, the process proceeds to Step S19 below. After the dexterous hand grasps the target and moves it, if the dexterous hand has effectively grasped and moved the target, then the target must have moved. If the target has not moved, the force applied to the target may be insufficient.

[0102] Step S16. The pressure sensor obtains the pressure signal of the target object on the dexterous hand in real time. The pressure signal is an electrical signal. If the pressure signal (electrical signal) shows a characteristic frequency of relative sliding, it is determined that relative sliding occurs between the target object and the dexterous hand, and the process goes to the following step S24. If the pressure signal (electrical signal) does not show a characteristic frequency of relative sliding, it is determined that relative sliding does not occur between the target object and the dexterous hand, and the process goes to the following step S17. It is judged whether the pressure sensor (flexible pressure sensor) on the dexterous hand has a sliding feature. Specifically, it is judged whether the pressure sensor (flexible pressure sensor) has a sliding feature by judging the frequency of the pressure signal. Alternatively, it can also be judged from parameters such as the amplitude or phase of the pressure signal. If the pressure signal sensed by the flexible pressure sensor shows a sliding frequency, it means that the flexible pressure sensor has a sliding feature, that is, the target object has slipped relative to the dexterous hand on the dexterous hand.

[0103] Step S17. The image capture device acquires an image of the target object's surface. Based on the acquired image of the target object's surface, it is determined whether the contact surface of the target object with the dexterous hand has deformed within a specified range. Having previously determined whether the target object has shifted, if the target object has shifted and there is no slippage between the target object and the dexterous hand, it indicates that the dexterous hand has effectively grasped the target object. Further, it is necessary to determine the magnitude of the grasping force applied by the dexterous hand on the target object. Successful grasping of the target object by the dexterous hand is ensured only when the grasping force applied by the dexterous hand on the target object is appropriate. Neither excessive nor insufficient grasping force is acceptable. If the contact surface of the target object with the dexterous hand has deformed within a specified range, the process proceeds to step S18. If the contact surface of the target object with the dexterous hand has deformed outside the specified range, the process proceeds to step S25.

[0104] Step S18: If the contact surface of the target object with the dexterous hand is deformed, and the deformation size is greater than the set minimum deformation value and less than the set maximum deformation value, it is determined that the grasping is successful and a grasping is completed.

[0105] Step S25. If the deformation of the contact surface of the target object contacted by the dexterous hand is greater than the set maximum deformation value, the grasping force is reduced, and the process returns to step S13 for re-grasping. In step S15, if the target object is displaced, it indicates that the dexterous hand has grasped the target object, and the grasping force applied by the dexterous hand to the target object is greater than the set minimum grasping force. In step S15, if the grasping force applied by the dexterous hand to the target object is less than the set minimum grasping force, the target object cannot be displaced. Therefore, in step S17, the judgment of whether the deformation of the contact surface of the target object contacted by the dexterous hand is within the specified range mainly has the following two cases. First, the deformation of the contact surface of the target object contacted by the dexterous hand is greater than the set minimum deformation value, and the deformation is less than the set maximum deformation value. Second, the deformation of the contact surface of the target object contacted by the dexterous hand is greater than the set maximum deformation value. If the deformation of the contact surface of the target object contacted by the dexterous hand is greater than the set maximum deformation value, the grasping force is reduced, and the process returns to step S13 for re-grasping.

[0106] In step S15, it is judged whether the target object is displaced. If the target object is not displaced, the process proceeds to step S19. In step S19, the image extraction device (remote camera) acquires the position of the dexterous hand in real time, and judges whether the dexterous hand is displaced based on the position of the dexterous hand. If the dexterous hand is displaced, the process proceeds to step S20. If the dexterous hand is not displaced, the process proceeds to step S22.

[0107] Before step S13, the image extraction device (remote camera) acquires the initial position of the dexterous hand, and sends the initial position to the controller. In step S19, the image extraction device (remote camera) acquires the actual position of the dexterous hand, denoted as the second position, and sends the second position information to the controller. The controller judges whether the dexterous hand is displaced based on the initial position and the second position of the dexterous hand. If the dexterous hand is displaced, the process proceeds to step S20. If the dexterous hand is not displaced, the process proceeds to step S22.

[0108] Step S20: If it is determined that the grasping force is less than the set grasping force or the dexterous hand fails to grasp the target object, the process proceeds to step S21. In step S14, a pressure signal is generated between the dexterous hand and the pressure sensor, indicating that the dexterous hand has grasped the target object. In step S15, the target object has not shifted. This may be because the grasping force applied by the dexterous hand to the target object is insufficient and the dexterous hand cannot drag the target object to move. Further, in step S19, the dexterous hand shifts, indicating that the dexterous hand has performed a dragging action. Therefore, in step S20, it is determined that the grasping force applied by the dexterous hand to the target object is less than the set grasping force. Alternatively, it may be that the dexterous hand grasps nothing during grasping, and the interaction between the fingers of the dexterous hand causes the pressure signal generated by the pressure sensor on the dexterous hand in step S14. Therefore, in this step S20, it is determined that the grasping force is less than the set grasping force or that the dexterous hand has failed to grasp the target object.

[0109] Step S21. Increase the gripping force based on the pressure signal characteristics obtained by the pressure sensor, and return to step S13. In step S20, if it is determined that the gripping force is less than the set gripping force, then increase the gripping force and return to step S13 to re-grasp. In step S20, if it is determined that the dexterous hand has failed to grasp the target object, it is also necessary to re-grasp the target object, preferably increasing the gripping force to re-grasp. Based on the frequency, amplitude, and phase changes of the pressure signal obtained by the pressure sensor, increase the gripping force and re-grasp.

[0110] The frequency of a pressure signal refers to the number of periodic changes in the pressure signal per unit time. Frequency describes the speed of the periodic changes in the pressure signal. A high frequency indicates a fast pressure change, while a low frequency indicates a slow pressure change.

[0111] The amplitude of a pressure signal refers to its maximum or peak value. It can be the maximum range of pressure change or the average range of pressure change. A high amplitude indicates a large pressure change, while a low amplitude indicates a small pressure change.

[0112] The phase of a pressure signal refers to its position or offset in time. It describes the signal's starting point or time offset. Two pressure signals of the same frequency may have different phases, indicating a time offset.

[0113] Step S22: Determine that the moving force is less than the set minimum moving force, and proceed to Step S23. In Step S14, the dexterous hand senses pressure from the pressure sensor, indicating that the dexterous hand has grasped the target object. In Step S15, the target object has not moved. This may be because the dexterous hand's grasping force on the target object is insufficient, and the dexterous hand cannot drag the target object. Furthermore, in Step S19, the dexterous hand has not moved either. Therefore, in Step S22, it is determined that the moving force applied by the dexterous hand on the target object is less than the set minimum moving force. The moving force applied by the dexterous hand on the target object is insufficient, and the moving force is less than the set minimum moving force, so the process proceeds to Step S23.

[0114] Step S23: Increase the moving force and return to step S13. In step S22, it is determined that the moving force applied by the dexterous hand to the target object is less than the set minimum moving force. Therefore, it is necessary to increase the moving force to grasp the target object again. Of course, it is also possible to increase the moving force in this step and directly move the target object.

[0115] Step S24. Increase the grasping force and return to step S13. After the previous steps S14 and S15, it means that the dexterous hand has effectively grasped the target object. If in step S16, the target object slides relatively on the dexterous hand, that is, the pressure signal on the flexible pressure sensor has a sliding feature, the target object may be detached from the dexterous hand. At this time, the grasping force applied by the dexterous hand to the target object may be insufficient, and the grasping force needs to be increased. Step S24. Increase the grasping force according to the change of the sliding feature of the pressure signal, and grasp again, that is, adjust the grasping force according to the frequency, amplitude and phase characteristics of the pressure signal, and grasp again.

[0116] In some embodiments, before the dexterous hand grasps and moves the target object, the following operations are performed:

[0117] Step S10: Establish a database of the target object's shape, size, weight, volume, and surface roughness, and estimate the rated grasping force and rated moving force required for the target object with the corresponding information.

[0118] Step S11: Obtain basic information of the target object, which at least includes the size, volume, weight and surface roughness of the target object.

[0119] Step S12: Estimate the rated grasping force and rated moving force required to grasp the target object based on the obtained basic information of the target object.

[0120] Based on the basic information of the target object, the rated grasping force and rated moving force required to grasp the target object are estimated, and the dexterous hand is tried to ensure that the target object can be grasped in one go, or the number of times the target object is re-grasped is minimized to improve the efficiency of target grasping.

[0121] Before acquiring basic object information, a database of the object's shape, size, weight, volume, and surface roughness is established, and the rated gripping force and rated moving force required for the corresponding object are estimated. Establishing this database allows for rapid access to this information, shortening the time required to estimate the rated gripping force and rated moving force required for grasping the object, improving their accuracy, and ensuring that the dexterous hand can grasp the object in a single attempt, or minimizing the number of re-grasps required to grasp the object, thereby improving grasping efficiency.

[0122] The technical solution of this application is described above in conjunction with the embodiments and drawings. Obviously, the specific implementation of this application is not limited to the above-mentioned method. As long as various non-substantial improvements are made using the method concept and technical solution of this application, or the concept and technical solution of the application are directly applied to other occasions without improvement, they are all within the scope of protection of this application.

Claims

1. A robot dexterous hand grasping control system, characterized in that: It includes a controller, a dexterous hand, an image acquisition device and a pressure sensor; The pressure sensor is installed on each finger of the dexterous hand, and the image capture device is installed on the robot and placed in a position where the dexterous hand and the target object can be fully observed; The dexterous hand is connected to a dexterous hand motion control device, and the dexterous hand motion control device, the image capture device and each of the pressure sensors are all connected to the controller signal.

2. The robot dexterous hand grasping control system according to claim 1, characterized in that: The pressure sensor is a flexible pressure sensor, and each finger of the dexterous hand is covered with a flexible pressure sensor; when the dexterous hand grasps a target object, the target object applies pressure to the flexible pressure sensor.

3. A robot dexterous hand grasping method, characterized in that: The grasping control system of the robot dexterous hand according to claim 1 or 2 is used for grasping, comprising the following steps: The dexterous hand first grasps and moves the target object; then it determines whether the pressure sensor receives a pressure signal, whether the target object moves, and whether relative sliding occurs between the target object and the dexterous hand; finally, it determines whether the contact surface between the target object and the dexterous hand deforms within a specified range; Only when the contact surface between the target object and the dexterous hand deforms within a specified range is the grasping judged to be successful; otherwise, it is judged to be unsuccessful.

4. The robot dexterous hand grasping method according to claim 3, characterized in that: If the pressure sensor does not obtain a pressure signal, it is determined that the grasping is unsuccessful; if the pressure sensor obtains a pressure signal, it is determined whether the target object has moved.

5. The robot dexterous hand grasping method according to claim 3 or 4, characterized in that: If the target object does not move, it is determined that the grasping is unsuccessful; if the target object moves, it is determined whether relative sliding occurs between the target object and the dexterous hand.

6. The robot dexterous hand grasping method according to claim 3, characterized in that: If relative sliding occurs between the target object and the dexterous hand, the grasping force is changed and the grasping continues; if relative sliding does not occur between the target object and the dexterous hand, it is judged whether the contact surface between the target object and the dexterous hand has deformed within a specified range.

7. The robot hand grasping method according to claim 6, characterized in that: If the contact surface of the target object with the dexterous hand is deformed, and the deformation is greater than the set maximum deformation value or less than the set minimum deformation value, it is judged as unsuccessful grasping; If the contact surface of the target object with the dexterous hand is deformed, and the deformation size is greater than the set minimum deformation value and less than the set maximum deformation value, it is judged that the grasping is successful.

8. The robot hand grasping method according to claim 6, characterized in that: After the dexterous hand grasps the target object, the pressure sensor obtains the pressure signal of the target object on the dexterous hand in real time. The pressure signal is an electrical signal. If the electrical signal of the pressure sensor shows a characteristic frequency of relative sliding, it is determined that relative sliding occurs between the target object and the dexterous hand. If the electrical signal of the pressure sensor does not show a characteristic frequency of relative sliding, it is determined that no relative sliding occurs between the target object and the dexterous hand.

9. The robot dexterous hand grasping method according to claim 3, characterized in that: The following steps are involved: Step S13: The dexterous hand grasps and moves the target object; Step S14: Determine whether the pressure sensor senses a pressure signal; if the pressure sensor does not sense a pressure signal, return to step S13; if the pressure sensor senses a pressure signal, proceed to the following step S15; Step S15. The image capture device obtains the position of the target object and determines whether the target object has moved; if the target object has moved, the process proceeds to the following step S16; if the target object has not moved, the process proceeds to the following step S19; Step S16. The pressure sensor acquires in real time a pressure signal from the target object to the dexterous hand, the pressure signal being an electrical signal. If the electrical signal from the pressure sensor exhibits a characteristic frequency of relative sliding, it is determined that relative sliding has occurred between the target object and the dexterous hand, and the process proceeds to Step S24. If the electrical signal from the pressure sensor does not exhibit a characteristic frequency of relative sliding, it is determined that relative sliding has not occurred between the target object and the dexterous hand, and the process proceeds to Step S17. Step S17. The image capture device acquires an image of the surface of the target object and, based on the acquired image of the surface of the target object, determines whether the contact surface of the target object with the dexterous hand has deformed within a specified range. If the contact surface of the target object with the dexterous hand has deformed within the specified range, the process proceeds to Step S18. If the contact surface of the target object with the dexterous hand has deformed outside the specified range, the process proceeds to Step S25. Step S18: If the contact surface of the target object with the dexterous hand deforms, and the deformation is greater than the set minimum deformation value and less than the set maximum deformation value, then it is determined that the grasping is successful and the grasping is completed; Step S19. The image capture device obtains the position of the dexterous hand in real time and determines whether the dexterous hand has moved based on the position of the dexterous hand; if the dexterous hand has moved, the process proceeds to the following step S20; if the dexterous hand has not moved, the process proceeds to the following step S22; Step S20: If it is determined that the grasping force is less than the set grasping force or the dexterous hand fails to grasp the target object, proceed to step S21; Step S21: increasing the gripping force according to the pressure signal characteristics obtained by the pressure sensor, and returning to step S13; Step S22: If the moving force is less than the set minimum moving force, proceed to step S23; Step S23. Increase the moving force and return to step S13; Step S24: Increase the gripping force and return to step S13; Step S25: If the contact surface of the target object with the dexterous hand is deformed and the deformation is greater than the set maximum deformation value, the grasping force is reduced and the process returns to step S13.

10. The robot dexterous hand grasping method according to claim 9, characterized in that: Before the dexterous hand grasps and moves the target object, it also performs the following operations: Step S10: Establish a database of the target object's shape, size, weight, volume, and surface roughness, and estimate the rated gripping force and rated moving force required for the target object corresponding to the information; Step S11. Obtaining basic information of the target object, wherein the basic information of the target object includes at least the size, volume, weight and surface roughness of the target object; Step S12: Estimate the rated grasping force and rated moving force required to grasp the target object based on the obtained basic information of the target object.

Images