Machine hand self-adaptive grasping method and system

Abstract

The application provides a machine hand self-adaptive grabbing method and system, the method comprising: controlling the machine hand to approach a grabbed object and setting an initial load to contact the grabbed object; increasing the load according to the monitoring data when the machine hand loads the grabbed object with the current load until the loading load is determined, and then controlling the machine hand to grab the grabbed object with the loading load; adjusting the current load according to the monitoring data when the machine hand holds and moves the grabbed object until the holding load is determined, and monitoring whether the machine hand moves the grabbed object above a target position; after the machine hand moves the grabbed object above the target position, controlling the machine hand to place the grabbed object on the target position.

Description

Technical Field

[0001] The present invention relates to the field of robot arm control, and in particular to a robot arm adaptive grasping method and system. Background Art

[0002] Robotic arms can be used in a variety of industries. Currently, the automated grasping strategies of robotic arms are usually designed for some fixed application scenarios, enabling them to grasp specific objects. Obviously, the grasping strategies of robotic arms for a certain application scenario are difficult to be transferred to other application scenarios.

[0003] In order to make the robotic arm suitable for a variety of application scenarios and to grasp a variety of different objects, it is necessary to design a highly adaptable grasping strategy. Currently, some grasping strategies use a small load to grasp the object and move it, and then increase the load by monitoring the relative displacement between the object and the robotic arm after the movement. The problem with this strategy is that for some soft objects, the robotic arm may start moving without actually grasping the target. Although the displacement can be detected during the movement, increasing the load speed cannot prevent the object from completely slipping. Conversely, some grasping strategies use a large load to grasp the object and move it, and then reduce the load based on some feedback data to avoid damaging the object. The problem with this strategy is that for some fragile objects, the large load may directly cause the object to break when grasped.

[0004] This shows that the existing robotic grasping strategies are not adaptable enough, are difficult to cope with diverse grasping objects, and have a low grasping success rate. Summary of the Invention

[0005] In view of this, the present application provides a robot arm adaptive grasping method, comprising:

[0006] Control the robot arm to approach the grasped object and set the initial load to contact the grasped object;

[0007] increasing the load according to monitoring data when the robot arm loads the grasped object with the current load until the loaded load is determined, and then controlling the robot arm to grasp the grasped object with the loaded load;

[0008] Adjust the current load based on the monitoring data when the robot grasps and moves the grasped object until the grasping load is determined, and monitor whether the robot moves the grasped object above the target position;

[0009] After the robot arm moves the grasped object to above the target position, the robot arm is controlled to place the grasped object to the target position.

[0010] Optionally, the monitoring data includes feedback data of the contact point between the robot arm and the grasped object and feedback data of a drive device that changes the load.

[0011] Optionally, increasing the load based on monitoring data when the robot arm is loading the grasped object with the current load includes:

[0012] Extracting a global eigenvector from the feedback data using a neural network model, and determining upper and lower load limit information based on the global eigenvector;

[0013] Determining whether the current load will damage the grasped object based on the load upper limit information;

[0014] If it is determined that the current load will not damage the grasped object, then determining whether the current load is sufficient to grasp the grasped object based on the load lower limit information;

[0015] If it is determined that the current load is insufficient to pick up the object, the load is increased.

[0016] Optionally, the feedback data of the contact portion between the robotic hand and the grasped object includes pressure data collected by an electronic skin provided at the contact portion between the robotic hand and the grasped object.

[0017] Optionally, the feedback data of the contact portion between the robotic hand and the grasped object includes temperature data collected by an electronic skin provided at the contact portion between the robotic hand and the grasped object.

[0018] Optionally, the feedback data of the drive device for changing the load includes displacement data and force data of each drive device of the robot arm.

[0019] Optionally, determining the load upper limit information and the load lower limit information according to the global eigenvector includes:

[0020] Classify the object according to the global feature vector to obtain a classification vector for the category of the grasped object;

[0021] Determining the probability values ​​of the grasped object belonging to various categories based on the classification vector;

[0022] The upper load limit information and the lower load limit information are determined according to the probability values ​​of the grasped object belonging to various categories and the upper load limits and lower load limits corresponding to the various categories.

[0023] Optionally, the load upper limit information includes load upper limits corresponding to various categories and probabilities that the grasped objects belong to various categories; the load lower limit information includes load lower limits corresponding to various categories and probabilities that the grasped objects belong to various categories.

[0024] Optionally, judging whether the current load will damage the grasped object according to the load upper limit information includes:

[0025] Comparing the current load with the load upper limit of each category respectively, and adding the probabilities of the categories exceeding the load upper limit to obtain a first cumulative probability;

[0026] Determining whether the first cumulative probability is lower than a first probability threshold;

[0027] When the first cumulative probability is lower than a first probability threshold, determining that the current load will not damage the grasped object;

[0028] Determining whether the current load is sufficient to grasp the object according to the load lower limit information includes:

[0029] Comparing the current load with the load lower limit of each category respectively, and adding the probabilities of the categories exceeding the load lower limit to obtain a second cumulative probability;

[0030] Determining whether the second cumulative probability is lower than a second probability threshold;

[0031] When the second cumulative probability is lower than a second probability threshold, it is determined that the current load is insufficient to grasp the object.

[0032] Optionally, the current load is adjusted based on monitoring data while the robot grips and moves the object, including:

[0033] It is determined whether the grasped object is sliding according to the monitoring data at the current moment and the monitoring data at the previous moment, and the load is increased when the grasped object is sliding.

[0034] Optionally, the current load is adjusted based on monitoring data while the robot grips and moves the object, including:

[0035] Predicting the sliding distance of the grasped object relative to the robotic arm based on the monitoring data at multiple moments;

[0036] Determining whether the sliding distance exceeds a first distance threshold;

[0037] When the sliding distance exceeds the first distance threshold, determining a load increase amount according to the sliding distance;

[0038] A target load is set using the load increase amount and the current load, and the load is increased until the target load is reached.

[0039] Optionally, the current load is adjusted based on monitoring data while the robot grips and moves the object, including:

[0040] A first adjustment operation is triggered according to a first cycle, and a second adjustment operation is triggered according to a second cycle. The first adjustment operation includes determining whether the grasped object slides based on the monitoring data at the current moment and the monitoring data at the previous moment, and increasing the load when the grasped object slides; the second adjustment operation includes predicting the sliding distance of the grasped object relative to the robot arm based on the monitoring data at multiple moments; determining whether the sliding distance exceeds a first distance threshold; when the sliding distance exceeds the first distance threshold, determining the load increase amount based on the sliding distance; using the load increase amount and the current load to set a target load, and increasing the load until the target load is reached.

[0041] Optionally, the monitoring data includes pressure data collected by an electronic skin provided at the contact portion between the robotic arm and the grasped object, and driving force data of each driving device of the robotic arm.

[0042] Optionally, the pressure data is array data collected by electronic skins respectively provided on the palm and multiple fingers of the robotic hand.

[0043] Optionally, predicting the sliding distance of the grasped object relative to the robotic arm based on the monitoring data at multiple moments includes:

[0044] Predicting the pressure feature vector at the next moment based on the array data at multiple moments by a time series prediction module;

[0045] Extracting a vertical driving force component from the driving force data, and using a time series prediction module to predict a driving force feature vector at a next moment based on the vertical driving force component at multiple moments;

[0046] Connecting the pressure eigenvector and the driving force eigenvector to obtain a global eigenvector;

[0047] A sliding distance is obtained according to the global feature vector.

[0048] Optionally, after the second adjustment operation is triggered, before the current load reaches the target load, triggering of the first adjustment operation is suspended.

[0049] Optionally, the load increase amount when the first adjustment operation is triggered is a fixed amount, and the fixed amount is smaller than the load increase amount in the second adjustment operation.

[0050] Accordingly, the present application provides a control device, characterized in that it includes: a processor and a memory connected to the processor; wherein the memory stores instructions that can be executed by the processor, and the instructions are executed by the processor so that the processor executes the above-mentioned robot arm adaptive grasping method.

[0051] The present application also provides an adaptive grasping system, comprising a robotic arm and the above-mentioned control device; wherein the robotic arm is provided with a monitoring device for collecting the monitoring data.

[0052] The adaptive grasping method and equipment of the robotic arm provided in this application divide the grasping process into four stages. In the positioning stage, a small initial load is used to try to contact the grasped object; in the loading stage, the load is gradually increased to ensure that the target is grasped without damaging the target; in the grasping stage, the load is continuously and dynamically adjusted during the movement to prevent the variables generated by the movement from causing the object to slip; in the placement stage, the target is placed after ensuring that the grasped object is moved above the target position. The processing methods of the above four stages can handle a variety of grasped objects and improve the success rate of grasping. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 is a flow chart of a method for adaptive grasping by a robotic arm in an embodiment of the present invention;

[0055] Figure 2 A schematic diagram of a neural network model architecture according to an embodiment of the present invention;

[0056] Figure 3 Schematic diagram of multi-step joint data processing in an embodiment of the present invention;

[0057] Figure 4 is a load upper limit and probability distribution diagram in an embodiment of the present invention;

[0058] Figure 5 is a load lower limit and probability distribution diagram in an embodiment of the present invention;

[0059] Figure 6 is a schematic diagram of another neural network model architecture according to an embodiment of the present invention;

[0060] Figure 7 Schematic diagram of the structure of the adaptive grasping system in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0062] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0063] The embodiment of the present invention provides a method for adaptive grasping of a robot arm, which can be executed by an electronic device such as a computer or a server. Figure 1 The following operations are shown:

[0064] S1: Control the robotic arm to approach the object and set an initial load to contact it. This step is called the positioning phase. First, the object's position must be determined. This can be achieved by using optical positioning or radar signal positioning. After identifying the object's location in space, the robotic arm is controlled to move toward it. Alternatively, a fixed-point contact approach can be used, where the object's location is pre-fixed and the robotic arm moves to a predetermined position.

[0065] The load in this application refers to the force applied by the robotic arm to the object being grasped. The greater the load, the stronger the grip. When the robotic arm and the object interact, sensors on the arm can detect the forces at various locations. These forces can be used to determine the actual load of the robotic arm. The initial load is configured to a relatively small value. This step is completed when the actual load at the time the robotic arm contacts the object reaches the initial load.

[0066] In one embodiment, a visual positioning solution is adopted. A two-dimensional image camera or a three-dimensional camera that can measure point clouds can be used to collect images of the position of the grasped object. First, the user's requirements for the target object are obtained, and visual imaging is performed through the camera. The target object is selected based on the visual imaging results, and the robot arm is moved close to the target object position. Then, the load is set to a very small value F0, and trial touch and adjustment are performed until the optimal state is reached.

[0067] S2: Based on the monitoring data of the robot arm loading the grasped object with the current load, the load is increased until the loaded load is determined, and then the robot arm is controlled to grasp the grasped object using the loaded load. This step is called the loading phase. In this phase, the robot arm attempts to grasp the grasped object in the original position without moving the grasped object. The current load when entering the loading phase is the initial load, and the current load will gradually increase during the execution of this step. What needs to be determined in this phase is the load value that can be grasped without damaging the grasped object. The loaded load is greater than the initial load.

[0068] Monitoring data refers to data generated by the interaction between the robotic arm and the grasped object, such as force, displacement, and temperature. Specifically, when the load changes, the pressure on the robotic arm will change, and the position of the robotic arm's movable joints will shift. In this step, based on the corresponding relationship between this monitoring data and the load, the load is dynamically adjusted until a value is determined that can lift the grasped object but damage it. This can be achieved using algorithms such as mathematical models or neural network models.

[0069] In addition, it may be impossible to determine the loading load at this stage. In this case, the load can be set to 0 and the process can be returned to step S1. The robot arm's posture or the contact point with the grasped object can be adjusted, and then step S2 can be re-entered. The number of loading retries can also be recorded. If the number of attempts exceeds a threshold, the grasping process is abandoned.

[0070] In step S3, the current load is adjusted based on the monitoring data while the robot grasps and moves the object until the grasping load is determined. The robot then monitors whether it has moved the object above the target position. Once the robot has moved the object above the target position, step S4 is executed. This step is called the grasping phase, during which the robot grasps the object and moves it (upward or laterally above the target position). The current load at the time of entering the grasping phase is the loading load determined in step S2. During this step, the current load may gradually increase or remain constant, i.e., loading load ≥ loading load.

[0071] In this step, as the robotic arm moves while holding the object, the monitoring data changes. Changes in the load also cause changes in the monitoring data. This step determines whether the object will slip from the grip based on these changes in monitoring data. If slippage occurs or is likely, the load is increased promptly to prevent it. This step ultimately stabilizes the load, which is referred to as the grip load.

[0072] It should be noted that there can be multiple types of monitoring data. The same data or partially the same data can be used in step S2 and step S3, or completely different data can be used respectively. For example, force and displacement can be used in step S2, and only force can be used in step S3.

[0073] S4: Control the robotic arm to place the object at the target location. This step is called the placement phase. During this phase, the load is not increased. The robotic arm moves downward until the object contacts the load surface, and then the load is reset to zero, ending the grasping process.

[0074] The adaptive grasping method of the robotic arm provided by the embodiment of the present invention divides the grasping process into four stages: in the positioning stage, the robot attempts to contact the grasped object with a small initial load; in the loading stage, the load is gradually increased to ensure that the target is grasped without damaging the target; in the grasping stage, the load is continuously and dynamically adjusted during the movement to prevent the variables generated by the movement from causing the object to slip; in the placement stage, the target is placed after ensuring that the grasped object is moved above the target position. The processing methods of the above four stages can handle a variety of grasped objects and improve the success rate of grasping.

[0075] In the loading phase, i.e., step S2 above, in one embodiment, the load is increased as follows:

[0076] S21, obtaining monitoring data when the robot arm loads the grasped object with the current load, the monitoring data including feedback data of the contact part between the robot arm and the grasped object and feedback data of the driving device that changes the load.

[0077] Loading in this method refers to the process of the robotic arm grasping and holding an object on a supporting surface (e.g., the ground, a tabletop, or other platform). It does not include moving the object upward or in other directions after grasping. This method can be executed synchronously with the load changes in the robotic arm. The initial load is denoted as F0. The robotic arm can contact the grasped object with load F0, and then gradually increase the load. As the load changes, the corresponding monitoring data will also change.

[0078] The contact parts between the robotic hand and the grasped object and the driving device for changing the load depend on the structure of the robotic hand. Taking the humanoid mobile phone robotic hand as an example, the contact parts are the surfaces of five fingers and one palm. The driving device for changing the load can be a device for driving the movement of the finger joints. Specifically, each movable joint can correspond to an independent driving device.

[0079] Feedback data from the contact area between the robotic hand and the grasped object may specifically include force. For a humanoid robotic hand, this refers to the force (pressure or intensity) exerted by the grasped object on the surface of the fingers and palm; in optional embodiments, it may also include temperature.

[0080] The feedback data of the drive device that changes the load may specifically include force and / or displacement. For an anthropomorphic robot hand, it refers to the force borne by the drive device and / or the displacement provided to generate the current load.

[0081] As an example, for the load at any time (time step), the monitoring data obtained can specifically include 、 、 and ,in represents the force at the ith contact point, represents the temperature of the ith contact point, represents the displacement of the i-th driving device, represents the force of the i-th driving device.

[0082] Load can be understood as a characteristic value related to force. The characteristic value can be one or more, specifically force data from monitoring data or a value calculated based on the force data. For example, the current load can be the value of a single force from the surface of the finger or palm, or the sum of these forces, or the force of a driving device or the sum of the forces of driving devices, etc.

[0083] S22, using a neural network model to extract a global feature vector from the feedback data, and determining the upper and lower load limits based on the global feature vector. The feedback data is at least two pieces of data from two different parts of the robot hand or the drive device. Based on the above example, there may be dozens of pieces of feedback data. To extract the feature vector, these feedback data need to be fused. The neural network model can use various network structures composed of units such as CNN (Convolutional Neural Network) and MLP (Multi-Layer Perceptron) to achieve feature extraction and fusion. There are many specific implementation methods, for example, it can include a ResNet (Residual Network) structure.

[0084] The upper load limit refers to the load above which the object is likely to be broken (it can be called the breaking load); the lower load limit refers to the load above which it is worth trying to grab the object, or it can be interpreted as the lowest load that can be used to grab the object.

[0085] It should be noted that the so-called upper load limit and lower load limit in this application are force values, and the upper load limit information and lower load limit information can be force values, or information calculated based on the force values, such as statistical information.

[0086] The upper load limit information and the lower load limit information can be the output result of the neural network model, or the mapping information of the output result of the neural network model. For example, the neural network model can be configured to directly output the value of the upper load limit and the value of the lower load limit, and a large amount of sample data is used to train it before use so that it can output two numerical values ​​based on the above-mentioned input feedback data; the neural network model can be configured to output information about the grasped object, such as the category of the grasped object, and a corresponding relationship library between this information and the upper load limit and the lower load limit is pre-established. When the neural network model outputs the information of the grasped object during use, the corresponding upper load limit and the lower load limit can be queried in the corresponding relationship library.

[0087] S23, determine whether the current load will damage the object according to the upper load limit information. If it is determined that the current load will not damage the object, step S24 is executed; otherwise, as an optional solution, step S26 can be executed, or the method is stopped.

[0088] An optional way is to determine the object belongs to a certain category by hard classification in step S22, and then obtain the corresponding upper load limit, or directly output the upper load limit by the neural network in step S22. In step S23, it is determined whether the current load is lower than the upper load limit, and if the current load is lower than the upper load limit, it is determined that the current load will not damage the object.

[0089] S24, determine whether the current load is sufficient to grasp the object according to the lower load limit information. If it is determined that the current load is not sufficient to grasp the object, step S25 is executed, otherwise, the current load is used to attempt to grasp the object, that is, enter the grasping stage.

[0090] An optional way is to determine the object belongs to a certain category by hard classification in step S22, and then obtain the corresponding lower load limit, or directly output the lower load limit by the neural network in step S22. In step S24, it is determined whether the current load is lower than the lower load limit, and if the current load is lower than the upper load limit, it is determined that the current load is not sufficient to grasp the object.

[0091] S25, increase the load. Then return to step S21 to re-execute the method until the current load meets the condition of step S24. The amount of load increase can be fixed, or can be calculated according to the difference between the current load and the lower load limit information, such as the difference between the current load and the lower load limit is larger, the increase is larger, and the like.

[0092] S26, set the load to the initial value, and then return to step S21 to re-execute the method. Further, before re-executing the method, the pose of the robot hand can be fine-tuned to re-attempt the loading process from a different angle or position. Fine-tuning the pose of the robot hand can be fine-tuning only the angle of the mechanical joint, or re-selecting the contact position.

[0093] According to the robot hand loading method provided by the embodiment of the application, the corresponding monitoring data is obtained as the grasping load changes, and then the upper load limit information and the lower load limit information corresponding to the current monitoring data are determined, the load is adjusted according to the relationship between the current load and the upper load limit information and the lower load limit information, the adaptive grasping loading process of loading, identifying and deciding is realized, the object is grasped with a suitable load, and the object can be effectively avoided from falling or being damaged in the subsequent grasping process, and the adaptability is strong.

[0094] Regarding the monitoring data in the above steps, in one embodiment, the feedback data from the contact area between the robotic hand and the grasped object includes pressure data collected by the electronic skin installed at the contact area between the robotic hand and the grasped object. The pressure data is array data collected by the electronic skin installed on the palm and multiple fingers of the robotic hand.

[0095] As an example, The first electronic skin Rank Column record .

[0096] In one embodiment, the feedback data from the contact area between the robotic hand and the grasped object includes temperature data collected by the electronic skin installed at the contact area between the robotic hand and the grasped object. The temperature data is array data collected by the electronic skin installed on the palm and multiple fingers of the robotic hand.

[0097] As an example, The first electronic skin Rank Column record .

[0098] According to the above example, the size of the electronic skin at different locations in the robot hand or the number of measurement points therein can be different, so the number of rows and columns of the array is different. In one embodiment, the feedback data of the drive device that changes the load includes the displacement data and force data of each drive device of the robot hand. As an example, for the first The displacement data and force data of each driving device are recorded as .

[0099] The above-mentioned various feedback data can accurately reflect the gripping state. By extracting features from these feedback data, the accuracy of the upper load limit information and the lower load limit information can be improved.

[0100] There are many specific structures of the neural network model in the above step S22. This embodiment provides a specific implementation method. Figure 2 As shown, step S22 includes the following operations:

[0101] S221, extracting the pressure data collected from each electronic skin to obtain the electronic skin force distribution feature, extracting the temperature data collected from each electronic skin to obtain the electronic skin temperature distribution feature, and fusing the electronic skin force distribution feature and the electronic skin temperature distribution feature to obtain the electronic skin feature vector.

[0102] Specifically, each piece of electronic skin force monitoring value array data Input into CNN layer respectively, and then pass through several residual modules respectively. The processed results are averaged to obtain the electronic skin force distribution characteristics; the array data of each electronic skin temperature monitoring value is ... electronic skin force distribution characteristics Input into CNN layer respectively, and then pass through several residual modules respectively. The processed results are averaged to obtain the temperature distribution characteristics of the electronic skin.

[0103] In some embodiments, the scales of the electronic skin temperature distribution feature and the electronic skin force distribution feature may be the same, so they can be directly fused to obtain the electronic skin feature vector. If the scales of the two distribution features are different, adjustments need to be made:

[0104] Pooling is performed on the one with a higher pixel density between the electronic skin force distribution feature and the electronic skin temperature distribution feature to adjust the number of pixels to be consistent (connect after unifying the pixel number); the adjusted electronic skin force distribution feature and electronic skin temperature distribution feature are fused (fusion array); and the fusion results are averaged in the array dimension (array average) to obtain the electronic skin feature vector.

[0105] The above layers process each piece of electronic skin separately, and the parameters of the layers are shared.

[0106] S222, extract the driving device feature vector based on the displacement data and force data of the driving device. The displacement data of the driving device at the previous moment Take the difference; compare the difference result with the drive device force data In this embodiment, the MLP module is specifically used to obtain the robot drive device feature vector.

[0107] S223, connecting the electronic skin feature vector and the driving device feature vector to obtain a global feature vector.

[0108] It should be noted that the above-described modular architecture is only one of many possible approaches and is not the only feasible solution. For example, normalization, dropout, and other processing can be implemented at appropriate locations in the above process, and other modules can be used to replace the above-described MLP and CNN modules to form a modified architecture. The differential between the displacement and force of the drive device reflects the relationship between force and displacement along the clamping direction, the force distribution of the electronic skin reflects the force distribution characteristics along the contact tangent direction, and the temperature reflects information such as heat capacity and heat transfer. The above-described embodiment integrates this rich information, and the result can fully reflect the material properties of the grasped object, thereby making the classification results more accurate.

[0109] In one embodiment, step S22 includes the following operations:

[0110] S224: Classify the global feature vector to obtain a classification vector for the category of the grasped object. Specifically, the global feature vector can be obtained by using the method of steps S221-S223 above, or other network structures such as LSTM network structures can be used.

[0111] S225: Determine the probability values ​​of the grasped object belonging to various categories based on the classification vector. An optional method is to take a softmax of the component vectors output by the neural network model to obtain the probability of each category.

[0112] S226: Determine the upper load limit information and the lower load limit information based on the probability values ​​of the grasped object belonging to various categories and the upper load limits and lower load limits corresponding to the various categories.

[0113] In a preferred embodiment, a multi-step joint approach is used to obtain the probability of each category. Figure 3 Step S225 specifically includes the following operations:

[0114] The multi-step joint data is obtained based on the classification vectors at the current moment and at least one previous moment. There is more than one specific calculation method. In this embodiment, the classification vector at each moment is multiplied by a preset coefficient. The preset coefficient in this embodiment is , where M is the number of time steps (the total number of the current moment and at least one previous moment), K and A are preset constants, and then the multiplication results of each moment are summed up, and the summed result is combined with the preset background value B to obtain multi-step joint data, specifically a vector with a length of the number of time steps + 1. Based on the multi-step joint data, the probability values ​​of the grasped object belonging to various categories are obtained. This embodiment adopts the method of taking softmax of the above processing results and dividing by the constant The values ​​corresponding to the items other than are the probability values ​​of the corresponding categories. The classification vector of Interpreted as the current moment, Interpreted as the initial moment or The component vectors obtained through steps S221-S224 can be regarded as feature information obtained from the feedback data at only one moment, based on which the category of the grasped object at that moment can be obtained. In actual applications, since the feedback data will change over time, different categories of grasped objects will be obtained at different moments. The above embodiment adopts a multi-step joint approach to integrate the feature information of the current moment and multiple previous moments to obtain the classification result, which can make the classification result more stable and more accurate.

[0115] Assuming there are four possible categories, A, B, C, and D, the probabilities of the grasped object belonging to each of these categories are obtained through the above process and are recorded as PA, PB, PC, and PD. An optional implementation is to determine the maximum of these probabilities in step S226 to further determine the category to which the grasped object belongs; and to obtain the upper and lower load limits corresponding to the category to which the grasped object belongs.

[0116] For example, assuming the maximum value among PA, PB, PC, and PD is PA, the grasped object is determined to belong to category A. The upper and lower load limits F1A and F2A associated with category A are obtained from a pre-stored database. In the following steps, the relationship between the current load and F1A and F2A is directly determined to determine whether to increase the load, reset, or enter the grasping phase (see steps S23-S26 above for details, which will not be repeated here). This approach requires minimal computation and offers fast response, meeting the needs of general applications.

[0117] In a preferred embodiment, the upper load limit information determined in step S226 includes the upper load limits corresponding to each category and the probability that the grasped object belongs to each category. The lower load limit information includes the lower load limits corresponding to each category and the probability that the grasped object belongs to each category. For example, the upper load limit F1A and lower load limit F2A for category A, the upper load limit F1B and lower load limit F2B for category B, the upper load limit F1C and lower load limit F2C for category C, and the upper load limit F1D and lower load limit F2D for category D. Through the above processing, the probabilities of grasped objects belonging to the four categories are obtained as PA, PB, PC, and PD.

[0118] Furthermore, step S23 specifically includes: comparing the current load with the load upper limit of each category respectively, adding the probabilities of the categories that exceed the load upper limit to obtain a first cumulative probability; judging whether the first cumulative probability is lower than a first probability threshold; when the first cumulative probability is lower than the first probability threshold, judging that the current load will not damage the grasped object.

[0119] pass Figure 4 Expressing the relationship between the load limit, probability, and current load, the horizontal axis represents the load and the vertical axis represents the probability. Since the current load is greater than the load limit F1A of category A and greater than the load limit F1B of category B, PA and PB are added together to obtain the first cumulative probability. If PA + PB is greater than the first probability threshold, it is determined that the current load will damage the grasped object; otherwise, it is determined that the current load will not damage the grasped object.

[0120] Step S24 specifically includes: comparing the current load with the load lower limit of each category respectively, adding the probabilities of the categories exceeding the load lower limit to obtain a second cumulative probability; judging whether the second cumulative probability is lower than the second probability threshold; when the second cumulative probability is lower than the second probability threshold, judging that the current load is insufficient to grab the object.

[0121] pass Figure 5 The relationship between the load lower limit, probability, and current load is expressed. The horizontal axis represents the load and the vertical axis represents the probability. Since the current load is greater than the load lower limit F2A of category A and greater than the load lower limit F2B of category B, PA and PB are added to obtain the second cumulative probability. If PA + PB is greater than the second probability threshold, it is determined that the current load can grasp the object; otherwise, it is determined that the current load is insufficient to grasp the object.

[0122] The above-mentioned method of calculating the cumulative probability and comparing it with the probability threshold to judge the current load can improve the accuracy of the judgment result, especially when the obtained probabilities of the grasped objects belonging to various categories are close or inconsistent with the actual situation. The above-mentioned preferred scheme can avoid entering the grasping stage incorrectly.

[0123] In one embodiment, the current load includes two different values, so as to be applied to the above steps S23 and S24 respectively. Specifically, in this embodiment, the force of each piece of electronic skin of the robot hand is array data , Indicates the number of electronic skins, j and k indicate the rows and columns of the array. For the sake of convenience, assuming that each of the five fingers is equipped with one electronic skin and the palm is equipped with one electronic skin, then the first characteristic contact force value is the maximum value of all array data, i.e., max ( The second characteristic contact force value is the smaller of the maximum value in the array data collected by the electronic skin of the thumb and the maximum value in the array data collected by the electronic skin of other fingers. The maximum value in the array data of the thumb is recorded as max ( ), the maximum value in the array data of the other four fingers is recorded as max ( , , , ), then the second characteristic contact force value is min (max ( ), max( 、 、 、 )).

[0124] In step S23, the current load can be max( ), in the above step S24, the current load can be min (max ( ), max( 、 、 、 )), in step S26, the initial value of the load can be set to any one of the above two characteristic values.

[0125] For the grasping stage, i.e., step S3 above, in one embodiment, the load is increased as follows:

[0126] S31A: Monitoring data is collected as the robotic arm grasps and moves the object. In this embodiment, the monitoring data is force feedback data, specifically the pressure or stress at the contact point between the robotic arm and the object, as well as the force of the drive device that changes the load. This method can collect force feedback data at a fixed periodicity, and the value of the force feedback data will change with the longitudinal or lateral movement of the robotic arm.

[0127] S32A, predicting the sliding distance of the grasped object relative to the robot arm based on the force feedback data at multiple moments. The multiple moments may include the current acquisition moment and one or more acquisition moments before it. For example, the initial moment of entering the grasping phase is t0, and the current acquisition moment is t i , then multiple moments can be t0~t i (i.e. all moments from the initial moment to the current moment), or t i-n、 t i (i.e. the current moment and the n moments before it), or t j ~t i (t j Refers to the last time the load was changed or set).

[0128] The sliding distance can be calculated using a pre-established mathematical model. For example, the corresponding sliding distance can be calculated based on the changing trend and rate of change of force feedback data at multiple moments. Alternatively, a neural network model suitable for processing sequence data, such as LSTM (Long Short-Term Memory), can be used to take force feedback data at multiple moments as input and output the sliding distance.

[0129] It should be noted that the sliding distance obtained in step S32A refers to the distance the grasped object is expected to slide, rather than the distance it has slid before.

[0130] S33A: Determine whether the sliding distance exceeds a first distance threshold. If the sliding distance exceeds the first distance threshold, it indicates that the object is expected to slide beyond a certain distance under the current load, thereby posing a risk of falling from the robot's grip. Therefore, step S34A is executed. Otherwise, the load remains unchanged and the process returns to step S31A.

[0131] In an optional embodiment, step S33A further includes determining whether the sliding distance exceeds a second distance threshold (the second distance threshold is greater than the first distance threshold, for example, the first distance threshold is 2 mm and the second distance threshold is 7 mm); when the sliding distance does not exceed the second distance threshold, determining whether the sliding distance exceeds the first distance threshold; when the sliding distance exceeds the second distance threshold, it indicates that the sliding distance of the grasped object is expected to be large under the current load, and even if the load is increased, it may not be possible to stop the object from sliding. In this case, the robot arm can be moved downward to place the grasped object back on the load-bearing plane in preparation for re-grasping.

[0132] S34A: Determine the load increase based on the sliding distance. A correspondence between sliding distance and load increase can be pre-established, for example, with different sliding distance intervals corresponding to different load increases. If the sliding distance falls within a certain interval, the load increase corresponds to that interval. Alternatively, a mathematical calculation model or a neural network model can be used to calculate the load increase based on the sliding distance.

[0133] S35A: Set a target load using the load increment and the current load, and increase the load until the target load is reached. For example, target load = current load + load increment. Load can be understood as a force-related characteristic value that characterizes the grip strength of the robotic arm on the object. The characteristic value can be one or more, specifically, a portion of the force feedback data or a value calculated based on the force feedback data.

[0134] The method for setting the gripping load of a robotic arm provided in an embodiment of the present invention uses feedback data at multiple moments to predict the sliding distance of an object relative to the robotic arm. It can obtain a target load suitable for the current grasped object before the object actually slides, thereby effectively preventing the sliding trend. In addition, this scheme sets the load increment based on the predicted sliding distance, so that the size of the target load is reasonable, avoiding damage to the object while preventing sliding.

[0135] In one embodiment, to further prevent the possibility of damage to the grasped object, the sum of the current load and the load increase (the total load) can be calculated to determine whether the total load exceeds an upper load limit. The upper load limit is the load above which the grasped object is likely to be damaged (this can be called the damage load). The upper load limit is a preset value, which can be calculated by a set algorithm during the loading phase or manually set. If the total load does not exceed the upper load limit, the total load is set to the target load; if the total load exceeds the upper load limit, the upper load limit is set to the target load.

[0136] In one embodiment, the force feedback data includes pressure data collected by the electronic skin installed at the contact point between the robotic hand and the grasped object, as well as driving force data of each of the robotic hand's actuators. Furthermore, the pressure data is array data collected by the electronic skin installed on the palm and multiple fingers of the robotic hand.

[0137] The contact parts between the robotic hand and the grasped object and the driving device for changing the load depend on the structure of the robotic hand. Taking the humanoid mobile phone robotic hand as an example, the contact parts are the surfaces of five fingers and one palm. The driving device for changing the load can be a device for driving the movement of the finger joints. Specifically, each movable joint can correspond to an independent driving device.

[0138] For a humanoid robotic hand, the pressure data collected by the electronic skin refers to the force (pressure or stress) exerted by the grasped object on the surface of the fingers and palm. The driving force data refers to the force exerted on the driving device.

[0139] As an example, The first electronic skin Rank The pressure data is recorded as ; Driving force data .

[0140] According to the above example, the size of the electronic skin at different positions in the robotic hand or the number of measurement points therein may be different, and thus the number of rows and columns of the array may be different.

[0141] Furthermore, regarding the load (current load and target load), assuming that each of the five fingers is provided with one piece of electronic skin and the palm is provided with one piece of electronic skin, the load can be the maximum value of all array data, that is, max ( The load can also be the smaller of the maximum value in the array data collected by the electronic skin of the thumb and the maximum value in the array data collected by the electronic skin of other fingers. The maximum value in the array data of the thumb is recorded as max ( ), the maximum value in the array data of the other four fingers is recorded as max ( , , , ), then the load is min(max( ), max( 、 、 、 )).

[0142] like Figure 6 As shown, in one embodiment, step S32A uses a neural network model to predict the sliding distance, which specifically includes the following processing:

[0143] The time series prediction module predicts the pressure feature vector at the next time point according to the array data at multiple time points. Specifically, several ConvLSTM (Convolutional Long Short-Term Memory) are used to normalize the pressure data of each block of electronic skin and input the data into several continuous ConvLSTM networks. This layer extracts spatial features at each time step and captures dynamic changes between time steps, and outputs new features and normalizes them. Then, through the Flatten layer, the multi-dimensional data is flattened into a one-dimensional vector (the pressure feature vector is obtained by taking the last time step and flattening it). The above layers process each block of electronic skin respectively, and the parameters of the layers are shared.

[0144] The vertical driving force component is extracted from the driving force data, and the time series prediction module predicts the driving force feature vector at the next time point according to the vertical driving force component at multiple time points. Specifically, several LSTM are used to input the vertical driving force component in the driving force data into several LSTM layers, and output the driving force feature vector after transformation by the several LSTM layers.

[0145] The global feature vector is obtained by coupling the pressure feature vector and the driving force feature vector.

[0146] The sliding distance is obtained according to the global feature vector. Specifically, an MLP (Multi-Layer Perceptron) is used to input the global feature vector into the MLP as input data, and output the sliding distance.

[0147] After obtaining the target load through the above steps S31A-S35A, the current load of the robot hand is adjusted until the target load is reached, which is referred to as the second adjustment operation in the present application; in one embodiment, another load adjustment operation based on threshold comparison is also used to jointly adjust the load in the gripping stage.

[0148] The present embodiment provides a load adjustment method based on threshold comparison (referred to as the first adjustment operation in the present application), which can be executed by a computer or a server or other electronic device, comprising:

[0149] S31B, according to the monitoring data at the current time and the monitoring data at the previous time, it is judged whether the grabbed object slides or not, and when the grabbed object slides, step S32B is executed, otherwise the detection is continued. Unlike S32A, S31B only gives a yes or no judgment result without calculating the sliding distance.

[0150] ​​The monitoring data in this embodiment can utilize the same force-related data as in step S31A above, including pressure data collected by the electronic skin located at the contact point between the robotic hand and the grasped object, as well as driving force data from each of the robotic hand's actuators. Furthermore, the pressure data comprises array data collected by the electronic skin located on the palm and multiple fingers of the robotic hand.

[0151] To improve computational efficiency, the force feedback data used in this embodiment is the sum of the pressure data collected by each electronic skin, as well as the vertical component extracted from the driving force data of a specific driving device. As a preferred embodiment, step S31B includes: calculating the difference in total pressure based on the sum of the pressure data at the current moment and the previous moment, and calculating the difference in vertical component based on the vertical driving force components at the current moment and the previous moment; determining whether either the difference in total pressure or the difference in vertical component exceeds a corresponding set threshold; and determining that the grasped object has slipped when either the difference in total pressure or the difference in vertical component exceeds a corresponding set threshold.

[0152] S32B, increase the load, the load increase amount is fixed.

[0153] On this basis, step S3 can simultaneously adopt the above two adjustment operations, specifically including:

[0154] During the process of the robot arm grasping and moving the grasped object, the first adjustment operation is triggered according to the first cycle and the second adjustment operation is triggered according to the second cycle. The first adjustment operation is detailed in steps S31B~S32B, and the second adjustment operation is detailed in steps S31A~S35A.

[0155] The triggering periods of the two different adjustment operations are different to ensure that they are not triggered simultaneously at the same time step. Preferably, the second period is greater than the first period. The load increase amount in the first adjustment operation is a fixed amount, while the load increase amount in the second adjustment operation is a variable amount. In a preferred embodiment, the fixed amount is configured to be less than the minimum value of the variable amount, for example, different orders of magnitude can be used.

[0156] As can be seen, when the first adjustment operation is triggered and it is determined that the load needs to be increased, a smaller increase is applied; when the second adjustment operation is triggered and it is determined that the load needs to be increased, a relatively large and non-fixed increase is applied. Furthermore, after the second adjustment operation is triggered, the first adjustment operation is suspended until the current load reaches the target load.

[0157] For example, the initial load is 1.0N, the force feedback data collection period is 10ms, the first period is 20ms, and the second period is 100ms. Two data sets M1 and M2 are set. M1 is used to store the force feedback data B required for the second adjustment operation (the pressure of each electronic skin, the vertical component of the driving force of each driving device), and M2 is used to store the force feedback data A required for the first adjustment operation (the total pressure, the vertical component of the specific driving device). The initial time of entering the grasping stage is recorded as t0. An example of adjusting the load is as follows:

[0158] t0+10ms: The force feedback data A of the first time step is added to the dataset M1.

[0159] t0+20ms: Trigger the first adjustment operation. At this time, data set M2 is empty, and the load is not adjusted. Force feedback data B is calculated based on force feedback data A at the first time step and stored in M2.

[0160] t0+30ms: The force feedback data A of the second time step is added to M1.

[0161] t0+40ms: The first adjustment operation is triggered. The force feedback data B stored in M2 is compared with the force feedback data B calculated from the second time step, and the difference between the total pressure and the vertical component is calculated to determine whether slip has occurred (steps S31B-S32B). If no slip has occurred, no load adjustment is made, and the force feedback data B calculated from the second time step overwrites M2. This calculation is completed within a few milliseconds.

[0162] …

[0163] t0+80ms: Trigger the first adjustment operation. Calculate the difference between the total pressure and the vertical component of the force feedback data B stored in M2 and the force feedback data B calculated from the fourth time step. Determine whether slip has occurred (steps S31B-S32B). If slip is determined to have occurred, increase the load to 1.02N (with a fixed increment of 0.02N). Overwrite M2 with the force feedback data B calculated from the fourth time step.

[0164] t0+90ms: The force feedback data A of the fifth time step is added to M1.

[0165] t0+100ms: Trigger the first adjustment operation. Assume that the load is increased to 1.04 N. Overwrite M2 with the force feedback data B calculated from the fifth time step data.

[0166] t0+105ms: The second adjustment operation is triggered. In a separate thread, the target load is set based on the force feedback data A from t0 to the triggering moment. Steps S31A-S35A are now used to determine the target load. In practice, since this calculation process takes time, the first adjustment operation is continued for several time steps until the result is obtained. It should be noted that the current load in step S35A refers to the load at the time the second adjustment operation is triggered, i.e., 1.04N.

[0167] t0+110ms: The force feedback data A of the sixth time step is added to M1.

[0168] t0+120ms: Trigger the first adjustment operation. Assume that the load is increased to 1.06 N. Overwrite M2 with the force feedback data B calculated from the sixth time step data.

[0169] t0+130ms: The force feedback data A of the 7th time step is added to M1.

[0170] 0+140ms: Trigger the first adjustment operation. Assume that the load is increased to 1.08 N. Overwrite M2 with the force feedback data B calculated from the 7th time step data.

[0171] t0+145ms: Steps S31A-S35A are completed, and the output sliding distance is 3mm, which is greater than the first threshold of 2mm but less than the second threshold of 7mm. The load increase is determined to be 0.2N. The target load is equal to the load at the time the second adjustment operation was triggered (t0+105ms): 1.04N + 0.2N = 1.24N. The current load is 1.08N, and the target load is 1.24N. The load is increased until the target load is reached. The first adjustment operation is not triggered again until the adjustment is complete.

[0172] t0+150ms: The force feedback data A of the 8th time step is added to M1.

[0173] t0+170ms: The force feedback data A of the 9th time step is added to M1.

[0174] The above examples are only for illustrating how to use two adjustment operations to control the load at the same time. The specific values ​​of the load value, cycle value, etc. used in the examples are only for reference and do not limit the values ​​of the relevant data.

[0175] According to the robot arm grasping load adjustment method provided by an embodiment of the present invention, two different adjustment operations are used to adjust the robot arm load. Through the first adjustment operation with a faster response speed, it is ensured that the load is increased in time when the grasped object slides. Through the second adjustment operation with a higher accuracy, the sliding distance is predicted and a suitable target load is set, thereby stopping the sliding trend. This solution takes into account both response speed and accuracy, and improves the adaptability of the robot arm in grasping objects.

[0176] like Figure 7 As shown, an embodiment of the present application provides an adaptive grasping system, including a robotic arm 110 and a control device 400. The control device 400 is used to execute the above grasping method. The robotic arm 110 is provided with a monitoring device for collecting monitoring data. In this embodiment, the monitoring device 210 is provided on the front ends of the fingers and the palm, which is used to collect pressure, temperature, etc. In addition, sensors for collecting driving force and displacement can be installed inside the robotic arm (not shown in the figure). Such sensors can be installed at the movable joints of the robotic arm.

[0177] This embodiment adopts a visual positioning method, and for this purpose, an image acquisition device 300 is provided. The device can be replaced by other positioning devices, such as radar equipment, etc.; in the embodiment of fixed-point grasping, the positioning device may also be omitted.

[0178] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0179] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0180] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0181] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0182] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will readily appreciate that other variations or modifications based on the above descriptions are possible. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A robotic arm adaptive grasping method, characterized in that: include: Control the robot arm to approach the grasped object and set the initial load to contact the grasped object; A neural network model is used to extract global feature vectors from monitoring data when the robot arm is loading a grasped object with a current load, and classification is performed based on the global feature vectors to obtain a classification vector for the grasped object category; Determining the probability values ​​of the grasped object belonging to various categories based on the classification vector; Determining the upper and lower load limits based on the probability values ​​of the grasped object belonging to various categories and the upper and lower load limits corresponding to the various categories; and determining whether the current load will damage the grasped object based on the upper load limit information. If it is determined that the current load will not damage the grasped object, then determining whether the current load is sufficient to grasp the grasped object based on the load lower limit information; if it is determined that the current load is insufficient to grasp the grasped object, then increasing the load; otherwise, attempting to grasp the grasped object with the current load; wherein the monitoring data includes feedback data of the contact point between the robot hand and the grasped object and feedback data of the drive device that changes the load; Adjusting the current load based on monitoring data when the robotic arm grasps and moves the grasped object, including determining whether the grasped object is slipping based on the monitoring data at the current moment and the monitoring data at the previous moment, increasing the load when the grasped object slips, until the grasping load is determined, and monitoring whether the robotic arm has moved the grasped object above the target position; After the robot arm moves the grasped object to above the target position, the robot arm is controlled to place the grasped object to the target position.

2. The method according to claim 1, characterized in that The feedback data of the contact part between the robot hand and the grasped object includes pressure data collected by the electronic skin provided at the contact part between the robot hand and the grasped object.

3. The method according to claim 1, characterized in that The feedback data of the contact part between the robot hand and the grasped object includes temperature data collected by the electronic skin provided at the contact part between the robot hand and the grasped object.

4. The method according to claim 1, wherein The feedback data of the drive devices that change the load include displacement data and force data of each drive device of the robot arm.

5. The method according to claim 1, wherein The load upper limit information includes the load upper limits corresponding to various categories and the probability that the grasped object belongs to various categories; the load lower limit information includes the load lower limits corresponding to various categories and the probability that the grasped object belongs to various categories.

6. The method according to claim 5, characterized in that Determining whether the current load will damage the grasped object based on the load upper limit information includes: Comparing the current load with the load upper limit of each category respectively, and adding the probabilities of the categories exceeding the load upper limit to obtain a first cumulative probability; Determining whether the first cumulative probability is lower than a first probability threshold; When the first cumulative probability is lower than a first probability threshold, determining that the current load will not damage the grasped object; Determining whether the current load is sufficient to grasp the object according to the load lower limit information includes: Comparing the current load with the load lower limit of each category respectively, and adding the probabilities of the categories exceeding the load lower limit to obtain a second cumulative probability; Determining whether the second cumulative probability is lower than a second probability threshold; When the second cumulative probability is lower than a second probability threshold, it is determined that the current load is insufficient to grasp the object.

7. The method according to claim 1, characterized in that Adjust the current load based on the monitoring data when the robot grasps and moves the grasped object, including: Predicting the sliding distance of the grasped object relative to the robotic arm based on the monitoring data at multiple moments; Determining whether the sliding distance exceeds a first distance threshold; When the sliding distance exceeds the first distance threshold, determining a load increase amount according to the sliding distance; A target load is set using the load increase amount and the current load, and the load is increased until the target load is reached.

8. The method according to claim 1, characterized in that Adjust the current load based on the monitoring data when the robot grasps and moves the grasped object, including: A first adjustment operation is triggered according to a first cycle, and a second adjustment operation is triggered according to a second cycle. The first adjustment operation includes determining whether the grasped object slides based on the monitoring data at the current moment and the monitoring data at the previous moment, and increasing the load when the grasped object slides; the second adjustment operation includes predicting the sliding distance of the grasped object relative to the robot arm based on the monitoring data at multiple moments; determining whether the sliding distance exceeds a first distance threshold; when the sliding distance exceeds the first distance threshold, determining the load increase amount based on the sliding distance; using the load increase amount and the current load to set a target load, and increasing the load until the target load is reached.

9. The method according to claim 7 or 8, characterized in that The monitoring data includes pressure data collected by the electronic skin provided at the contact portion between the robotic hand and the grasped object, and driving force data of each driving device of the robotic hand.

10. The method according to claim 9, characterized in that The pressure data is array data collected by electronic skins respectively provided on the palm and multiple fingers of the robotic hand.

11. The method according to claim 10, characterized in that Predicting the sliding distance of the grasped object relative to the robotic arm based on the monitoring data at multiple moments includes: Predicting the pressure feature vector at the next moment based on the array data at multiple moments by a time series prediction module; Extracting a vertical driving force component from the driving force data, and using a time series prediction module to predict a driving force feature vector at a next moment based on the vertical driving force component at multiple moments; Connecting the pressure eigenvector and the driving force eigenvector to obtain a global eigenvector; A sliding distance is obtained according to the global feature vector.

12. The method according to claim 8, characterized in that After the second adjustment operation is triggered, before the current load reaches the target load, triggering of the first adjustment operation is suspended.

13. The method according to claim 12, characterized in that The load increase amount when the first adjustment operation is triggered is a fixed amount, and the fixed amount is smaller than the load increase amount in the second adjustment operation.

14. A control device, characterized in that: include: A processor and a memory connected to the processor; wherein the memory stores instructions that can be executed by the processor, and the instructions are executed by the processor to enable the processor to perform the robot hand adaptive grasping method according to any one of claims 1 to 13.

15. An adaptive grasping system, characterized in that: It comprises a robot arm and the control device according to claim 14; wherein the robot arm is provided with a monitoring device for collecting the monitoring data.

Images