Self-adaptive grabbing method and system for robot arm

Abstract

The invention provides a robot arm self-adaptive grabbing method and system, and the method comprises the steps: controlling a robot arm to approach a grabbed object, and setting an initial load to contact the grabbed object; the load is increased according to monitoring data when the robot arm loads the grabbed object with the current load until the loaded load is determined, and then the robot arm is controlled to grasp the grabbed object with the loaded load; the current load is adjusted according to monitoring data when the robot arm grabs and moves the grabbed object till the grabbed load is determined, and whether the robot arm moves the grabbed object to the position above the target position or not is monitored; and after the robot arm moves the grabbed object to the position above the target position, the robot arm is controlled to place the grabbed object to the target position.

Description

Technical Field

[0001] The present invention relates to the field of robot arm control, and in particular to a robot arm adaptive grasping method and system. Background Art

[0002] Robotic arms can be used in a variety of industries. Currently, the automated grasping strategies of robotic arms are usually designed for some fixed application scenarios, so that they can grasp specific objects. Obviously, the grasping strategies of robotic arms for a certain application scenario are difficult to be transferred to other application scenarios.

[0003] In order to make the robot arm suitable for a variety of application scenarios and to grasp a variety of different objects, it is necessary to design a grasping strategy with strong adaptability. At present, some grasping strategies use a smaller load to grasp the object and move it, and increase the load by monitoring the relative displacement between the object and the robot arm after the movement. The problem with this strategy is that for some soft objects, the robot arm may start to move without actually grasping the target. Although the displacement can be detected during the movement, increasing the load speed cannot prevent the object from completely slipping off; on the contrary, some grasping strategies use a larger load to grasp the object and move it, and then reduce the load according to some feedback data to avoid damaging the object. The problem with this strategy is that for some fragile objects, the larger load may directly cause the object to break when grasping the object.

[0004] This shows that the existing robotic grasping strategies are not adaptable enough, are difficult to handle with diverse grasped objects, and have a low grasping success rate. Summary of the invention

[0005] In view of this, the present application provides a robot arm adaptive grasping method, comprising: Control the robot arm to approach the grasped object and set the initial load to contact the grasped object; Increasing the load according to monitoring data when the robot arm loads the grasped object with the current load until the loading load is determined, and then controlling the robot arm to grasp the grasped object with the loading load; The current load is adjusted according to the monitoring data when the robot grasps and moves the grasped object until the grasping load is determined, and the robot is monitored to see whether the grasped object is moved above the target position; After the robot arm moves the grasped object to above the target position, the robot arm is controlled to place the grasped object to the target position.

[0006] Optionally, the monitoring data includes feedback data of the contact point between the robot arm and the grasped object and feedback data of a drive device for changing the load.

[0007] Optionally, increasing the load according to monitoring data when the robot arm loads the grasped object with the current load includes: Extracting a global feature vector from the feedback data using a neural network model, and determining load upper limit information and load lower limit information according to the global feature vector; Determining whether the current load will damage the grasped object according to the load upper limit information; If it is determined that the current load will not damage the grasped object, determining whether the current load is sufficient to grasp the grasped object according to the load lower limit information; If it is determined that the current load is not sufficient to pick up the object, the load is increased.

[0008] Optionally, the feedback data of the contact portion between the robot hand and the grasped object includes pressure data collected by an electronic skin arranged at the contact portion between the robot hand and the grasped object.

[0009] Optionally, the feedback data of the contact portion between the robot hand and the grasped object includes temperature data collected by an electronic skin arranged at the contact portion between the robot hand and the grasped object.

[0010] Optionally, the feedback data of the drive device for changing the load includes displacement data and force data of each drive device of the robot.

[0011] Optionally, determining the load upper limit information and the load lower limit information according to the global eigenvector includes: Classifying according to the global feature vector to obtain a classification vector for the category of the grasped object; Determining the probability values ​​of the grasped object belonging to various categories according to the classification vector; The upper load limit information and the lower load limit information are determined according to the probability values ​​of the grasped objects belonging to various categories and the upper load limits and lower load limits corresponding to the various categories.

[0012] Optionally, the load upper limit information includes load upper limits corresponding to various categories and probabilities that the grasped objects belong to various categories; the load lower limit information includes load lower limits corresponding to various categories and probabilities that the grasped objects belong to various categories.

[0013] Optionally, judging whether the current load will damage the grasped object according to the load upper limit information includes: Comparing the current load with the load upper limit of each category respectively, and adding the probabilities of the categories exceeding the load upper limit to obtain a first cumulative probability; Determining whether the first cumulative probability is lower than a first probability threshold; When the first cumulative probability is lower than a first probability threshold, it is determined that the current load will not damage the grasped object; Judging whether the current load is sufficient to grasp the grasped object according to the load lower limit information includes: Comparing the current load with the load lower limit of each category respectively, and adding the probabilities of the categories exceeding the load lower limit to obtain a second cumulative probability; Determining whether the second cumulative probability is lower than a second probability threshold; When the second cumulative probability is lower than a second probability threshold, it is determined that the current load is insufficient to grasp the grasped object.

[0014] Optionally, the current load is adjusted based on monitoring data when the robot grips and moves the grasped object, including: Whether the grasped object is sliding is determined based on the monitoring data at the current moment and the monitoring data at the previous moment, and the load is increased when the grasped object is sliding.

[0015] Optionally, the current load is adjusted based on monitoring data when the robot grips and moves the grasped object, including: Predicting the sliding distance of the grasped object relative to the robot hand based on the monitoring data at multiple moments; Determining whether the sliding distance exceeds a first distance threshold; When the sliding distance exceeds the first distance threshold, determining a load increase amount according to the sliding distance; A target load is set using the load increase amount and the current load, and the load is increased until the target load is reached.

[0016] Optionally, the current load is adjusted based on monitoring data when the robot grips and moves the grasped object, including: A first adjustment operation is triggered according to a first cycle, and a second adjustment operation is triggered according to a second cycle, wherein the first adjustment operation includes determining whether the grasped object slides according to the monitoring data at a current moment and the monitoring data at a previous moment, and increasing the load when the grasped object slides; the second adjustment operation includes predicting a sliding distance of the grasped object relative to the robot arm according to the monitoring data at multiple moments; determining whether the sliding distance exceeds a first distance threshold; determining a load increase amount according to the sliding distance when the sliding distance exceeds the first distance threshold; setting a target load using the load increase amount and the current load, and increasing the load until the target load is reached.

[0017] Optionally, the monitoring data includes pressure data collected by an electronic skin disposed at a contact portion between the robot arm and the grasped object, and driving force data of each driving device of the robot arm.

[0018] Optionally, the pressure data is array data collected by electronic skins respectively arranged on the palm and multiple fingers of the robotic hand.

[0019] Optionally, predicting the sliding distance of the grasped object relative to the robot hand according to the monitoring data at multiple moments includes: Predicting the pressure feature vector at the next moment according to the array data at multiple moments by a time series prediction module; Extracting a vertical driving force component from the driving force data, and using a time series prediction module to predict a driving force feature vector at a next moment based on the vertical driving force component at multiple moments; Connecting the pressure characteristic vector and the driving force characteristic vector to obtain a global characteristic vector; The sliding distance is obtained according to the global feature vector.

[0020] Optionally, after the second adjustment operation is triggered, before the current load reaches the target load, triggering of the first adjustment operation is paused.

[0021] Optionally, the load increase amount when the first adjustment operation is triggered is a fixed amount, and the fixed amount is smaller than the load increase amount in the second adjustment operation.

[0022] Accordingly, the present application provides a control device, characterized in that it includes: a processor and a memory connected to the processor; wherein the memory stores instructions that can be executed by the processor, and the instructions are executed by the processor so that the processor executes the above-mentioned robot arm adaptive grasping method.

[0023] The present application also provides an adaptive grasping system, comprising a robot arm and the above-mentioned control device; wherein the robot arm is provided with a monitoring device for collecting the monitoring data.

[0024] The adaptive grasping method and device for a robotic arm provided in the present application divide the grasping process into four stages: in the positioning stage, a small initial load is used to attempt to contact the grasped object; in the loading stage, the load is gradually increased to ensure that the target is grasped without damaging it; in the grasping stage, the load is continuously and dynamically adjusted during the movement to prevent the variables generated by the movement from causing the object to slip; in the placement stage, the target is placed after ensuring that the grasped object is moved above the target position. The processing methods of the above four stages can handle a variety of grasped objects and improve the grasping success rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 is a flow chart of a method for adaptive grasping by a robot hand in an embodiment of the present invention; Figure 2 A schematic diagram of a neural network model architecture in an embodiment of the present invention; Figure 3 A schematic diagram of multi-step joint data processing in an embodiment of the present invention; Figure 4 is a load upper limit and probability distribution diagram in an embodiment of the present invention; Figure 5 is a load lower limit and probability distribution diagram in an embodiment of the present invention; Figure 6 is a schematic diagram of another neural network model architecture in an embodiment of the present invention; Figure 7 Schematic diagram of the structure of the adaptive grasping system in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0029] The embodiment of the present invention provides a robot hand adaptive grasping method, which can be executed by an electronic device such as a computer or a server. Figure 1 The following operations are shown: S1, control the robot arm to approach the grasped object and set the initial load to contact the grasped object. This step is called the positioning stage. First, the position of the grasped object needs to be determined. For example, an optical positioning solution or a radar signal positioning solution can be used to identify the position of the grasped object in space and then control the robot arm to move toward the grasped object; or a fixed-point contact method can be used, that is, the position of the grasped object is fixed in advance, and the robot arm moves to the predetermined position.

[0030] The load mentioned in this application refers to the force applied by the robot hand to the grasped object. The greater the load, the greater the gripping force of the robot hand on the object. When the robot hand contacts the object and generates interaction force, the force of each part can be collected by the sensor set on the robot hand. The actual load of the robot hand can be determined based on these forces. The initial load is configured as a smaller value. When the actual load of the robot hand when it contacts the grasped object reaches the initial load, this step is completed.

[0031] In one embodiment, a visual positioning solution is adopted. A two-dimensional image camera or a three-dimensional camera that can measure point clouds can be used to collect images of the position of the grasped object. First, the user's requirements for the target object are obtained, and visual imaging is performed through the camera. The target object is selected based on the visual imaging results, and the robot arm is moved close to the target object position. Then, the load is set to a very small value F0, and trial touch and adjustment are performed until the optimal state is reached.

[0032] S2, increase the load according to the monitoring data when the robot arm loads the grasped object with the current load until the loading load is determined, and then control the robot arm to grasp the grasped object with the loaded load. This step is called the loading stage. In this stage, the robot arm tries to grasp the grasped object in the original place where the grasped object is located without moving the position of the grasped object. The current load when entering the loading stage is the initial load, and the current load will gradually increase during the execution of this step. What needs to be determined in this stage is the load value that can be grasped but not damaged. The loading load is greater than the initial load.

[0033] Monitoring data refers to the data generated by the interaction between the robot and the grasped object, such as force, displacement, temperature, etc. Specifically, when the load changes, the pressure on the robot will change, and the position of the movable joints of the robot will be displaced. In this step, according to the corresponding relationship between these monitoring data and the load, the load is dynamically adjusted until a value is determined that can grasp but damage the grasped object. This can be implemented by using algorithms such as mathematical models or neural network models.

[0034] In addition, the loading load may not be determined at this stage. In this case, the load can be set to 0 and the process returns to step S1, where the robot arm's position or the contact point with the grasped object is adjusted and step S2 is re-entered. The number of retry attempts can also be recorded, and the grasping process is abandoned if the number of attempts exceeds a threshold.

[0035] S3, adjust the current load according to the monitoring data when the robot grasps and moves the grasped object, until the grasping load is determined, and monitor whether the robot moves the grasped object above the target position. When the robot moves the grasped object above the target position, execute step S4. This step is called the grasping stage. In this stage, the robot grasps the object and moves (upward, horizontally above the target position). The current load when entering the grasping stage is the loading load determined in step S2. During the execution of this step, the current load may gradually increase or remain unchanged, that is, the loading load ≥ the loading load.

[0036] In this step, as the robot moves while holding the object, the monitoring data will change. The change in load will also cause the monitoring data to change. This step needs to determine whether the object will slip from the robot hand based on the change in monitoring data, and increase the load in time when it slips or is likely to slip, to avoid the slipping event. This step will eventually stabilize the load, and the result is called the gripping load.

[0037] It should be noted that there may be multiple types of monitoring data. The same data or partially the same data may be used in step S2 and step S3, or completely different data may be used respectively. For example, force and displacement may be used in step S2, and only force may be used in step S3.

[0038] S4, control the robot arm to place the grasped object at the target position. This step is called the placement phase. In this phase, the load is not increased. The robot arm is controlled to move downward so that the grasped object contacts the load plane and then the load is set to 0 to end the grasping process.

[0039] The adaptive grasping method of a robotic arm provided in an embodiment of the present invention divides the grasping process into four stages: in the positioning stage, a small initial load is used to try to contact the grasped object; in the loading stage, the load is gradually increased to ensure that the target is grasped without damaging the target; in the grasping stage, the load is continuously and dynamically adjusted during the movement to prevent the variables generated by the movement from causing the object to slip; in the placement stage, the target is placed after ensuring that the grasped object is moved above the target position. The processing methods of the above four stages can deal with a variety of grasped objects and improve the success rate of grasping.

[0040] For the loading stage, i.e., the above step S2, in one embodiment, the load is increased in the following manner: S21, obtaining monitoring data when the robot arm loads the grasped object with the current load, wherein the monitoring data includes feedback data of the contact part between the robot arm and the grasped object and feedback data of the driving device for changing the load.

[0041] The loading in this method refers to the process of the robot arm grasping and holding an object on a supporting surface (such as the ground, a table or other platform, etc.), and does not include the action of moving the object upward or in other directions after grasping. This method can be executed synchronously with the change process of the robot arm load. The initial load is recorded as F0. The robot arm can contact the grasped object with load F0, and then gradually increase the load. As the load changes, the corresponding monitoring data also changes.

[0042] The contact parts between the robot hand and the grasped object and the driving device for changing the load depend on the structure of the robot hand. Taking the humanoid mobile robot hand as an example, the contact parts are the surfaces of five fingers and one palm. The driving device for changing the load can be a device for driving the movement of finger joints, and specifically, each movable joint can correspond to an independent driving device.

[0043] The feedback data of the contact point between the robot hand and the grasped object may specifically include force. For a humanoid robot hand, it refers to the force (pressure or intensity) exerted by the grasped object on the surface of the fingers and palm; in an optional embodiment, it may also include temperature.

[0044] The feedback data of the drive device for changing the load may specifically include force and / or displacement. For the anthropomorphic robot hand, it refers to the force borne by the drive device and / or the displacement provided to generate the current load.

[0045] As an example, for the load at any time (time step), the monitoring data obtained can specifically include , , and ,in represents the force at the ith contact point, represents the temperature of the ith contact point, represents the displacement of the ith driving device, represents the force of the ith driving device.

[0046] The load can be understood as a characteristic value related to the force, and the characteristic value can be one or more, specifically, it can be the force data in the monitoring data, or a value calculated based on the force data. For example, the current load can be the value of a certain force among the forces acting on the surface of the finger and the palm, or it can be the sum of these forces, or it can be the force of a certain driving device or the sum of the forces of the driving devices, etc.

[0047] S22, extracting global feature vectors from feedback data using a neural network model, and determining upper and lower load limit information based on the global feature vectors. Feedback data is at least two data from two different parts or drive devices in the robot hand. According to the above example, there may be dozens of feedback data. In order to extract feature vectors, these feedback data need to be fused. The neural network model can use various network structures composed of units such as CNN (Convolutional Neural Network) and MLP (Multi-Layer Perceptron) to realize feature extraction and fusion. There are many specific implementation methods, for example, it can include ResNet (Residual Network) structure.

[0048] The upper load limit refers to the load above which the object is likely to be destroyed (it can be called the destruction load); the lower load limit refers to the load above which it is worth trying to grasp the object, or it can be interpreted as the lower load limit being considered to be the lowest load that can grasp the object.

[0049] It should be noted that the so-called upper load limit and lower load limit in this application are force values, and the upper load limit information and lower load limit information can be force values, or can be information calculated based on the force values, such as statistical information.

[0050] The upper load limit information and the lower load limit information can be the output result of the neural network model, or the mapping information of the output result of the neural network model. For example, the neural network model can be configured to directly output the value of the upper load limit and the value of the lower load limit, and a large amount of sample data is used to train it before use, so that it can output two values ​​according to the above-mentioned feedback data input; the neural network model can be configured to output information about the grasped object, such as the category of the grasped object, and a corresponding relationship library between this information and the upper load limit and the lower load limit is pre-established. When the neural network model outputs the information of the grasped object during use, the corresponding upper load limit and the lower load limit can be queried in the corresponding relationship library.

[0051] S23, judging whether the current load will damage the grasped object according to the load upper limit information. If it is determined that the current load will not damage the grasped object, executing step S24; otherwise, as an optional solution, executing step S26, or stopping the execution of this method.

[0052] An optional method is to use hard classification in step S22 to determine whether the grasped object belongs to a certain category, and then obtain the corresponding load upper limit, or directly output the load upper limit by the neural network in step S22. In step S23, it is determined whether the current load is lower than the load upper limit. If the current load is lower than the load upper limit, it is determined that the current load will not damage the grasped object.

[0053] S24, judging whether the current load is sufficient to grasp the grasped object according to the load lower limit information. If it is determined that the current load is insufficient to grasp the grasped object, executing step S25, otherwise trying to grasp the object with the current load, that is, entering the grasping stage.

[0054] An optional method is to use hard classification in step S22 to determine whether the grasped object belongs to a certain category, and then obtain the corresponding load lower limit, or directly output the load lower limit by the neural network in step S22. In step S24, it is determined whether the current load is lower than the load lower limit. If the current load is lower than the load upper limit, it is determined that the current load is insufficient to grasp the grasped object.

[0055] S25, increase the load. Then return to step S21 and re-execute the method until the current load meets the condition of step S24. The load increase amount can be fixed or calculated based on the difference between the current load and the load lower limit information. For example, if the difference between the current load and the load lower limit is large, the increase amount is large.

[0056] S26, set the load to the initial value, then return to step S21 and re-execute the method. Further, before re-execution of the method, the posture of the robot arm can be fine-tuned so that it can re-try the loading process from a different angle or position. Fine-tuning the posture of the robot arm can be fine-tuning only the angle of the mechanical joint, or re-selecting the contact position.

[0057] According to the robot arm loading method provided by the embodiment of the present invention, the corresponding monitoring data is obtained as the grasping load changes, and then the load upper limit information and the load lower limit information corresponding to the current monitoring data are determined, and the load is adjusted according to the relationship between the current load and the load upper limit information and the load lower limit information, to achieve an adaptive grasping and loading process of loading, identifying, and making decisions at the same time. Grasping objects with appropriate loads can effectively avoid slipping or damage when the objects are subsequently grasped, and has strong adaptability.

[0058] Regarding the monitoring data in the above steps, in one embodiment, the feedback data of the contact part between the robot hand and the grasped object includes pressure data collected by the electronic skin set at the contact part between the robot hand and the grasped object. The pressure data is array data collected by the electronic skin set at the palm and multiple fingers of the robot hand.

[0059] As an example, The first electronic skin Line Column record .

[0060] In one embodiment, the feedback data of the contact part between the robot hand and the grasped object includes temperature data collected by the electronic skin arranged at the contact part between the robot hand and the grasped object. The temperature data is array data collected by the electronic skin arranged at the palm and multiple fingers of the robot hand.

[0061] As an example, The first electronic skin Line Column record .

[0062] According to the above example, the size of the electronic skin at different positions in the robot hand or the number of measurement points therein may be different, so the number of rows and columns of the array is different. In one embodiment, the feedback data of the drive device for changing the load includes the displacement data and force data of each drive device of the robot hand. As an example, for the first The displacement data and force data of each driving device are recorded as .

[0063] The above-mentioned various feedback data can accurately reflect the gripping state. By extracting features from these feedback data, the accuracy of the upper load limit information and the lower load limit information can be improved.

[0064] There are many specific structures of the neural network model in the above step S22. This embodiment provides a specific implementation method. Figure 2 As shown, step S22 includes the following operations: S221, extracting the pressure data collected from each electronic skin to obtain the electronic skin force distribution feature, extracting the temperature data collected from each electronic skin to obtain the electronic skin temperature distribution feature, and fusing the electronic skin force distribution feature and the electronic skin temperature distribution feature to obtain the electronic skin feature vector.

[0065] Specifically, each electronic skin force monitoring value array data They are input into the CNN layer respectively, and then pass through several residual modules respectively. The processed results are averaged to obtain the electronic skin force distribution characteristics; the array data of each electronic skin temperature monitoring value is averaged. They are input into the CNN layer respectively, and then pass through several residual modules respectively. The processed results are averaged to obtain the temperature distribution characteristics of the electronic skin.

[0066] In some embodiments, the scales of the electronic skin temperature distribution feature and the electronic skin force distribution feature may be the same, so they can be directly merged to obtain the electronic skin feature vector. If the scales of the two distribution features are different, adjustments need to be made: The one with higher pixel density between the electronic skin force distribution feature and the electronic skin temperature distribution feature is pooled to adjust the number of pixels to be consistent (connect after unifying the number of pixels); the adjusted electronic skin force distribution feature and electronic skin temperature distribution feature are fused (fusion array); the fusion results are averaged in the array dimension (array average) to obtain the electronic skin feature vector.

[0067] The above layers process each piece of electronic skin separately, and the parameters of the layers are shared.

[0068] S222, extract the data based on the displacement data and force data of the driving device to obtain the driving device feature vector. The displacement data of the drive device at the previous moment Take the difference; compare the difference result with the drive device force data In this embodiment, the MLP module is specifically used to obtain the robot driving device feature vector.

[0069] S223, connecting the electronic skin feature vector and the driving device feature vector to obtain a global feature vector.

[0070] It should be noted that the above module architecture is only one of many optional methods, and is not the only feasible solution. For example, at least normalization, dropout and other processing can be set at appropriate positions in the above process, and other modules can be used to replace the above MLP module and CNN module to form a deformed architecture. The difference between the displacement and force of the driving device reflects the relationship between the force and displacement along the clamping direction, the force distribution of the electronic skin reflects the force distribution characteristics along the contact tangent direction, and the temperature reflects the heat capacity, heat transfer and other information. The above embodiments integrate this rich information, and the result can fully reflect the material properties of the grasped object, thereby making the classification result more accurate.

[0071] In one embodiment, step S22 includes the following operations: S224: Classify according to the global feature vector to obtain a classification vector for the category of the grasped object. Specifically, the global feature vector can be obtained by adopting the above steps S221-S223, or by adopting other network structures, such as LSTM network structures, etc.

[0072] S225, determining the probability value of the grasped object belonging to various categories according to the classification vector. An optional method is to take softmax of the component vectors output by the neural network model to obtain the probability of each category.

[0073] S226: Determine the upper load limit information and the lower load limit information according to the probability values ​​of the grasped object belonging to various categories and the upper load limits and lower load limits corresponding to the various categories.

[0074] In a preferred embodiment, a multi-step joint approach is used to obtain the probability of each category. Figure 3 The step S225 shown specifically includes the following operations: The multi-step joint data is obtained based on the classification vectors at the current moment and at least one previous moment. There is more than one specific calculation method. In this embodiment, the classification vector at each moment is multiplied by a preset coefficient. The preset coefficient in this embodiment is , where M is the number of time steps (the total number of the current moment and at least one previous moment), K and A are preset constants, and then the multiplication results of each moment are summed, and the summation result is combined with the preset background value B to obtain multi-step joint data, specifically a vector with a length of time steps + 1. The probability values ​​of the grasped object belonging to various categories are obtained based on the multi-step joint data. This embodiment adopts the method of taking softmax of the above processing results, dividing the constant The values ​​corresponding to the items other than are the probability values ​​of the corresponding classifications. The classification vector of Interpreted as the current moment, Interpreted as the initial moment or The component vector obtained through steps S221-S224 can be regarded as feature information obtained from feedback data at only one moment, based on which the category of the grasped object at that moment can be obtained. In practical applications, since the feedback data will change over time, different categories of grasped objects will be obtained at different moments. The above embodiment adopts a multi-step joint method to integrate the feature information of the current moment and multiple moments before to obtain the classification result, which can make the classification result more stable and more accurate.

[0075] Assuming that there are four possible categories, A, B, C, and D, the probability values ​​of the grasped object belonging to the four categories obtained through the above processing are recorded as PA, PB, PC, and PD. An optional implementation is to determine the maximum value of the probability value in step S226, and then determine the category to which the grasped object belongs; obtain the upper and lower load limits corresponding to the category to which the grasped object belongs.

[0076] As an example, assuming that the maximum value among PA, PB, PC, and PD is PA, it is determined that the grasped object belongs to category A, and the upper load limit F1A and the lower load limit F2A associated with category A are obtained in the pre-stored relationship library. In the next step, the relationship between the current load and F1A and F2A can be directly determined to decide to increase the load, reset, or enter the grasping stage (see the above steps S23 to S26 for details, which will not be repeated here). This method has a small amount of calculation and a fast response speed, and can meet the needs of general application scenarios.

[0077] In a preferred embodiment, the upper load limit information determined in step S226 includes the upper load limits corresponding to various categories and the probability that the grasped object belongs to various categories, and the lower load limit information includes the lower load limits corresponding to various categories and the probability that the grasped object belongs to various categories. For example, the upper load limit F1A and the lower load limit F2A of category A, the upper load limit F1B and the lower load limit F2B of category B, the upper load limit F1C and the lower load limit F2C of category C, and the upper load limit F1D and the lower load limit F2D of category D. Through the above processing, the probability values ​​of grasped objects belonging to four categories are obtained as PA, PB, PC, and PD.

[0078] Furthermore, step S23 specifically includes: comparing the current load with the load upper limit of each category respectively, adding the probabilities of the categories exceeding the load upper limit to obtain a first cumulative probability; judging whether the first cumulative probability is lower than a first probability threshold; when the first cumulative probability is lower than the first probability threshold, judging that the current load will not damage the grasped object.

[0079] pass Figure 4Express the relationship between the load upper limit, probability, and current load. The horizontal axis represents the load and the vertical axis represents the probability. Since the current load is greater than the load upper limit F1A of category A and greater than the load upper limit F1B of category B, PA and PB are added to obtain the first cumulative probability. If PA+PB is greater than the first probability threshold, it is determined that the current load will destroy the grasped object, otherwise it is determined that the current load will not destroy the grasped object.

[0080] Step S24 specifically includes: comparing the current load with the load lower limit of each category respectively, adding the probabilities of the categories exceeding the load lower limit to obtain a second cumulative probability; judging whether the second cumulative probability is lower than the second probability threshold; when the second cumulative probability is lower than the second probability threshold, judging that the current load is insufficient to grab the object.

[0081] pass Figure 5 The relationship between the load lower limit, probability and current load is expressed. The horizontal axis represents the load and the vertical axis represents the probability. Since the current load is greater than the load lower limit F2A of category A and greater than the load lower limit F2B of category B, PA and PB are added to obtain the second cumulative probability. If PA+PB is greater than the second probability threshold, it is determined that the current load can grab the object, otherwise it is determined that the current load is insufficient to grab the object.

[0082] The above method of calculating the cumulative probability and comparing it with the probability threshold to judge the current load can improve the accuracy of the judgment result, especially when the obtained probabilities of the grasped objects belonging to various categories are close or inconsistent with the actual situation, the above preferred scheme can avoid entering the grasping stage incorrectly.

[0083] In one embodiment, the current load includes two different values, so as to be applied to the above steps S23 and S24 respectively. Specifically, in this embodiment, the force of each piece of electronic skin of the robot hand is array data , represents the number of electronic skins, j and k represent the rows and columns of the array. For the sake of convenience, it is assumed that each of the five fingers is provided with one piece of electronic skin and the palm is provided with one piece of electronic skin. Then the first characteristic contact force value is the maximum value of all array data, that is, max ( The second characteristic contact force value is the smaller of the maximum value in the array data collected by the electronic skin of the thumb and the maximum value in the array data collected by the electronic skin of other fingers, and the maximum value in the array data of the thumb is recorded as max ( ), the maximum value in the array data of the other four fingers is recorded as max ( , , , ), then the second characteristic contact force value is min (max ( ), max( , , , )).

[0084] In step S23, the current load can be max ( ), in the above step S24, the current load can be min (max ( ), max( , , , )), in step S26, the initial value of the load can be set to any one of the above two characteristic values.

[0085] For the grasping stage, i.e., the above step S3, in one embodiment, the load is increased in the following manner: S31A, collecting monitoring data when the robot grasps and moves the grasped object. The monitoring data in this embodiment uses force feedback data, which can be the pressure or pressure at the contact point between the robot and the grasped object, as well as the force of the driving device that changes the load. This method can collect force feedback data according to a certain period, and the value of the force feedback data will change with the longitudinal or lateral movement of the robot.

[0086] S32A, predicting the sliding distance of the grasped object relative to the robot hand based on the force feedback data at multiple moments. The multiple moments may include the current acquisition moment and one or more acquisition moments before it. For example, the initial moment of entering the grasping phase is t0, and the current acquisition moment is t i , then multiple moments can be t0~t i (i.e. all moments from the initial moment to the current moment), or t i-n、 t i (i.e. the current moment and the n moments before it), or t j ~t i (t j refers to the last time the load was changed or set).

[0087] The sliding distance can be calculated using a pre-established mathematical model, for example, the corresponding sliding distance can be calculated based on the changing trend and rate of change of force feedback data at multiple moments; or a neural network model suitable for processing sequence data such as LSTM (Long Short-Term Memory) can be used, with force feedback data at multiple moments as input to output the sliding distance.

[0088] It should be noted that the sliding distance obtained in step S32A refers to the distance that the grasped object is expected to slide, rather than the distance that has already slid before.

[0089] S33A, determining whether the sliding distance exceeds a first distance threshold. When the sliding distance exceeds the first distance threshold, it indicates that the grasped object is expected to slide over a certain distance under the grip of the current load, thereby posing a risk of slipping from the robot hand, and thus step S34A is executed, otherwise the load is kept unchanged and the process returns to step S31A.

[0090] In an optional embodiment, step S33A further includes determining whether the sliding distance exceeds a second distance threshold (the second distance threshold is greater than the first distance threshold, for example, the first distance threshold is 2 mm and the second distance threshold is 7 mm); when the sliding distance does not exceed the second distance threshold, determining whether the sliding distance exceeds the first distance threshold; when the sliding distance exceeds the second distance threshold, it indicates that the sliding distance of the grasped object under the current load is expected to be large, and even if the load is increased, it may not be possible to stop the object from sliding. In this case, the robot arm can be moved downward to place the grasped object back on the load-bearing plane to prepare for re-grasping.

[0091] S34A, determining the load increase according to the sliding distance. A correspondence between the sliding distance and the load increase can be established in advance, for example, different sliding distance intervals correspond to different load increases, and if the sliding distance is in a certain interval, the interval corresponds to the load increase; or a mathematical calculation model and a neural network model are used to calculate the load increase according to the sliding distance.

[0092] S35A, using the load increase and the current load to set the target load, and increase the load until the target load is reached. For example, the target load = the current load + the load increase. The load can be understood as a characteristic value related to the force, which can characterize the gripping force of the robot hand on the grasped object. The characteristic value can be one or more, specifically part of the force feedback data, or a value calculated based on the force feedback data.

[0093] The method for setting the gripping load of a robot arm provided in an embodiment of the present invention utilizes feedback data at multiple moments to predict the sliding distance of an object relative to the robot arm, and can obtain a target load suitable for the current grasped object before the object actually slides, thereby effectively stopping the sliding trend. In addition, this scheme sets the load increment based on the predicted sliding distance, so that the size of the target load is reasonable, thereby preventing sliding while avoiding damage to the object.

[0094] In one embodiment, in order to further prevent the grasped object from being damaged, the sum of the current load and the load increase (the total load) can be calculated first to determine whether the total load exceeds the upper load limit. The upper load limit means that the grasped object is likely to be damaged above this load (which can be called the damage load). The upper load limit is a preset value, which can be calculated by a set algorithm during the loading stage or can be set manually. When the total load does not exceed the upper load limit, the total load is set to the target load; when the total load exceeds the upper load limit, the upper load limit is set to the target load.

[0095] In one embodiment, the force feedback data includes pressure data collected by the electronic skin disposed at the contact part between the robot hand and the grasped object, and driving force data of each driving device of the robot hand. Further, the pressure data is array data collected by the electronic skin disposed at the palm and multiple fingers of the robot hand.

[0096] The contact parts between the robot hand and the grasped object and the driving device for changing the load depend on the structure of the robot hand. Taking the humanoid mobile robot hand as an example, the contact parts are the surfaces of five fingers and one palm. The driving device for changing the load can be a device for driving the movement of finger joints, and specifically, each movable joint can correspond to an independent driving device.

[0097] For the humanoid robot hand, the pressure data collected by the electronic skin refers to the force (pressure or intensity) exerted by the grasped object on the surface of the fingers and palm. The driving force data refers to the force borne by the driving device.

[0098] As an example, The first electronic skin Line The pressure data is recorded as ; Driving force data .

[0099] According to the above example, the size of the electronic skin at different positions in the robot hand or the number of measurement points therein may be different, so the number of rows and columns of the array is different.

[0100] Furthermore, regarding the load (current load and target load), assuming that each of the five fingers is provided with one piece of electronic skin and the palm is provided with one piece of electronic skin, the load can be the maximum value of all array data, that is, max ( The load may also be the smaller of the maximum value in the array data collected by the electronic skin of the thumb and the maximum value in the array data collected by the electronic skin of other fingers. The maximum value in the array data of the thumb is recorded as max ( ), the maximum value in the array data of the other four fingers is recorded as max ( , , , ), then the load is min (max ( ), max( , , , )).

[0101] like Figure 6 As shown, in one embodiment, step S32A uses a neural network model to predict the sliding distance, which specifically includes the following processing: The time series prediction module predicts the pressure feature vector at the next moment based on the array data at multiple moments. Specifically, several ConvLSTM (convolutional long short-term memory networks) are used to transform the pressure data of each piece of electronic skin into a Normalized and used as input data for several consecutive ConvLSTM networks. This layer extracts spatial features in each time step, captures dynamic changes between time steps, outputs new features and normalizes them. Then, through the Flatten layer, the multidimensional data is flattened into a one-dimensional vector (the last time step is taken and flattened to obtain the pressure feature vector). The above layers process each piece of electronic skin separately, and the parameters of the layers are shared.

[0102] The vertical driving force component is extracted from the driving force data, and the driving force feature vector of the next moment is predicted based on the vertical driving force component at multiple moments through the time series prediction module. The vertical driving force component in is used as the input data of several LSTM layers, and the driving force feature vector is output after being transformed by several LSTM layers.

[0103] The pressure eigenvector and the driving force eigenvector are connected to obtain the global eigenvector.

[0104] The sliding distance is obtained according to the global feature vector. Specifically, MLP (Multi-Layer Perceptron) is used, and the global feature vector is used as the input data of MLP to output the sliding distance.

[0105] After obtaining the target load through the above steps S31A~S35A, the operation of adjusting the current load of the robot arm until the target load is reached is referred to as the second adjustment operation in this application; in one embodiment, another load adjustment operation based on threshold comparison is also used to adjust the load in the grasping stage.

[0106] This embodiment provides a load adjustment method based on threshold comparison (referred to as a first adjustment operation in this application), which can be executed by an electronic device such as a computer or a server, including: S31B, judging whether the grasped object slides according to the monitoring data at the current moment and the monitoring data at the previous moment, and executing step S32B when the grasped object slides, otherwise continuing to detect. Different from S32A, S31B only gives a yes or no judgment result without calculating the sliding distance.

[0107] The monitoring data of this embodiment can adopt the same force-related data as in the above step S31A, including the pressure data collected by the electronic skin provided at the contact part between the robot hand and the grasped object, and the driving force data of each driving device of the robot hand. Furthermore, the pressure data is array data collected by the electronic skin provided at the palm and multiple fingers of the robot hand.

[0108] In order to improve the calculation efficiency, the force feedback data used in this embodiment is the sum of the pressure data collected by each electronic skin and the vertical component extracted from the driving force data of the specific driving device. As a preferred embodiment, step S31B includes: calculating the difference in total pressure according to the sum of the pressure data at the current moment and the previous moment, and calculating the difference in vertical component according to the vertical driving force component at the current moment and the previous moment; judging whether any one of the difference in total pressure and the difference in vertical component exceeds the corresponding set threshold; when any one of the difference in total pressure and the difference in vertical component exceeds the corresponding set threshold, judging that the grasped object has slipped.

[0109] S32B, increase the load, the load increase amount is a fixed amount.

[0110] On this basis, step S3 can simultaneously adopt the above two adjustment operations, specifically including: In the process of the robot arm grasping and moving the grasped object, the first adjustment operation is triggered according to the first cycle and the second adjustment operation is triggered according to the second cycle. The first adjustment operation is detailed in steps S31B~S32B, and the second adjustment operation is detailed in steps S31A~S35A.

[0111] The triggering periods of the two different adjustment operations are different, so as to ensure that they are not triggered at the same time step. Preferably, the second period is greater than the first period. The load increase amount in the first adjustment operation is a fixed amount, and the load increase amount in the second adjustment operation is a non-fixed amount. In a preferred embodiment, the fixed amount is configured to be less than the minimum value of the non-fixed amount, for example, different orders of magnitude can be used.

[0112] It can be seen that when the first adjustment operation is triggered and it is determined that the load needs to be increased, a smaller increase will be applied; when the second adjustment operation is triggered and it is determined that the load needs to be increased, a relatively large and non-fixed increase will be applied. Further, after the second adjustment operation is triggered, before the current load reaches the target load, the triggering of the first adjustment operation is suspended.

[0113] For example, the initial load is 1.0N, the collection period of force feedback data is 10ms, the first period is 20ms, and the second period is 100ms. Two data sets M1 and M2 are set, M1 is used to store the force feedback data B required for the second adjustment operation (the pressure of each electronic skin, the vertical component of the driving force of each driving device), and M2 is used to store the force feedback data A required for the first adjustment operation (the total pressure, the vertical component of the specific driving device). The initial moment of entering the grasping stage is recorded as t0. The example of adjusting the load is as follows: t0+10ms: The force feedback data A of the first time step is added to the data set M1.

[0114] t0+20ms: Trigger the first adjustment operation. At this time, the data set M2 is empty, the load is not adjusted, and the force feedback data B is calculated based on the force feedback data A of the first time step and stored in M2.

[0115] t0+30ms: The force feedback data A of the second time step is added to M1.

[0116] t0+40ms: Trigger the first adjustment operation. Calculate the force feedback data B stored in M2 and the force feedback data B calculated from the second time step data, calculate the difference in total pressure and vertical component, and determine whether sliding occurs (steps S31B~S32B). Assuming that no sliding occurs, the load is not adjusted, and the force feedback data B calculated from the second time step data covers M2. The above calculation can be completed within a few milliseconds.

[0117] … t0+80ms: Trigger the first adjustment operation. Calculate the force feedback data B stored in M2 and the force feedback data B calculated from the 4th time step data, calculate the difference in total pressure and vertical component, and determine whether sliding occurs (steps S31B~S32B). Assuming that sliding occurs, increase the load to 1.02N (the increase is a fixed value of 0.02N). Overwrite M2 with the force feedback data B calculated from the 4th time step data.

[0118] t0+90ms: The force feedback data A of the fifth time step is added to M1.

[0119] t0+100ms: Trigger the first adjustment operation. Assume that the load is increased to 1.04 N. Overwrite M2 with the force feedback data B calculated from the fifth time step data.

[0120] t0+105ms: trigger the second adjustment operation. In an independent thread, set the target load according to the force feedback data A from t0 to the triggering moment. At this time, the target load is determined using steps S31A~S35A. In actual application, since this calculation process takes a certain amount of time, the first adjustment operation is still triggered in the next several time steps until the result is obtained. It should be noted that the current load in step S35A refers to the load when the second adjustment operation is triggered, that is, 1.04N.

[0121] t0+110ms: The force feedback data A of the sixth time step is added to M1.

[0122] t0+120ms: Trigger the first adjustment operation. Assume that the load is increased to 1.06 N. Overwrite M2 with the force feedback data B calculated from the 6th time step data.

[0123] t0+130ms: The force feedback data A of the 7th time step is added to M1.

[0124] 0+140ms: Trigger the first adjustment operation. Assume that the load is increased to 1.08 N. Overwrite M2 with the force feedback data B calculated from the 7th time step data.

[0125] t0+145ms: Steps S31A~S35A are completed and the output sliding distance is 3mm, which is greater than the first threshold of 2mm but less than the second threshold of 7mm. The load increase is determined to be 0.2N, and the target load = the load when the second adjustment operation is triggered (t0+105ms) 1.04N+0.2N=1.24N. The current load is 1.08N, and the target load is 1.24N. The load is increased until the target load is reached. Before the adjustment is completed, the first adjustment operation is no longer triggered.

[0126] t0+150ms: The force feedback data A of the 8th time step is added to M1.

[0127] t0+170ms: The force feedback data A of the 9th time step is added to M1.

[0128] The above examples are only for illustrating how to use two adjustment operations to control the load at the same time. The specific values ​​of the load value, cycle value, etc. used in the examples are only for reference, and are not intended to limit the values ​​of the relevant data.

[0129] According to the robot arm grasping load adjustment method provided by an embodiment of the present invention, two different adjustment operations are used to adjust the robot arm load. Through the first adjustment operation with a faster response speed, it is ensured that the load is increased in time when a sliding event occurs in the grasped object. Through the second adjustment operation with a higher accuracy, the sliding distance is predicted and a suitable target load is set, thereby stopping the sliding trend. This solution takes into account both response speed and accuracy, and improves the adaptability of the robot arm to grasp objects.

[0130] like Figure 7 As shown, the embodiment of the present application provides an adaptive grasping system, including a robot 110 and a control device 400. The control device 400 is used to execute the above grasping method, and the robot 110 is provided with a monitoring device for collecting monitoring data, which is specifically a measuring piece 210 set at the front end of the finger and the palm in this embodiment, which is used to collect pressure, temperature, etc. In addition, a sensor for collecting driving force and displacement can be set inside the robot (not shown in the figure), and such a sensor can be set at the movable joint of the robot.

[0131] This embodiment adopts a visual positioning method, and for this purpose, an image acquisition device 300 is provided, which can be replaced by other positioning devices, such as radar equipment, etc.; in the embodiment of fixed-point grasping, the positioning device may also not be provided.

[0132] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0134] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0136] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A robot hand adaptive grasping method, characterized in that: include: Control the robot arm to approach the grasped object and set the initial load to contact the grasped object; Increasing the load according to monitoring data when the robot arm loads the grasped object with the current load until the loading load is determined, and then controlling the robot arm to grasp the grasped object with the loading load; The current load is adjusted according to the monitoring data when the robot grasps and moves the grasped object until the grasping load is determined, and the robot is monitored to see whether the grasped object is moved above the target position; After the robot arm moves the grasped object to above the target position, the robot arm is controlled to place the grasped object to the target position.

2. The method according to claim 1, characterized in that The monitoring data includes feedback data of the contact point between the robot arm and the grasped object and feedback data of the driving device for changing the load.

3. The method according to claim 2, characterized in that Increase the load based on the monitoring data when the robot is loading the grasped object with the current load, including: Extracting a global feature vector from the feedback data using a neural network model, and determining load upper limit information and load lower limit information according to the global feature vector; Determining whether the current load will damage the grasped object according to the load upper limit information; If it is determined that the current load will not damage the grasped object, determining whether the current load is sufficient to grasp the grasped object according to the load lower limit information; If it is determined that the current load is not sufficient to pick up the object, the load is increased.

4. The method according to claim 3, characterized in that The feedback data of the contact part between the robot hand and the grasped object includes the pressure data collected by the electronic skin arranged at the contact part between the robot hand and the grasped object.

5. The method according to claim 4, characterized in that The feedback data of the contact part between the robot hand and the grasped object includes temperature data collected by the electronic skin arranged at the contact part between the robot hand and the grasped object.

6. The method according to claim 3, characterized in that The feedback data of the drive devices for changing the load include displacement data and force data of each drive device of the robot.

7. The method according to claim 3, characterized in that Determining load upper limit information and load lower limit information according to the global eigenvector includes: Classifying according to the global feature vector to obtain a classification vector for the category of the grasped object; Determining the probability values ​​of the grasped object belonging to various categories according to the classification vector; The upper load limit information and the lower load limit information are determined according to the probability values ​​of the grasped objects belonging to various categories and the upper load limits and lower load limits corresponding to the various categories.

8. The method according to claim 7, characterized in that The load upper limit information includes the load upper limits corresponding to various categories and the probability that the grasped object belongs to various categories; the load lower limit information includes the load lower limits corresponding to various categories and the probability that the grasped object belongs to various categories.

9. The method according to claim 8, characterized in that Judging whether the current load will damage the grasped object according to the load upper limit information includes: Comparing the current load with the load upper limit of each category respectively, and adding the probabilities of the categories exceeding the load upper limit to obtain a first cumulative probability; Determining whether the first cumulative probability is lower than a first probability threshold; When the first cumulative probability is lower than a first probability threshold, it is determined that the current load will not damage the grasped object; Judging whether the current load is sufficient to grasp the grasped object according to the load lower limit information includes: Comparing the current load with the load lower limit of each category respectively, and adding the probabilities of the categories exceeding the load lower limit to obtain a second cumulative probability; Determining whether the second cumulative probability is lower than a second probability threshold; When the second cumulative probability is lower than a second probability threshold, it is determined that the current load is insufficient to grasp the grasped object.

10. The method according to claim 1, characterized in that Adjust the current load based on the monitoring data when the robot grasps and moves the grasped object, including: Whether the grasped object is sliding is determined based on the monitoring data at the current moment and the monitoring data at the previous moment, and the load is increased when the grasped object is sliding.

11. The method according to claim 1, characterized in that: Adjust the current load based on the monitoring data when the robot grasps and moves the grasped object, including: Predicting the sliding distance of the grasped object relative to the robot hand based on the monitoring data at multiple moments; Determining whether the sliding distance exceeds a first distance threshold; When the sliding distance exceeds the first distance threshold, determining a load increase amount according to the sliding distance; A target load is set using the load increase amount and the current load, and the load is increased until the target load is reached.

12. The method according to claim 1, characterized in that Adjust the current load based on the monitoring data when the robot grasps and moves the grasped object, including: A first adjustment operation is triggered according to a first cycle, and a second adjustment operation is triggered according to a second cycle, wherein the first adjustment operation includes determining whether the grasped object slides according to the monitoring data at a current moment and the monitoring data at a previous moment, and increasing the load when the grasped object slides; the second adjustment operation includes predicting a sliding distance of the grasped object relative to the robot arm according to the monitoring data at multiple moments; determining whether the sliding distance exceeds a first distance threshold; determining a load increase amount according to the sliding distance when the sliding distance exceeds the first distance threshold; setting a target load using the load increase amount and the current load, and increasing the load until the target load is reached.

13. The method according to claim 11 or 12, characterized in that: The monitoring data includes pressure data collected by the electronic skin arranged at the contact part between the robot hand and the grasped object, and driving force data of each driving device of the robot hand.

14. The method according to claim 13, characterized in that The pressure data is array data collected by electronic skins respectively arranged on the palm and multiple fingers of the robot hand.

15. The method according to claim 14, characterized in that Predicting the sliding distance of the grasped object relative to the robot hand according to the monitoring data at multiple moments includes: Predicting the pressure feature vector at the next moment according to the array data at multiple moments by a time series prediction module; Extracting a vertical driving force component from the driving force data, and using a time series prediction module to predict a driving force feature vector at a next moment based on the vertical driving force component at multiple moments; Connecting the pressure characteristic vector and the driving force characteristic vector to obtain a global characteristic vector; The sliding distance is obtained according to the global feature vector.

16. The method according to claim 12, characterized in that After the second adjustment operation is triggered, before the current load reaches the target load, triggering of the first adjustment operation is suspended.

17. The method according to claim 16, characterized in that The load increase amount when the first adjustment operation is triggered is a fixed amount, and the fixed amount is smaller than the load increase amount in the second adjustment operation.

18. A control device, characterized in that: include: A processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor so that the processor executes the robot arm adaptive grasping method as described in any one of claims 1-17.

19. An adaptive grasping system, characterized in that: It comprises a robot arm and the control device as claimed in claim 18; wherein the robot arm is provided with a monitoring device for collecting the monitoring data.

Images