Robot arm loading method and equipment

Abstract

The invention provides a robot arm loading method and equipment, and the method comprises the steps: obtaining monitoring data when a robot arm loads a grabbed object with a current load, the monitoring data comprising feedback data of a contact part of the robot arm and the grabbed object and feedback data of a driving device for changing the load; extracting a global feature vector from the feedback data by using a neural network model, and determining load upper limit information and load lower limit information according to the global feature vector; according to the load upper limit information, whether the current load can damage a grabbed object or not is judged; if it is judged that the current load cannot damage the grabbed object, whether the current load is enough to grab the grabbed object or not is judged according to the load lower limit information; and if the current load is not enough to grab the grabbed object, the load is increased.

Description

Technical Field

[0001] The present invention relates to the field of robot arm control, and in particular to a robot arm loading method and device. Background Art

[0002] Robotic arms can be used in a variety of industries. Currently, the automated grasping strategies of robotic arms are usually designed for some fixed application scenarios, so that they can grasp specific objects. In some application scenarios, the grasped objects may be highly diverse. If the grasping strategy is not adaptable enough, it may fail to grasp the target object or damage the target object during grasping.

[0003] A typical adaptive grasping strategy is to grasp the object with a set smaller load (usually referred to as the loading stage), and then dynamically adjust the load according to the grasping situation. The above-mentioned prior art may fail to grasp the object at all due to the initial set load being too small, or the load during loading directly causes the object to be damaged. It can be seen that the existing adaptive grasping strategy is not adaptable enough. Summary of the invention

[0004] In view of this, the present application provides a robot arm loading method, comprising: Acquire monitoring data when the robot arm loads the grasped object with the current load, wherein the monitoring data includes feedback data of the contact part between the robot arm and the grasped object and feedback data of the driving device for changing the load; Extracting a global feature vector from the feedback data using a neural network model, and determining load upper limit information and load lower limit information according to the global feature vector; Determining whether the current load will damage the grasped object according to the load upper limit information; If it is determined that the current load will not damage the grasped object, determining whether the current load is sufficient to grasp the grasped object according to the load lower limit information; If it is determined that the current load is not sufficient to pick up the object, the load is increased.

[0005] Optionally, the feedback data of the contact portion between the robot hand and the grasped object includes pressure data collected by an electronic skin arranged at the contact portion between the robot hand and the grasped object.

[0006] Optionally, the pressure data is array data collected by electronic skins respectively arranged on the palm and multiple fingers of the robotic hand.

[0007] Optionally, the current load includes a first characteristic contact force value and a second characteristic contact force value, wherein the first characteristic contact force value is a maximum value in all array data, and the second characteristic contact force value is a relatively smaller maximum value in array data collected by the electronic skin of the thumb and a maximum value in array data collected by the electronic skin of other fingers; The first characteristic contact force value is used in the step of judging whether the current load will damage the grasped object according to the load upper limit information, and the second characteristic contact force value is used in the step of judging whether the current load is sufficient to grasp the grasped object according to the load lower limit information.

[0008] Optionally, the feedback data of the contact portion between the robot hand and the grasped object includes temperature data collected by an electronic skin arranged at the contact portion between the robot hand and the grasped object.

[0009] Optionally, the temperature data is array data collected by electronic skins respectively arranged on the palm and multiple fingers of the robotic hand.

[0010] Optionally, the feedback data of the drive device for changing the load includes displacement data and force data of each drive device of the robot.

[0011] Optionally, extracting a global feature vector from the feedback data using a neural network model includes: Extracting the pressure data collected from each electronic skin to obtain the electronic skin force distribution feature, extracting the temperature data collected from each electronic skin to obtain the electronic skin temperature distribution feature, and fusing the electronic skin force distribution feature and the electronic skin temperature distribution feature to obtain the electronic skin feature vector; Extracting data based on the connection between displacement data and force data of the driving device to obtain a characteristic vector of the driving device; The electronic skin feature vector and the driving device feature vector are connected to obtain a global feature vector.

[0012] Optionally, fusing the electronic skin force distribution feature and the electronic skin temperature distribution feature to obtain an electronic skin feature vector includes: Performing pooling processing on the one of the electronic skin force distribution feature and the electronic skin temperature distribution feature having a higher pixel density, so as to adjust the number of pixels to be consistent; fusing the adjusted electronic skin force distribution characteristics and the electronic skin temperature distribution characteristics; The fusion results are averaged in the array dimension to obtain the electronic skin feature vector.

[0013] Optionally, extracting data based on the connection between displacement data and force data of the driving device to obtain a driving device feature vector includes: Taking the difference between the displacement data at the current moment and the displacement data at the previous moment; The differential result is connected with the force data to extract the drive device characteristic vector.

[0014] Optionally, determining the load upper limit information and the load lower limit information according to the global eigenvector includes: Classifying according to the global feature vector to obtain a classification vector for the category of the grasped object; Determining the probability values ​​of the grasped object belonging to various categories according to the classification vector; The upper load limit information and the lower load limit information are determined according to the probability values ​​of the grasped objects belonging to various categories and the upper load limits and lower load limits corresponding to the various categories.

[0015] Optionally, determining the probability values ​​of the grasped object belonging to various categories according to the classification vector includes: Obtain multi-step joint data according to the classification vector at the current moment and at least one previous moment; The probability values ​​of the grasped object belonging to various categories are obtained according to the multi-step joint data.

[0016] Optionally, obtaining multi-step joint data according to the classification results at the current moment and at least one previous moment includes: Multiplying the classification vector at each moment by a preset coefficient respectively, and then summing the multiplication results at each moment; The summation result is combined with a preset background value to obtain the multi-step joint data.

[0017] Optionally, the upper load limit information and the lower load limit information are determined according to probability values ​​of the grasped object belonging to various categories and upper load limits and lower load limits corresponding to various categories, including: Determine the maximum value among the probability values, and then determine the category to which the grasped object belongs; Get the upper and lower load limits corresponding to the category of the grasped object.

[0018] Optionally, judging whether the current load will damage the grasped object according to the load upper limit information includes: Determine whether the current load is lower than the load upper limit; If the current load is lower than the upper load limit, it is determined that the current load will not damage the grasped object; Judging whether the current load is sufficient to grasp the grasped object according to the load lower limit information includes: Determine whether the current load is lower than the load lower limit; If the current load is lower than the load lower limit, it is determined that the current load is insufficient to grasp the grasped object.

[0019] Optionally, the load upper limit information includes load upper limits corresponding to various categories and probabilities that the grasped objects belong to various categories; the load lower limit information includes load lower limits corresponding to various categories and probabilities that the grasped objects belong to various categories.

[0020] Optionally, judging whether the current load will damage the grasped object according to the load upper limit information includes: Comparing the current load with the load upper limit of each category respectively, and adding the probabilities of the categories exceeding the load upper limit to obtain a first cumulative probability; Determining whether the first cumulative probability is lower than a first probability threshold; When the first cumulative probability is lower than a first probability threshold, it is determined that the current load will not damage the grasped object; Judging whether the current load is sufficient to grasp the grasped object according to the load lower limit information includes: Comparing the current load with the load lower limit of each category respectively, and adding the probabilities of the categories exceeding the load lower limit to obtain a second cumulative probability; Determining whether the second cumulative probability is lower than a second probability threshold; When the second cumulative probability is lower than a second probability threshold, it is determined that the current load is insufficient to grasp the grasped object.

[0021] Accordingly, the present application provides a robotic arm loading device, characterized in that it includes: a processor and a memory connected to the processor; wherein the memory stores instructions that can be executed by the processor, and the instructions are executed by the processor so that the processor executes the above-mentioned robotic arm loading method.

[0022] According to the robot arm loading method and equipment provided by the present application, the corresponding monitoring data is obtained as the grasping load changes, and then the load upper limit information and load lower limit information corresponding to the current monitoring data are determined, and the load is adjusted according to the relationship between the current load and the load upper limit information and the load lower limit information, to achieve an adaptive grasping and loading process of loading, identifying, and making decisions at the same time. Grasping objects with appropriate loads can effectively avoid slipping or damage when the objects are subsequently grasped, and has strong adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 is a flow chart of a robot arm loading method in an embodiment of the present invention; Figure 2 Schematic diagram of a neural network model architecture in an embodiment of the present invention; Figure 3 A schematic diagram of multi-step joint data processing in an embodiment of the present invention; Figure 4is a load upper limit and probability distribution diagram in an embodiment of the present invention; Figure 5 Graph showing a load lower limit and probability distribution in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0027] The process of a robot grasping an object can be divided into four stages, namely the positioning stage, the loading stage, the grasping stage and the placement stage. The positioning stage refers to the robot determining the location of the grasped object, moving, approaching until it touches the grasped object; the loading stage refers to adjusting the load of the robot after the robot touches the grasped object, and determining the load value that can be grasped without destroying the grasped object; the grasping stage refers to grasping the object and moving it (moving upward, moving horizontally above the target position); the placement stage refers to placing the object at the target position.

[0028] For the loading stage, this embodiment provides a robot loading method, which can be executed by an electronic device such as a computer or a server, such as Figure 1 The following operations are shown: S21, obtaining monitoring data when the robot arm loads the grasped object with the current load, wherein the monitoring data includes feedback data of the contact part between the robot arm and the grasped object and feedback data of the driving device for changing the load.

[0029] The loading in this method refers to the process of the robot arm grasping and holding an object on a supporting surface (such as the ground, a table or other platform, etc.), and does not include the action of moving the object upward or in other directions after grasping. This method can be executed synchronously with the change process of the robot arm load. The initial load is recorded as F 0 , the robot can be operated with a load F 0 Contact the grasped object and then gradually increase the load. As the load changes, the corresponding monitoring data also changes.

[0030] The contact parts between the robot hand and the grasped object and the driving device for changing the load depend on the structure of the robot hand. Taking the humanoid mobile robot hand as an example, the contact parts are the surfaces of five fingers and one palm. The driving device for changing the load can be a device for driving the movement of finger joints, and specifically, each movable joint can correspond to an independent driving device.

[0031] The feedback data of the contact point between the robot hand and the grasped object may specifically include force. For a humanoid robot hand, it refers to the force (pressure or intensity) exerted by the grasped object on the surface of the fingers and palm; in an optional embodiment, it may also include temperature.

[0032] The feedback data of the drive device for changing the load may specifically include force and / or displacement. For the anthropomorphic robot hand, it refers to the force borne by the drive device and / or the displacement provided to generate the current load.

[0033] As an example, for the load at any time (time step), the monitoring data obtained can specifically include , , and ,in represents the force at the ith contact point, represents the temperature of the ith contact point, represents the displacement of the ith driving device, represents the force of the ith driving device.

[0034] The load can be understood as a characteristic value related to the force, and the characteristic value can be one or more, specifically, it can be the force data in the monitoring data, or a value calculated based on the force data. For example, the current load can be the value of a certain force among the forces acting on the surface of the finger and the palm, or it can be the sum of these forces, or it can be the force of a certain driving device or the sum of the forces of the driving devices, etc.

[0035] S22, extracting global feature vectors from feedback data using a neural network model, and determining upper and lower load limit information based on the global feature vectors. Feedback data is at least two data from two different parts or drive devices in the robot hand. According to the above example, there may be dozens of feedback data. In order to extract feature vectors, these feedback data need to be fused. The neural network model can use various network structures composed of units such as CNN (Convolutional Neural Network) and MLP (Multi-Layer Perceptron) to realize feature extraction and fusion. There are many specific implementation methods, for example, it can include ResNet (Residual Network) structure.

[0036] The upper load limit refers to the load above which the object is likely to be destroyed (it can be called the destruction load); the lower load limit refers to the load above which it is worth trying to grasp the object, or it can be interpreted as the lower load limit being considered to be the lowest load that can grasp the object.

[0037] It should be noted that the so-called upper load limit and lower load limit in this application are force values, and the upper load limit information and lower load limit information can be force values, or can be information calculated based on the force values, such as statistical information.

[0038] The upper load limit information and the lower load limit information can be the output result of the neural network model, or the mapping information of the output result of the neural network model. For example, the neural network model can be configured to directly output the value of the upper load limit and the value of the lower load limit, and a large amount of sample data is used to train it before use, so that it can output two values ​​according to the above-mentioned feedback data input; the neural network model can be configured to output information about the grasped object, such as the category of the grasped object, and a corresponding relationship library between this information and the upper load limit and the lower load limit is pre-established. When the neural network model outputs the information of the grasped object during use, the corresponding upper load limit and the lower load limit can be queried in the corresponding relationship library.

[0039] S23, judging whether the current load will damage the grasped object according to the load upper limit information. If it is determined that the current load will not damage the grasped object, executing step S24; otherwise, as an optional solution, executing step S26, or stopping the execution of this method.

[0040] An optional method is to use hard classification in step S22 to determine whether the grasped object belongs to a certain category, and then obtain the corresponding load upper limit, or directly output the load upper limit by the neural network in step S22. In step S23, it is determined whether the current load is lower than the load upper limit. If the current load is lower than the load upper limit, it is determined that the current load will not damage the grasped object.

[0041] S24, judging whether the current load is sufficient to grasp the grasped object according to the load lower limit information. If it is determined that the current load is insufficient to grasp the grasped object, executing step S25, otherwise trying to grasp the object with the current load, that is, entering the grasping stage.

[0042] An optional method is to use hard classification in step S22 to determine whether the grasped object belongs to a certain category, and then obtain the corresponding load lower limit, or directly output the load lower limit by the neural network in step S22. In step S24, it is determined whether the current load is lower than the load lower limit. If the current load is lower than the load upper limit, it is determined that the current load is insufficient to grasp the grasped object.

[0043] S25, increase the load. Then return to step S21 and re-execute the method until the current load meets the condition of step S24. The load increase amount can be fixed or calculated based on the difference between the current load and the load lower limit information. For example, if the difference between the current load and the load lower limit is large, the increase amount is large.

[0044] S26, set the load to the initial value, then return to step S21 and re-execute the method. Further, before re-execution of the method, the posture of the robot arm can be fine-tuned so that it can re-try the loading process from a different angle or position. Fine-tuning the posture of the robot arm can be fine-tuning only the angle of the mechanical joint, or re-selecting the contact position.

[0045] According to the robot arm loading method provided by the embodiment of the present invention, the corresponding monitoring data is obtained as the grasping load changes, and then the load upper limit information and the load lower limit information corresponding to the current monitoring data are determined, and the load is adjusted according to the relationship between the current load and the load upper limit information and the load lower limit information, to achieve an adaptive grasping and loading process of loading, identifying, and making decisions at the same time. Grasping objects with appropriate loads can effectively avoid slipping or damage when the objects are subsequently grasped, and has strong adaptability.

[0046] Regarding the monitoring data in the above steps, in one embodiment, the feedback data of the contact part between the robot hand and the grasped object includes pressure data collected by the electronic skin set at the contact part between the robot hand and the grasped object. The pressure data is array data collected by the electronic skin set at the palm and multiple fingers of the robot hand.

[0047] As an example, The first electronic skin Line Column record .

[0048] In one embodiment, the feedback data of the contact part between the robot hand and the grasped object includes temperature data collected by the electronic skin arranged at the contact part between the robot hand and the grasped object. The temperature data is array data collected by the electronic skin arranged at the palm and multiple fingers of the robot hand.

[0049] As an example, The first electronic skin Line Column record .

[0050] According to the above example, the size of the electronic skin at different positions in the robot hand or the number of measurement points therein may be different, so the number of rows and columns of the array is different. In one embodiment, the feedback data of the drive device for changing the load includes the displacement data and force data of each drive device of the robot hand. As an example, for the first The displacement data and force data of each driving device are recorded as .

[0051] The above-mentioned various feedback data can accurately reflect the gripping state. By extracting features from these feedback data, the accuracy of the upper load limit information and the lower load limit information can be improved.

[0052] There are many specific structures of the neural network model in the above step S22. This embodiment provides a specific implementation method. Figure 2 As shown, step S22 includes the following operations: S221, extracting the pressure data collected from each electronic skin to obtain the electronic skin force distribution feature, extracting the temperature data collected from each electronic skin to obtain the electronic skin temperature distribution feature, and fusing the electronic skin force distribution feature and the electronic skin temperature distribution feature to obtain the electronic skin feature vector.

[0053] Specifically, each electronic skin force monitoring value array data They are input into the CNN layer respectively, and then pass through several residual modules respectively. The processed results are averaged to obtain the electronic skin force distribution characteristics; the array data of each electronic skin temperature monitoring value is averaged. They are input into the CNN layer respectively, and then pass through several residual modules respectively. The processed results are averaged to obtain the temperature distribution characteristics of the electronic skin.

[0054] In some embodiments, the scales of the electronic skin temperature distribution feature and the electronic skin force distribution feature may be the same, so they can be directly merged to obtain the electronic skin feature vector. If the scales of the two distribution features are different, adjustments need to be made: The one with higher pixel density between the electronic skin force distribution feature and the electronic skin temperature distribution feature is pooled to adjust the number of pixels to be consistent (connect after unifying the number of pixels); the adjusted electronic skin force distribution feature and electronic skin temperature distribution feature are fused (fusion array); the fusion results are averaged in the array dimension (array average) to obtain the electronic skin feature vector.

[0055] The above layers process each piece of electronic skin separately, and the parameters of the layers are shared.

[0056] S222, extract the data based on the displacement data and force data of the driving device to obtain the driving device feature vector. The displacement data of the drive device at the previous moment Take the difference; compare the difference result with the drive device force data In this embodiment, the MLP module is specifically used to obtain the robot driving device feature vector.

[0057] S223, connecting the electronic skin feature vector and the driving device feature vector to obtain a global feature vector.

[0058] It should be noted that the above module architecture is only one of many optional methods, and is not the only feasible solution. For example, at least normalization, dropout and other processing can be set at appropriate positions in the above process, and other modules can be used to replace the above MLP module and CNN module to form a deformed architecture. The difference between the displacement and force of the driving device reflects the relationship between the force and displacement along the clamping direction, the force distribution of the electronic skin reflects the force distribution characteristics along the contact tangent direction, and the temperature reflects the heat capacity, heat transfer and other information. The above embodiments integrate this rich information, and the result can fully reflect the material properties of the grasped object, thereby making the classification result more accurate.

[0059] In one embodiment, step S22 includes the following operations: S224: Classify according to the global feature vector to obtain a classification vector for the category of the grasped object. Specifically, the global feature vector can be obtained by adopting the above steps S221-S223, or by adopting other network structures, such as LSTM network structures, etc.

[0060] S225, determining the probability value of the grasped object belonging to various categories according to the classification vector. An optional method is to take softmax of the component vectors output by the neural network model to obtain the probability of each category.

[0061] S226: Determine the upper load limit information and the lower load limit information according to the probability values ​​of the grasped object belonging to various categories and the upper load limits and lower load limits corresponding to the various categories.

[0062] In a preferred embodiment, a multi-step joint approach is used to obtain the probability of each category. Figure 3 The step S225 shown specifically includes the following operations: The multi-step joint data is obtained based on the classification vectors at the current moment and at least one previous moment. There is more than one specific calculation method. In this embodiment, the classification vector at each moment is multiplied by a preset coefficient. The preset coefficient in this embodiment is , where M is the number of time steps (the total number of the current moment and at least one previous moment), K and A are preset constants, and then the multiplication results of each moment are summed, and the summation result is combined with the preset background value B to obtain multi-step joint data, specifically a vector with a length of time steps + 1. The probability values ​​of the grasped object belonging to various categories are obtained based on the multi-step joint data. This embodiment adopts the method of taking softmax of the above processing results, dividing the constant The values ​​corresponding to the items other than are the probability values ​​of the corresponding classifications. The classification vector of Interpreted as the current moment, Interpreted as the initial moment or The component vector obtained through steps S221-S224 can be regarded as feature information obtained from feedback data at only one moment, based on which the category of the grasped object at that moment can be obtained. In practical applications, since the feedback data will change over time, different categories of grasped objects will be obtained at different moments. The above embodiment adopts a multi-step joint method to integrate the feature information of the current moment and multiple moments before to obtain the classification result, which can make the classification result more stable and more accurate.

[0063] Assuming that there are four possible categories, A, B, C, and D, the probability values ​​of the grasped object belonging to the four categories obtained through the above processing are recorded as PA, PB, PC, and PD. An optional implementation is to determine the maximum value of the probability value in step S226, and then determine the category to which the grasped object belongs; obtain the upper and lower load limits corresponding to the category to which the grasped object belongs.

[0064] As an example, assuming that the maximum value among PA, PB, PC, and PD is PA, it is determined that the grasped object belongs to category A, and the upper load limit F1A and the lower load limit F2A associated with category A are obtained in the pre-stored relationship library. In the next step, the relationship between the current load and F1A and F2A can be directly determined to decide to increase the load, reset, or enter the grasping stage (see the above steps S23 to S26 for details, which will not be repeated here). This method has a small amount of calculation and a fast response speed, and can meet the needs of general application scenarios.

[0065] In a preferred embodiment, the upper load limit information determined in step S226 includes the upper load limits corresponding to various categories and the probability that the grasped object belongs to various categories, and the lower load limit information includes the lower load limits corresponding to various categories and the probability that the grasped object belongs to various categories. For example, the upper load limit F1A and the lower load limit F2A of category A, the upper load limit F1B and the lower load limit F2B of category B, the upper load limit F1C and the lower load limit F2C of category C, and the upper load limit F1D and the lower load limit F2D of category D. Through the above processing, the probability values ​​of grasped objects belonging to four categories are obtained as PA, PB, PC, and PD.

[0066] Furthermore, step S23 specifically includes: comparing the current load with the load upper limit of each category respectively, adding the probabilities of the categories exceeding the load upper limit to obtain a first cumulative probability; judging whether the first cumulative probability is lower than a first probability threshold; when the first cumulative probability is lower than the first probability threshold, judging that the current load will not damage the grasped object.

[0067] pass Figure 4 Express the relationship between the load upper limit, probability, and current load. The horizontal axis represents the load and the vertical axis represents the probability. Since the current load is greater than the load upper limit F1A of category A and greater than the load upper limit F1B of category B, PA and PB are added to obtain the first cumulative probability. If PA+PB is greater than the first probability threshold, it is determined that the current load will destroy the grasped object, otherwise it is determined that the current load will not destroy the grasped object.

[0068] Step S24 specifically includes: comparing the current load with the load lower limit of each category respectively, adding the probabilities of the categories exceeding the load lower limit to obtain a second cumulative probability; judging whether the second cumulative probability is lower than the second probability threshold; when the second cumulative probability is lower than the second probability threshold, judging that the current load is insufficient to grab the object.

[0069] pass Figure 5 The relationship between the load lower limit, probability and current load is expressed. The horizontal axis represents the load and the vertical axis represents the probability. Since the current load is greater than the load lower limit F2A of category A and greater than the load lower limit F2B of category B, PA and PB are added to obtain the second cumulative probability. If PA+PB is greater than the second probability threshold, it is determined that the current load can grab the object, otherwise it is determined that the current load is insufficient to grab the object.

[0070] The above method of calculating the cumulative probability and comparing it with the probability threshold to judge the current load can improve the accuracy of the judgment result, especially when the obtained probabilities of the grasped objects belonging to various categories are close or inconsistent with the actual situation, the above preferred scheme can avoid entering the grasping stage incorrectly.

[0071] In one embodiment, the current load includes two different values, so as to be applied to the above steps S23 and S24 respectively. Specifically, in this embodiment, the force of each piece of electronic skin of the robot hand is array data , represents the number of electronic skins, j and k represent the rows and columns of the array. For the sake of convenience, it is assumed that each of the five fingers is provided with one piece of electronic skin and the palm is provided with one piece of electronic skin. Then the first characteristic contact force value is the maximum value of all array data, that is, max ( The second characteristic contact force value is the smaller of the maximum value in the array data collected by the electronic skin of the thumb and the maximum value in the array data collected by the electronic skin of other fingers, and the maximum value in the array data of the thumb is recorded as max ( ), the maximum value in the array data of the other four fingers is recorded as max ( , , , ), then the second characteristic contact force value is min (max ( ), max( , , , )).

[0072] In step S23, the current load can be max ( ), in the above step S24, the current load can be min (max ( ), max( , , , )), in step S26, the initial value of the load can be set to any one of the above two characteristic values.

[0073] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0074] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0075] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0077] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A robot loading method, characterized in that: include: Acquire monitoring data when the robot arm loads the grasped object with the current load, wherein the monitoring data includes feedback data of the contact part between the robot arm and the grasped object and feedback data of the driving device for changing the load; Extracting a global feature vector from the feedback data using a neural network model, and determining load upper limit information and load lower limit information according to the global feature vector; Determining whether the current load will damage the grasped object according to the load upper limit information; If it is determined that the current load will not damage the grasped object, determining whether the current load is sufficient to grasp the grasped object according to the load lower limit information; If it is determined that the current load is not sufficient to pick up the object, the load is increased.

2. The method according to claim 1, characterized in that The feedback data of the contact part between the robot hand and the grasped object includes the pressure data collected by the electronic skin arranged at the contact part between the robot hand and the grasped object.

3. The method according to claim 2, characterized in that The pressure data is array data collected by electronic skins respectively arranged on the palm and multiple fingers of the robot hand.

4. The method according to claim 3, characterized in that The current load includes a first characteristic contact force value and a second characteristic contact force value, wherein the first characteristic contact force value is a maximum value in all array data, and the second characteristic contact force value is a relatively smaller maximum value in array data collected by the electronic skin of the thumb and a maximum value in array data collected by the electronic skin of other fingers; The first characteristic contact force value is used in the step of judging whether the current load will damage the grasped object according to the load upper limit information, and the second characteristic contact force value is used in the step of judging whether the current load is sufficient to grasp the grasped object according to the load lower limit information.

5. The method according to claim 2, characterized in that: The feedback data of the contact part between the robot hand and the grasped object includes temperature data collected by the electronic skin arranged at the contact part between the robot hand and the grasped object.

6. The method according to claim 5, characterized in that The temperature data is array data collected by electronic skins respectively arranged on the palm and multiple fingers of the robot hand.

7. The method according to claim 5, characterized in that The feedback data of the drive devices for changing the load include displacement data and force data of each drive device of the robot.

8. The method according to claim 7, characterized in that Extracting a global feature vector from the feedback data using a neural network model includes: Extracting the pressure data collected from each electronic skin to obtain the electronic skin force distribution feature, extracting the temperature data collected from each electronic skin to obtain the electronic skin temperature distribution feature, and fusing the electronic skin force distribution feature and the electronic skin temperature distribution feature to obtain the electronic skin feature vector; Extracting data based on the connection between displacement data and force data of the driving device to obtain a characteristic vector of the driving device; The electronic skin feature vector and the driving device feature vector are connected to obtain a global feature vector.

9. The method according to claim 8, characterized in that The electronic skin force distribution feature and the electronic skin temperature distribution feature are integrated to obtain an electronic skin feature vector, including: Performing pooling processing on the one of the electronic skin force distribution feature and the electronic skin temperature distribution feature having a higher pixel density, so as to adjust the number of pixels to be consistent; fusing the adjusted electronic skin force distribution characteristics and the electronic skin temperature distribution characteristics; The fusion results are averaged in the array dimension to obtain the electronic skin feature vector.

10. The method according to claim 8, characterized in that The characteristic vector of the driving device is obtained by extracting the data based on the connection between the displacement data and the force data of the driving device, including: Taking the difference between the displacement data at the current moment and the displacement data at the previous moment; The differential result is connected with the force data to extract the drive device characteristic vector.

11. The method according to claim 1, characterized in that: Determining load upper limit information and load lower limit information according to the global eigenvector includes: Classifying according to the global feature vector to obtain a classification vector for the category of the grasped object; Determining the probability values ​​of the grasped object belonging to various categories according to the classification vector; The upper load limit information and the lower load limit information are determined according to the probability values ​​of the grasped objects belonging to various categories and the upper load limits and lower load limits corresponding to the various categories.

12. The method according to claim 11, characterized in that Determine the probability value of the grasped object belonging to various categories according to the classification vector, including: Obtain multi-step joint data according to the classification vector at the current moment and at least one previous moment; The probability values ​​of the grasped object belonging to various categories are obtained according to the multi-step joint data.

13. The method according to claim 12, characterized in that The multi-step joint data is obtained according to the classification results at the current moment and at least one previous moment, including: Multiplying the classification vector at each moment by a preset coefficient respectively, and then summing the multiplication results at each moment; The summation result is combined with a preset background value to obtain the multi-step joint data.

14. The method according to claim 11, characterized in that The upper load limit information and the lower load limit information are determined according to the probability values ​​of the grasped objects belonging to various categories and the upper load limits and the lower load limits corresponding to the various categories, including: Determine the maximum value among the probability values, and then determine the category to which the grasped object belongs; Get the upper and lower load limits corresponding to the category of the grasped object.

15. The method according to claim 14, characterized in that Judging whether the current load will damage the grasped object according to the load upper limit information includes: Determine whether the current load is lower than the load upper limit; If the current load is lower than the upper load limit, it is determined that the current load will not damage the grasped object; Judging whether the current load is sufficient to grasp the grasped object according to the load lower limit information includes: Determine whether the current load is lower than the load lower limit; If the current load is lower than the load lower limit, it is determined that the current load is insufficient to grasp the grasped object.

16. The method according to claim 11, characterized in that The load upper limit information includes the load upper limits corresponding to various categories and the probability that the grasped object belongs to various categories; the load lower limit information includes the load lower limits corresponding to various categories and the probability that the grasped object belongs to various categories.

17. The method according to claim 16, characterized in that Judging whether the current load will damage the grasped object according to the load upper limit information includes: Comparing the current load with the load upper limit of each category respectively, and adding the probabilities of the categories exceeding the load upper limit to obtain a first cumulative probability; Determining whether the first cumulative probability is lower than a first probability threshold; When the first cumulative probability is lower than a first probability threshold, it is determined that the current load will not damage the grasped object; Judging whether the current load is sufficient to grasp the grasped object according to the load lower limit information includes: Comparing the current load with the load lower limit of each category respectively, and adding the probabilities of the categories exceeding the load lower limit to obtain a second cumulative probability; Determining whether the second cumulative probability is lower than a second probability threshold; When the second cumulative probability is lower than a second probability threshold, it is determined that the current load is insufficient to grasp the grasped object.

18. A robot loading device, characterized in that: include: A processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor so that the processor executes the robot arm loading method as described in any one of claims 1-17.

Images