Mechanical arm control method, device, equipment and storage medium

Abstract

Embodiments of the present application disclose a mechanical arm control method, device and equipment and a storage medium. After an image containing a target object and a mechanical arm to be controlled is obtained, point cloud data of the target object and a target gripper pose of a dexterous hand are obtained based on the image, and then the output torque of each motor in the mechanical arm is adjusted multiple times directly based on a motor control model trained by reinforcement learning, so as to control the dexterous hand to gradually move to the target object until the dexterous hand reaches the target gripper pose and the dexterous hand does not touch the target object. When the dexterous hand reaches the target gripper pose, a gripping action of the dexterous hand is triggered again, so that the dexterous hand holds the target object. The motor control model trained by reinforcement learning is introduced to control the motor in the mechanical arm, and an optimization algorithm is no longer called to plan a path and then control the motor in the mechanical arm, so that the control accuracy of the mechanical arm is improved, and the success rate of the mechanical arm is improved.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and more specifically, to a robotic arm control method, apparatus, device, and storage medium. Background Technology

[0002] In the field of robotics, a robotic arm includes a robotic arm and a dexterous hand at one end of the robotic arm. By controlling the robotic arm, the dexterous hand can be manipulated to grasp objects.

[0003] One current approach to controlling robotic arms involves first predicting the target gripper posture that the dexterous hand can grasp. Then, an optimization algorithm (such as MoveIt) plans the motion path from the current gripper posture to the target gripper posture. This motion path is converted into output signals from the motors in the robotic arm, which in turn control the arm's movement to ensure the dexterous hand reaches the target gripper posture. However, for robotic arms with insufficient degrees of freedom or specific structures, optimization algorithms often fail to find a reasonable motion path, resulting in a low grasping success rate. Furthermore, this method suffers from low accuracy in the converted motor output signals, causing the gripper posture controlled based on these signals to fail to accurately reflect the target gripper posture, further reducing the robotic arm's grasping success rate. Summary of the Invention

[0004] In view of this, this application provides a robotic arm control method, apparatus, device, and storage medium to improve the success rate of robotic arm grasping.

[0005] To achieve the above objectives, the following solution is proposed:

[0006] A robotic arm control method, comprising:

[0007] Acquire images, which include the target object and the robotic arm to be controlled;

[0008] Based on the image, the point cloud data of the target object and the target gripper pose of the robotic arm's dexterous hand are obtained;

[0009] Based on the point cloud data, the target gripper pose, and the real-time angle signals of each motor in the robotic arm, the output torque of each motor is adjusted multiple times using a motor control model trained by reinforcement learning, so as to control the dexterous hand to move gradually toward the target object until the dexterous hand reaches the target gripper pose; the dexterous hand does not touch the target object during the process of moving toward the target object;

[0010] When the dexterous hand reaches the target gripper position, it is triggered to perform a grasping action so that the dexterous hand can grasp the target object.

[0011] Optionally, the process of adjusting the output torque of each motor multiple times in the above method includes:

[0012] The motor control model is used to predict the output torque of each motor in the robotic arm multiple times. During each prediction, the motor control model processes the point cloud data, the target gripper pose, and the real-time angle signals of each motor to determine the next target torque to be output by each motor and the identification information corresponding to the target torque. The identification information indicates whether the dexterous hand has reached the target gripper pose.

[0013] Each time the target torque of each motor is obtained, control each motor to output the target torque;

[0014] If the identification information indicates that the dexterous hand has not reached the target gripper pose, the output torque of each motor is predicted again.

[0015] Optionally, the motor control model described above is trained using the following reinforcement learning method:

[0016] A simulation model of the robotic arm is generated based on the model file of the robotic arm.

[0017] For any object simulation model among several object simulation models with randomly set postures, obtain the point cloud data of the object simulation model and the random target gripper posture of the robotic arm simulation model corresponding to the object simulation model.

[0018] The point cloud data of any object simulation model, the pose of the random target gripper, and the real-time angle signals of each motor model in the robotic arm simulation model are input into the motor control model to obtain the next output torque of each motor model output by the motor control model and the identification information corresponding to the output torque. The identification information indicates whether the dexterous hand of the robotic arm simulation model has reached the pose of the random target gripper.

[0019] The parameters of the motor control model are updated with the goal of the dexterous hand of the robotic arm simulation model reaching the random target gripper pose without touching the point cloud data of any object simulation model. The update continues until the identification information corresponding to the torque to be output indicates that the dexterous hand of the robotic arm simulation model has reached the random target gripper pose.

[0020] Optionally, in the above method, obtaining the point cloud data of the target object based on the image includes:

[0021] Obtain the name information of the target object;

[0022] Based on the name information, object recognition is performed on the image to determine a rectangular region in the image containing the target object;

[0023] The rectangular region of the image is cropped from the image;

[0024] The image of the rectangular region is processed to determine the outline of the target object;

[0025] Point cloud data of the target object is obtained based on the contour of the target object and the depth information of the pixels in its internal region.

[0026] The above methods may optionally include:

[0027] Specific point recognition is performed on the image to identify the first coordinates of multiple specific points on the robotic arm in the camera coordinate system; the second coordinates of the multiple specific points in the robotic arm coordinate system are obtained; and a preset mapping matrix between the camera coordinate system and the robotic arm coordinate system is corrected based on the first and second coordinates to obtain the corrected mapping matrix.

[0028] Accordingly, the process of obtaining the point cloud data of the target object includes:

[0029] Based on the contour of the target object and the depth information of the pixels in its internal region, the point cloud data of the target object in the camera coordinate system is obtained.

[0030] Based on the corrected mapping matrix, the point cloud data of the target object in the camera coordinate system is mapped to the point cloud data of the target object in the robotic arm coordinate system, which is then used as the final point cloud data of the target object.

[0031] Optionally, the process of correcting the preset mapping matrix in the above method includes:

[0032] The preset mapping matrix is ​​summed with the correction matrix to be solved to obtain the sum matrix;

[0033] Based on the sum matrix, the first coordinates of the plurality of specific points are mapped to the third coordinates of the specific points in the coordinate system of the robotic arm.

[0034] Given that the change in the mapping matrix before and after correction, determined based on the first three rows of the correction matrix, is less than the target value, and the fourth row of the correction matrix is ​​zero, find the value of the correction matrix that minimizes the difference between the third coordinate and the second coordinate.

[0035] The values ​​of the corrected matrix obtained by solving are summed with the preset mapping matrix to obtain the corrected mapping matrix.

[0036] Optionally, the method described above, which involves obtaining the target gripper pose of the robotic arm's dexterous hand based on the image, includes:

[0037] The point cloud data of the target object in the camera coordinate system is processed by a pre-trained gripper pose prediction model to obtain the target gripper pose.

[0038] The gripper pose prediction model is trained using a target training dataset. The target training dataset includes baseline training samples and augmented training samples. The baseline training samples are point cloud data of real objects with randomly set poses in the camera coordinate system. The labels of the baseline training samples are the gripper poses of the dexterous hand when the robotic arm's dexterous hand is moved near the real object by manual operation. The point cloud data in any augmented training sample is obtained by translating or rotating the point cloud data in a baseline training sample. The labels of any augmented training sample are obtained by translating or rotating the labels of the baseline training sample accordingly.

[0039] A robotic arm control device, comprising:

[0040] The acquisition module is used to acquire images, which include the target object and the robotic arm to be controlled;

[0041] The acquisition module is used to acquire point cloud data of the target object and the target gripper pose of the dexterous hand of the robotic arm based on the image.

[0042] The control module is used to adjust the output torque signals of each motor multiple times based on the point cloud data, the target gripper pose, and the real-time angle signals of each motor in the robotic arm, using a motor control model trained by reinforcement learning. This controls the dexterous hand to gradually move towards the target object until it reaches the target gripper pose. During the movement towards the target object, the dexterous hand does not touch the target object. When the dexterous hand reaches the target gripper pose, it is triggered to perform a grasping action so that it can grasp the target object.

[0043] A robotic arm control device includes a memory and a processor;

[0044] The memory is used to store programs;

[0045] The processor is used to execute the program to implement the various steps of the robotic arm control method as described in any of the preceding claims.

[0046] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the robotic arm control method as described in any of the preceding claims.

[0047] As can be seen from the above technical solutions, the robotic arm control method, device, equipment, and storage medium provided in this application, after obtaining an image containing a target object and a robotic arm to be controlled, obtains point cloud data of the target object and the target gripper pose of the dexterous hand based on the image. Then, based on the motor control model trained by reinforcement learning, it directly adjusts the output torque of each motor in the robotic arm multiple times, thereby controlling the dexterous hand to gradually move towards the target object until the dexterous hand reaches the target gripper pose without touching the target object. When the dexterous hand reaches the target gripper pose, it is then triggered to perform a grasping action, allowing the dexterous hand to grasp the target object. This application directly controls the motors in the robotic arm by introducing a motor control model trained by reinforcement learning, without calling an optimization algorithm to plan the path before controlling the motors in the robotic arm, thereby improving the control accuracy of the robotic arm and thus increasing the success rate of grasping. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0049] Figure 1 This is a flowchart illustrating an implementation of the robotic arm control method disclosed in an embodiment of this application.

[0050] Figure 2 This is an example of an image acquired by the image acquisition device disclosed in this application when no target object is placed near the robotic arm;

[0051] Figure 3 This is a flowchart illustrating an implementation of reinforcement learning training for a first motor control model as disclosed in an embodiment of this application.

[0052] Figure 4 This is a flowchart illustrating an implementation of obtaining point cloud data of a target object based on an image, as disclosed in an embodiment of this application.

[0053] Figure 5 This is a flowchart illustrating an implementation of correcting a preset mapping matrix as disclosed in an embodiment of this application.

[0054] Figure 6 This is a schematic diagram of a robotic arm control device disclosed in an embodiment of this application;

[0055] Figure 7 This is a hardware structure block diagram of the robotic arm control device disclosed in an embodiment of this application. Detailed Implementation

[0056] Before describing the solution proposed in this application, the relevant concepts will be explained.

[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0058] This application proposes a solution to improve the success rate of gripping by robotic arms.

[0059] like Figure 1 The diagram shown is a flowchart of one implementation of the robotic arm control method provided in this application, which may include:

[0060] Step S101: Acquire an image, which includes the target object and the robotic arm to be controlled.

[0061] The image acquisition device (i.e., camera) used to capture images is usually located at the main body of the robotic arm, corresponding to the human head. The lens of the image acquisition device can be pointed in front of or below the main body, which means that the image acquisition device can look at eye level or downwards. Since the robotic arm is in front of the main body and can move in front of the main body, by adjusting the orientation of the lens of the image acquisition device, the image acquisition device can capture images of the robotic arm. When a target object to be grasped is placed in front of the main body, the captured image can also include the target object.

[0062] The image acquisition device can be an RGB camera or a depth camera, etc.

[0063] like Figure 2 The image shown is an example of an image acquired by the image acquisition device provided in this application embodiment when no target object is placed near the robotic arm.

[0064] Step S102: Obtain point cloud data of the target object and the target gripper pose of the robotic arm's dexterous hand based on the image.

[0065] The point cloud data of the target object refers to the point cloud data of the target object in the robot arm coordinate system, which can be obtained by mapping the point cloud data of the target object in the camera coordinate system.

[0066] The target gripper pose indicates that the dexterous hand has reached the vicinity of the target object but has not yet made contact with it. However, as long as the dexterous hand performs a grasping action, it can make contact with the target object and successfully grasp it.

[0067] Information representing the pose of the target gripper of a dexterous hand may include, but is not limited to, the spatial position of the dexterous hand, rotation angle, and degree of opening and closing.

[0068] Step S103: Based on point cloud data and real-time angle signals of each motor in the robotic arm, the output torque of each motor is adjusted multiple times using the motor control model obtained by reinforcement learning training, so as to control the dexterous hand to move gradually toward the target object until the dexterous hand reaches the target gripper pose; the dexterous hand does not touch the target object during the process of moving toward the target object.

[0069] This application pre-trains a motor control model (referred to as the first motor control model for ease of description and distinction) using reinforcement learning. During the control of the robotic arm, the angle signals of each motor in the robotic arm are monitored in real time. After obtaining the target gripper pose of the dexterous hand, the trained first motor control model is used to adjust the output torque of each motor in the robotic arm multiple times based on the point cloud data of the target object, the target gripper pose, and the real-time angle signals of each motor in the robotic arm. This controls the dexterous hand to gradually approach the target object until it reaches the target gripper pose where it can grasp the target object simply by performing a grasping action. During the multiple adjustments of the output torque of each motor based on the first motor control model, the dexterous hand will not touch the target object. This avoids the dexterous hand touching the target object before reaching the target gripper pose, which could cause the target object to translate, rotate, or tip over, making it impossible for the dexterous hand to grasp the target object even after reaching the target gripper pose and performing a grasping action.

[0070] Step S104: When the dexterous hand reaches the target gripper position, trigger the dexterous hand to perform a grasping action so that the dexterous hand can grasp the target object.

[0071] The dexterous hand can be controlled to perform grasping actions by controlling the output torque of the motor in the dexterous hand.

[0072] In an optional embodiment, a preset motor control algorithm (an algorithm for controlling the dexterous hand to perform grasping actions) can be used to control the motor in the dexterous hand to output torque in a preset manner, so that the dexterous hand can perform grasping actions. The specific control algorithm can be found in the prior art, but since it is not the focus of this application, it will not be described in detail here.

[0073] In an optional embodiment, a motor control model (referred to as the second motor control model for ease of description) for controlling the dexterous hand to perform grasping actions can be trained in advance using reinforcement learning training methods. Based on the point cloud data of the target object, the pose of the target gripper, and the real-time angle signals of each motor in the dexterous hand, the output torque of each motor in the dexterous hand is adjusted multiple times using the second motor control model trained by reinforcement learning, so as to control each finger in the dexterous hand to move closer to the target object until each finger touches the target object.

[0074] The robotic arm control method provided in this application, after obtaining an image containing a target object and the robotic arm to be controlled, obtains point cloud data of the target object and the target gripper pose of the dexterous hand based on the image. Then, based on a motor control model trained by reinforcement learning, it directly adjusts the output torque of each motor in the robotic arm multiple times, thereby controlling the dexterous hand to gradually move towards the target object until the dexterous hand reaches the target gripper pose without touching the target object. When the dexterous hand reaches the target gripper pose, it is triggered to perform a grasping action, allowing the dexterous hand to grasp the target object. This application improves the control accuracy of the robotic arm and thus increases the grasping success rate by introducing a motor control model trained by reinforcement learning to directly control the motors in the robotic arm, instead of calling an optimization algorithm to plan the path first and then control the motors in the robotic arm.

[0075] In an optional embodiment, one way to implement the above-mentioned multiple adjustments to the output torque of each motor in the robotic arm can be:

[0076] The output torque of each motor in the robotic arm is predicted multiple times using the first motor control model. During each prediction, the first motor control model processes the point cloud data of the target object, the pose of the target gripper, and the real-time angle signals of each motor to determine the next target torque to be output by each motor in the robotic arm and the corresponding identification information. This identification information indicates whether the dexterous hand has reached the target gripper pose.

[0077] In other words, in this application, each time the first motor control model makes a prediction, in addition to predicting the target torque to be output by each motor in the robotic arm, it also outputs identification information indicating whether the dexterous hand has reached the target gripper pose when each motor in the robotic arm outputs the target torque.

[0078] Each time the target torque of each motor in the robotic arm is obtained, the target torque is output by controlling each motor in the robotic arm.

[0079] If the identification information indicates that the dexterous hand has not reached the target gripper pose, the real-time angle signals of each motor in the robotic arm are obtained again, and the output torque of each motor is predicted again. That is, the point cloud data of the target object, the target gripper pose and the real-time angle signals of each motor are processed using the first motor control model to determine the next target torque to be output by each motor in the robotic arm and the identification information corresponding to the target torque.

[0080] If the identification information indicates that the dexterous hand has reached the target gripper's pose, the prediction of the output torque of each motor ends, that is, the adjustment of the output torque of each motor in the robotic arm stops.

[0081] In an optional embodiment, a flowchart of an implementation of reinforcement learning training for a first motor control model provided in this application is shown below. Figure 3 As shown, it may include:

[0082] Step S301: Generate a robotic arm simulation model based on the robotic arm's model file.

[0083] The model file for the robotic arm can be a URDF (Unified Robot Description) file, which describes the mechanical structure of the robotic arm and dexterous hand and its relationship with the operable motors (i.e., the motors inside the robotic arm).

[0084] The model file of the robotic arm can be imported into the simulator, which will then generate a simulation model of the robotic arm based on the model file. This simulation model of the robotic arm can then be used for simulation control.

[0085] This application generates a robotic arm simulation model from the model file of the actual robotic arm, which is also very compatible with robotic arms with insufficient degrees of freedom or specific structures, further improving the grasping effect of the robotic arm.

[0086] Step S302: For any object simulation model among several object simulation models with randomly set postures, obtain the point cloud data of the object simulation model and the random target gripper posture of the robotic arm simulation model corresponding to the object simulation model.

[0087] This application constructs several object simulation models and randomly sets the posture of each object simulation model. Each time, an object simulation model is placed near the robotic arm, that is, there is always an object simulation model near the robotic arm. Different object simulation models may be in the same position or different positions near the robotic arm.

[0088] For any object simulation model, obtain the point cloud data of the object simulation model in the robot arm coordinate system, operate the robot arm simulation model so that the dexterous hand of the robot arm simulation model moves to a position near the object simulation model. At this position, the dexterous hand of the robot arm simulation model does not touch the object simulation model, and the dexterous hand can grasp the object simulation model by performing a grasping action. Since there are multiple positions that meet the above conditions, the dexterous hand of the robot arm simulation model can be randomly moved to a position near the object simulation model that can perform a grasping action (for ease of description, it is denoted as random target gripper posture), and the random target gripper posture is recorded.

[0089] Step S303: Input the point cloud data of any object simulation model, the pose of the random target gripper, and the real-time angle signals of each motor model in the robotic arm simulation model into the first motor control model to obtain the next output torque of each motor model (i.e., the motor simulation model) output by the first motor control model and the corresponding identification information of the output torque. The identification information indicates whether the dexterous hand of the robotic arm simulation model has reached the pose of the random target gripper.

[0090] The first motor control model can be a neural network model, including but not limited to any of the following: Deep Neural Networks (DNN), Convolutional Neural Networks (CNN), Long Short Term Memory (LSTM), Residual Networks (ResNet), Large Language Model, etc.

[0091] Step S304: With the goal of the dexterous hand of the robotic arm simulation model reaching the random target gripper pose without touching the point cloud data of any object simulation model, the parameters of the first motor control model are updated until the identification information corresponding to the output torque indicates that the dexterous hand of the robotic arm simulation model has reached the random target gripper pose.

[0092] In other words, during the reinforcement learning training of the first motor control model, this application uses the dexterous hand reaching the pose of a random target gripper as the proxy task, and adds the constraint that the dexterous hand does not collide with the object point cloud.

[0093] Optionally, the dexterous hand of the robotic arm simulation model can be controlled to move gradually toward any object simulation model at a frequency of 50 movements per second until the dexterous hand of the robotic arm simulation model reaches the random target gripper pose.

[0094] During the movement of the dexterous hand in the control simulation model of the robotic arm, if the dexterous hand has not yet reached the random target gripper pose, the value of the identification information is empty, or the value of the identification information is a specific value (referred to as the second specific value for ease of description and distinction); if the dexterous hand reaches the random target gripper pose, the value of the identification information is another specific value (referred to as the first specific value for ease of description and distinction).

[0095] Optionally, the simulator can compare the real-time gripper pose of the dexterous hand of the robotic arm simulation model with the random target gripper pose. If the distance between the two is greater than or equal to a distance threshold, it is considered that the dexterous hand of the robotic arm simulation model has not yet reached the random target gripper pose; otherwise, it is considered that the dexterous hand of the robotic arm simulation model has reached the random target gripper pose.

[0096] In an optional embodiment, a flowchart illustrating one method for obtaining point cloud data of a target object based on an image is shown below. Figure 4 As shown, it may include:

[0097] Step S401: Obtain the name information of the target object.

[0098] The name information of the target object can be entered by the user via a text input device or via a voice input device. In the case of voice input, the received voice can be converted into text for subsequent object recognition.

[0099] Step S402: Perform object recognition on the image based on name information to determine the rectangular region in the image that contains the target object.

[0100] Optionally, the name information and image can be input into the trained recognition model to obtain the position of the target object in the image. This position can be represented by the vertex coordinates of the smallest rectangle containing the target object (specifically, the coordinates of the two diagonal vertices).

[0101] When inputting name information into the recognition model, the name information can be in text format or in speech format. This application does not make any specific limitations.

[0102] As an example, the recognition model can be, but is not limited to: Grounding DINO model, etc.

[0103] Step S403: Crop out the rectangular region of the image from the image.

[0104] Step S404: Process the image of the rectangular area to determine the outline of the target object.

[0105] Optionally, a large image segmentation model can be used to segment the image of the rectangular region, thereby segmenting the target object from the image of the rectangular region, that is, extracting the outline of the target object.

[0106] Step S405: Obtain point cloud data of the target object based on the contour of the target object and the depth information of the pixels in its internal region.

[0107] In other words, this application first segments the target object from the image, and then uses the depth information of the pixels of the segmented target object to obtain the point cloud data of the target object.

[0108] The depth information of a pixel can be determined from images captured by a stereo camera or a depth camera. For specific implementation methods, please refer to existing solutions, which will not be detailed here.

[0109] When determining pixel depth using a stereo camera, the aforementioned image can be captured by one of the stereo cameras. When determining depth information based on an image captured by a depth camera, the aforementioned image can be captured by either a depth camera or an RGB image.

[0110] In an optional embodiment, point cloud data of the target object in the camera coordinate system can be obtained first based on the contour of the target object and the depth information of the pixels in its internal region. Then, the point cloud data of the target object in the camera coordinate system is mapped to the point cloud data of the target object in the robot arm coordinate system based on a mapping matrix, which serves as the final point cloud data of the target object. That is, when the output torque of each motor in the robot arm is adjusted multiple times using the first motor control model, the point cloud data of the target object in the robot arm coordinate system is used.

[0111] This application research found that, typically, based on the camera's mounting position and angle, a mapping matrix between the camera coordinate system and the robotic arm coordinate system (denoted as the preset mapping matrix for ease of description and distinction) can be pre-calculated. However, due to factors such as installation errors and hardware errors, the pre-calculated mapping matrix often contains certain errors, which negatively impacts high-precision grasping. Therefore, it is necessary to correct the aforementioned preset mapping matrix. Based on this, the robotic arm control method provided in the embodiments of this application may further include:

[0112] Specific point recognition is performed on the image to identify the first coordinates of multiple specific points on the robotic arm in the camera coordinate system.

[0113] Specific points on a robotic arm can include, but are not limited to: the joints of the robotic arm, the joints of a dexterous hand, and the fingertips of a dexterous hand. For example... Figure 2 The small circles in the diagram are shown.

[0114] The identification of specific points can be achieved using existing feature engineering techniques, which will not be detailed here.

[0115] Obtain the second coordinates of multiple specific points in the robotic arm coordinate system. Since the position of each specific point on the robotic arm is fixed, the coordinates of these points in the robotic arm coordinate system can be obtained by acquiring the motor status (e.g., motor angle information) and combining it with the hardware details of the robotic arm (e.g., shoulder width, arm length, finger length).

[0116] The preset mapping matrix between the camera coordinate system and the robotic arm coordinate system is corrected based on the first and second coordinates to obtain the corrected mapping matrix.

[0117] Accordingly, the process of obtaining point cloud data of the target object described above may include:

[0118] Based on the contour of the target object and the depth information of the pixels in its internal region, the point cloud data of the target object in the camera coordinate system is obtained.

[0119] Based on the corrected mapping matrix, the point cloud data of the target object in the camera coordinate system is mapped to the point cloud data of the target object in the robot arm coordinate system, which serves as the final point cloud data of the target object.

[0120] In an optional embodiment, a flowchart of an implementation for correcting a preset mapping matrix provided in this application is shown below. Figure 5 As shown, it may include:

[0121] Step S501: Summate the preset mapping matrix with the correction matrix to be solved to obtain the sum matrix.

[0122] Assume the preset mapping matrix is The correction matrix to be solved is The sum matrix is ​​then:

[0123] RT0+ΔRT (1)

[0124] Step S502: Based on the sum matrix, map the first coordinates of multiple specific points to the third coordinates of the specific points in the robot arm coordinate system.

[0125] In this embodiment of the application, it is assumed that there are N specific points, and the first coordinate of the i-th (i = 1, 2, 3, ..., N) specific point in the camera coordinate system is denoted as . The third coordinate in the robotic arm coordinate system is defined as Since the mapping matrix is ​​a 4×4 matrix, and the coordinates of a specific point are 3-dimensional vectors, to facilitate calculation, the coordinates of the specific point are padded to a 4-dimensional vector. Specifically, the value of the 4th dimension can be set to 1. For example, the first 4-dimensional coordinate of the i-th specific point in the camera coordinate system is: The following formula can be used to map the 4D first coordinates of N specific points to the 4D third coordinates of those specific points in the robot arm coordinate system:

[0126] P′ arm =(RT0+ΔRT)P cam (2)

[0127] Among them, P cam P′ is a 4×N matrix consisting of the 4-dimensional first coordinates of N specific points. arm P′ is a 4×N matrix consisting of the 4-dimensional third coordinates of N specific points. arm In the diagram, the first three rows of the j-th column (j = 1, 2, 3, ..., N) represent the 3D third coordinates of the j-th specific point.

[0128] Step S503: Under the condition that the change in the mapping matrix before and after correction is less than the target value, and the fourth row of the correction matrix is ​​zero, solve for the value of the correction matrix that minimizes the difference between the third coordinate and the second coordinate.

[0129] The second 3D coordinate system in the robotic arm coordinate system is defined as follows: Its corresponding 4D second coordinate is:

[0130] Optionally, an objective function can be constructed, which aims to minimize the difference between the 4D third coordinate and the 4D second coordinate of N coordinate points. The objective function is:

[0131]

[0132] Because the correction only requires fine-tuning the error, and the mapping matrix only needs to optimize the first 3×4 elements, this application sets the following two constraints when optimizing the objective function:

[0133] First condition:

[0134] Second condition:

[0135] Among them, in the first condition, This represents the change in the mapping matrix before and after correction, determined by the first three rows of the correction matrix; that is, the change represented by the first three rows of the correction matrix. ε is the target value, which may differ for different robotic arms. It is related to installation and hardware errors and is usually a small value. The second condition indicates that the fourth row of the correction matrix is ​​zero.

[0136] Based on this, the process of solving the objective function in this application is to find the ΔRT that minimizes the objective function among the ΔRTs that satisfy the first and second conditions.

[0137] The objective function can be solved using the least squares method.

[0138] Step S504: Summate the values ​​of the obtained correction matrix with the preset mapping matrix to obtain the corrected mapping matrix.

[0139] In an optional embodiment, one way to obtain the target gripper pose of the robotic arm's dexterous hand based on the image described above can be:

[0140] The point cloud data of the target object in the camera coordinate system is processed by a pre-trained gripper pose prediction model to obtain the target gripper pose.

[0141] The acquisition of point cloud data of the target object in the camera coordinate system can be referred to the aforementioned embodiments, and will not be repeated here.

[0142] The gripper pose prediction model can be a neural network model, including but not limited to any of the following: deep neural network, convolutional neural network, long short-term memory network, residual network, large language model, etc.

[0143] The gripper pose prediction model is trained using a target training dataset, which includes baseline training samples and augmented training samples. The baseline training samples are point cloud data of real objects with randomly set poses in the camera coordinate system. The labels of the baseline training samples are the gripper poses of the dexterous hand when the robotic arm's dexterous hand is manually moved near the real object (at which position the robotic arm can perform a grasping action to grab the real object). The point cloud data in any augmented training sample is obtained by translating or rotating the point cloud data in a baseline training sample. The labels of any augmented training sample are obtained by translating or rotating the labels of the corresponding baseline training samples.

[0144] By enhancing the benchmark training samples, the diversity of the training dataset is increased, ensuring that the trained gripper pose prediction model has high generalization and stability, and is capable of predicting the gripper pose of different objects. Furthermore, since the input to the gripper pose prediction model does not contain RGB images, but only point cloud information, the trained gripper pose prediction model has high robustness, unaffected by factors such as the type of robotic arm and the texture of the object, further improving the generalization and stability of the gripper pose prediction model.

[0145] Corresponding to the method embodiments, this application also provides a robotic arm control device. A schematic diagram of a robotic arm control device provided in this application embodiment is shown below. Figure 6 As shown, it may include:

[0146] The acquisition module 601, the acquisition module 602, and the control module 603; wherein,

[0147] The acquisition module 601 is used to acquire images, which include the target object and the robotic arm to be controlled;

[0148] The acquisition module 602 is used to acquire point cloud data of the target object and the target gripper pose of the dexterous hand of the robotic arm based on the image;

[0149] The control module 603 is used to adjust the output torque signals of each motor multiple times based on the point cloud data, the target gripper pose, and the real-time angle signals of each motor in the robotic arm, using a motor control model trained by reinforcement learning. This controls the dexterous hand to gradually move towards the target object until it reaches the target gripper pose. During the movement towards the target object, the dexterous hand does not touch the target object. When the dexterous hand reaches the target gripper pose, it is triggered to perform a grasping action so that it can grasp the target object.

[0150] The robotic arm control device provided in this application, after obtaining an image containing a target object and the robotic arm to be controlled, obtains point cloud data of the target object and the target gripper pose of the dexterous hand based on the image. Then, based on a motor control model trained by reinforcement learning, it directly adjusts the output torque of each motor in the robotic arm multiple times, thereby controlling the dexterous hand to gradually move towards the target object until the dexterous hand reaches the target gripper pose without touching the target object. When the dexterous hand reaches the target gripper pose, it is triggered to perform a grasping action, allowing the dexterous hand to grasp the target object. This application improves the control accuracy of the robotic arm and thus increases the grasping success rate by introducing a motor control model trained by reinforcement learning to directly control the motors in the robotic arm, instead of calling an optimization algorithm to plan the path first and then control the motors in the robotic arm.

[0151] In an optional embodiment, when the control module 603 adjusts the output torque of each motor multiple times, it is used to:

[0152] The motor control model is used to predict the output torque of each motor in the robotic arm multiple times. During each prediction, the motor control model processes the point cloud data, the target gripper pose, and the real-time angle signals of each motor to determine the next target torque to be output by each motor and the identification information corresponding to the target torque. The identification information indicates whether the dexterous hand has reached the target gripper pose.

[0153] Each time the target torque of each motor is obtained, control each motor to output the target torque;

[0154] If the identification information indicates that the dexterous hand has not reached the target gripper pose, the output torque of each motor is predicted again.

[0155] In an optional embodiment, this application further includes a training module, specifically used for:

[0156] A simulation model of the robotic arm is generated based on the model file of the robotic arm.

[0157] For any object simulation model among several object simulation models with randomly set postures, obtain the point cloud data of the object simulation model and the random target gripper posture of the robotic arm simulation model corresponding to the object simulation model.

[0158] The point cloud data of any object simulation model, the pose of the random target gripper, and the real-time angle signals of each motor model in the robotic arm simulation model are input into the motor control model to obtain the next output torque of each motor model output by the motor control model and the identification information corresponding to the output torque. The identification information indicates whether the dexterous hand of the robotic arm simulation model has reached the pose of the random target gripper.

[0159] The parameters of the motor control model are updated with the goal of the dexterous hand of the robotic arm simulation model reaching the random target gripper pose without touching the point cloud data of any object simulation model. The update continues until the identification information corresponding to the torque to be output indicates that the dexterous hand of the robotic arm simulation model has reached the random target gripper pose.

[0160] In an optional embodiment, when the obtaining module 602 obtains the point cloud data of the target object based on the image, it is used to:

[0161] Obtain the name information of the target object;

[0162] Based on the name information, object recognition is performed on the image to determine a rectangular region in the image containing the target object;

[0163] The rectangular region of the image is cropped from the image;

[0164] The image of the rectangular region is processed to determine the outline of the target object;

[0165] Point cloud data of the target object is obtained based on the contour of the target object and the depth information of the pixels in its internal region.

[0166] In an optional embodiment, this application further includes a correction module for:

[0167] Specific point recognition is performed on the image to identify the first coordinates of multiple specific points on the robotic arm in the camera coordinate system; the second coordinates of the multiple specific points in the robotic arm coordinate system are obtained; and a preset mapping matrix between the camera coordinate system and the robotic arm coordinate system is corrected based on the first and second coordinates to obtain the corrected mapping matrix.

[0168] Accordingly, when the obtaining module 602 obtains the point cloud data of the target object, it is used to:

[0169] Based on the contour of the target object and the depth information of the pixels in its internal region, the point cloud data of the target object in the camera coordinate system is obtained.

[0170] Based on the corrected mapping matrix, the point cloud data of the target object in the camera coordinate system is mapped to the point cloud data of the target object in the robotic arm coordinate system, which is then used as the final point cloud data of the target object.

[0171] In an optional embodiment, when the correction module corrects the preset mapping matrix, it is used to:

[0172] The preset mapping matrix is ​​summed with the correction matrix to be solved to obtain the sum matrix;

[0173] Based on the sum matrix, the first coordinates of the plurality of specific points are mapped to the third coordinates of the specific points in the coordinate system of the robotic arm.

[0174] Given that the change in the mapping matrix before and after correction, determined based on the first three rows of the correction matrix, is less than the target value, and the fourth row of the correction matrix is ​​zero, find the value of the correction matrix that minimizes the difference between the third coordinate and the second coordinate.

[0175] The values ​​of the corrected matrix obtained by solving are summed with the preset mapping matrix to obtain the corrected mapping matrix.

[0176] In an optional embodiment, when the obtaining module 602 obtains the target gripper pose of the robotic arm's dexterous hand based on the image, it is used to:

[0177] The point cloud data of the target object in the camera coordinate system is processed by a pre-trained gripper pose prediction model to obtain the target gripper pose.

[0178] The gripper pose prediction model is trained using a target training dataset. The target training dataset includes baseline training samples and augmented training samples. The baseline training samples are point cloud data of real objects with randomly set poses in the camera coordinate system. The labels of the baseline training samples are the gripper poses of the dexterous hand when the robotic arm's dexterous hand is moved near the real object by manual operation. The point cloud data in any augmented training sample is obtained by translating or rotating the point cloud data in a baseline training sample. The labels of any augmented training sample are obtained by translating or rotating the labels of the baseline training sample accordingly.

[0179] The robotic arm control device provided in this application embodiment can be applied to robotic arm control equipment, such as PC terminals, robots, cloud platforms, servers, and server clusters. Optionally, Figure 7 The hardware structure block diagram of the robotic arm control device is shown, with reference to... Figure 7 The hardware structure of the robotic arm control device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;

[0180] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;

[0181] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0182] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0183] The memory stores a program, which the processor can call. The program is used for:

[0184] Acquire images, which include the target object and the robotic arm to be controlled;

[0185] Based on the image, the point cloud data of the target object and the target gripper pose of the robotic arm's dexterous hand are obtained;

[0186] Based on the point cloud data, the target gripper pose, and the real-time angle signals of each motor in the robotic arm, the output torque of each motor is adjusted multiple times using a motor control model trained by reinforcement learning, so as to control the dexterous hand to move gradually toward the target object until the dexterous hand reaches the target gripper pose; the dexterous hand does not touch the target object during the process of moving toward the target object;

[0187] When the dexterous hand reaches the target gripper position, it is triggered to perform a grasping action so that the dexterous hand can grasp the target object.

[0188] Optionally, the refined and extended functions of the program can be found in the description above.

[0189] This application embodiment also provides a storage medium that can store a program suitable for execution by a processor, the program being used for:

[0190] Acquire images, which include the target object and the robotic arm to be controlled;

[0191] Based on the image, the point cloud data of the target object and the target gripper pose of the robotic arm's dexterous hand are obtained;

[0192] Based on the point cloud data, the target gripper pose, and the real-time angle signals of each motor in the robotic arm, the output torque of each motor is adjusted multiple times using a motor control model trained by reinforcement learning, so as to control the dexterous hand to move gradually toward the target object until the dexterous hand reaches the target gripper pose; the dexterous hand does not touch the target object during the process of moving toward the target object;

[0193] When the dexterous hand reaches the target gripper position, it is triggered to perform a grasping action so that the dexterous hand can grasp the target object.

[0194] Optionally, the refined and extended functions of the program can be found in the description above.

[0195] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0196] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0197] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0198] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0199] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0200] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0201] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0202] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A robotic arm control method, characterized in that, include: Acquire images, which include the target object and the robotic arm to be controlled; Based on the image, the point cloud data of the target object and the target gripper pose of the robotic arm's dexterous hand are obtained; Based on the point cloud data, the target gripper pose, and the real-time angle signals of each motor in the robotic arm, the output torque of each motor in the robotic arm is predicted multiple times using a motor control model. During each prediction, the motor control model processes the point cloud data, the target gripper pose, and the real-time angle signals of each motor to determine the next target torque to be output by each motor and the corresponding identification information of the target torque. The identification information indicates whether the dexterous hand has reached the target gripper pose. Each time the target torque of each motor is obtained, control each motor to output the target torque; If the identification information indicates that the dexterous hand has not reached the target gripper posture, the output torque of each motor is predicted again until the dexterous hand reaches the target gripper posture; The dexterous hand does not touch the target object while moving towards it; The motor control model is obtained through reinforcement learning training with the goal of the dexterous hand of the robotic arm simulation model reaching the random target gripper pose corresponding to the object simulation model, and the constraint that the dexterous hand does not collide with the point cloud data of the object simulation model; the robotic arm simulation model is generated based on the model file of the robotic arm. When the dexterous hand reaches the target gripper position, it is triggered to perform a grasping action so that the dexterous hand can grasp the target object.

2. The method according to claim 1, characterized in that, The motor control model was trained using the following reinforcement learning method: For any object simulation model among several object simulation models with randomly set postures, obtain the point cloud data of the object simulation model and the random target gripper posture of the robotic arm simulation model corresponding to the object simulation model. The point cloud data of any object simulation model, the pose of the random target gripper, and the real-time angle signals of each motor model in the robotic arm simulation model are input into the motor control model to obtain the next output torque of each motor model and the identification information corresponding to the output torque. The identification information indicates whether the dexterous hand of the robotic arm simulation model has reached the pose of the random target gripper. The parameters of the motor control model are updated with the goal of the dexterous hand of the robotic arm simulation model reaching the random target gripper pose without touching the point cloud data of any object simulation model. The update continues until the identification information corresponding to the torque to be output indicates that the dexterous hand of the robotic arm simulation model has reached the random target gripper pose.

3. The method according to claim 1, characterized in that, The step of obtaining point cloud data of the target object based on the image includes: Obtain the name information of the target object; Based on the name information, object recognition is performed on the image to determine a rectangular region in the image containing the target object; The rectangular region of the image is cropped from the image; The image of the rectangular region is processed to determine the outline of the target object; Point cloud data of the target object is obtained based on the contour of the target object and the depth information of the pixels in its internal region.

4. The method according to claim 3, characterized in that, Also includes: Specific point recognition is performed on the image to identify the first coordinates of multiple specific points on the robotic arm in the camera coordinate system; Obtain the second coordinates of the plurality of specific points in the robot arm coordinate system; based on the first and second coordinates, correct the preset mapping matrix between the camera coordinate system and the robot arm coordinate system to obtain the corrected mapping matrix; Accordingly, the process of obtaining the point cloud data of the target object includes: Based on the contour of the target object and the depth information of the pixels in its internal region, the point cloud data of the target object in the camera coordinate system is obtained. Based on the corrected mapping matrix, the point cloud data of the target object in the camera coordinate system is mapped to the point cloud data of the target object in the robotic arm coordinate system, which is then used as the final point cloud data of the target object.

5. The method according to claim 4, characterized in that, The process of correcting the preset mapping matrix includes: The preset mapping matrix is ​​summed with the correction matrix to be solved to obtain the sum matrix; Based on the sum matrix, the first coordinates of the plurality of specific points are mapped to the third coordinates of the specific points in the coordinate system of the robotic arm. Given that the change in the mapping matrix before and after correction, determined based on the first three rows of the correction matrix, is less than the target value, and the fourth row of the correction matrix is ​​zero, find the value of the correction matrix that minimizes the difference between the third coordinate and the second coordinate. The values ​​of the corrected matrix obtained by solving are summed with the preset mapping matrix to obtain the corrected mapping matrix.

6. The method according to claim 1, characterized in that, Obtaining the target gripper pose of the robotic arm's dexterous hand based on the image includes: The point cloud data of the target object in the camera coordinate system is processed by a pre-trained gripper pose prediction model to obtain the target gripper pose. The gripper pose prediction model is trained using a target training dataset. The target training dataset includes baseline training samples and augmented training samples. The baseline training samples are point cloud data of real objects with randomly set poses in the camera coordinate system. The labels of the baseline training samples are the gripper poses of the dexterous hand when the robotic arm's dexterous hand is moved near the real object by manual operation. The point cloud data in any augmented training sample is obtained by translating or rotating the point cloud data in a baseline training sample. The labels of any augmented training sample are obtained by translating or rotating the labels of the baseline training sample accordingly.

7. A robotic arm control device, characterized in that, include: The acquisition module is used to acquire images, which include the target object and the robotic arm to be controlled; The acquisition module is used to acquire point cloud data of the target object and the target gripper pose of the dexterous hand of the robotic arm based on the image. The control module is used to predict the output torque of each motor in the robotic arm multiple times based on the point cloud data, the target gripper pose, and the real-time angle signals of each motor in the robotic arm using a motor control model. During each prediction, the motor control model processes the point cloud data, the target gripper pose, and the real-time angle signals of each motor to determine the next target torque to be output by each motor and the identification information corresponding to the target torque. The identification information indicates whether the dexterous hand has reached the target gripper pose. Each time the target torque of each motor is obtained, control each motor to output the target torque; If the identification information indicates that the dexterous hand has not reached the target gripper pose, the output torque of each motor is predicted again until the dexterous hand reaches the target gripper pose; the dexterous hand does not touch the target object during its movement toward the target object; When the dexterous hand reaches the target gripper position, it is triggered to perform a grasping action so that the dexterous hand can grasp the target object; The motor control model is obtained through reinforcement learning training with the goal of the dexterous hand of the robotic arm simulation model reaching the random target gripper pose corresponding to the object simulation model, and the constraint that the dexterous hand does not collide with the point cloud data of the object simulation model; the robotic arm simulation model is generated based on the model file of the robotic arm.

8. A robotic arm control device, characterized in that, Including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement the various steps of the robotic arm control method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the various steps of the robotic arm control method as described in any one of claims 1-6.

Images