Prediction method and device for compensating joint angle
By using predictive models on the robotic arm to predict joint angles, the problem of expensive and uncommon use of equipment such as laser trackers is solved, and efficient compensation for unexpected movements of the robotic arm base is achieved, improving work efficiency and applicability.
Patent Information
- Application Number
- CN202411126525.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-16
AI Technical Summary
In the prior art, coordinate acquisition equipment such as laser trackers are expensive and cannot be flexibly applied to various scenarios, and requires multiple operations and complex calculations to compensate for the unexpected movement of the robotic arm base.
Provide a prediction method and device for compensating joint angle. By installing a camera on the robot arm to capture marking points on the calibration plate, the joint angle and calibration plate position of the robot arm are obtained, and the joint angle prediction model of the trained target compensation joint angle prediction model is predicted. Compensating joint angle after movement is planned to achieve accurate compensation for the unexpected movement of the robotic arm base.
Accurate compensation of joint angle prediction values without multiple fine-tuning and complex operations, greatly improving work efficiency, reducing costs, and maintaining effectiveness in various scenarios.
Smart Images

Figure CN118650632B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of robot arm precision compensation, and in particular to a prediction method and device for compensating joint angles. Background Art
[0002] When the robotic arm moves with different joint configurations, in actual situations, the base of the robotic arm will move unexpectedly, causing the mark points photographed by the camera installed on the robotic arm to move relative to the base. However, the amount of change cannot be learned through various algorithms such as the controller. Therefore, we can only compare the differences and slightly adjust the joint angles of the robotic arm so that when the mark points photographed by the camera are converted to the base, they are consistent with the values of the mark points photographed with other joint angles with larger differences. In other words, the unexpected movement of the base is compensated by the change in the joint angles.
[0003] At present, in the related technology, coordinate acquisition equipment such as laser trackers are used to calibrate the position of the TCP at the end of the robot arm in full space and collect model errors, and then the model is trained using a lookup table method to capture the unexpected movement of the robot arm base, thereby determining the joint angle used to compensate for the unexpected movement. However, the above method is relatively expensive, and the trained model cannot be used in various general scenarios. Summary of the invention
[0004] The present application provides a prediction method and device for compensating joint angles, which at least to some extent solves one of the technical problems in the related art. The technical solution of the present application is as follows:
[0005] According to a first aspect of an embodiment of the present application, a method for predicting a compensated joint angle is provided, comprising: when a camera installed on a robotic arm has photographed a marked point on a calibration plate, obtaining a first joint angle of the robotic arm when the camera is photographing, and obtaining a posture of the calibration plate in the camera coordinate system on which the robotic arm is installed; obtaining a second joint angle of the robotic arm in a movement to be planned after the first joint angle; and using a trained target compensated joint angle prediction model to predict a predicted value of the compensated joint angle used to compensate for unexpected movement of the robotic arm base after the robotic arm executes the second joint angle of the movement to be planned, based on the first joint angle of the robotic arm after photographing by the camera, the posture of the calibration plate and the second joint angle.
[0006] According to the second aspect of an embodiment of the present application, a prediction device for compensating joint angles is provided, comprising: a first acquisition module, for acquiring, when a camera installed on the robotic arm has captured a mark point on a calibration plate, the first joint angle of the robotic arm when the camera is photographed, and acquiring the calibration plate posture in the camera coordinate system on which the robotic arm is installed; a second acquisition module, for acquiring the second joint angle of the robotic arm in a movement to be planned after the first joint angle; a prediction module, for predicting, based on the first joint angle of the robotic arm after the camera is photographed, the calibration plate posture and the second joint angle, a predicted value of the compensation joint angle used to compensate for the unexpected movement of the robotic arm base after the robotic arm executes the second joint angle of the movement to be planned, using a trained target compensation joint angle prediction model.
[0007] According to the third aspect of the embodiments of the present application, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the method for predicting compensated joint angles as described in the embodiment of the first aspect of the present application.
[0008] According to the fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the prediction method for compensating joint angles as described in the embodiment of the first aspect of the present application.
[0009] According to a fifth aspect of the embodiments of the present application, a computer program product is provided, comprising: a computer program, which, when executed by a processor, implements the method for predicting the compensated joint angle as described in the embodiments of the first aspect of the present application.
[0010] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:
[0011] The present application provides a method and device for predicting the compensated joint angle, in which, when a camera installed on a robotic arm has photographed a marked point on a calibration plate, the first joint angle of the robotic arm when the camera is photographed, and the posture of the calibration plate in the camera coordinate system on which the robotic arm is installed, and the second joint angle of the robotic arm to be planned for movement is obtained; based on the first joint angle of the robotic arm after the camera is photographed, the posture of the calibration plate and the second joint angle of the robotic arm, a trained target compensated joint angle prediction model is used to predict the predicted value of the compensated joint angle used to compensate for the unexpected movement of the robotic arm base after the robotic arm executes the second joint angle of the movement to be planned, thereby obtaining the first joint angle of the robotic arm when the camera is photographed, the posture of the calibration plate in the camera coordinate system on which the robotic arm is installed, and the posture of the robotic arm after the first joint angle. The second joint angle of the movement to be planned is input into the trained target compensation joint angle prediction model to predict the compensation joint angle prediction value used to compensate for the unexpected movement of the robot base after the second joint angle of the movement to be planned is executed. Therefore, it is possible to predict the compensation joint angle for compensating for the unexpected movement of the robot base after the robot base executes the joint angle of the movement to be planned by simply inputting the joint angle of the robot after the joint configuration of the robot arm is moved, the position of the robot arm after executing the joint angle, the posture of the calibration plate in the camera coordinate system installed on the robot arm, and the joint angle of the robot arm to be planned into the target compensation joint angle prediction model. This can predict the compensation joint angle for compensating for the unexpected movement of the robot base after the robot arm executes the joint angle of the movement to be planned. There is no need for the staff to perform multiple fine-tuning and complex calculations to obtain accurate compensation joint angle prediction values, thereby greatly improving work efficiency.
[0012] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0014] Figure 1 A schematic diagram of a method for predicting a compensated joint angle provided in an embodiment of the present application Figure 1 ;
[0015] Figure 2 A schematic diagram of a method for predicting a compensated joint angle provided in an embodiment of the present application Figure 2 ;
[0016] Figure 3 A schematic diagram of a method for predicting a compensated joint angle provided in an embodiment of the present application Figure 3 ;
[0017] Figure 4 A schematic diagram of a method for predicting a compensated joint angle provided in an embodiment of the present application Figure 4 ;
[0018] Figure 5 A schematic diagram of the structure of a prediction device for compensating joint angles provided in an embodiment of the present application;
[0019] Figure 6 It is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION
[0020] In order to enable ordinary persons in the art to better understand the technical solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.
[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.
[0022] It should be noted that in the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are all carried out with the user's consent, and are in compliance with relevant laws and regulations and do not violate public order and good morals.
[0023] At present, in the related technology, the method of using laser tracker and other coordinate acquisition equipment to calibrate the position of the whole space and determine the compensation joint angle has the following problems:
[0024] 1. Coordinate acquisition equipment such as laser trackers are expensive and heavily dependent on the accuracy of the equipment.
[0025] 2. Large amounts of complex data require staff to perform multiple operations and calculations.
[0026] 3. It cannot be flexibly applied to various scenarios.
[0027] In response to the above problems, the present application proposes a prediction method and device for compensating joint angles.
[0028] Figure 1 A schematic diagram of a method for predicting a compensated joint angle provided in an embodiment of the present application Figure 1 .
[0029] Step 101, when the camera installed on the robot arm has photographed the marking points on the calibration plate, obtain the first joint angle of the robot arm when the camera photographs, and obtain the position and posture of the calibration plate in the camera coordinate system installed on the robot arm.
[0030] Among them, the robotic arm can be a robotic arm with different joint configurations, for example, a robotic arm with five joint configurations, which is not specifically limited in this embodiment.
[0031] In addition, it should be noted that the first joint angle of the robot arm is used to represent the rotation angles of each joint of the robot arm.
[0032] Among them, the installation method of the camera installed on the robotic arm can be divided into two methods. One method is to install the camera at the end of the robotic arm, and the other method is to fix the camera at a set position. The camera does not move with the robotic arm. This embodiment does not make specific limitations.
[0033] In order to accurately obtain the calibration plate pose in the camera coordinate system installed on the robot, as a possible implementation, the calibration plate pose in the camera coordinate system is determined based on the installation method of the robot camera.
[0034] As an example, based on the installation method of the robot arm camera, a corresponding calibration method is selected. Based on the corresponding calibration method, when the camera installed on the robot arm has photographed the marking points on the calibration plate, the position and posture of the calibration plate in the camera coordinate system is determined.
[0035] Step 102, obtaining a second joint angle of the robot arm for planned movement after the robot arm is at the first joint angle.
[0036] Step 103, based on the first joint angle, calibration plate posture and second joint angle of the robotic arm after being photographed by the camera, a trained target compensation joint angle prediction model is used to predict the predicted value of the compensation joint angle used to compensate for the unexpected movement of the robotic arm base after the robotic arm executes the second joint angle of the planned movement.
[0037] It should be understood that when the joint configuration of the robot arm moves, that is, when the robot arm executes the joint angle of the planned movement, in actual circumstances, the robot arm base supporting the robot arm will undergo unexpected changes, which will cause the distance between the robot arm base and the marking point on the calibration plate to change, resulting in the relative position between the calibration plate and the robot arm base to change, and this change is difficult to observe through the camera, so that the subsequent robot arm cannot accurately reach the target position after the joint configuration moves. Therefore, in order to enable the robot arm to accurately reach the target position, after the robot arm executes the joint angle of the planned movement, the joint angle is fine-tuned to compensate for the unexpected movement of the robot arm base.
[0038] The method for predicting the compensated joint angle provided in the present application obtains, when a camera installed on the robot arm has photographed the marked points on the calibration plate, the first joint angle of the robot arm when the camera is photographed, and the posture of the calibration plate in the camera coordinate system on which the robot arm is installed; obtains the second joint angle of the robot arm to be planned for movement after the first joint angle is obtained; and according to the first joint angle of the robot arm after the camera is photographed, the posture of the calibration plate and the second joint angle, a trained target compensated joint angle prediction model is used to predict the predicted value of the compensated joint angle used to compensate for the unexpected movement of the robot arm base after the robot arm executes the second joint angle for movement to be planned, thereby obtaining the first joint angle of the robot arm when the camera is photographed, the posture of the calibration plate in the camera coordinate system on which the robot arm is installed, and the posture of the calibration plate in the camera coordinate system on which the robot arm is installed, and the second joint angle of the robot arm to be planned after the first joint angle is obtained. The second joint angle of the planned movement is input into the trained target compensation joint angle prediction model to predict the compensation joint angle prediction value used to compensate for the unexpected movement of the robot base after the second joint angle of the movement to be planned is executed. Therefore, it is only necessary to input the joint angle of the robot after the joint configuration of the robot is moved, the position of the robot after the joint angle is executed, the posture of the calibration plate in the camera coordinate system installed on the robot, and the joint angle of the robot to be planned into the target compensation joint angle prediction model, so as to predict the compensation joint angle for compensating for the unexpected movement of the robot base after the robot executes the joint angle of the movement to be planned. There is no need for the staff to perform multiple fine-tuning and complex calculations, and accurate compensation joint angle prediction values can be obtained, thereby greatly improving work efficiency.
[0039] In order to clearly illustrate how the target compensation joint angle prediction model is determined in the above embodiment, the present application proposes another prediction method for the compensation joint angle.
[0040] Figure 2 A schematic diagram of a method for predicting a compensated joint angle provided in an embodiment of the present application Figure 2 .
[0041] like Figure 2 As shown, the prediction method of the compensated joint angle may include the following steps:
[0042] Step 201, obtain the initial joint angles, initial calibration plate pose and planned joint angle sequence of the robot arm, wherein each planned joint angle in the planned joint angle sequence is used to indicate the joint angle of each planned movement of the robot arm, wherein the initial calibration plate pose is used to indicate the calibration plate pose in the camera coordinate system obtained by photographing when the robot arm is at the initial joint angle.
[0043] It can be understood that the initial joint angle may be the current joint angle of the robotic arm, or the joint angle after the operator moves the robotic arm.
[0044] In addition, it should be noted that the planned joint angle sequence can be preset according to actual conditions, or the initial joint angles can be calculated to determine the planned joint angle sequence.
[0045] Step 202 , based on the initial joint angles and the initial calibration plate posture, and the planned joint angle sequence, at least one first compensation joint angle for compensating for the unexpected movement of the robot arm base is measured.
[0046] In order to accurately obtain at least one first compensation joint angle for compensating for the unexpected movement of the robot arm base, as a possible implementation method, based on the initial joint angle and the initial calibration plate posture, and the planned joint angle sequence, multiple rounds of forward iteration process are performed to obtain at least one first compensation joint angle for compensating for the unexpected movement of the robot arm base.
[0047] Each time a round of forward iteration is performed, the planned joint angles in the planned joint angle sequence output in the forward arrangement order corresponding to the current round of forward iteration are determined.
[0048] For example, if the planned joint angle sequence is (joint-1, joint-2, ..., joint-N), the planned joint angles in the planned joint angle sequence are output in forward arrangement order. For example, the first round of forward iteration process is executed to determine the first planned joint angle in the planned joint angle sequence output in forward arrangement order corresponding to this round of forward iteration process, namely, joint-1.
[0049] Step 203, obtain the last planned joint angle in the planned joint angle sequence output in the forward arrangement order, and the calibration plate pose in the camera coordinate system obtained when the robot arm is at the position after executing the last planned joint angle.
[0050] For example, if the planned joint angle sequence is (joint-1, joint-2, ..., joint-N), the last planned joint angle in the planned joint angle sequence, namely joint-N, is output in forward arrangement order.
[0051] Step 204 , based on the last planned joint angle and calibration plate pose, the initial joint angle and the initial calibration plate pose, and the planned joint angle sequence, at least one second compensation joint angle for compensating for the unexpected movement of the robot arm base is measured.
[0052] In order to accurately obtain at least one second compensating joint angle for compensating for the unexpected movement of the robot arm base, as a possible implementation method, multiple rounds of negative iteration process are performed based on the last planned joint angle and calibration plate pose, the initial joint angle and the initial calibration plate pose, and the planned joint angle sequence to obtain at least one second compensating joint angle for compensating for the unexpected movement of the robot arm base.
[0053] Each time a negative iteration process is performed, the planned joint angles in the planned joint angle sequence output in the negative arrangement order corresponding to the current negative iteration process are determined.
[0054] For example, if the planned joint angle sequence is (joint-1, joint-2, ..., joint-N-1, joint-N), the planned joint angles in the planned joint angle sequence are output in negative order. For example, when the first round of negative iteration process is executed, since the robot arm has executed the last planned joint angle, the first round of negative iteration process corresponds to the output of the second planned joint angle in the planned joint angle sequence in negative order, namely, joint-N-1.
[0055] Step 205, input the initial joint angle and the initial calibration plate posture, as well as each planned joint angle in the planned joint angle sequence into the compensation joint angle prediction model to obtain the first compensation joint angle prediction value used to compensate for the unexpected movement of the robot arm base after the robot arm executes the corresponding planned joint angle.
[0056] Step 206, input the last planned joint angle and calibration plate posture, the initial joint angle and initial calibration plate posture, and each planned joint angle in the planned joint angle sequence into the compensation joint angle prediction model to obtain a second compensation joint angle prediction value used to compensate for the unexpected movement of the robot arm base after the robot arm executes the corresponding planned joint angle.
[0057] Step 207, training the compensation joint angle prediction model according to the difference between the first compensation joint angle prediction value and the first compensation joint angle, and the difference between the second compensation joint angle prediction value and the second compensation joint angle, to obtain a trained target compensation joint angle prediction model.
[0058] The present application provides a method for predicting a compensated joint angle, which obtains an initial joint angle, an initial calibration plate pose, and a planned joint angle sequence of a robotic arm. Based on the initial joint angle, the initial calibration plate pose, and the planned joint angle sequence, at least one first compensated joint angle for compensating for unexpected motion of a robotic arm base is measured. The last planned joint angle in the planned joint angle sequence output in a forward arrangement order and the calibration plate pose in a camera coordinate system obtained when the robotic arm is at the position after executing the last planned joint angle are obtained. Based on the last planned joint angle and the calibration plate pose, The initial joint angle and the initial calibration plate posture, as well as the planned joint angle sequence, are used to measure at least one second compensation joint angle for compensating for the unexpected movement of the robot base, and the initial joint angle and the initial calibration plate posture, as well as each planned joint angle in the planned joint angle sequence are input into the compensation joint angle prediction model to obtain a first compensation joint angle prediction value used by the robot to compensate for the unexpected movement of the robot base after the robot executes the corresponding planned joint angle, and the last planned joint angle and the calibration plate posture, the initial joint angle and the initial calibration plate posture, as well as each planned joint angle in the planned joint angle sequence are input into the compensation joint angle prediction model. The compensation joint angle prediction model is introduced into the compensation joint angle prediction model to obtain a second compensation joint angle prediction value used to compensate for the unexpected movement of the robot arm base after the robot arm executes the corresponding planned joint angle. According to the difference between the first compensation joint angle prediction value and the first compensation joint angle, and the difference between the second compensation joint angle prediction value and the second compensation joint angle, the compensation joint angle prediction model is trained to obtain a trained target compensation joint angle prediction model. Thus, based on the initial joint angle and the initial calibration plate posture, as well as the planned joint angle sequence, repeated measurements are performed to obtain the first compensation joint angle used to compensate for the unexpected movement of the robot arm base, and based on the last planned joint angle and the calibration plate posture, the initial joint angle and the initial calibration plate posture, as well as the planned joint angle sequence, repeated measurements are performed to obtain the second compensation joint angle used to compensate for the unexpected movement of the robot arm base. Thus, the compensation joint angle prediction model can be trained by multiple sets of data, so that the compensation joint angle prediction model can accurately determine the relationship between the joint angle of the robot arm after the camera is shot, the calibration plate posture and the joint angle of the movement to be planned and the compensation joint angle, so that the target compensation joint angle prediction model can accurately predict the compensation joint angle.
[0059] In order to clearly illustrate that the above embodiment is based on the initial joint angle and the initial calibration plate posture, and the planned joint angle sequence, to measure at least one first compensation joint angle for compensating for the unexpected movement of the robot arm base, the present application proposes another method for predicting the compensation joint angle.
[0060] Figure 3 A schematic diagram of a method for predicting a compensated joint angle provided in an embodiment of the present application Figure 3 .
[0061] like Figure 3 As shown, the prediction method of the compensated joint angle may include the following steps:
[0062] Step 301 , based on the initial joint angles and the initial calibration plate pose, as well as the planned joint angle sequence, execute multiple rounds of forward iteration process to obtain at least one first compensation joint angle for compensating for the unexpected movement of the robot arm base.
[0063] It can be understood that, each time a round of forward iteration process is executed, the planned joint angles in the planned joint angle sequence output in the forward arrangement order corresponding to the current round of forward iteration process are determined.
[0064] Step 302, each time a round of forward iteration process is performed, based on the initial joint angles and the initial calibration plate posture, an initial transformation matrix from the coordinate system of the robot base to the marking point to be photographed by the camera is obtained, and based on the planned joint angles, a first transformation matrix from the coordinate system of the robot base to the marking point is obtained.
[0065] In order to accurately obtain the initial transformation matrix, as an implementation method, based on the initial joint angles and the initial calibration plate posture, the initial transformation matrix from the coordinate system of the robot base to the marked point is obtained.
[0066] As an example, the initial joint angle is substituted into the forward solution algorithm to obtain the transformation matrix from the coordinate system of the robot base to the center point of the flange at the end of the robot arm; based on the initial joint angle, the hand-eye calibration algorithm is used to obtain the transformation matrix from the center point of the flange at the end of the robot arm to the marker point; according to the transformation matrix from the coordinate system of the robot base to the center point of the flange at the end of the robot arm and the transformation matrix from the center point of the flange at the end of the robot arm to the camera, the transformation matrix from the coordinate system of the robot base to the camera is obtained; according to the transformation matrix from the coordinate system of the robot base to the camera and the initial calibration plate posture, the initial transformation matrix from the coordinate system of the robot base to the marker point is obtained.
[0067] In order to accurately obtain the first transformation matrix, as an implementation method, when the robot is in the position after executing the planned joint angle, the calibration plate posture in the camera coordinate system installed on the robot is obtained, and based on the planned joint angle and the calibration plate posture, the first transformation matrix from the coordinate system of the robot base to the marked point is obtained.
[0068] As an example, the planned joint angle is substituted into the forward solution algorithm to obtain the transformation matrix from the coordinate system of the robot base to the center point of the flange at the end of the robot arm; based on the planned joint angle, the hand-eye calibration algorithm is used to obtain the transformation matrix from the center point of the flange at the end of the robot arm to the marker point; according to the transformation matrix from the coordinate system of the robot base to the center point of the flange at the end of the robot arm and the transformation matrix from the center point of the flange at the end of the robot arm to the camera, the transformation matrix from the coordinate system of the robot base to the camera is obtained; according to the transformation matrix from the coordinate system of the robot base to the camera and the posture of the calibration plate, the first transformation matrix from the coordinate system of the robot base to the marker point is obtained.
[0069] For example, the transformation matrix from the coordinate system of the robot base to the camera can be expressed by the formula T_base_camera=T_base_fcp*T_fcp_camera, where base is used to represent the robot base, and fcp is used to represent the center point of the flange at the end of the robot. Then T_base_fcp is used to represent the transformation matrix from the coordinate system of the robot base to the center point of the flange at the end of the robot, T_fcp_camera is used to represent the transformation matrix from the coordinate system of the robot base to the center point of the flange at the end of the robot to the marker point, and T_base_camera is used to represent the transformation matrix from the coordinate system of the robot base to the camera.
[0070] Step 303: Obtain the planned joint angles used in the forward iteration process performed in each round before the current round and the first transformation matrix obtained.
[0071] Step 304: If the difference between the initial transformation matrix and the first transformation matrix, or the difference between the first transformation matrix obtained by the forward iteration process in the previous rounds and the first transformation matrix obtained by the forward iteration process in the current round is not within the preset error range, the joint angles of the robot arm are adjusted at least once to obtain the first target transformation matrix from the coordinate system of the robot arm base to the marked point.
[0072] In addition, it should be noted that each time the joint angle of the robot arm is adjusted, the adjustment position of the robot arm after the joint angle is adjusted and the joint angle of the robot arm after adjustment are obtained.
[0073] In order to accurately obtain the first target transformation matrix from the coordinate system of the robot base to the marked point, as a possible implementation method, the adjustment position of the robot after adjusting the joint angle and the joint angle of the robot after adjustment are used to obtain the first target transformation matrix from the coordinate system of the robot base to the marked point.
[0074] As an example, for each adjustment, when the robot arm is adjusting its position, a first adjustment transformation matrix from the coordinate system of the robot arm base to the mark point is obtained; a judgment is made as to whether the difference between the first adjustment transformation matrix and the initial transformation matrix is within a preset error range, or a judgment is made as to whether the difference between the first adjustment transformation matrix and the first transformation matrix obtained by performing a forward iteration process in each round before this round is within a preset error range, so as to obtain a first judgment result; when the first judgment result indicates that the difference between the first adjustment transformation matrix and the initial transformation matrix is within a preset error range, the first adjustment transformation matrix is used as the first target transformation matrix, and the joint angle after adjustment of the robot arm is used as the first actual joint angle of the robot arm; when the first judgment result indicates that the difference between the first adjustment transformation matrix and the first transformation matrix obtained by performing a forward iteration process in each round before this round is within a preset error range, the first adjustment transformation matrix is used as the first target transformation matrix, and the joint angle after adjustment of the robot arm is used as the second actual joint angle of the robot arm.
[0075] Correspondingly, when the first judgment result indicates that the difference between the first adjustment transformation matrix and the initial transformation matrix is not within the preset error range, the joint angles of the robot arm are readjusted to obtain a second adjustment transformation matrix from the coordinate system of the robot arm base to the marked point; when the first judgment result indicates that the difference between the first adjustment transformation matrix and the first transformation matrix obtained by performing the forward iteration process in each round before this round is not within the preset error range, the joint angles of the robot arm are readjusted to obtain a second adjustment transformation matrix from the coordinate system of the robot arm base to the marked point.
[0076] As an example, it is judged whether the difference between the second adjustment transformation matrix and the initial transformation matrix is within a preset error range, or whether the difference between the second adjustment transformation matrix and the first transformation matrix obtained by performing a forward iteration process in each round before the current round is within a preset error range to obtain a second judgment result; when the second judgment result indicates that the difference between the second adjustment transformation matrix and the initial transformation matrix is within a preset error range, or the difference between the second adjustment transformation matrix and the first transformation matrix obtained by performing a forward iteration process in each round before the current round is within a preset error range, the second adjustment transformation matrix is used as the first target transformation matrix.
[0077] It should be noted that the preset error range is the difference between the preset initial transformation matrix and the first transformation matrix, or the critical value of the difference between the first transformation matrix obtained by the forward iteration process in each round before the current round and the first transformation matrix obtained by the forward iteration process in the current round. The value of the preset error range can be preset according to actual needs. This embodiment does not limit the specific value of the preset error range.
[0078] Step 305: If the difference between the first target transformation matrix and the initial transformation matrix is within the preset error range, the current first actual joint angle of the robot arm is obtained; if the difference between the first target transformation matrix and the first transformation matrix obtained by executing the forward iteration process in each round before this round is within the preset error range, the current second actual joint angle of the robot arm is obtained.
[0079] Step 306, based on the difference between the first actual joint angle and the initial joint angle and the difference between the second actual joint angle and the planned joint angle corresponding to the forward iteration process performed in each round before this round, obtain at least one first compensation joint angle corresponding to this round of forward iteration process for compensating for the unexpected movement of the robot arm base.
[0080] It can be understood that when executing the first round of forward iteration process, the first planned joint angle in the planned joint angle sequence output in the forward arrangement order corresponding to the first round of forward iteration process is determined.
[0081] As an example, when executing the first round of forward iteration, if the difference between the initial transformation matrix and the first transformation matrix is not within a preset error range, the joint angle of the robot arm is adjusted at least once to obtain the first target transformation matrix from the coordinate system of the robot arm base to the marked point; if the difference between the first target transformation matrix and the initial transformation matrix is within the preset error range, the current first actual joint angle of the robot arm is obtained; based on the difference between the first actual joint angle and the initial joint angle, a first compensation joint angle corresponding to the first round of forward iteration is obtained for compensating for the unexpected movement of the robot arm base.
[0082] In order to clearly illustrate that in the above embodiment, at least one second compensation joint angle for compensating for the unexpected movement of the robot arm base is measured based on the last planned joint angle and calibration plate posture, the initial joint angle and the initial calibration plate posture, and the planned joint angle sequence, the present application proposes another method for predicting the compensation joint angle.
[0083] Figure 4 A schematic diagram of a method for predicting a compensated joint angle provided in an embodiment of the present application Figure 4 .
[0084] like Figure 4 As shown, the prediction method of the compensated joint angle may include the following steps:
[0085] Step 401, based on the last planned joint angle and calibration plate pose, the initial joint angle and initial calibration plate pose, and the planned joint angle sequence, perform multiple rounds of negative iteration process to obtain at least one second compensation joint angle for compensating for the unexpected movement of the robot arm base.
[0086] Each time a negative iteration process is performed, the planned joint angles in the planned joint angle sequence output in the negative arrangement order corresponding to the current negative iteration process are determined.
[0087] Step 402, each time a negative iteration process is performed, the third transformation matrix from the coordinate system of the robot base to the marked point is obtained based on the last planned joint angle and the calibration plate posture, and the second transformation matrix from the coordinate system of the robot base to the marked point is obtained based on the planned joint angle.
[0088] In order to accurately obtain the second transformation matrix, as an implementation method, when the robot arm is in the position after executing the last planned joint angle, the calibration plate pose in the camera coordinate system installed on the robot arm is obtained, and based on the last planned joint angle and the calibration plate pose, the third transformation matrix from the coordinate system of the robot arm base to the marked point is obtained, and based on the planned joint angle, the second transformation matrix from the coordinate system of the robot arm base to the marked point is obtained.
[0089] As an example, the planned joint angle is substituted into the forward solution algorithm to obtain the transformation matrix from the coordinate system of the robot base to the center point of the flange at the end of the robot arm; based on the planned joint angle, the hand-eye calibration algorithm is used to obtain the transformation matrix from the center point of the flange at the end of the robot arm to the marker point; according to the transformation matrix from the coordinate system of the robot base to the center point of the flange at the end of the robot arm and the transformation matrix from the center point of the flange at the end of the robot arm to the camera, the transformation matrix from the coordinate system of the robot base to the camera is obtained; according to the transformation matrix from the coordinate system of the robot base to the camera and the posture of the calibration plate, the second transformation matrix from the coordinate system of the robot base to the marker point is obtained.
[0090] Step 403: Obtain the planned joint angles corresponding to the negative iteration process performed in each round before the current round and the obtained second transformation matrix.
[0091] Step 404: If the difference between the third transformation matrix and the second transformation matrix is not within the preset error range, or the difference between the second transformation matrix obtained by the negative iteration process in the previous rounds and the second transformation matrix obtained by the negative iteration process in this round is not within the preset error range, the joint angle of the robot arm is adjusted at least once to obtain the second target transformation matrix from the coordinate system of the robot arm base to the marked point.
[0092] In order to accurately obtain the second target transformation matrix from the coordinate system of the robot arm base to the marked point, as a possible implementation method, the adjustment position of the robot arm after adjusting the joint angle and the joint angle of the robot arm after adjustment are used to obtain the second target transformation matrix from the coordinate system of the robot arm base to the marked point.
[0093] As an example, for each adjustment, when the robot arm is adjusting its position, a third adjustment transformation matrix from the coordinate system of the robot arm base to the mark point is obtained; a judgment is made as to whether the difference between the third adjustment transformation matrix and the third transformation matrix is within a preset error range, or a judgment is made as to whether the difference between the third adjustment transformation matrix and the second transformation matrix obtained by performing a negative iteration process in each round before this round is within a preset error range, so as to obtain a third judgment result; when the third judgment result indicates that the difference between the third adjustment transformation matrix and the third transformation matrix is within a preset error range, the third adjustment transformation matrix is used as the second target transformation matrix, and the joint angle after adjustment of the robot arm is used as the fourth actual joint angle of the robot arm; when the third judgment result indicates that the difference between the third adjustment transformation matrix and the second transformation matrix obtained by performing a negative iteration process in each round before this round is within a preset error range, the third adjustment transformation matrix is used as the second target transformation matrix, and the joint angle after adjustment of the robot arm is used as the third actual joint angle of the robot arm.
[0094] Correspondingly, when the second judgment result indicates that the difference between the third adjustment transformation matrix and the third transformation matrix is not within the preset error range, the joint angles of the robot arm are readjusted to obtain the fourth adjustment transformation matrix from the coordinate system of the robot arm base to the marked point; when the third judgment result indicates that the difference between the third adjustment transformation matrix and the second transformation matrix obtained by performing the negative iteration process in the rounds before this round is not within the preset error range, the joint angles of the robot arm are readjusted to obtain the fourth adjustment transformation matrix from the coordinate system of the robot arm base to the marked point.
[0095] As an example, it is judged whether the difference between the fourth adjustment transformation matrix and the third transformation matrix is within a preset error range, or whether the difference between the fourth adjustment transformation matrix and the second transformation matrix obtained by performing a negative iteration process in each round before the current round is within the preset error range, so as to obtain a fourth judgment result; when the fourth judgment result indicates that the difference between the fourth adjustment transformation matrix and the third transformation matrix is within the preset error range, or the difference between the fourth adjustment transformation matrix and the second transformation matrix obtained by performing a negative iteration process in each round before the current round is within the preset error range, the fourth adjustment transformation matrix is used as the second target transformation matrix.
[0096] Step 405: If the difference between the second target transformation matrix and the second transformation matrix obtained by performing the negative iteration process in each round before this round is within the preset error range, the current third actual joint angle of the robot arm is obtained; if the difference between the second target transformation matrix and the third transformation matrix is within the preset error range, the current fourth actual joint angle of the robot arm is obtained.
[0097] Step 406, based on the difference between the third actual joint angle and the planned joint angles corresponding to the negative iteration processes performed in the rounds before this round, and the difference between the fourth actual joint angle and the last planned joint angle, obtain at least one second compensation joint angle corresponding to this round of negative iteration process for compensating for the unexpected movement of the robot arm base.
[0098] It can be understood that when executing the first round of negative iteration process, the second planned joint angle in the planned joint angle sequence output in the negative arrangement order corresponding to the first round of negative iteration process is determined.
[0099] As an example, when executing the first round of negative iteration process, determine the second planned joint angle in the planned joint angle sequence output in the negative arrangement order corresponding to the first round of negative iteration process; based on the last planned joint angle and the calibration plate posture, obtain the third transformation matrix from the coordinate system of the robot base to the marked point, and based on the second planned joint angle, obtain the second transformation matrix from the coordinate system of the robot base to the marked point; if the difference between the third transformation matrix and the second transformation matrix is not within the preset error range, adjust the joint angle of the robot arm at least once to obtain the second target transformation matrix from the coordinate system of the robot base to the marked point; if the difference between the second target transformation matrix and the third transformation matrix is within the preset error range, obtain the fourth actual joint angle of the robot arm.
[0100] It can be understood that when executing the last round of negative iteration process, it is determined that the last round of negative iteration process corresponds to the initial joint angle.
[0101] As an example, when executing the last round of negative iteration process, the initial joint angle corresponding to the last round of negative iteration process is determined; based on the last planned joint angle and the calibration plate posture, the third transformation matrix from the coordinate system of the robot base to the marked point is obtained, and based on the initial joint angle and the initial calibration plate posture, the initial transformation matrix from the coordinate system of the robot base to the marked point is obtained; if the difference between the third transformation matrix and the initial transformation matrix is not within the preset error range, or the second transformation matrix and the initial transformation matrix obtained by executing the negative iteration process in each round before this round are not within the preset error range, the joint angle of the robot is adjusted at least once to obtain the second target transformation matrix from the coordinate system of the robot base to the marked point; if the difference between the second target transformation matrix and the second transformation matrix obtained by executing the negative iteration process in each round before this round is within the preset error range, the third actual joint angle of the robot is obtained; if the difference between the second target transformation matrix and the third transformation matrix is within the preset error range, the fourth actual joint angle of the robot is obtained.
[0102] Corresponding to the prediction method for compensating joint angle provided in the above-mentioned embodiment, the present application also provides a prediction device for compensating joint angle. Since the prediction device for compensating joint angle provided in the embodiment of the present application corresponds to the prediction method for compensating joint angle provided in the above-mentioned embodiment, the implementation method of the prediction method for compensating joint angle is also applicable to the prediction device for compensating joint angle provided in the embodiment of the present application, and will not be described in detail in the embodiment of the present application.
[0103] Figure 5 It is a structural schematic diagram of a prediction device for compensating joint angle provided in an embodiment of the present application.
[0104] like Figure 5 As shown, the prediction device 500 for compensating joint angles includes: a first acquisition module 501, a second acquisition module 502 and a prediction module 503.
[0105] A first acquisition module is used to acquire the first joint angle of the robotic arm when the camera is photographed, and to acquire the position and posture of the calibration plate in the camera coordinate system installed on the robotic arm, when the camera installed on the robotic arm has photographed the marking point on the calibration plate;
[0106] A second acquisition module is used to acquire a second joint angle of the robot arm to be planned for movement after the robot arm is at the first joint angle;
[0107] The prediction module is used to predict the compensation joint angle prediction value used to compensate for the unexpected movement of the robot arm base after the robot arm executes the second joint angle of the planned movement based on the first joint angle, calibration plate posture and second joint angle of the robot arm after being photographed by the camera, using the trained target compensation joint angle prediction model.
[0108] As a possible implementation of the embodiment of the present application, the prediction device 500 for compensating joint angles further includes: a training module
[0109] The training module includes: a first acquisition unit, which is used to acquire the initial joint angle of the robot arm, the initial calibration plate posture and the planned joint angle sequence before determining the target compensation joint angle prediction model, wherein each planned joint angle in the planned joint angle sequence is used to indicate the joint angle of each planned movement of the robot arm, wherein the initial calibration plate posture is used to indicate the calibration plate posture in the camera coordinate system obtained by shooting when the robot arm is at the initial joint angle;
[0110] A first measuring unit is used to measure and obtain at least one first compensation joint angle for compensating for unexpected motion of a robot arm base based on an initial joint angle and an initial calibration plate posture, and a planned joint angle sequence;
[0111] A second acquisition unit is used to acquire the last planned joint angle in the planned joint angle sequence output in a forward arrangement order, and the calibration plate pose in the camera coordinate system obtained when the robot arm is at the position after executing the last planned joint angle;
[0112] A second measuring unit is used to measure and obtain at least one second compensation joint angle for compensating for unexpected motion of the robot arm base based on the last planned joint angle and calibration plate posture, the initial joint angle and the initial calibration plate posture, and the planned joint angle sequence;
[0113] A first obtaining unit is used to input the initial joint angle and the initial calibration plate posture, and each planned joint angle in the planned joint angle sequence into a compensation joint angle prediction model to obtain a first compensation joint angle prediction value used to compensate for the unexpected movement of the robot arm base after the robot arm executes the corresponding planned joint angle;
[0114] A second obtaining unit is used to input the last planned joint angle and the calibration plate posture, the initial joint angle and the initial calibration plate posture, and each planned joint angle in the planned joint angle sequence into the compensation joint angle prediction model to obtain a second compensation joint angle prediction value used to compensate for the unexpected movement of the robot arm base after the robot arm executes the corresponding planned joint angle;
[0115] The training unit is used to train the compensation joint angle prediction model according to the difference between the first compensation joint angle prediction value and the first compensation joint angle, and the difference between the second compensation joint angle prediction value and the second compensation joint angle, so as to obtain a trained target compensation joint angle prediction model.
[0116] In one embodiment of the application, the first obtaining unit is specifically used to:
[0117] Based on the initial joint angles and the initial calibration plate posture, as well as the planned joint angle sequence, multiple rounds of forward iteration process are performed to obtain at least one first compensation joint angle for compensating for the unexpected motion of the robot arm base, wherein each round of forward iteration process is performed to determine the planned joint angle in the planned joint angle sequence output in the forward arrangement order corresponding to the current round of forward iteration process;
[0118] Based on the initial joint angles and the initial calibration plate posture, an initial transformation matrix from the coordinate system of the robot base to the marker point to be photographed by the camera is obtained, and based on the planned joint angles, a first transformation matrix from the coordinate system of the robot base to the marker point is obtained;
[0119] Obtain the planned joint angles used in the forward iteration process in each round before this round and the first transformation matrix obtained;
[0120] If the difference between the initial transformation matrix and the first transformation matrix, or the difference between the first transformation matrix obtained by the forward iteration process in each round before this round and the first transformation matrix obtained by the forward iteration process in this round is not within the preset error range, the joint angle of the manipulator is adjusted at least once to obtain the first target transformation matrix from the coordinate system of the manipulator base to the marked point;
[0121] If the difference between the first target transformation matrix and the initial transformation matrix is within the preset error range, the current first actual joint angle of the robot arm is obtained; if the difference between the first target transformation matrix and the first transformation matrix obtained by performing the forward iteration process in each round before this round is within the preset error range, the current second actual joint angle of the robot arm is obtained;
[0122] Based on the difference between the first actual joint angle and the initial joint angle and the difference between the second actual joint angle and the planned joint angle corresponding to the forward iteration process performed in each round before this round, at least one first compensation joint angle corresponding to this round of forward iteration process is obtained to compensate for the unexpected movement of the robot arm base.
[0123] In one embodiment of the application, the second obtaining unit is specifically used for:
[0124] Based on the last planned joint angle and the calibration plate pose, the initial joint angle and the initial calibration plate pose, and the planned joint angle sequence, multiple rounds of negative iteration process are performed to obtain at least one second compensation joint angle for compensating for the unexpected motion of the robot arm base, wherein each round of negative iteration process is performed to determine the planned joint angle in the planned joint angle sequence output in the negative arrangement order corresponding to the current round of negative iteration process;
[0125] Based on the last planned joint angle and the calibration plate posture, a third transformation matrix from the coordinate system of the robot base to the marked point is obtained, and based on the planned joint angle, a second transformation matrix from the coordinate system of the robot base to the marked point is obtained;
[0126] Obtain the planned joint angles corresponding to the negative iteration process performed in each round before this round and the obtained second transformation matrix;
[0127] If the difference between the third transformation matrix and the second transformation matrix is not within the preset error range, or the difference between the second transformation matrix obtained by the negative iteration process in each round before this round and the second transformation matrix obtained by the negative iteration process in this round is not within the preset error range, the joint angle of the manipulator is adjusted at least once to obtain the second target transformation matrix from the coordinate system of the manipulator base to the marked point;
[0128] If the difference between the second target transformation matrix and the second transformation matrix obtained by performing the negative iteration process in each round before this round is within the preset error range, the current third actual joint angle of the robot arm is obtained; if the difference between the second target transformation matrix and the third transformation matrix is within the preset error range, the current fourth actual joint angle of the robot arm is obtained;
[0129] Based on the difference between the third actual joint angle and the planned joint angles corresponding to the negative iteration processes performed in the rounds before this round, and the difference between the fourth actual joint angle and the last planned joint angle, at least one second compensation joint angle corresponding to this round of negative iteration process is obtained to compensate for the unexpected movement of the robot arm base.
[0130] In one embodiment of the application, when executing the first round of forward iteration process, the first obtaining unit is specifically used to:
[0131] If the difference between the initial transformation matrix and the first transformation matrix is not within the preset error range, the joint angles of the robot arm are adjusted at least once to obtain a first target transformation matrix from the coordinate system of the robot arm base to the marking point;
[0132] If the difference between the first target transformation matrix and the initial transformation matrix is within a preset error range, obtaining the current first actual joint angle of the robotic arm;
[0133] Based on the difference between the first actual joint angle and the initial joint angle, a first compensation joint angle corresponding to the first round of forward iteration process is obtained to compensate for the unexpected movement of the robot arm base.
[0134] In one embodiment of the application, the first obtaining unit is further specifically used for:
[0135] Based on the initial joint angles and the initial calibration plate pose, the initial transformation matrix from the coordinate system of the robot base to the marker point is obtained;
[0136] When the robot arm is in the position after executing the planned joint angle, the position and posture of the calibration plate in the camera coordinate system installed on the robot arm are obtained;
[0137] Based on the planned joint angles and the calibration plate pose, the first transformation matrix from the coordinate system of the robot base to the marked point is obtained.
[0138] In one embodiment of the application, the first obtaining unit is further specifically used for:
[0139] Substitute the planned joint angles into the forward solution algorithm to obtain the transformation matrix from the coordinate system of the robot base to the center point of the flange at the end of the robot arm;
[0140] Based on the planned joint angles, the hand-eye calibration algorithm is used to obtain the transformation matrix from the center point of the flange at the end of the robot to the marked point;
[0141] According to the transformation matrix from the coordinate system of the robot base to the center point of the flange at the end of the robot and the transformation matrix from the center point of the flange at the end of the robot to the camera, the transformation matrix from the coordinate system of the robot base to the camera is obtained;
[0142] According to the transformation matrix from the coordinate system of the robot base to the camera and the position and posture of the calibration plate, the first transformation matrix from the coordinate system of the robot base to the marked point is obtained.
[0143] In one embodiment of the application, the first obtaining unit is further specifically used for:
[0144] Each time the joint angle of the robot arm is adjusted, the adjustment position of the robot arm after the joint angle is adjusted and the joint angle of the robot arm after adjustment are obtained;
[0145] When the robot arm is adjusting its position, a first adjustment transformation matrix from the coordinate system of the robot arm base to the marking point is obtained;
[0146] Determine whether the difference between the first adjusted transformation matrix and the initial transformation matrix is within a preset error range, or determine whether the difference between the first adjusted transformation matrix and the first transformation matrix obtained by performing the forward iteration process in each round before the current round is within a preset error range, so as to obtain a first determination result;
[0147] When the first judgment result indicates that the difference between the first adjustment transformation matrix and the initial transformation matrix is within a preset error range, the first adjustment transformation matrix is used as the first target transformation matrix, and the adjusted joint angle of the robotic arm is used as the first actual joint angle of the robotic arm;
[0148] When the first judgment result indicates that the difference between the first adjustment transformation matrix and the first transformation matrix obtained by performing the forward iteration process in each round before the current round is within a preset error range, the first adjustment transformation matrix is used as the first target transformation matrix, and the joint angle of the manipulator after adjustment is used as the second actual joint angle of the manipulator;
[0149] When the first judgment result indicates that the difference between the first adjustment transformation matrix and the initial transformation matrix is not within the preset error range, the joint angles of the robot arm are readjusted to obtain a second adjustment transformation matrix from the coordinate system of the robot arm base to the marking point;
[0150] When the first judgment result indicates that the difference between the first adjustment transformation matrix and the first transformation matrix obtained by executing the forward iteration process in each round before this round is not within the preset error range, the joint angles of the robot arm are readjusted to obtain a second adjustment transformation matrix from the coordinate system of the robot arm base to the marked point.
[0151] In one embodiment of the application, the first obtaining unit is further specifically used for:
[0152] Determine whether the difference between the second adjusted transformation matrix and the initial transformation matrix is within a preset error range, or determine whether the difference between the second adjusted transformation matrix and the first transformation matrix obtained by performing the forward iteration process in each round before the current round is within a preset error range, so as to obtain a second determination result;
[0153] When the second judgment result indicates that the difference between the second adjusted transformation matrix and the initial transformation matrix is within a preset error range, or the difference between the second adjusted transformation matrix and the first transformation matrix obtained by performing the forward iteration process in each round before this round is within a preset error range, the second adjusted transformation matrix is used as the first target transformation matrix.
[0154] In one embodiment of the application, when executing the first round of negative iteration process, the second obtaining unit is further specifically used for:
[0155] Determine the second planned joint angle in the planned joint angle sequence output in the negative arrangement order corresponding to the first round of negative iteration process;
[0156] Based on the last planned joint angle and the calibration plate posture, the third transformation matrix from the coordinate system of the robot base to the marked point is obtained, and based on the second planned joint angle, the second transformation matrix from the coordinate system of the robot base to the marked point is obtained;
[0157] If the difference between the third transformation matrix and the second transformation matrix is not within the preset error range, the joint angles of the robot arm are adjusted at least once to obtain a second target transformation matrix from the coordinate system of the robot arm base to the marking point;
[0158] If the difference between the second target transformation matrix and the third transformation matrix is within a preset error range, a fourth actual joint angle of the robotic arm is obtained.
[0159] In one embodiment of the application, when executing the last round of negative iteration process, the second obtaining unit is further specifically used for:
[0160] Determine the initial joint angle corresponding to the last round of negative iteration process;
[0161] Based on the last planned joint angle and the calibration plate posture, the third transformation matrix from the coordinate system of the robot base to the marked point is obtained, and based on the initial joint angle and the initial calibration plate posture, the initial transformation matrix from the coordinate system of the robot base to the marked point is obtained;
[0162] If the difference between the third transformation matrix and the initial transformation matrix is not within the preset error range, or the second transformation matrix and the initial transformation matrix obtained by the negative iteration process in the previous rounds are not within the preset error range, the joint angles of the manipulator are adjusted at least once to obtain the second target transformation matrix from the coordinate system of the manipulator base to the marked point;
[0163] If the difference between the second target transformation matrix and the second transformation matrix obtained by performing the negative iteration process in each round before the current round is within a preset error range, obtaining the third actual joint angle of the robot arm;
[0164] If the difference between the second target transformation matrix and the third transformation matrix is within a preset error range, a fourth actual joint angle of the robotic arm is obtained.
[0165] It should be noted that the aforementioned explanation of the embodiment of the method for predicting the compensated joint angle is also applicable to the prediction device for predicting the compensated joint angle of this embodiment, and will not be repeated here.
[0166] The device for predicting the compensated joint angle provided in the present application obtains the first joint angle of the robotic arm when the camera is photographed, and obtains the posture of the calibration plate in the camera coordinate system on which the robotic arm is installed; obtains the second joint angle of the robotic arm to be planned after the first joint angle is obtained; and uses the trained target compensated joint angle prediction model to predict the predicted value of the compensated joint angle used to compensate for the unexpected movement of the robotic arm base after the robotic arm executes the second joint angle to be planned, based on the first joint angle, the posture of the calibration plate and the second joint angle of the robotic arm after the camera is photographed. Thus, the first joint angle of the robotic arm when the camera is photographed, the posture of the calibration plate in the camera coordinate system on which the robotic arm is installed and the posture of the calibration plate in the camera coordinate system on which the robotic arm is installed and the posture of the calibration plate in the first joint angle to be planned after the robotic arm is The second joint angle of the planned movement is input into the trained target compensation joint angle prediction model to predict the compensation joint angle prediction value used to compensate for the unexpected movement of the robot base after the second joint angle of the movement to be planned is executed. Therefore, it is only necessary to input the joint angle of the robot after the joint configuration of the robot is moved, the position of the robot after the joint angle is executed, the posture of the calibration plate in the camera coordinate system installed on the robot, and the joint angle of the robot to be planned into the target compensation joint angle prediction model, so as to predict the compensation joint angle for compensating for the unexpected movement of the robot base after the robot executes the joint angle of the movement to be planned. There is no need for the staff to perform multiple fine-tuning and complex calculations, and accurate compensation joint angle prediction values can be obtained, thereby greatly improving work efficiency.
[0167] In an exemplary embodiment, an electronic device is also provided.
[0168] Among them, electronic equipment includes:
[0169] processor;
[0170] a memory for storing processor-executable instructions;
[0171] The processor is configured to execute instructions to implement the quality inspection method proposed in any of the aforementioned embodiments.
[0172] As an example, Figure 6 is a schematic diagram of the structure of an electronic device 600 shown in an exemplary embodiment of the present application. Figure 6 As shown, the electronic device 600 may further include:
[0173] The memory 610 and the processor 620, a bus 630 connecting different components (including the memory 610 and the processor 620), the memory 610 stores a computer program, and when the processor 620 executes the program, the quality inspection method described in the embodiment of the present application is implemented.
[0174] Bus 630 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.
[0175] The electronic device 600 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the electronic device 600, including volatile and non-volatile media, removable and non-removable media.
[0176] The memory 610 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 640 and / or cache memory 650. The server 600 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 660 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 not shown, often referred to as a "hard drive packager"). Although Figure 6Not shown in the figure, a disk driver package for reading and writing a removable non-volatile disk (such as a "floppy disk"), and an optical disk driver package for reading and writing a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM or other optical medium) can be provided. In these cases, each driver package can be connected to the bus 630 through one or more data medium interfaces. The memory 610 may include at least one program product, which has a set (such as at least one) of program modules, which are configured to perform the functions of each embodiment of the present application.
[0177] A program / utility 680 having a set (at least one) of program modules 670 may be stored, for example, in the memory 610, such program modules 670 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 670 generally perform the functions and / or methods of the embodiments described herein.
[0178] The electronic device 600 may also communicate with one or more external devices 660 (e.g., keyboard, pointing device, display 691, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication may be performed through an input / output (I / O) interface 692. In addition, the electronic device 600 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) through a network adapter 693. As shown, the network adapter 693 communicates with other modules of the electronic device 600 through a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device driver package, redundant processing unit, external disk driver package array, RAID system, tape driver package, and data backup storage system, etc.
[0179] The processor 620 executes various functional applications and data processing by running the programs stored in the memory 610 .
[0180] It should be noted that the implementation process and technical principles of the electronic device of this embodiment refer to the aforementioned explanation of the quality inspection method of the embodiment of the present application, and will not be repeated here.
[0181] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of an electronic device to complete the quality inspection method proposed in any of the above embodiments. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0182] In an exemplary embodiment, a computer program product is also provided, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the quality inspection method proposed in any of the above embodiments is implemented.
[0183] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.
[0184] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A method for predicting a compensated joint angle, characterized in that: The following steps are involved: When a camera installed on the robot arm has photographed a mark point on the calibration plate, obtaining a first joint angle of the robot arm when the camera photographs the mark point, and obtaining a position and posture of the calibration plate in a camera coordinate system installed on the robot arm; Acquire a second joint angle of the robot arm to be planned for movement after the first joint angle; According to the first joint angle of the robotic arm after being photographed by the camera, the calibration plate posture and the second joint angle, a trained target compensation joint angle prediction model is used to predict a predicted value of the compensation joint angle used to compensate for the unexpected movement of the robotic arm base after the robotic arm executes the second joint angle of the movement to be planned; The method further comprises: Before determining the target compensation joint angle prediction model, the initial joint angle, initial calibration plate pose and planned joint angle sequence of the robotic arm are obtained, wherein each planned joint angle in the planned joint angle sequence is used to indicate the joint angle of each movement to be planned of the robotic arm, and the initial calibration plate pose is used to indicate the calibration plate pose in the camera coordinate system obtained by photographing when the robotic arm is at the initial joint angle; Based on the initial joint angle and the initial calibration plate posture, and the planned joint angle sequence, at least one first compensation joint angle for compensating for unexpected movement of the robot arm base is measured; Obtaining the calibration plate pose in the camera coordinate system obtained by outputting the last planned joint angle in the planned joint angle sequence in a forward arrangement order and the position of the robot arm after executing the last planned joint angle; Based on the last of the planned joint angles and the calibration plate pose, the initial joint angles and the initial calibration plate pose, and the planned joint angle sequence, at least one second compensation joint angle for compensating for the unexpected movement of the robot arm base is measured; Inputting the initial joint angle and the initial calibration plate posture, as well as each of the planned joint angles in the planned joint angle sequence into a compensation joint angle prediction model to obtain a first compensation joint angle prediction value used by the robotic arm to compensate for unexpected motion of the robotic arm base after the robotic arm executes the corresponding planned joint angle; Inputting the last planned joint angle and the calibration plate pose, the initial joint angle and the initial calibration plate pose, and each planned joint angle in the planned joint angle sequence into the compensation joint angle prediction model to obtain a second compensation joint angle prediction value used by the robot arm to compensate for the unexpected movement of the robot arm base after the robot arm executes the corresponding planned joint angle; The compensation joint angle prediction model is trained according to the difference between the first compensation joint angle prediction value and the first compensation joint angle, and the difference between the second compensation joint angle prediction value and the second compensation joint angle to obtain the trained target compensation joint angle prediction model.
2. The method according to claim 1, characterized in that The method of obtaining at least one first compensation joint angle for compensating for unexpected motion of the robot arm base based on the initial joint angle, the initial calibration plate posture, and the planned joint angle sequence comprises: Based on the initial joint angle and the initial calibration plate posture, and the planned joint angle sequence, multiple rounds of forward iteration process are performed to obtain at least one first compensation joint angle for compensating for the unexpected movement of the robot arm base, wherein each round of the forward iteration process is performed to determine the planned joint angle in the planned joint angle sequence output in a forward arrangement order corresponding to the current round of the forward iteration process; Based on the initial joint angle and the initial calibration plate posture, an initial transformation matrix from the coordinate system of the robot base to the marking point to be photographed by the camera is obtained, and based on the planned joint angle, a first transformation matrix from the coordinate system of the robot base to the marking point is obtained; Obtaining the planned joint angles used in executing the forward iteration process in each round before the current round and the first transformation matrix obtained; If the difference between the initial transformation matrix and the first transformation matrix, or the difference between the first transformation matrix obtained by executing the forward iteration process in each round before the current round and the first transformation matrix obtained by executing the forward iteration process in the current round is not within a preset error range, the joint angles of the manipulator are adjusted at least once to obtain a first target transformation matrix from the coordinate system to which the manipulator base belongs to the marking point; If the difference between the first target transformation matrix and the initial transformation matrix is within the preset error range, the current first actual joint angle of the robot arm is obtained; if the difference between the first target transformation matrix and the first transformation matrix obtained by executing the forward iteration process in each round before the current round is within the preset error range, the current second actual joint angle of the robot arm is obtained; Based on the difference between the first actual joint angle and the initial joint angle and the difference between the second actual joint angle and the planned joint angle corresponding to the forward iteration process in each round before this round, at least one first compensation joint angle corresponding to the forward iteration process in this round is obtained, which is used to compensate for the unexpected movement of the robot arm base.
3. The method according to claim 1, characterized in that The method of obtaining at least one second compensation joint angle for compensating for unexpected motion of the robot arm base based on the last planned joint angle and the calibration plate pose, the initial joint angle and the initial calibration plate pose, and the planned joint angle sequence comprises: Based on the last planned joint angle and the calibration plate posture, the initial joint angle and the initial calibration plate posture, and the planned joint angle sequence, multiple rounds of negative iteration process are performed to obtain at least one second compensation joint angle for compensating for the unexpected movement of the robot arm base, wherein each round of the negative iteration process is performed to determine the planned joint angle in the planned joint angle sequence output in a negative arrangement order corresponding to the current round of the negative iteration process; Based on the last of the planned joint angles and the calibration plate posture, a third transformation matrix from the coordinate system of the robot base to the marked point is obtained, and based on the planned joint angles, a second transformation matrix from the coordinate system of the robot base to the marked point is obtained; Obtaining the planned joint angles corresponding to the negative iteration process performed in each round before the current round and the obtained second transformation matrix; If the difference between the third transformation matrix and the second transformation matrix is not within a preset error range, or the difference between the second transformation matrix obtained by executing the negative iteration process in each round before this round and the second transformation matrix obtained by executing the negative iteration process in this round is not within a preset error range, the joint angle of the robot arm is adjusted at least once to obtain a second target transformation matrix from the coordinate system to which the robot arm base belongs to the marking point; If the difference between the second target transformation matrix and the second transformation matrix obtained by performing the negative iteration process in each round before the current round is within the preset error range, the current third actual joint angle of the robot arm is obtained; if the difference between the second target transformation matrix and the third transformation matrix is within the preset error range, the current fourth actual joint angle of the robot arm is obtained; Based on the difference between the third actual joint angle and the planned joint angles corresponding to the negative iteration process performed in each round before this round, and the difference between the fourth actual joint angle and the last planned joint angle, at least one second compensation joint angle corresponding to the negative iteration process in this round is obtained, which is used to compensate for the unexpected movement of the robot arm base.
4. The method according to claim 2, characterized in that: When executing the first round of the forward iteration process, the method includes: If the difference between the initial transformation matrix and the first transformation matrix is not within a preset error range, adjusting the joint angles of the robotic arm at least once to obtain a first target transformation matrix from the coordinate system to which the robotic arm base belongs to the marking point; If the difference between the first target transformation matrix and the initial transformation matrix is within the preset error range, obtaining the current first actual joint angle of the robotic arm; Based on the difference between the first actual joint angle and the initial joint angle, a first compensation joint angle corresponding to the first round of the forward iteration process is obtained, which is used to compensate for the unexpected movement of the robot arm base.
5. The method according to claim 2, characterized in that: The method of obtaining an initial transformation matrix from the coordinate system of the robot base to the marking point to be photographed by the camera based on the initial joint angle and the initial calibration plate posture, and obtaining a first transformation matrix from the coordinate system of the robot base to the marking point based on the planned joint angle, includes: Based on the initial joint angle and the initial calibration plate posture, the initial transformation matrix from the coordinate system of the robot base to the marking point is obtained; When the robot arm is in the position after executing the planned joint angle, obtaining the position and posture of the calibration plate in the camera coordinate system installed on the robot arm; Based on the planned joint angles and the calibration plate posture, the first transformation matrix from the coordinate system to which the robot base belongs to the marking point is obtained.
6. The method according to claim 5, characterized in that The step of obtaining the first transformation matrix from the coordinate system to which the robot base belongs to the marked point based on the planned joint angle and the calibration plate posture includes: Substituting the planned joint angle into the forward solution algorithm, obtaining a transformation matrix from the coordinate system of the robot base to the center point of the flange at the end of the robot; Based on the planned joint angles, a hand-eye calibration algorithm is used to obtain a transformation matrix from the center point of the flange at the end of the robotic arm to the marking point; Obtain a transformation matrix from the coordinate system of the robot base to the camera according to a transformation matrix from the coordinate system of the robot base to the center point of the robot end flange and a transformation matrix from the center point of the robot end flange to the camera; According to the transformation matrix from the coordinate system to which the robotic arm base belongs to the camera and the position and posture of the calibration plate, the first transformation matrix from the coordinate system to which the robotic arm base belongs to the marking point is obtained.
7. The method according to claim 2, characterized in that If the difference between the initial transformation matrix and the first transformation matrix, or the difference between the first transformation matrix obtained by executing the forward iteration process in each round before this round and the first transformation matrix obtained by executing the forward iteration process in this round is not within a preset error range, the joint angle of the robot arm is adjusted at least once to obtain a first target transformation matrix from the coordinate system to which the robot arm base belongs to the marking point, include: Wherein, each time the adjustment is made, the joint angle of the robotic arm is adjusted to obtain the adjustment position of the robotic arm after the joint angle is adjusted and the joint angle of the robotic arm after the adjustment; When the robotic arm is in the adjustment position, a first adjustment transformation matrix from the coordinate system to which the robotic arm base belongs to the marking point is obtained; Determine whether the difference between the first adjusted transformation matrix and the initial transformation matrix is within the preset error range, or determine whether the difference between the first adjusted transformation matrix and the first transformation matrix obtained by performing the forward iteration process in each round before the current round is within the preset error range, so as to obtain a first determination result; When the first judgment result indicates that the difference between the first adjustment transformation matrix and the initial transformation matrix is within the preset error range, the first adjustment transformation matrix is used as the first target transformation matrix, and the joint angle of the robotic arm after adjustment is used as the first actual joint angle of the robotic arm; If the first judgment result indicates that the difference between the first adjustment transformation matrix and the first transformation matrix obtained by executing the forward iteration process in each round before the current round is within the preset error range, the first adjustment transformation matrix is used as the first target transformation matrix, and the joint angle of the robotic arm after adjustment is used as the second actual joint angle of the robotic arm; When the first judgment result indicates that the difference between the first adjustment transformation matrix and the initial transformation matrix is not within the preset error range, readjusting the joint angles of the robotic arm to obtain a second adjustment transformation matrix from the coordinate system to which the robotic arm base belongs to the marking point; When the first judgment result indicates that the difference between the first adjustment transformation matrix and the first transformation matrix obtained by executing the forward iteration process in each round before this round is not within the preset error range, the joint angle of the robot arm is readjusted to obtain a second adjustment transformation matrix from the coordinate system of the robot arm base to the mark point.
8. The method according to claim 7, characterized in that After the joint angles of the robotic arm are readjusted to obtain a second target transformation matrix from the coordinate system of the robotic arm base to the marking point, the method further includes: Determine whether the difference between the second adjusted transformation matrix and the initial transformation matrix is within the preset error range, or determine whether the difference between the second adjusted transformation matrix and the first transformation matrix obtained by performing the forward iteration process in each round before the current round is within the preset error range, so as to obtain a second determination result; When the second judgment result indicates that the difference between the second adjusted transformation matrix and the initial transformation matrix is within the preset error range, or the difference between the second adjusted transformation matrix and the first transformation matrix obtained by performing the forward iteration process in each round before this round is within the preset error range, the second adjusted transformation matrix is used as the first target transformation matrix.
9. The method according to claim 3, characterized in that: When executing the first round of the negative iteration process, the method comprises: Determine the second planned joint angle in the planned joint angle sequence output in a negative arrangement order corresponding to the first round of the negative iteration process; Based on the last of the planned joint angles and the calibration plate posture, obtain the third transformation matrix from the coordinate system of the robot base to the marked point, and based on the second of the planned joint angles, obtain the second transformation matrix from the coordinate system of the robot base to the marked point; If the difference between the third transformation matrix and the second transformation matrix is not within the preset error range, adjusting the joint angle of the robotic arm at least once to obtain the second target transformation matrix from the coordinate system of the robotic arm base to the marking point; If the difference between the second target transformation matrix and the third transformation matrix is within the preset error range, the fourth actual joint angle of the robotic arm is obtained.
10. The method according to claim 3, characterized in that When executing the last round of the negative iteration process, the method comprises: Determine the initial joint angle corresponding to the last round of the negative iteration process; Based on the last planned joint angle and the calibration plate posture, the third transformation matrix from the coordinate system of the robot base to the marked point is obtained, and based on the initial joint angle and the initial calibration plate posture, the initial transformation matrix from the coordinate system of the robot base to the marked point is obtained; If the difference between the third transformation matrix and the initial transformation matrix is not within the preset error range, or the second transformation matrix obtained by executing the negative iteration process in each round before the current round and the initial transformation matrix are not within the preset error range, the joint angle of the robot arm is adjusted at least once to obtain the second target transformation matrix from the coordinate system to which the robot arm base belongs to the marking point; If the difference between the second target transformation matrix and the second transformation matrix obtained by performing the negative iteration process in each round before the current round is within the preset error range, obtaining the third actual joint angle of the robotic arm; If the difference between the second target transformation matrix and the third transformation matrix is within the preset error range, the fourth actual joint angle of the robotic arm is obtained.
11. A prediction device for compensating joint angles, characterized in that: include: A first acquisition module is used to acquire the first joint angle of the robotic arm when the camera installed on the robotic arm has photographed the marking point on the calibration plate, and to acquire the position and posture of the calibration plate in the camera coordinate system installed on the robotic arm; A second acquisition module is used to acquire a second joint angle of the robot arm to be planned for movement after the first joint angle; A prediction module, for predicting a predicted value of a compensation joint angle used to compensate for unexpected movement of the base of the robotic arm after the robotic arm executes the second joint angle of the movement to be planned, based on the first joint angle of the robotic arm after being photographed by the camera, the calibration plate posture and the second joint angle, using a trained target compensation joint angle prediction model; The device also includes: Before determining the target compensation joint angle prediction model, the initial joint angle, initial calibration plate pose and planned joint angle sequence of the robotic arm are obtained, wherein each planned joint angle in the planned joint angle sequence is used to indicate the joint angle of each movement to be planned of the robotic arm, and the initial calibration plate pose is used to indicate the calibration plate pose in the camera coordinate system obtained by photographing when the robotic arm is at the initial joint angle; Based on the initial joint angle and the initial calibration plate posture, and the planned joint angle sequence, at least one first compensation joint angle for compensating for unexpected movement of the robot arm base is measured; Obtaining the calibration plate pose in the camera coordinate system obtained by outputting the last planned joint angle in the planned joint angle sequence in a forward arrangement order and the position of the robot arm after executing the last planned joint angle; Based on the last of the planned joint angles and the calibration plate pose, the initial joint angles and the initial calibration plate pose, and the planned joint angle sequence, at least one second compensation joint angle for compensating for the unexpected movement of the robot arm base is measured; Inputting the initial joint angle and the initial calibration plate posture, as well as each of the planned joint angles in the planned joint angle sequence into a compensation joint angle prediction model to obtain a first compensation joint angle prediction value used by the robotic arm to compensate for unexpected motion of the robotic arm base after the robotic arm executes the corresponding planned joint angle; Inputting the last planned joint angle and the calibration plate pose, the initial joint angle and the initial calibration plate pose, and each planned joint angle in the planned joint angle sequence into the compensation joint angle prediction model to obtain a second compensation joint angle prediction value used by the robot arm to compensate for the unexpected movement of the robot arm base after the robot arm executes the corresponding planned joint angle; The compensation joint angle prediction model is trained according to the difference between the first compensation joint angle prediction value and the first compensation joint angle, and the difference between the second compensation joint angle prediction value and the second compensation joint angle to obtain the trained target compensation joint angle prediction model.
12. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 10.
13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 10 when executed by a processor.
14. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 10 when being executed by a processor.
