Robot arm control method, apparatus, device, and computer program

Abstract

A method for controlling a robotic arm relating to the field of robotics and automatic control technology. The method includes the steps of: acquiring N time point pairs of historical data, each pair of historical data includes historical input data and historical output data, the historical input data being used to control the motion of the robotic arm (220) so that a first object (210) maintains equilibrium on the robotic arm (220), and the historical output data being used to represent the posture of the robotic arm (220) and the first object (210) after controlling the motion of the robotic arm (220) based on the historical input data, where N is a positive integer; determining predictive input data for the robotic arm (220) based on the N time point pairs of historical data; and controlling the motion of the robotic arm (220) based on the predictive input data so that the first object (210) balances during the motion of the robotic arm (220). The control response is faster because it predicts the future system state of the robotic arm (220) based on historical data, rather than relying on feedback variables of the robotic arm system. Furthermore, the present invention provides a control device, equipment, and storage medium for a robotic arm.

Description

[Technical Field]

[0001] This application claims priority to a Chinese patent application filed with the China National Patent Office on October 7, 2023, with application number 2023112991432, and the title of the invention is "Method, apparatus, device and storage medium for controlling a robot arm," the entire content of which is incorporated into this application by reference.

[0002] The embodiments of this application relate to the field of robotics and automatic control technology, and more particularly to the control of a robot arm. [Background technology]

[0003] Robots are automated control devices that simulate human movements, and robots in the form of robotic arms can mimic the movements of a human arm.

[0004] In related technologies, the task of balancing the bottle on the robot arm is completed by processing tactile information from the bottle on the robot arm to obtain the bottle's position on the robot arm, obtaining the bottle's velocity by calculating the difference, using the position and velocity of the bottle on the robot arm as inputs to a PID (Proportional Integral Differential) controller, and using the rotation angles of the robot arm's joints as outputs to the PID controller.

[0005] However, with the above method, the PID controller relies solely on the robot arm's feedback variables, resulting in a slow response speed. [Overview of the Initiative]

[0006] Embodiments of the present application provide a method, apparatus, robot, storage medium, and program product for controlling a robot arm. The technical means are as follows:

[0007] According to one embodiment of the present invention, a method for controlling a robot arm is provided. This method is A step of obtaining N time point pairs of historical data, wherein each historical data pair includes historical input data and historical output data, the historical input data being used to control the motion of the robot arm so that a first object maintains equilibrium on the robot arm, and the historical output data being used to represent the pose of the robot arm and the first object after controlling the motion of the robot arm based on the historical input data, where N is a positive integer. The steps include determining the predictive input data for the robot arm based on historical data pairs at N time points, The method includes the step of controlling the motion of a robotic arm based on predictive input data so that a first object balances during the motion of the robotic arm.

[0008] According to one embodiment of the present invention, a control device for a robot arm is provided. This device is An acquisition module for acquiring N time point pairs of historical data, wherein each historical data pair includes historical input data and historical output data, the historical input data being used to control the motion of a robotic arm so that a first object maintains equilibrium on the robotic arm, and the historical output data being used to represent the pose of the robotic arm and the first object after controlling the motion of the robotic arm based on the historical input data, where N is a positive integer. A decision module for determining predictive input data for a robot arm based on historical data pairs at N time points, The system includes a control module for controlling the movement of a robotic arm so that a first object maintains balance during the movement of the robotic arm, based on predictive input data.

[0009] According to one embodiment of the present invention, a robotic arm comprising a processor and memory is provided. A computer program is stored in the memory, and the computer program is loaded and executed by the processor to realize a method for controlling the robotic arm.

[0010] According to one embodiment of the present invention, a computer-readable storage medium in which a computer program is stored is provided. The computer program is loaded and executed by a processor to realize the control method of the robot arm described above.

[0011] According to one embodiment of the present invention, a computer program product is provided which includes a computer program stored in a computer-readable storage medium. The robot's processor reads the computer program from the computer-readable storage medium and executes it, thereby causing the robot to perform the robot arm control method described above.

[0012] The technical means relating to the embodiments of the present application can have the following advantageous effects. The task of balancing a first object on the robot arm is completed by acquiring historical data pairs at multiple time points, predicting input data for the robot arm, acquiring predicted input data, and controlling the robot arm's motion based on the predicted input data. Because the system predicts the future system state of the robot arm based on historical data, rather than relying solely on feedback variables of the robot arm system, the control response becomes faster. [Brief explanation of the drawing]

[0013] [Figure 1] This is a flowchart of a control method for a robot arm according to one embodiment of the present invention. [Figure 2] This is a schematic diagram of a task involving balancing a first object on a robot arm according to one embodiment of the present invention. [Figure 3] This is a schematic diagram of a task involving balancing a first object on a robot arm according to another embodiment of the present invention. [Figure 4] This is a schematic diagram of a control method for a robot arm according to one embodiment of the present invention. [Figure 5] This is a simplified schematic diagram of a dynamics model according to one embodiment of the present invention. [Figure 6]It is a block diagram of a method for controlling a robot arm using a kinetic model according to an embodiment of the present application. [Figure 7] It is a schematic diagram of a coordinate system of a robot arm according to an embodiment of the present application. [Figure 8] It is an architecture diagram of a method for controlling a robot arm according to an embodiment of the present application. [Figure 9] It is a schematic diagram of a 7-axis robot arm according to an embodiment of the present application. [Figure 10] It is a schematic diagram of a 3-axis shoulder joint according to an embodiment of the present application. [Figure 11] It is an overall schematic diagram of a shoulder joint according to an embodiment of the present application. [Figure 12] It is a front view and a side view of a differential mechanism according to an embodiment of the present application. [Figure 13] It is a schematic diagram showing the realization principle of a differential mechanism according to an embodiment of the present application. [Figure 14] It is a cross-sectional view of a differential mechanism according to an embodiment of the present application. [Figure 15] It is a schematic diagram of an upper arm drive module according to an embodiment of the present application. [Figure 16] It is a schematic diagram of a wrist joint / elbow joint motor drive module according to an embodiment of the present application. [Figure 17] It is a cross-sectional view showing the connection of an intermediate shaft according to an embodiment of the present application. [Figure 18] It is a block diagram of a control device for a robot arm according to an embodiment of the present application. [Figure 19] It is a block diagram of a control device for a robot arm according to another embodiment of the present application. [Figure 20] It is a block diagram of a robot according to an embodiment of the present application.

Modes for Carrying Out the Invention

[0014] Herein, exemplary embodiments are described in detail, and examples are shown in the drawings. In the following description, when the drawings are referred to, unless otherwise specified, the same numbers in different drawings indicate the same or similar elements. The embodiments described below in the exemplary embodiments are not representative of all embodiments consistent with the present application. On the contrary, they are merely examples of methods consistent with some aspects of the present application, which are described in detail in the appended claims.

[0015] Robot arms are widely used in industrial production and academic research, but in general use, the focus is often on performing specific manipulative tasks using the robot arm's end effector or robot hand. Few people use the rigid connecting members or housing of the robot arm to perform corresponding tasks, and even fewer use the housing to perform corresponding manipulative tasks. The main reasons for this are as follows: (1) The exterior of a robotic arm is typically designed with curved surfaces and does not have large flat surfaces. (2) There is no design for a gripping mechanism such as a hand, and the contact between the robot arm's appearance and external objects does not form a closed position and closing force, making control extremely difficult. (3) The housing of the robot arm basically lacks tactile feedback and therefore is difficult to control.

[0016] Unlike conventional technologies, the embodiment of the present invention provides an operation and method for achieving the operation of maintaining the balance of an object placed on a robot arm using the forearm housing of the robot arm, and preventing the object from falling off the forearm of the robot arm during this process. The specific method is as follows. (1) Design of the control architecture (2) Use of tactile sensors (3) Controller design

[0017] In the method according to the embodiments of the present application, the entity that performs each step may be a computer device, which may be a device for controlling a robotic arm or the robotic arm itself. When the computer device is the robotic arm itself, the robotic arm means an electronic device equipped with data calculation, processing, and storage functions. The robotic arm according to the embodiments of the present application can be used in industrial applications (such as industrial robots), service applications (such as serving robots), entertainment applications (such as performance robots), healthcare applications (such as medical robots), etc., but the embodiments of the present application are not particularly limited thereto.

[0018] The technical means relating to the embodiments of this application involve technologies such as automatic control of artificial intelligence and can realize the control of a robotic arm. Specifically, this will be explained by the following embodiments.

[0019] In related technologies, a PID controller is used to process tactile information from a bottle on a robotic arm to obtain the bottle's position and then obtain its velocity from the difference. By using the position and velocity as inputs to the PID controller and the rotation angles of the robotic arm's joints as outputs to the controller, the task of balancing the bottle is completed.

[0020] However, the method using a PID controller has the following limitations: (1) PID controllers are susceptible to disturbances and noise, which can degrade their control performance. (2) PID controllers can only handle a single variable; for multivariable systems, other control methods must be used. (3) Simple PID controllers lack feedforward, have slow response times, and the PID only performs its role as a control when an error occurs. (4) Velocity and acceleration in the input signals of the PID controller are calculated by difference, which results in large errors and generates noise, so the PID control amount obtained based on the input signal obtained by difference becomes oscillatory. (5) PID is a linear controller, and if the nonlinear elements of the controlled system are large, the PID controller cannot control the controlled system.

[0021] The embodiment of the present invention overcomes the above-mentioned problems by using DDMPC (Data Driven Model Predictive Control) and can perform the task of balancing an object (including, but not limited to, a bottle) on a robotic arm. The embodiment of the present invention can achieve the following effects. (1) The data-driven MPC controller takes noise interference into consideration, offers excellent adaptability and robustness, and the output will not oscillate due to input noise. (2) A data-driven MPC controller can control multiple control variables simultaneously. (3) Data-driven MPC not only relies on system feedback variables, but can also predict future system states based on historical data, resulting in faster control response for data-driven MPC. (4) Data-driven MPC can take into account realistic constraints such as upper and lower limits for the joint position of the robot arm, upper and lower limits for the joint velocity of the robot arm, and upper and lower limits for the joint acceleration of the robot arm, so that the control variables calculated by the controller satisfy realistic constraints.

[0022] Figure 1 shows a flowchart of a robot arm control method according to one embodiment of the present invention. In this embodiment, the method will be explained using the robot arm described above as an example. This method may include the following steps (110-130). For the sake of clarity, a bottle will be used as an example of the first object placed on the robot arm.

[0023] Step 110: Obtain N time point pairs of historical data, each pair containing historical input data and historical output data, where the historical input data is used to control the motion of the robot arm so that the first object maintains equilibrium on the robot arm, and the historical output data is used to represent the pose of the robot arm and the first object after controlling the motion of the robot arm based on the historical input data, where N is a positive integer.

[0024] Here, both the historical input data and the historical output data are based on the current time t. The historical input data is used to control the movement of the robot arm so that the first object on the robot arm maintains a state of equilibrium, and the historical output data is used to represent the posture of the robot arm and the first object after the robot arm has moved. Therefore, the historical input data can be recorded as past control data, and the historical output data can be recorded as past posture data. Historical input data and historical output data at the same historical time can be recorded as a historical data pair.

[0025] In some embodiments, for the current time t, historical data pairs from N time points prior to the current time t are obtained.

[0026] In some embodiments, for any given time t, historical data pairs from N time points prior to time t are obtained.

[0027] In some embodiments, the N times may be consecutive or non-consecutive, but the present invention is not limited thereto.

[0028] In some embodiments, the time interval between any two adjacent times among the N times is the same. For example, when N=3, the time interval between time 1 and time 2 is the same as the time interval between time 2 and time 3.

[0029] In some embodiments, the motion of the robot arm is controlled based on input data to prevent the first object from falling while maintaining balance on the robot arm during its movement.

[0030] In some embodiments, the first object can change its relative position to the robot arm as the robot arm moves.

[0031] This application does not limit the type or shape of the first object. For example, the first object may be a cylindrical object such as a bottle, or a spherical object such as a ping-pong ball.

[0032] In some embodiments, the input data corresponding to a certain time may include one or more data points, but this application is not limited thereto. For example, the robot arm may include shoulder joints, elbow joints, and wrist joints, and the input data may include the rotation angles of one or more joints.

[0033] In some embodiments, the output data corresponding to a certain time may include one or more data points, but this application is not limited thereto. For example, the output data may include one or more data points from the angle between the robot arm and the horizontal plane, the position of a first object on the robot arm, and the rotation angles of one or more joints of the robot arm.

[0034] Step 120: Based on N pairs of historical data at different time points, determine the predicted input data for the robot arm at the current time t.

[0035] In some embodiments, the input to the robot arm at the current time is predicted based on historical data pairs at N time points, and predicted input data for the robot arm is obtained.

[0036] In some embodiments, a DDMPC controller is used to determine the predicted input data for the robot arm based on historical data pairs at N time points.

[0037] In some embodiments, the predictive input data may be one or more parameters for controlling the motion of the robot arm, and it is possible to determine how to control the motion of the robot arm based on the predictive input data. For example, the predictive input data may be torque or the rotation angle of the robot arm's joints.

[0038] The roles of predictive input data and historical input data are similar; both are used to control the movement of the robot arm so that the first object maintains equilibrium on the robot arm. However, they differ in the timing of the control: predictive input data is used to control the movement of the robot arm at the current time t, while historical input data is used to control the movement of the robot arm at N time points prior to the current time t.

[0039] Step 130: Based on the predicted input data, control the movement of the robot arm so that the first object balances during the robot arm's movement.

[0040] In some embodiments, at the current time t, a robotic arm in a first form is controlled to move to a second form based on predicted input data. During this process, the first object is balanced on the robotic arm.

[0041] In some embodiments, the first object moves relative to the robot arm as the robot arm moves.

[0042] In the embodiments of this application, the motion of the robot arm refers to the motion of the robot arm relative to the ground. When the robot arm moves, the first object also moves relative to the ground, but because the mechanical state of the first object changes due to the change in the angle of the robot arm with respect to the horizontal plane, the first object may also move relative to the robot arm.

[0043] For example, as shown in Figure 2, the first object 210 is located on the forearm of the robot arm 220, and at this time the robot arm is in a first configuration. The robot arm 220 moves based on the predicted input data and reaches a third configuration as shown in Figure 3, at which point the relative positions of the first object 210 and the robot arm 220 have changed.

[0044] In some embodiments, the range of motion of the first object on the robot arm may include the entire robot arm or only a portion of the robot arm. For example, the first object may move in the upper arm, forearm, and end of the robot arm, or it may move only in the forearm of the robot arm.

[0045] The technical means according to the embodiment of the present invention completes the task of balancing a first object on a robot arm by acquiring historical data pairs at multiple time points, predicting input data for the robot arm, acquiring predicted input data, and controlling the motion of the robot arm based on the predicted input data. Since it predicts the future system state of the robot arm based on historical data, rather than relying solely on feedback variables of the robot arm system, the control response becomes faster.

[0046] In some embodiments, a robotic arm can be considered an LTI system (linear time-invariant system). The fundamental characteristics of an LTI system are linearity (homogeneity and additivity), time invariance, differentiability, and integrability.

[0047] In some embodiments, the system parameters of a linear time-invariant system do not change over time. Therefore, if the unit impulse response is known, the input data of the LTI system can be determined, and the corresponding output data can be obtained based on the system state of the LTI system. Conversely, if the input data, output data, and system state of the LTI system are known, the unit impulse response can be determined. In the embodiments of this application, the control of a robot arm joint (or rotary joint) is used as an example, so the robot arm can be considered as the LTI system, and the process of determining the predicted input data of the robot arm in the embodiments of this application can be considered as the process of determining the unit impulse response. In reality, errors and noise often affect the robot arm, and the unit impulse response obtained from different historical data pairs may differ. Therefore, when determining the unit impulse response, it is necessary to consider the constraints of real-world conditions, and to adjust the value of the unit impulse response based on the robot arm's feedback (output data corresponding to the second form of the robot arm after motion based on the predicted input data) and the robot arm's ideal equilibrium state. However, the unit impulse response is actually a unit impulse response sequence, and for the sake of simplicity, it is referred to as a unit impulse response in the embodiments of this application, but it can also be referred to as a unit impulse response sequence.

[0048] Furthermore, if it is necessary to control multiple joints (or rotary joints) of a robotic arm, a detailed analysis of the actual robotic arm is required, and it cannot simply be considered an LTI system.

[0049] In the embodiments of the present invention, the robot arm system can be considered as an LTI system shown in the following formula (1). TIFF2026512271000002.tif11170 formula, TIFF2026512271000003.tif5170 unit impulse response, TIFF2026512271000004.tif5170 Prediction input data, TIFF2026512271000005.tif6170 Prediction Output Data, TIFF2026512271000006.tif6170hankel (Hankel) queue, TIFF2026512271000007.tif5170 Predicted step length of robot arm, TIFF2026512271000008.tif6170 is the history trajectory of L+n-th order sustained excitations in a robotic arm system. The history trajectory of sustained excitations indicates that the system state and characteristics represented by this trajectory are stable, meaning that future trajectory points of length L can be predicted using the previous n trajectory points (history data pairs at a given time) (which, when mapped to a Hankel matrix, correspond to the previous n rows and the following L rows of the matrix).

[0050] In the embodiments of this application, input data or output data over a certain period of time is referred to as a trajectory. TIFF2026512271000009.tif17170In the formula, TIFF2026512271000010.tif5170 history data (this may be history input data or history output data), This is historical data for the N-L time point in TIFF2026512271000011.tif6170. This is the number of historical data entries for TIFF2026512271000012.tif5170.

[0051] In some embodiments, when the number of types of historical data is not 1, the dimension of the Hankel matrix is TIFF2026512271000013.tif5170 formula, TIFF2026512271000014.tif5170 Number of types of historical data, TIFF2026512271000015.tif5170 This is the system order of the robot arm.

[0052] Equation (1) above can be considered as a representation of the convolution sum in the time domain of a discrete LTI system. The meaning of this equation is that if there is an output when any α is multiplied by the Hankel matrix, then a certain LTI system I understand that it must be TIFF2026512271000016.tif6170.

[0053] The above formula implicitly represents the system model through the historical data of the LTI system, replacing M (model) in MPC with the historical data.

[0054] In some embodiments, step 120 can be implemented as at least one of the following steps 121 to 123 (not shown).

[0055] Step 121: Based on N pairs of historical data at different time points, construct a constraint function and a cost function for the robot arm. The constraint function is used to describe the constraints that the data must satisfy, and the cost function is used to show the difference between the predicted data and the equilibrium data. The equilibrium data includes the input and output data for the robot arm and the first object in an ideal equilibrium state. The predicted data includes predicted input data and predicted output data, and the predicted output data is predicted based on the historical output data.

[0056] In some embodiments, the constraint function is used to describe the real-world conditions that the robot arm must satisfy. In some embodiments, the cost function, also called the loss function, is used to describe the difference between the predicted data and the equilibrium data.

[0057] In some embodiments, the ideal equilibrium state means a state in which the robot arm does not move when the input data for the robot arm is 0, that is, a state in which the output data of the robot arm does not change compared to the output data at the previous time. Taking the example of a first object moving at the forearm of the robot arm, the ideal equilibrium state may be a state in which the forearm of the robot arm is parallel to the horizontal plane, the first object is located at the midpoint of the forearm of the robot arm, and the plane formed by the position of the center of mass of the first object and the midline of the robot arm is perpendicular to the horizontal plane.

[0058] In some embodiments, step 121 can be implemented as at least one of the following steps a to c (not shown).

[0059] Step a: Determine a first matrix and a second matrix, the first matrix containing the historical input data of historical data pairs at N time points, and the second matrix containing the historical output data of historical data pairs at N time points.

[0060] In some embodiments, the historical input data of N time points in the historical data pairs is determined as the elements of a first matrix, and the historical output data of N time points in the historical data pairs is determined as the elements of a second matrix.

[0061] In some embodiments, the historical input data from the (i-1)th to the NL-n+i-1th timestamp out of N timestamps is determined as the i-th row of the first matrix, and the historical output data from the (i-1)th to the NL-n+i-1th timestamp out of N timestamps is determined as the i-th row of the second matrix, where i is a positive integer.

[0062] In some embodiments, the first matrix is ​​the one in equation (1) above. TIFF2026512271000017.tif6170 The second matrix is ​​given by equation (1) above. TIFF2026512271000018.tif6170

[0063] Step b: Construct a constraint function for the robot arm based on the previous n rows (first n rows) in the first matrix and the previous n rows in the second matrix, where n is the system order of the robot arm and is a positive integer.

[0064] In some embodiments, the constraint function may include one or more, as it is used to describe constraints on the robot arm under real-world conditions.

[0065] In some embodiments, the constraint function includes a first constraint function, a second constraint function, a third constraint function, and a fourth constraint function.

[0066] In some embodiments, a first constraint function is constructed based on a slack variable, where the slack variable is used to describe the measurement noise, and the first constraint function is used to reduce the interference of the measurement noise with the predicted input data.

[0067] In some embodiments, a slack variable is introduced because measurement noise is present in actual scenarios. The purpose of introducing the slack variable is to ensure that the Hankel matrix equation (equation (1) above) remains valid even when y containing measurement noise is substituted (ideally, y in the Hankel matrix equation does not contain noise).

[0068] In some embodiments, the first constraint function is given by equation (3) below. TIFF2026512271000019.tif11170 formula, TIFF2026512271000020.tif5170 Slack variable, TIFF2026512271000021.tif5170 Prediction input data, TIFF2026512271000022.tif6170 Prediction output data, TIFF2026512271000023.tif5170 unit impulse response, TIFF2026512271000024.tif6170Hankel matrix consisting of historical input data at 170N time points, TIFF2026512271000025.tif6 is a Hankel matrix composed of historical output data at 170N time points.

[0069] In some embodiments, a second constraint function is constructed based on the previous n rows in the first matrix and the previous n rows in the second matrix, and this second constraint function is used to describe the historical data pairs for the previous n time points.

[0070] In some embodiments, the second constraint function is used to represent the following: that is, the range from -n to -1 at a given time t of the LTI system. TIFF2026512271000026.tif5170 and the history output data TIFF2026512271000027.tif6170 represents the previous n historical data pairs at the current time in the LTI system, and can be understood as different descriptions of the same LTI trajectory of length n.

[0071] In some embodiments, the second constraint function is given by equation (4) below. TIFF2026512271000028.tif11170 formula, Predicted input data from -n to -1 at time t in TIFF2026512271000029.tif5170, Historical output data from -n to -1 at time t in TIFF2026512271000030.tif5170. TIFF2026512271000031.tif6170 Previous n historical input data at the current time t, TIFF2026512271000032.tif6170 is the previous n historical output data at the current time t.

[0072] In some embodiments, a third constraint function is constructed based on the previous n rows of the first matrix, the previous n rows of the second matrix, and equilibrium data, and the third constraint function shows that the robot arm approaches the ideal equilibrium state over time.

[0073] In some embodiments, the third constraint function is a constraint corresponding to a quadratic programming problem in a data-driven MPC controller. The equality constraint here is called a terminal equality constraint. Terminal equality constraints are typically used to constrain the final value of an optimization variable in an optimization process to equal a specific target value. For example, in a control problem, they can bring the system to a specific target state. In embodiments of this application, this is done to bring the robot arm closer to an ideal equilibrium state.

[0074] In some embodiments, the third constraint function is given by equation (5) below. TIFF2026512271000033.tif11170 formula, TIFF2026512271000034.tif5170 is the predicted input data for the LTI system of length n at time t+Ln to t+L-1. TIFF2026512271000035.tif5170 is the predicted output data of the LTI system of length n at time t+Ln to t+L-1. TIFF2026512271000036.tif6170 is the equilibrium point of the LTI system at n time points, where, TIFF2026512271000037.tif5170 equilibrium point, TIFF2026512271000038.tif5170 represents writing the equilibrium points as column vectors of length n.

[0075] Since the trajectory from time t+Ln to t+L-1 in the future is unknown, predictions can only be made based on the characteristics of the LTI system and the history data of sustained excitations. Therefore, the control process updates this equation constraint by updating the trajectory of the previous n points at the current time in real time. The "end" in the terminal equation constraint can be understood as the final state (equilibrium point) that the system will reach. This terminal equation constraint can be written in the corresponding form (the form of multiplying the Hankel matrix by α) based on the Hankel matrix equation of the LTI system. Thus, the terminal equation constraint is transformed into an equation with respect to α.

[0076] In some embodiments, the above equilibrium point refers to the ideal equilibrium state.

[0077] In some embodiments, the ideal equilibrium state is the same at each time step.

[0078] In some embodiments, a fourth constraint function is constructed based on a slack variable, a first boundary point, and a unit impulse response, where the first boundary point is the upper limit of the slack variable, and the fourth constraint function is used to constrain the convergence of the process of predicting the predictive input data.

[0079] In some embodiments, the fourth constraint function is given by equation (6) below. TIFF2026512271000039.tif5170 formula, TIFF2026512271000040.tif6170 Slack variable, TIFF2026512271000041.tif5170 Upper limit of slack variable (first boundary point), TIFF2026512271000042.tif5170 is a unit impulse response.

[0080] In some examples, TIFF2026512271000043.tif6170 If it is sufficiently large, the regularization term of σ is TIFF2026512271000044.tif6170 Therefore, the penalty is imposed and becomes very small, so the fourth constraint function is perfectly satisfied. Thus, this inequality constraint is not considered in the quadratic programming problem. TIFF2026512271000045.tif6170 represents the penalty weight for slack variables.

[0081] Step c: Construct a cost function for the robot arm based on the last L rows in the first matrix and the last L rows in the second matrix, where L is a positive integer and is the predicted step length of the robot arm.

[0082] In some embodiments, a first norm is constructed based on the last L rows of a first matrix and the last L rows of a second matrix, the first norm including the square norm of the historical input data for the last L time points out of N time points and the square norm of the historical output data for the last L time points out of N time points. A first penalty term is constructed based on a first penalty weight, a first boundary point, and a first regularization term, the first regularization term being a regularization penalty on the unit impulse response, the first penalty weight being the penalty weight for the first regularization term, the first boundary point being the upper limit of the slack variable, the slack variable being used to describe the measurement noise. A second penalty term is constructed based on a second penalty weight and a second regularization term, the second regularization term being a regularization penalty on the slack variable, and the second penalty weight being the penalty weight for the second regularization term. We construct a cost function based on the first norm, the first penalty term, and the second penalty term.

[0083] In some embodiments, historical data pairs from n time points prior to the robot arm are used to predict input and output data from L time points after the robot arm.

[0084] In some embodiments, the cost function is given by equation (7) below. TIFF2026512271000046.tif28170 formula, TIFF2026512271000047.tif6170 is the square of the quadratic norm between the input data, output data, and their corresponding equilibrium points. TIFF2026512271000048.tif5170 is a penalty weight matrix of the square of the quadratic norm between the input data and its equilibrium point. This is the penalty weight matrix of the quadratic norm squared between the output data of TIFF2026512271000049.tif5170 and its equilibrium point.

[0085] This is a penalty for regularization in TIFF2026512271000050.tif6170α, and its purpose is to prevent overfitting in the process of solving quadratic programming problems and to reduce errors in equations composed of Hankel matrices. TIFF2026512271000051.tif6170 shows the penalty weights for this regularization term. TIFF2026512271000052.tif5170 represents the upper limit of the Slack variable.

[0086] TIFF2026512271000053.tif6170 This is a penalty for regularizing the slack variable σ. TIFF2026512271000054.tif6170 This shows the penalty weights for this regularization term. Slack variables are often used in optimization problems to transform inequality constraints into equality constraints, which expands the feasible domain and makes the optimization problem easier to solve. Finally, the cost function is transformed into an expression in terms of α by replacing the data sequences of u and y in the cost function with the Hankel matrix equality, which implicitly expresses the LTI system model (we use the L row after the Hankel matrix here to ensure that the matrix dimensions on both sides of the equality are correct).

[0087] Step 122: Based on the constraint function and cost function, calculate the unit impulse response of the robot arm, which is used to represent the robot arm's feedback on the predicted input data.

[0088] In some embodiments, both the cost function and the constraint function are transformed into functions of α, and the entire quadratic programming problem can be solved to find α that minimizes the cost function and satisfies the equality constraints.

[0089] Step 123: Determine the predicted input data for the robot arm based on the unit impulse response.

[0090] In some embodiments, the predicted input data for the robot arm is determined based on the unit impulse response and N pairs of historical input data at different time points.

[0091] In some embodiments, a third matrix is ​​determined based on historical input data pairs at N time points, the third matrix comprising a first matrix and a second matrix, the first matrix comprising historical input data for the N time points, and the second matrix comprising historical output data for the N time points. Predicted input data for the robot arm is calculated based on the unit impulse response and the third matrix.

[0092] In some embodiments, the third matrix is ​​the left-hand Hankel matrix of the Hankel matrix equation shown in equation (1) above, and the prediction input data is the right-hand side of the equation. TIFF2026512271000055.tif5170

[0093] The above method simplifies the process by treating the task of balancing a first object on a robot arm as a task of solving a discrete LTI system, obtaining the unit impulse response of the LTI system, and predicting the robot arm's input data based on the unit impulse response. Since this process considers the constraints of real-world conditions, the obtained predicted input data can satisfy these constraints. Furthermore, by describing the influence of measurement noise on the output data using a slack variable and considering noise interference, the method provides better adaptability and robustness.

[0094] Exemplary, Figure 4 shows a workflow diagram of a DDMPC controller according to one embodiment of the present invention. 1. Collect the system's history trajectory of length N. 2. Construct a Hankel matrix from the history trajectory. 3. Decompose the Hankel matrix (the first n rows and the last L columns of the Hankel matrix are used to form the terminal equality constraints, and the last L columns are used to form the Hessien matrix and Gradient matrix in the quadratic programming problem), Reference, and RQ penalty weight matrix into the cost function and terminal equality constraints to form a quadratic programming problem for α. 4. Use a quadratic programming solver (such as qpOASES) to solve for α, which minimizes the cost function and satisfies the equality constraints. 5. Substitute α into the Hankel matrix equality to obtain the corresponding input and output trajectories and extract the predicted input data u. 6. Send the predicted input data u to the robot arm for execution. 7. Use the previous n input and output trajectory points at the current time to update the terminal equality constraints as feedback. 8. Set t=t+1 and repeat step 4 until the system reaches equilibrium.

[0095] In some embodiments, the types of input and output data for the robot arm can be derived using a dynamics model, or they can be derived without using a dynamics model. Embodiments of the present invention will be illustrated by the following embodiments.

[0096] 1. When using a dynamics model In some embodiments, the dynamic equations for the robot arm are constructed based on the motion parameters of the first object and the motion parameters of the robot arm. The motion parameters include at least one parameter that affects kinetic or potential energy, and the dynamic equations include a sub-drive equation and a drive equation, the sub-drive equation being the equation where the input torque is zero, and the drive equation being the equation where the input torque is not zero.

[0097] In some embodiments, the motion parameters include at least one of the following: the length of the robot arm, the specifications of the first object, the mass of the robot arm, the mass of the first object, the moment of inertia of the robot arm, and the moment of inertia of the first object.

[0098] In some embodiments, the types of motion parameters included in the motion parameters of the first object and the types of motion parameters included in the motion parameters of the robot arm may be the same or different.

[0099] In some embodiments, a state-space equation for the robot arm is constructed based on the under-drive equation and a third matrix, and this state-space equation is used to describe the motion state of the robot arm. Based on the state-space equation and the drive equation, predictive input data is calculated.

[0100] In some embodiments, a first state-space equation for the robot arm is constructed based on a third matrix, and the first state-space equation is processed by the sub-drive equation to obtain a state-space equation.

[0101] In some embodiments, the input torque is calculated based on state-space equations and drive equations, and this input torque is used to control the joint rotation of the robot arm. The input torque is determined as the predicted input data.

[0102] In some embodiments, the initial dynamics equations for the robot arm are constructed based on the motion parameters of the bottle and the robot arm. Partial feedback linearization is performed on the initial dynamics equations to obtain the dynamics equations, and the partial feedback linearization is an approximate linearization process performed on the initial dynamics equations.

[0103] In some embodiments, partial feedback linearization involves initializing the parameters of the initial dynamics equations to parameters that satisfy the linearization requirements. For example, as θ approaches 0, sinθ is initialized to θ.

[0104] For example, consider a scenario where a robot arm balances a bottle on its forearm housing. The process of performing an abstracted dynamic model of the robot arm based on this scenario is described as follows:

[0105] First, the robot arm is in the world coordinate system. A spatial orthogonal coordinate system is constructed by defining the direction of extension of the robot arm's forearm as the positive y-axis, the direction perpendicular to the y-axis and parallel to the ground as the x-axis, the direction of the right hand when standing in the same direction as the robot arm as the positive x-axis, and the direction opposite to gravity, vertically upward and perpendicular to the ground as the z-axis. A 2D model can be created on the YOZ plane to control the robot arm to balance the bottle.

[0106] Figure 5 shows a simplified diagram of a two-dimensional model in the YOZ plane. The upper circle 510 represents the cross-section of the bottle, the lower rectangle 520 represents the side of the robot's forearm, and the circle 530 within rectangle 520 represents the axis of rotation that rotates the forearm (such as the elbow joint of the robot arm). The first problem is considered as a homogeneous rigid body, and the position of the bottle is determined by the center of mass of the bottle. The bottle is positioned approximately perpendicular to the forearm.

[0107] The physical quantities (motion parameters) used in the two-dimensional model and their positive directions are defined as follows. The distance along the horizontal direction of the side surface of the robot arm from the mass center of the bottle to the axis of the rotation axis of the robot arm is s. The torque for rotating the rotation axis of the robot arm is τ, and its positive direction is counterclockwise. The angle by which the forearm rotates with respect to the world coordinate system is θ. The length of the side surface of the forearm is l a , the radius of the bottle is r b , the mass of the forearm is m a , the mass of the bottle is m b , the moment of inertia of the forearm is I a , the moment of inertia of the bottle is I b is as follows.

[0108] Exemplarily, the physical quantities used in the two-dimensional model are defined as follows. Length of the forearm: l a = 0.27 m Radius of the bottle: r b = 0.0325 m Mass of the forearm: m a = 2.48 kg Mass of the bottle: m b = 0.24 kg Moment of inertia of the forearm: I a = 0.03 Moment of inertia of the bottle: I b = 2.535e-04

[0109] The square of the linear velocity of the bottle can be expressed as follows. TIFF2026512271000056.tif5170

[0110] The angular velocity of rotation of the bottle can be expressed as follows. TIFF2026512271000057.tif10170

[0111] According to the Euler-Lagrange equation, it is necessary to solve the kinetic energy and potential energy of all the rigid bodies in the system respectively. The sum of the kinetic energies of all the rigid bodies in the system is as follows. The sum of the potential energies of all rigid bodies in the TIFF2026512271000058.tif11170 system is as follows: TIFF2026512271000059.tif10170

[0112] The partial derivative of the kinetic energy with respect to state s (where s is the position of the bottle on the forearm) and θ in generalized coordinates is as follows: TIFF2026512271000060.tif39170 The partial derivative of the potential energy with respect to state s and θ in the generalized coordinate system is as follows: TIFF2026512271000061.tif18170

[0113] Using the Euler-Lagrange equations to calculate the above results, we can see that a simplified dynamic model of the system in the YOZ plane is given by the following equation. Here, the first line represents the dynamic equation for s degrees of freedom (position s of the bottle on the forearm). Since this direction is underdriven, the actual input torque on the right side of the equation is 0. The second line represents the dynamic equation for the degree of freedom θ, which is the rotation angle of the forearm. Since the drive in this degree of freedom is the motor torque τ, the right side of the equation in the second line is τ. TIFF2026512271000062.tif20170

[0114] Generally, to ensure that the state-space equations describing a system take into account the system's dynamics, the sub-driven part of the dynamical model (the equation where the right-hand side of the equality is zero) is introduced into the state-space equations.

[0115] Based on the inputs and outputs of the dynamics model, the input data u of the robot arm is for the rotational joint. Set to TIFF2026512271000063.tif5170, the output data y is the angle θ of the rotational joint and TIFF2026512271000064.tif5170 and the position s of the bottle on the forearm It is set to TIFF2026512271000065.tif5170. Expressed as a state-space equation, the first three lines are simple differential relations between state variables, and the last line of the state-space equation consists of the sub-driving part of the dynamics. The equilibrium point (ideal equilibrium state) ref is set such that the equilibrium point for all (angular) velocities and (angular) accelerations is 0 (and the corresponding units), the equilibrium point for joint rotation angles is 0 rad (the derived simplified dynamics model is 0 rad, and the corresponding actual robot arm is 0.465 rad), and the equilibrium point s of the bottle on the forearm is the midpoint of the forearm (the derived simplified dynamics model is 0.135 m, and the corresponding actual robot arm is 22 (this 22 means the 22nd column of the tactile sensor array placed on the robot arm)). Typically, when a robot arm's motor can be controlled using torque, a partial feedback linearization process is performed on the dynamic system model to obtain a linearized system model and expressions for torque τ and the new control variable v introduced by partial feedback linearization. The former is used in the controller design, and the latter is used to convert the controller's output v (the new control variable introduced by partial feedback linearization) into torque τ and transmit it to the robot arm. The entire control process is shown in Figure 6.

[0116] In the embodiments of this application, TIFF2026512271000066.tif6170 Let the above new control variable be v. The partial feedback linearization process is as follows: 1. When θ is close to 0 and the change is very small, sinθ is approximated and linearized to obtain θ. 2. When θ is close to 0 and the change is very small, TIFF2026512271000067.tif8170 is approximated and linearized to 0.

[0117] After partial feedback linearization, the dynamic sub-drive equations are introduced into the state-space equations. The state-space equations are shown as follows: TIFF2026512271000068.tif31170

[0118] A new control variable v is introduced into the dynamics drive equation by torque τ and partial feedback linearization. The expression will be TIFF2026512271000069.tif6170. TIFF2026512271000070.tif10170

[0119] In the above, the input and output data of the robot arm are determined based on a dynamic model. In this case, the input data of the robot arm is the rotational joint. The file is TIFF2026512271000071.tif5170, and the output data is the angle θ of the rotational joint and TIFF2026512271000072.tif5170 and the position s of the bottle on the forearm The filename is TIFF2026512271000073.tif5170.

[0120] Note that in the above formula TIFF2026512271000074.tif6170 This is the acceleration due to gravity.

[0121] In some embodiments, when using a dynamics model, the rotational angular velocity of the robot arm joints is calculated based on the unit impulse response and a third matrix, and the rotational angular velocity is determined as the predicted input data for the robot arm at the current time t.

[0122] 2. When a dynamics model is not used. When selecting the system's inputs and outputs without using a dynamic model, the input data is set to the rotation angle θ of the rotary joint, and the output data is related to the position of the first object on the forearm and the time of the error. TIFF2026512271000075.tif6170 and the position s of the first object on the forearm The value is set to TIFF2026512271000076.tif5170. The equilibrium point (ideal equilibrium state) ref is set such that the equilibrium point for all (angular) velocities is 0 (and the corresponding units), the equilibrium point for joint rotation angles is 0 rad (the derived simplified dynamics model is 0 rad, and the corresponding actual robot arm is 0.465 rad), and the equilibrium point position s of the bottle on the forearm is the midpoint of the forearm (the derived simplified dynamics model is 0.135 m, and the corresponding actual robot arm is 22 (where 22 means the 22nd row of the tactile sensor array placed on the robot arm)).

[0123] θ represents the angle in the pitch direction when the forearm of the robot arm rotates around the y-axis. TIFF2026512271000077.tif6170 The angular velocity in the pitch direction when the forearm of the robot arm rotates around the y-axis is represented, and τ represents the control torque applied in the pitch direction when the forearm of the robot arm rotates around the y-axis. s represents the offset in the y-direction between the center of mass of the first object and the center of gravity of the forearm of the robot arm. TIFF2026512271000078.tif4170 represents the offset velocity in the y-direction between the center of mass of the bottle and the center of gravity of the forearm of the robot arm.

[0124] In some embodiments, due to hardware limitations, Since it is not possible to accurately collect and transmit TIFF2026512271000079.tif5170 to the robot arm, a method of selecting the system's inputs and outputs without using a dynamics model can be used.

[0125] In some embodiments, for the current time t, historical data pairs are obtained from multiple consecutive historical time points prior to the current time t, and historical data pairs from multiple consecutive historical time points are uniformly sampled to obtain historical data pairs for N time points.

[0126] For example, if we collect historical data pairs of length N=1000 (including historical input and output data), and N is too large, the Hankel matrix becomes excessively large, significantly increasing the computational load and computation time of the computer device. Therefore, by employing interval sampling, sampling one input and one output data point every four points, and compressing the system's historical data of length 1000 to 200 before inputting it to the DDMPC controller, the computational load and computation time of the computer device can be significantly reduced. Furthermore, the length of the prediction horizon is set to 50 to adjust all penalty weights related to DDMPC.

[0127] Finally, when the first object is placed on the forearm of the robot arm and a disturbance is applied to the first object, the DDMPC outputs the rotation angle θ of the rotary joint and completes the task of controlling the robot arm to balance the first object on the robot arm.

[0128] Regarding a method for obtaining historical output data at N time points, this application provides an exemplary embodiment.

[0129] In some embodiments, the orientation of a first object can be recognized using a visual recognition method. However, this method may result in linear errors of 1-2 centimeters and angular errors of 5-10 degrees. Furthermore, the calculation time is relatively long, approximately 100 ms (10 Hz).

[0130] In some embodiments, an engineering vision solution can be employed, specifically, a lightweight data image processing method can be used to determine the position of the center of mass of a first object in the x-direction. While there are numerous specific implementation methods, this application is not limited to them. For example, if the first object is a bottle, cluster analysis can be performed based on the color difference between the bottle and surrounding objects to determine the bottle's geometric center, and model calibration can be performed on the positions of the bottle's geometric center and center of mass. Alternatively, if the first object is a bottle, feature recognition can be performed to determine the bottle's position in the image, thereby determining its geometric center, and model calibration can be performed on the positions of the bottle's geometric center and center of mass. Because this solution is computationally intensive, it is fast, typically taking only about 10ms (100Hz) per operation, and has relatively controllable linear errors, usually around 1cm, with angular errors of about 5 degrees. However, a drawback of this method is that it introduces large errors in the depth direction of the camera, i.e., in the y-direction of the bottle's world coordinate system. Details of the above coordinate system (such as the x and y directions) are omitted here, as they should be explained in the example of the dynamics model described above.

[0131] This error is one of the advantages of tactile sensors. While tactile sensors can only measure the absolute position of the contact point between the first object and the robot arm, and cannot accurately measure its orientation in the x-direction, they are very accurate in measuring the position of the robot arm in the y-direction at the contact point between the first object and the robot arm. Therefore, the state of the first object can be estimated by utilizing the advantages of both sight and touch.

[0132] In several embodiments, a visual-tactile fusion solution can be employed. Exemplarily, a tactile sensor provides the bottle's y-direction position and pitch orientation. Furthermore, lightweight visual data image processing and comparison with previous photographic data determine the position of the bottle's center of mass in the x-direction. This solution has a computation time of approximately 10 ms (100 Hz), a linear error of approximately 1 cm, and an angular error of approximately 5 degrees. While similar to engineering vision solutions, it overcomes the drawback of engineering vision solutions, which cannot accurately measure the center of mass of a first object in the y-direction.

[0133] For example, as shown in Figure 7, the engineering vision solution can obtain a relatively accurate orientation of the first object 710 in the x-direction, and the visual-tactile fusion solution can obtain a relatively accurate orientation of the first object 710 in the y-direction. By combining these two, the precise location of the center of mass of the first object 710 can be obtained. Table 1 shows the cycles and corresponding errors required for the three solutions described above.

[0134] TIFF2026512271000080.tif90170

[0135] Taking the above visual-tactile fusion solution as an example, the step of obtaining historical data pairs at N historical time points may include at least one of the following steps 1-7. Step 1: For the j-th time point out of N time points, obtain the historical input data for the j-th time point, where j is a positive integer less than or equal to N. Step 2: Determine the posture of the robot arm after movement based on the historical input data at the j-th time point. Step 3: Based on the tactile sensor, obtain the first position of the first object on the robot arm. Step 4: Obtain an image of the first object at the j-th time point. Step 5: Based on the image of the first object, obtain the second position of the first object on the robot arm. Step 6: Based on the first and second positions, determine the position of the center of mass of the first object. Step 7: Based on the posture of the robot arm after motion using the historical input data at the j-th time point, and the position of the center of mass of the first object, the historical output data of the robot arm at the j-th time point is determined.

[0136] In some embodiments, the first and second positions are expressed in the form of coordinates (x,y), where the coordinates corresponding to the first position are (x1,y1) and the coordinates corresponding to the second position are (x2,y2).

[0137] As described above for the visual-tactile fusion solution, since the first position is relatively accurate in the y-direction and the second position is relatively accurate in the x-direction, the position of the center of mass of the first object can be determined as (x2, y1).

[0138] This application does not limit the method for obtaining the second position of the first object on the robot arm based on an image of the first object in step 5. For example, a feature recognition method can be used to determine the position of the first object in an image of the first object, then determine the geometric center of the first object, and then determine the second position based on the geometric center of the first object.

[0139] In some embodiments, the types of data included in the hierarchical output data may be determined by whether a dynamics model is used. For example, if a dynamics model is used, the hierarchical output data may include the position of the center of mass of the first object, the velocity of the first object, the angle of the robot arm's rotary joint, the angular velocity of the robot arm's rotary joint, and the angular acceleration of the robot arm's rotary joint. If a dynamics model is not used, the hierarchical output data may include the position of the center of mass of the first object, the velocity of the first object, and the integral of the position of the center of mass of the first object with respect to time.

[0140] The above method can acquire a relatively accurate position of the first object on the robotic arm and strongly support the task of balancing the first object on the robotic arm.

[0141] Figure 8 shows the overall control architecture that realizes the above functions. The ultimate goal is to control the task of balancing a first object on the robot arm by inputting commands to the joint motors of the robot arm. Each joint motor of the robot arm is equipped with a joint encoder that provides feedback information on the rotation angle, angular velocity, and current of each joint motor. This information can be used to estimate the state of the robot arm. In addition, tactile sensors are attached to the fingers, palm, and some links of the robot arm. Based on the tactile information and the position and orientation of the bottle obtained by state estimation, a two-dimensional or three-dimensional system dynamics model of the first object and the forearm can be obtained. This system dynamics model may be two-dimensional or three-dimensional. Since the inputs and outputs of the robot arm can be determined based on the system dynamics model, the necessary data for the Data Driven MPC controller can be collected based on the inputs and outputs of the robot arm determined by the dynamics model. Based on the collected historical data pairs, the Data Driven MPC controller is designed. When a first object is placed on the forearm of a robot arm and a disturbance is applied, the Data Driven MPC completes the task of balancing the first object on the robot arm by outputting corresponding predictive input data. Depending on the type of robot arm, these predictive input data may be variables in joint space or variables in Cartesian coordinate space. If the predictive input data are variables in Cartesian coordinate space, the joint angles of each joint must be calculated using inverse kinematics solutions. In this case, as time changes, the controller outputs a sequence of the terminal posture of the robot arm or the posture of the center of mass of a link. As a result corresponding to the series of inverse kinematics solutions, a sequence of joint angles and joint angular velocities of each joint of the robot arm is obtained. By transmitting this sequence of joint angles and joint angular velocities to the robot arm, it is possible to control the terminal posture of the robot arm or the position and posture of the center of mass of a link.If the predicted input data consists of joint space variables such as joint angle, angular velocity, and angular acceleration, the task of balancing a first object on the robot arm can be completed by sending the predicted input data directly to the robot.

[0142] In the embodiment of this invention, a real-world experiment was conducted using a bottle as the first object. In the real-world experiment, the bottle rolled back and forth on the forearm of the robot arm, and its motion state could change in response to changes in the absolute posture between the forearm of the robot arm and the ground. The rolling of the bottle on the forearm could also be achieved by controlling the joint motors of the robot arm. During the experiment, the bottle maintained its balance while moving on the forearm of the robot arm and did not fall to the ground. The real-world experiment demonstrated the motion sequence described in the embodiment of this invention and demonstrated the effectiveness and stability of the control architecture and controller described in the embodiment of this invention.

[0143] The embodiments of this application further provide information regarding the hardware of a robot arm, and all of the control methods for the robot arm according to the above embodiments can be implemented with this robot arm.

[0144] Figure 9 shows a humanoid 7-degree-of-freedom robotic arm. The control motors for the elbow and wrist are located in the hollow part of the third joint of the shoulder. Wire-driven elbows and wrists transmit power from the motors in the third joint of the shoulder to pulleys via belts, and the pulleys control the corresponding movements of the elbows and wrists via transmission wires. The structure of a 3-degree-of-freedom shoulder joint is shown in Figure 10. The number of joints in a robotic mechanism capable of independent movement is called the degree of freedom of motion of the robotic mechanism, and is abbreviated as Degree of Freedom (DOF). In control methods currently used in industrial robots, each joint of the robotic arm is a separate servo mechanism; that is, each axis corresponds to a servo, each servo is controlled via a bus, and a controller controls and adjusts the movement of the servos.

[0145] The low-inertia differential shoulder joint structure used in the 7-degree-of-freedom robot arm employs a differential wire drive mechanism in the shoulder and positions the motor module rearward, resulting in not only a lighter mechanism but also the ability to superimpose torque under certain conditions. The third degree of freedom of the shoulder joint uses two pulleys of different sizes and transmits power via wire drive, improving transmission accuracy and reducing weight. Furthermore, by placing the drive modules for the wrist and elbow joints in the upper arm module of the shoulder joint, the overall weight of the robot arm is reduced. These structures are easily modularized, allowing for simplified manufacturing processes.

[0146] Figure 11 shows an overall view of the shoulder joint. The shoulder joint is mainly divided into three modules: the differential mechanism module 1, the cross-roller bearing rotation module 2, and the humeral end drive module 3. Of these, the cross-roller bearing rotation module 2 is an intermediate module that connects the differential mechanism module 1 and the humeral end drive module 3. These three modules will be explained in detail below.

[0147] Figure 12 shows the front and side views of the differential mechanism. Figure 12 shows each of the main components, and below, we will explain each of the components with complex shapes and how they are connected. As can be seen from Figure 12, the differential mechanism consists of a rotary encoder 1.1, a differential mechanism mounting seat 1.2, a large pulley 1.3, a motor protective cover 1.4, a differential mechanism inner ring pulley shaft 1.5, a motor 1.6, a differential mechanism outer ring pulley shaft 1.7, a motor seat 1.8, a cross roller bearing outer ring end cap 1.9, a cross roller inner ring mounting seat 1.10, a differential mechanism small pulley 1.11, a connecting block 1.12, a connecting shaft 1.13, a rotary encoder 1.14, a differential mechanism large pulley 1.15, a bearing end cap 1.16, a small pulley 1.17, a wire 1.18, a bearing end cap 1.19, and an upper arm connecting seat 1.20. Below, we will introduce the specific configuration and operating principle of the differential mechanism. The operating principle of the differential mechanism is similar to that of a differential mechanism with three bevel gears. Figure 13 illustrates the operating principle of the differential mechanism. Figure 13a shows the body of the differential mechanism, which includes a large pulley 1.3, a differential mechanism inner ring pulley shaft 1.5, a differential mechanism outer ring pulley shaft 1.7, a differential mechanism small pulley 1.11, a differential mechanism large pulley 1.15, and wires wound around them. The differential mechanism small pulley 1.11 and the differential mechanism large pulley 1.15 are fixed to the upper arm connecting seat 1.20 with screws. Figure 13b shows the connection between the differential mechanism inner ring pulley shaft 1.5 and the differential mechanism small pulley 1.11, realizing the connection of the differential mechanism gear pair. It is driven by the large pulley 1.3 and the corresponding small pulley 1.17, which are connected to the differential mechanism inner ring pulley shaft 1.5 with screws. Figure 3c shows another gear pair consisting of the differential mechanism outer ring pulley shaft 1.7 and the differential mechanism large pulley 1.15. The differential mechanism's inner ring pulley shaft 1.5 and the differential mechanism's outer ring pulley shaft 1.7 are connected by shaft fitting, the specific structure of which will be explained later. The transmission shaft 1.21 in Figure 13a is connected to the connecting block 1.12 and plays the role of transmitting the rotation of the entire differential mechanism to the rotary encoder 1.1.

[0148] Figure 14 is a cross-sectional view of the differential mechanism. Since the differential mechanism small pulley 1.11 and the differential mechanism large pulley 1.15 are connected to the upper arm connecting seat 1.20 by screws, the method of connection between them is not described in Figure 14. In Figure 14, the outer ring of the rotary encoder 1.1 is fixed to the connecting pressure plate 1.22 with a nut, and the inner ring of the rotary encoder 1.1 is sandwiched between the nut and the shoulder of the transmission shaft 1.21. As a result, the inner ring and the transmission shaft 1.21 rotate together, and the clamping plate 1.23 presses against the outer ring of the deep groove ball bearing 1.25 (all bearings in Figure 14 are deep groove ball bearings, so they are represented by only one symbol) together with the connecting pressure plate 1.22. The connection between the differential mechanism inner ring pulley shaft 1.5 and the differential mechanism outer ring pulley shaft 1.7 is the same as the connection described above. Two sets of deep groove ball bearings 1.25 are provided at the upper and lower ends of the differential mechanism outer ring pulley shaft 1.7. The inner ring of the bearing is fixed to a snap ring 1.24 via the shoulder of the differential mechanism inner ring pulley shaft 1.5, and the outer ring is fixed to a bearing pressure plate 1.29 via the shoulder of the differential mechanism outer ring pulley shaft 1.7. The connection between the differential mechanism inner ring pulley shaft 1.5 and the cross roller inner ring fixing seat 1.10 and the connecting block 1.12 is almost the same as the connection described above, and is achieved by crimping the inner and outer rings of the bearings, so a detailed explanation is omitted. The connecting shaft 1.13 and the connecting block 1.12 are fixed and their positions are restricted by a key 1.28 and a screw. In Figure 14, the cross roller bearing 1.27 is clamped and fixed by the cross roller inner ring fixing seat 1.10 and the cross roller bearing inner ring cover 1.26, while the outer ring is clamped and fixed by the differential mechanism fixing seat 1.2 and the cross roller bearing outer ring end cap 1.9. Similarly, a deep groove ball bearing is used to connect the upper arm connecting seat 1.20 and the connecting shaft 1.13, and is fixed by pressing the inner and outer rings of the deep groove ball bearing together.

[0149] Figure 15 is a schematic diagram of the upper arm drive module. Because this part has a complex structure, it will be explained in two parts: the shoulder joint large and small pulley drive module and the wrist and elbow joint motor drive module. The shoulder joint large and small pulley drive module mainly consists of a motor protection housing 2.1, a motor 2.2, a motor fixing seat 2.3, a small pulley 2.4, a large pulley 2.5, an upper cover plate 2.6 for fixing the outer ring of the cross roller bearing, a lower cover plate 2.7 for fixing the inner ring of the cross roller bearing, and a lower cover plate 2.8 for fixing the outer ring of the cross roller bearing. In fact, the structure of this module is almost the same as the structure of the differential mechanism described above, with the large pulley 2.5 positioned tangentially with the small pulley 2.4, the small pulley 2.4 being connected to the motor 2.2 to form a drive source, and the motor 2.2 being fixed to the upper arm connecting seat 1.20. The rotating module is similarly composed of cross roller bearings 2.22. The lower end of the large pulley 2.5 and the lower cover plate 2.7 for fixing the inner ring of the cross roller bearing sandwich the inner ring of the cross roller bearing, and the upper cover plate 2.6 for fixing the outer ring of the cross roller bearing and the lower cover plate 2.8 for fixing the outer ring of the cross roller bearing sandwich the outer ring of the cross roller bearing. Furthermore, the upper cover plate 2.6 for fixing the outer ring of the cross roller bearing is fixed to the upper arm connecting seat 1.20 with screws, thereby completing the entire rotating mechanism. The wrist and elbow joint motor drive modules will be described below.

[0150] Figure 16 shows the wrist and elbow joint motor drive module. This part has a symmetrical structure, so only half needs to be explained. This module mainly consists of a drive motor mounting seat 2.9, a drive motor 2.10, a snap ring 2.11, a bearing end cap 2.12, a pulley shaft mounting seat 2.13, a deep groove ball bearing 2.14, a first small pulley 2.15, a pulley bearing 2.16, a stud 2.17, a screw 2.18, a first synchronous pulley shaft 2.19, a second small pulley 2.20, a synchronous belt 2.21, a cross roller bearing 2.22, a first intermediate shaft 2.23, a pulley cover plate 2.24, a lock nut 2.25, a second synchronous pulley shaft 2.26, and a flange bearing 2.27. The motor mounting seat 2.9 and the pulley shaft mounting seat 2.13 are connected to the lower cover plate 2.7 for fixing the inner ring of the cross roller bearing shown in Figure 15, and rotate together with the large pulley 2.5. The drive motor 2.10 is screwed to the motor mounting seat 2.9. The drive motor 2.10 is connected to the first small pulley 2.15 and the second small pulley 2.20, and transmits power to the lower pulley shaft via the synchronous belt 2.21. The synchronous belt 2.21 is tensioned by a combination of pulley bearings 2.16 and studs 2.17. The first synchronous pulley shaft 2.19 and the second synchronous pulley shaft 2.26 are connected in the same manner as the synchronous pulley shaft mounting seat 2.13, and the two synchronous pulley shafts are fixed by two flange bearings 2.27 and the flanges of the synchronous pulley shaft mounting seat 2.13. The synchronous pulleys are integrally molded with the shafts, and pulley cover plates 2.24 are attached to both sides of each synchronous pulley to prevent the synchronous belt 2.21 from falling off, and are secured with lock nuts 2.25 and screws 2.18. The connection of the intermediate shaft is via a shaft fitting structure, which will be explained separately below.

[0151] As shown in Figure 17, the first intermediate shaft 2.23 and the second intermediate shaft 2.30 are connected by two deep groove ball bearings 2.28, allowing relative rotation of the two shafts. The inner ring of the bearing is locked by a snap ring 2.31 and the shoulder of the second intermediate shaft 2.30, and the outer ring is locked by the collar of the first intermediate shaft 2.23 and the pulley cover plate 2.24. The outside of the two shafts are connected to a deep groove ball bearing 2.29, the inner ring of which is locked by the shoulder and snap ring, and the outer ring is locked by the pulley shaft fixing seat 2.13 and the bearing end cap 2.12. The overall structure is compact and the transmission accuracy is high.

[0152] The following are embodiments of the apparatus of the present application, which can be used to carry out embodiments of the method of the present application. For details not described in the embodiments of the apparatus of the present application, please refer to the embodiments of the method of the present application.

[0153] Figure 18 shows a block diagram of a control device for a robot arm according to one embodiment of the present invention. This device has the function of implementing an example of the control method for the robot arm described above. This function may be implemented by hardware, or by running corresponding software on hardware. This device may be the robot arm described above, or it may be mounted on the robot arm. This device 1800 may include an acquisition module 1810, a determination module 1820, and a control module 1830.

[0154] The acquisition module 1810 is used to acquire historical data pairs at N time points. Each historical data pair includes historical input data and historical output data. The historical input data is used to control the motion of the robot arm so that the first object maintains equilibrium on the robot arm, and the historical output data is used to represent the pose of the robot arm and the first object after controlling the motion of the robot arm based on the historical input data. N is a positive integer. The decision module 1820 is used to determine the predicted input data for the robot arm at the current time t, based on historical data pairs at N time points. The control module 1830 is used to control the movement of the robot arm so that the first object balances during the movement of the robot arm, based on predictive input data.

[0155] In some embodiments, as shown in Figure 19, the decision module 1820 comprises a construction unit 1821, a calculation unit 1822, and a decision unit 1823. The construction unit 1821 is used to construct constraint and cost functions for the robot arm based on historical data pairs at N time points. The constraint function is used to describe the constraints that the data must satisfy, and the cost function is used to show the difference between the predicted data and the equilibrium data. The equilibrium data includes input and output data for the robot arm and the first object in an ideal equilibrium state, and the predicted data includes predicted input data and predicted output data. The predicted output data is predicted based on the historical output data. The computing unit 1822 is used to calculate the unit impulse response of the robot arm based on constraint and cost functions. The unit impulse response is used to represent the robot arm's feedback with respect to the predicted input data. The decision unit 1823 is used to determine the predicted input data for the robot arm at the current time t, based on the unit impulse response.

[0156] In some embodiments, the construction unit 1821 determines a first matrix and a second matrix, the first matrix containing historical input data of historical data pairs at N time points, and the second matrix containing historical output data of historical data pairs at N time points; constructs a constraint function for the robot arm based on the previous n rows in the first matrix and the previous n rows in the second matrix, where n is the system order of the robot arm and is a positive integer; and constructs a cost function for the robot arm based on the subsequent L rows in the first matrix and the subsequent L rows in the second matrix, where L is the predicted step length of the robot arm and is a positive integer. Here, the historical data pairs at the previous n time points of the robot arm are used to predict the input and output data at the subsequent L time points of the robot arm.

[0157] In some embodiments, the construction unit 1821 is used to determine the i-th row of a first matrix from the i-1th to the NL-n+i-1th timestamp out of N timestamps, and to determine the i-th row of a second matrix from the i-1th to the NL-n+i-1th timestamp out of N timestamps, where i is a positive integer.

[0158] In some embodiments, the constraint function includes a first constraint function, a second constraint function, a third constraint function, and a fourth constraint function. The construction unit 1821 constructs a first constraint function based on a slack variable, where the slack variable is used to describe measurement noise, and the first constraint function is used to reduce the interference of measurement noise with predictive input data; constructs a second constraint function based on the previous n rows in a first matrix and the previous n rows in a second matrix, where the second constraint function is used to describe historical data pairs at the previous n time points; constructs a third constraint function based on the previous n rows in a first matrix, the previous n rows in a second matrix, and equilibrium data, where the third constraint function indicates that the robot arm approaches an ideal equilibrium state over time; and constructs a fourth constraint function based on a slack variable, a first boundary point, and a unit impulse response, where the first boundary point is the upper limit of the slack variable, and the fourth constraint function is used to constrain the convergence of the process of predicting predictive input data.

[0159] In some embodiments, the construction unit 1821 constructs a first norm based on the last L rows of a first matrix and the last L rows of a second matrix, wherein the first norm includes the square norm of the historical input data at the last L time points out of N time points and the square norm of the historical output data at the last L time points out of N time points, and constructs a first penalty term based on a first penalty weight, a first boundary point, and a first regularization term, wherein the first regularization term is a regularization penalty for the unit impulse response, and the first penalties are... The Narti weights are penalty weights for the first regularization term, the first boundary point is the upper limit of the slack variable, the slack variable is used to describe the measurement noise, and the second penalty term is constructed based on the second penalty weights and the second regularization term, the second regularization term is the regularization penalty for the slack variable, the second penalty weights are penalty weights for the second regularization term, and the cost function is constructed based on the first norm, the first penalty term, and the second penalty term.

[0160] In some embodiments, the decision unit 1823 is used to determine a third matrix based on historical input data pairs at N time points, wherein the third matrix includes a first matrix and a second matrix, the first matrix including historical input data for the historical data pairs at N time points, and the second matrix including historical output data for the historical data pairs at N time points, and to calculate predicted input data for the robot arm at the current time t based on the unit impulse response and the third matrix.

[0161] In some embodiments, the determination unit 1823 constructs a dynamic equation for the robot arm based on the motion parameters of a first object and the motion parameters of the robot arm, wherein the motion parameters include at least one parameter that affects kinetic energy or potential energy, the dynamic equation includes a subdrive equation and a drive equation, wherein the subdrive equation is an equation where the input torque is 0 and the drive equation is an equation where the input torque is not 0, and constructs a state-space equation for the robot arm based on the subdrive equation and a third matrix, wherein the state-space equation is used to describe the motion state of the robot arm, and calculates predicted input data for the robot arm at current time t based on the state-space equation and the drive equation. Here, the motion parameters include at least one of the length of the robot arm, the specifications of the first object, the mass of the robot arm, the moment of inertia of the robot arm, and the moment of inertia of the first object.

[0162] In some embodiments, the decision unit 1823 calculates the input torque based on the state-space equations and the drive equations, the input torque being used to control the joint rotation of the robot arm, and the input torque being determined as the predicted input data for the robot arm at the current time t.

[0163] In some embodiments, the decision unit 1823 is used to construct an initial dynamic equation for the robot arm based on the motion parameters of a first object and the motion parameters of the robot arm, and to obtain a dynamic equation by performing a partial feedback linearization process on the initial dynamic equation, wherein the partial feedback linearization process is an approximate linearization process on the initial dynamic equation.

[0164] In some embodiments, the determination unit 1823 is used to calculate the rotational angular velocity of the robot arm joints based on the unit impulse response and a third matrix, and to determine the rotational angular velocity as the predicted input data for the robot arm at the current time t.

[0165] In some embodiments, as shown in Figure 19, the apparatus 1800 further comprises an acquisition module 1840. The acquisition module 1840 is used to acquire historical data pairs for multiple consecutive historical time points prior to the current time point t, and to uniformly sample the historical data pairs from multiple consecutive historical time points to acquire historical data pairs for N time points.

[0166] In some embodiments, the acquisition module 1810 is used to acquire historical input data for the j-th time out of N time points, where j is a positive integer less than or equal to N; to determine the posture of the robot arm after motion based on the historical input data at the j-th time point; to acquire a first position of a first object on the robot arm based on a tactile sensor; to acquire an image of the first object at the j-th time point; to acquire a second position of the first object on the robot arm based on the image of the first object; to determine the position of the center of mass of the first object based on the first and second positions; and to determine the historical output data of the robot arm at the j-th time point based on the posture of the robot arm after motion based on the historical input data at the j-th time point and the position of the center of mass of the first object.

[0167] The technical means according to the embodiment of the present invention completes the task of balancing a first object on a robot arm by acquiring historical data pairs at multiple time points, predicting input data for the robot arm, acquiring predicted input data, and controlling the motion of the robot arm based on the predicted input data. Since it predicts the future system state of the robot arm based on historical data, rather than relying solely on feedback variables of the robot arm system, the control response becomes faster.

[0168] It should be noted that the above-described embodiment of the device is merely an example of how the partitions of each functional module are used to explain how to realize its functions. In actual applications, the above functions may be assigned to different functional modules as needed, that is, all or some of the functions described above may be realized by partitioning the internal structure of the device into different functional modules. Furthermore, the above-described embodiment of the device belongs to the same concept as the embodiment of the method, and details of its specific implementation process can be found in the embodiment of the method, so the explanation is omitted here.

[0169] Figure 20 shows a structural block diagram of a robot arm according to an exemplary embodiment of the present invention. This robot arm 2000 may be the robot arm described above.

[0170] Typically, the robot arm 2000 includes a processor 2001 and memory 2002.

[0171] The processor 2001 may include one or more processing cores, such as a 4-core processor or a 20-core processor. The processor 2001 may be implemented in the form of at least one hardware component from among a DSP (Digital Signal Processing), an FPGA (Field Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 2001 may include a main processor and a coprocessor, the main processor being a processor for processing data in a wake-up state and also called a CPU (Central Processing Unit), and the coprocessor being a low-power processor for processing data in a standby state. In some embodiments, the processor 2001 may integrate a GPU (Graphics Processing Unit), which is used for rendering and drawing content that needs to be displayed on a display screen. In some embodiments, the processor 2001 may further include an AI (Artificial Intelligence) processor, which is used for processing computational operations related to machine learning.

[0172] Memory 2002 may include one or more computer-readable storage media, which may be tangible and non-temporary. Memory 2002 may further include high-speed random-access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-temporary computer-readable storage media in memory 2002 store a computer program that is loaded and executed by the processor 2001 to realize the robot arm control method according to each embodiment of the above method.

[0173] In an exemplary embodiment, a computer-readable storage medium in which a computer program is stored is further provided. This computer program, when executed by a processor, realizes the method for controlling the robot arm described above.

[0174] Optionally, computer-readable storage media may include ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives), or optical discs. Of these, random-access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0175] In exemplary embodiments, a computer program product is further provided which includes a computer program stored on a computer-readable storage medium. The robot's processor reads the computer program from the computer-readable storage medium and executes it, thereby causing the robot to perform the robot arm control method described above.

[0176] In this specification, "plural" means two or more. "And / or" describes the relationship between related objects and indicates that there may be three possible relationships. For example, "A and / or B" can indicate three cases: A exists only, both A and B exist, or B exists only. The symbol " / " usually indicates that the preceding and following related objects are in an "or" relationship.

[0177] The above description is merely an illustrative example of the present application and is not intended to limit the application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present application shall all be within the scope of protection of the present application.

Claims

1. A method for controlling a robotic arm, which is performed by a computer device, A step of obtaining N time point pairs of historical data, wherein each historical data pair includes historical input data and historical output data, the historical input data being used to control the motion of the robot arm so that a first object maintains a state of equilibrium on the robot arm, and the historical output data being used to represent the posture of the robot arm and the first object after the motion of the robot arm has been controlled based on the historical input data, where N is a positive integer; The steps include determining the predicted input data for the robot arm at the current time t based on the N pairs of historical data at the aforementioned time points, A step of controlling the movement of the robot arm so that the first object maintains balance during the movement of the robot arm, based on the predicted input data; A method for controlling a robotic arm, including the control method.

2. The step of determining the predicted input data for the robot arm at the current time t based on the N pairs of historical data at each time point is as follows: A step of constructing a constraint function and a cost function for the robot arm based on the N pairs of historical data at the aforementioned time points, wherein the constraint function is used to describe the constraints that the data must satisfy, the cost function is used to show the difference between the predicted data and the equilibrium data, the equilibrium data includes input and output data for the robot arm and the first object in an ideal equilibrium state, the predicted data includes the predicted input data and predicted output data, and the predicted output data is predicted based on the historical output data, A step of calculating the unit impulse response of the robot arm based on the constraint function and the cost function, wherein the unit impulse response is used to represent the feedback of the robot arm with respect to the predicted input data, The steps include determining the predicted input data for the robot arm at the current time t based on the unit impulse response, The method according to claim 1, including the method described in claim 1.

3. The step of constructing a constraint function and a cost function for the robot arm based on the N time point pairs is: A step of determining a first matrix and a second matrix, wherein the first matrix includes historical input data of historical data pairs at the N time points, and the second matrix includes historical output data of historical data pairs at the N time points. Steps include: constructing a constraint function for the robot arm based on the previous n rows of the first matrix and the previous n rows of the second matrix, where n is the system order of the robot arm and is a positive integer; A step of constructing a cost function for the robot arm based on the last L rows of the first matrix and the last L rows of the second matrix, wherein L is the predicted step length of the robot arm and is a positive integer, and the steps are... Includes, The method according to claim 2, wherein the historical data pairs at n time points prior to the robot arm are used to predict the input and output data at L time points after the robot arm.

4. The step of determining the first matrix and the second matrix is: The steps include determining the i-th to N-L-n+i-1 time points among the N time points as the i-th row of the first matrix, The steps include determining the history output data for the i-1th to N-L-n+i-1th time points among the N time points as the i-th row of the second matrix, Includes, The method according to claim 3, wherein i is a positive integer.

5. The constraint function includes a first constraint function, a second constraint function, a third constraint function, and a fourth constraint function. The step of constructing the constraint function of the robot arm based on the previous n rows of the first matrix and the previous n rows of the second matrix is: A step of constructing a first constraint function based on a slack variable, wherein the slack variable is used to describe measurement noise, and the first constraint function is used to reduce the interference of the measurement noise with the predicted input data. Steps include constructing a second constraint function based on the previous n rows of the first matrix and the previous n rows of the second matrix, wherein the second constraint function is used to describe historical data pairs at the previous n time points, A step of constructing a third constraint function based on the previous n rows of the first matrix, the previous n rows of the second matrix, and equilibrium data, wherein the third constraint function indicates that the robot arm approaches the ideal equilibrium state over time. A step of constructing a fourth constraint function based on the slack variable, a first boundary point, and the unit impulse response, wherein the first boundary point is the upper limit of the slack variable, and the fourth constraint function is used to constrain the convergence of the process of predicting the predict input data. The method according to claim 3, including the method described in claim 3.

6. The step of constructing a cost function for the robot arm based on the last L rows in the first matrix and the last L rows in the second matrix is: A step of constructing a first norm based on the last L rows of the first matrix and the last L rows of the second matrix, wherein the first norm includes the square norm of the historical input data at the last L time points out of the N time points, and the square norm of the historical output data at the last L time points out of the N time points. Steps include constructing a first penalty term based on a first penalty weight, a first boundary point, and a first regularization term, wherein the first regularization term is a regularization penalty to the unit impulse response, the first penalty weight is a penalty weight to the first regularization term, the first boundary point is an upper limit of a slack variable, and the slack variable is used to describe measurement noise, and A step of constructing a second penalty term based on a second penalty weight and a second regularization term, wherein the second regularization term is a regularization penalty on the slack variable, and the second penalty weight is a penalty weight on the second regularization term. The steps of constructing the cost function based on the first norm, the first penalty term, and the second penalty term, The method according to claim 3, including the method described in claim 3.

7. The step of determining the predicted input data for the robot arm at the current time t based on the unit impulse response is: A step of determining a third matrix based on the historical input data pairs at the N time points, wherein the third matrix includes a first matrix and a second matrix, the first matrix includes the historical input data of the historical data pairs at the N time points, and the second matrix includes the historical output data of the historical data pairs at the N time points. A step of calculating the predicted input data for the robot arm at the current time t based on the unit impulse response and the third matrix, The method according to claim 2, including the method described in claim 2.

8. The step of calculating the predicted input data for the robot arm at the current time t based on the unit impulse response and the third matrix is ​​as follows: Steps of constructing a dynamic equation for the robot arm based on the motion parameters of the first object and the motion parameters of the robot arm, wherein the motion parameters include at least one parameter that affects kinetic energy or potential energy, the dynamic equation includes a sub-drive equation and a drive equation, the sub-drive equation is an equation where the input torque is zero, and the drive equation is an equation where the input torque is not zero, A step of constructing a state-space equation for the robot arm based on the under-driven equation and the third matrix, wherein the state-space equation is used to describe the motion state of the robot arm. The steps include: calculating the predicted input data for the robot arm at the current time t based on the state-space equation and the drive equation; Includes, The method according to claim 7, wherein the motion parameter includes at least one of the length of the robot arm, the specifications of the first object, the mass of the robot arm, the mass of the first object, the moment of inertia of the robot arm, and the moment of inertia of the first object.

9. The step of calculating the predicted input data for the robot arm at the current time t based on the state-space equation and the drive equation is as follows: A step of calculating an input torque based on the state-space equation and the drive equation, wherein the input torque is used to control the joint rotation of the robot arm. The steps include determining the input torque as the predicted input data for the robot arm at the current time t, The method according to claim 8, including the method described in claim 8.

10. The step of constructing the dynamic equations for the robot arm based on the motion parameters of the first object and the motion parameters of the robot arm is: The steps include constructing the initial dynamic equations for the robot arm based on the motion parameters of the first object and the motion parameters of the robot arm, A step of performing a partial feedback linearization process on the initial dynamic equations to obtain the dynamic equations, wherein the partial feedback linearization process is an approximate linearization process on the initial dynamic equations. The method according to claim 8, including the method described in claim 8.

11. The step of calculating the predicted input data for the robot arm at the current time t based on the unit impulse response and the third matrix is ​​as follows: A step of calculating the rotational angular velocity of the joints of the robot arm based on the unit impulse response and the third matrix, The steps include determining the rotational angular velocity as the predicted input data for the robot arm at the current time t, The method according to claim 7, including the method described in claim 7.

12. The aforementioned method, The steps include: obtaining historical data pairs for multiple consecutive historical time points prior to the current time t, The steps include: uniformly sampling historical data pairs at the plurality of consecutive historical time points to obtain historical data pairs at the N time points; The method according to claim 1, further comprising:

13. The step of obtaining historical data pairs at N time points is: A step of obtaining historical input data for the j-th time out of N time points, wherein j is a positive integer less than or equal to N. The steps include determining the posture of the robot arm after it has moved based on the historical input data at the j-th time, A step of obtaining a first position of the first object on the robot arm based on a tactile sensor, The steps include obtaining an image of the first object at the j-th time, A step of obtaining a second position of the first object on the robot arm based on an image of the first object, A step of determining the position of the center of mass of the first object based on the first position and the second position, A step of determining the history output data of the robot arm at the j-th time based on the posture of the robot arm after it has moved based on the history input data at the j-th time and the position of the center of mass of the first object, The method according to claim 1, including the method described in claim 1.

14. A control device for a robot arm, An acquisition module for acquiring N time point pairs of historical data, wherein each historical data pair includes historical input data and historical output data, the historical input data being used to control the motion of the robot arm so that a first object maintains a state of equilibrium on the robot arm, and the historical output data being used to represent the pose of the robot arm and the first object after the motion of the robot arm has been controlled based on the historical input data, where N is a positive integer. A decision module for determining the predicted input data of the robot arm at the current time t based on the aforementioned N pairs of historical data at different time points, A control module for controlling the movement of the robot arm so that the first object maintains balance during the movement of the robot arm, based on the predicted input data, A control device for a robotic arm, equipped with the following features.

15. A robotic arm comprising a processor and memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to perform the robotic arm control method according to any one of claims 1 to 13.

16. A computer-readable storage medium in which a computer program is stored, wherein the computer program is loaded and executed by a processor to perform the robot arm control method described in any one of claims 1 to 13.

17. A computer program product comprising a computer program stored in a computer-readable storage medium, wherein the processor reads the computer program from the computer-readable storage medium and executes it to perform the robot arm control method according to any one of claims 1 to 13.

Images