Method and apparatus for controlling robotic arm, and device, storage medium and program product

Abstract

A method for controlling a robotic arm, relating to the technical fields of robots and automatic control. The method comprises: acquiring historical-data pairs at N moments, wherein each historical-data pair comprises historical input data and historical output data, the historical input data is used for controlling a robotic arm (220) to move, such that a first object (210) is kept in a balanced state on the robotic arm (220), and the historical output data is used for representing a pose of the robotic arm (220) and a pose of the first object (210) after the robotic arm (220) is controlled, on the basis of the historical input data, to move, N being a positive integer; on the basis of the historical-data pairs at the N moments, determining predicted input data of the robotic arm (220); and on the basis of the predicted input data, controlling the robotic arm (220) to move, wherein the first object (210) maintains balance during the motion of the robotic arm (220). The prediction of a future system state of the robotic arm (220) is performed on the basis of historical data, and does not only rely on feedback variables of a robotic arm system, so that a control response is faster. Further provided are a robotic arm control apparatus, and a device and a storage medium.

Description

Robotic arm control method, device, equipment, storage medium and program product

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on October 7, 2023, with application number 2023112991432 and application name “Robotic Arm Control Method, Device, Equipment and Storage Medium”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of robotics and automatic control technology, and in particular to robotic arm control. Background Art

[0003] A robot is an automatically controlled machine that simulates human operations, and a robot in the form of a robotic arm can imitate the movements of a human arm.

[0004] In related technologies, the tactile information of the bottle on the robotic arm is processed to obtain the position of the bottle on the robotic arm, and then the speed of the bottle is obtained through differential means. The position and speed of the bottle on the robotic arm are used as the input of a PID (Proportional Integral Differential) controller, and the rotation angle of the robotic arm joint is used as the output of the PID controller to complete the task of keeping the bottle balanced on the robotic arm.

[0005] However, in the above method, the PID controller only relies on the feedback variable of the robot arm and has a slow response speed.

[0006] Summary of the Invention

[0007] The present invention provides a method, device, robot, storage medium, and program product for controlling a robotic arm. The technical solution is as follows:

[0008] According to one aspect of an embodiment of the present application, a method for controlling a robotic arm is provided, the method comprising:

[0009] Acquire historical data pairs at N moments, the historical data pairs including historical input data and historical output data, the historical input data being used to control the movement of the robotic arm so that the first object maintains a balanced state on the robotic arm, and the historical output data being used to represent the postures of the robotic arm and the first object after the movement of the robotic arm is controlled based on the historical input data, where N is a positive integer;

[0010] Determining predicted input data for the robotic arm based on the N historical data pairs at each moment;

[0011] The movement of the robotic arm is controlled based on the predicted input data, and the first object maintains balance during the movement of the robotic arm.

[0012] According to one aspect of an embodiment of the present application, a robotic arm control device is provided, the device comprising:

[0013] an acquisition module, configured to acquire historical data pairs at N moments, the historical data pairs comprising historical input data and historical output data, the historical input data being used to control the movement of the robotic arm so that the first object maintains a balanced state on the robotic arm, and the historical output data being used to represent the postures of the robotic arm and the first object after the movement of the robotic arm is controlled based on the historical input data, where N is a positive integer;

[0014] a determination module, configured to determine predicted input data of the robotic arm based on the historical data pairs at the N moments;

[0015] A control module is configured to control the movement of the robotic arm based on the predicted input data, wherein the first object maintains balance during the movement of the robotic arm.

[0016] According to one aspect of an embodiment of the present application, a robotic arm is provided, comprising a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the above-mentioned robotic arm control method.

[0017] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the above-mentioned robotic arm control method.

[0018] According to one aspect of an embodiment of the present application, a computer program product is provided, the computer program product including a computer program stored in a computer-readable storage medium. A processor of a robot reads the computer program from the computer-readable storage medium and executes the computer program, causing the robot to perform the above-described robotic arm control method.

[0019] The technical solutions provided by the embodiments of the present application may have the following beneficial effects:

[0020] By acquiring historical data pairs at multiple moments, the robot's input data is predicted, generating predicted input data. Based on this predicted input data, the robot's motion is controlled to achieve the task of balancing the first object on the robot. This prediction of the robot's future system state based on historical data, rather than relying solely on the robot's system's feedback variables, results in faster control response. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] FIG1 is a flow chart of a method for controlling a robotic arm according to an embodiment of the present application;

[0022] FIG2 is a schematic diagram of a balancing task of a first object on a robotic arm provided by one embodiment of the present application;

[0023] FIG3 is a schematic diagram of a balancing task of a first object on a robotic arm provided by another embodiment of the present application;

[0024] FIG4 is a schematic diagram of a robotic arm control method provided by one embodiment of the present application;

[0025] FIG5 is a simplified schematic diagram of a kinetic model provided by one embodiment of the present application;

[0026] FIG6 is a block diagram of a method for controlling a robotic arm using a dynamic model according to an embodiment of the present application;

[0027] FIG7 is a schematic diagram of a coordinate system of a robotic arm provided in one embodiment of the present application;

[0028] FIG8 is an architecture diagram of a robotic arm control method provided by one embodiment of the present application;

[0029] FIG9 is a schematic diagram of a 7-DOF robotic arm provided by one embodiment of the present application;

[0030] FIG10 is a schematic diagram of a 3-DOF shoulder joint provided by one embodiment of the present application;

[0031] FIG11 is an overall schematic diagram of a shoulder joint provided by one embodiment of the present application;

[0032] FIG12 is a front view and a side view of a differential mechanism provided by one embodiment of the present application;

[0033] FIG13 is a schematic diagram illustrating an implementation principle of a differential mechanism provided by an embodiment of the present application;

[0034] FIG14 is a cross-sectional view of a differential mechanism provided in one embodiment of the present application;

[0035] FIG15 is a schematic diagram of a large arm drive module provided by one embodiment of the present application;

[0036] FIG16 is a schematic diagram of a wrist joint and elbow joint motor drive module provided in one embodiment of the present application;

[0037] FIG17 is a cross-sectional view of an intermediate shaft connection provided by one embodiment of the present application;

[0038] FIG18 is a block diagram of a robotic arm control device provided by one embodiment of the present application;

[0039] FIG19 is a block diagram of a robotic arm control device provided by another embodiment of the present application;

[0040] FIG20 is a block diagram of a robot provided in accordance with an embodiment of the present application. DETAILED DESCRIPTION

[0041] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numbers in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present application. Rather, they are merely examples of methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0042] Robotic arms are widely used in industrial production and academic research, but most applications focus on using the end of the arm or the robot hand to complete some operational tasks. Few people use the rigid body connectors and shell of the robot arm to complete the corresponding tasks, and few people use the shell of the robot arm to complete the corresponding operational tasks. The main reasons are:

[0043] (1) The appearance of the robotic arm is generally a curved design, without a large flat surface;

[0044] (2) There is no grasping mechanism design such as hands, and the contact between the appearance of the robot arm and the external objects will not form a shape closure and force closure, which makes control too difficult;

[0045] (3) The robot arm shell has basically no tactile perception, and the lack of sensory feedback makes control difficult to achieve.

[0046] Different from the previous technology, the embodiments of the present application provide a method for balancing an object placed on the arm housing of a robotic arm, preventing the object from sliding off the arm during the process, and a method for achieving the same. The specific method includes:

[0047] (1) Design of control architecture;

[0048] (2) Use of tactile sensors;

[0049] (3) Controller design.

[0050] In the method provided in the embodiments of the present application, the execution subject of each step can be a computer device, which can be a device for controlling a robotic arm or the robotic arm itself. When the computer device is the robotic arm itself, the robotic arm refers to an electronic device with data calculation, processing, and storage capabilities. The robotic arm provided in the embodiments of the present application can be used in scenarios such as industry (such as industrial robots), services (such as food serving robots), entertainment (such as performance robots), and medical care (such as medical robots), and the embodiments of the present application do not specifically limit this.

[0051] The solution provided in the embodiments of the present application involves technologies such as automated control of artificial intelligence, which can realize the control of the robotic arm, and is specifically explained through the following embodiments.

[0052] In related technologies, a PID controller is used to process the tactile information of the bottle on the robotic arm to determine the bottle's position, which is then differentiated to determine its velocity. Position and velocity serve as inputs to the PID controller, while the rotation angles of the robotic arm's joints serve as the controller's output to balance the bottle.

[0053] However, the PID controller approach is subject to the following limitations:

[0054] (1) PID controllers are sensitive to external disturbances and noise, which may lead to a decrease in control performance.

[0055] (2) PID controller can only handle single variable situations. For multi-variable systems, other control methods need to be used.

[0056] (3) A simple PID controller has no feedforward and responds slowly. PID will only play its control role when an error occurs.

[0057] (4) The speed and even acceleration in the input signal of the PID controller are calculated by differential calculation, which has a large error and introduces noise. Therefore, the PID control quantity obtained based on the input signal obtained by differential calculation will be very jittery.

[0058] (5) PID is a linear controller. When the nonlinear factors of the controlled system are large, the PID controller cannot control the controlled object.

[0059] This embodiment of the application uses DD MPC (Data Driven Model Predictive Control) to overcome the above problems and achieve the task of balancing an object (including but not limited to a bottle) on a robotic arm. This embodiment of the application can achieve the following effects:

[0060] (1) The Data Driven MPC controller takes noise interference into account, has better adaptability and robustness, and the output will not jitter due to input noise.

[0061] (2) Data driven MPC controller can control multiple controlled variables simultaneously.

[0062] (3) Data driven MPC can predict the future system state based on historical data, not just relying on the system's feedback variables. Therefore, Data driven MPC has a faster control response.

[0063] (4) Data driven MPC can take into account some real-world constraints, such as the upper and lower limits of the robot arm's joint positions, the upper and lower limits of the robot arm's joint velocities, the upper and lower limits of the robot arm's joint accelerations, etc. The control variables calculated by the controller meet the real-world constraints.

[0064] Please refer to Figure 1, which shows a flow chart of a method for controlling a robotic arm provided by one embodiment of the present application. In this embodiment, the method is applied to the robotic arm described above as an example. The method may include the following steps (110-130). For ease of description, the first object placed on the robotic arm is mainly described as a bottle:

[0065] Step 110, obtain historical data pairs at N moments, where the historical data pairs include historical input data and historical output data. The historical input data is used to control the movement of the robotic arm so that the first object maintains a balanced state on the robotic arm. The historical output data is used to characterize the posture of the robotic arm and the first object after the movement of the robotic arm is controlled based on the historical input data. N is a positive integer.

[0066] The historical input data and historical output data are both relative to the current time t. The historical input data is used to control the robot arm to move so that the first object on the robot arm maintains a balanced state, while the historical output data is used to identify the posture of the robot arm and the first object after the robot arm moves. Therefore, the historical input data can also be recorded as prior control data, and the historical output data can also be recorded as prior posture data. The historical input data and historical output data at the same historical time can be recorded as a historical data pair.

[0067] In some embodiments, for a current moment t, historical data pairs of N moments before the current moment t are obtained.

[0068] In some embodiments, for any time t, historical data pairs of N time points before the time t are obtained.

[0069] In some embodiments, the above-mentioned N moments may be N consecutive moments or N discontinuous moments, which is not limited in this application.

[0070] In some embodiments, the time interval between any two adjacent moments in the N moments is the same. For example, N=3, where the time interval between moment 1 and moment 2 is the same as the time interval between moment 2 and moment 3.

[0071] In some embodiments, the movement of the robotic arm is controlled based on the input data, and during the movement of the robotic arm, the first object remains balanced on the robotic arm and does not fall.

[0072] In some embodiments, the first object may change its relative position to the robotic arm as the robotic arm moves.

[0073] This application does not limit the type and shape of the first object. For example, the first object can be a cylindrical object, such as a bottle, or a spherical object, such as a ping-pong ball.

[0074] In some embodiments, the input data corresponding to a certain moment may include one or more data, which is not limited in this application. For example, the robotic arm may include a shoulder joint, an elbow joint, and a wrist joint, and the input data may include the rotation angle of one or more joints.

[0075] In some embodiments, the output data corresponding to a particular moment may include one or more data, which is not limited in this application. For example, the output data may include one or more data of the angle between the robotic arm and the horizontal plane, the position of the first object on the robotic arm, and the rotation angle of one or more joints of the robotic arm.

[0076] Step 120 : Determine the predicted input data of the robot arm at the current time t based on the historical data pairs at N time moments.

[0077] In some embodiments, based on historical data pairs at N moments, the input of the robotic arm at the current moment is predicted to obtain predicted input data of the robotic arm.

[0078] In some embodiments, a DD MPC controller is used to determine predicted input data of the robot arm based on historical data pairs at N moments.

[0079] In some embodiments, the predicted input data may be one or more parameters for controlling the motion of the robotic arm, and how to control the motion of the robotic arm may be determined based on the predicted input data. For example, the predicted input data may be a torque or a rotation angle of a joint of the robotic arm.

[0080] The functions of the predicted input data and the historical input data are similar. Both are used to control the movement of the robotic arm so that the first object maintains a balanced state on the robotic arm. The difference lies in the moment of control. The predicted input data is used to control the movement of the robotic arm at the current moment t, and the historical input data is used to control the movement of the robotic arm at N moments before the current moment t.

[0081] Step 130 : Control the movement of the robotic arm based on the predicted input data, so that the first object maintains balance during the movement of the robotic arm.

[0082] In some embodiments, at a current time t, the robotic arm is in a first configuration, and the robotic arm is controlled to move to a second configuration according to the predicted input data. During this process, the first object maintains balance on the robotic arm.

[0083] In some embodiments, the first object moves relative to the robotic arm as the robotic arm moves.

[0084] It should be noted that the movement of the robotic arm mentioned in the embodiment of the present application refers to the movement of the robotic arm relative to the ground. During the movement of the robotic arm, the first object will also move relative to the ground. However, due to the change in the angle of the robotic arm relative to the horizontal plane, the force condition of the first object changes, and the first object may also move relative to the robotic arm.

[0085] Exemplarily, as shown in Figure 2, the first object 210 is located on the forearm of the robotic arm 220. At this time, the robotic arm is in a first form. The robotic arm 220 moves based on the predicted input data to reach a third form as shown in Figure 3. At this time, the relative position between the first object 210 and the robotic arm 220 has changed.

[0086] In some embodiments, the range of motion of the first object on the robotic arm may include the entire robotic arm or only a portion of the robotic arm. For example, the first object may move on the upper arm, lower arm, and distal end of the robotic arm, or may move only on the lower arm.

[0087] The technical solution provided in the embodiments of the present application predicts the input data of the robotic arm by acquiring historical data pairs at multiple time points, obtaining predicted input data, and controlling the movement of the robotic arm based on the predicted input data to complete the task of balancing a first object on the robotic arm. Predicting the future system state of the robotic arm based on historical data does not rely solely on the feedback variables of the robotic arm system, thereby achieving faster control response.

[0088] In some embodiments, the robotic arm can be viewed as an LTI (linear time-invariant system) system. The basic properties of an LTI system are: linearity (homogeneity and additivity), time invariance, differentiability, and integration.

[0089] In some embodiments, the system parameters of the linear time-invariant system do not change with time. Therefore, when the unit impulse response is known, the input data of the LTI system is determined, and the corresponding output data can be obtained based on the system state of the LTI system. Conversely, when the input data, output data and system state of the LTI system are known, the unit impulse response can be solved. In the embodiment of the present application, an example of controlling a joint (or a rotary joint) of a robotic arm is used for illustrative purposes. Therefore, the robotic arm can be regarded as an LTI system. The process of determining the predicted input data of the robotic arm in the embodiment of the present application can be regarded as the process of solving the unit impulse response. Considering that there are many errors, noises, etc. in reality that will affect the robotic arm, different historical data may not have the same unit impulse response to the solved unit impulse response. Therefore, in the process of solving the unit impulse response, it is necessary to consider the constraints of the actual conditions. It is also necessary to adjust the value of the unit impulse response based on the feedback of the robotic arm (the output data corresponding to the second form of the robotic arm after the predicted input data moves) and the ideal equilibrium state of the robotic arm. Among them, the unit impulse response is actually a unit impulse response sequence. To simplify the expression, it is called a unit impulse response in the embodiment of the present application, and it can also be called a unit impulse response sequence.

[0090] In addition, if it is necessary to control multiple joints (or rotary joints) of the robot arm, a specific analysis of the actual situation of the robot arm is required, and it cannot be simply regarded as an LTI system.

[0091] For the embodiment of the present application, the robotic arm system can be regarded as an LTI system as shown in the following formula (1):

[0092] Where α is the unit impulse response, is the predicted input data, Refers to the predicted output data, H L (u d ), H L (y d ) is the Hankel matrix, L is the predicted step length of the robot arm, u d 、y dThis refers to a historical trajectory of a robotic arm system with L+n-order continuous excitations. A historical trajectory of continuous excitation indicates that the system state and properties represented by this trajectory are stable. This allows us to predict future trajectory points of length L using the first n trajectory points (trajectory points are simply historical data pairs at a specific moment in time). (Mapping to the Hankel matrix corresponds to the first n and last L rows of the matrix.)

[0093] It should be noted that, in the embodiment of the present application, input data or output data for a period of time is referred to as a trajectory.

[0094] in,

[0095] Among them, x refers to historical data (which can be historical input data or historical output data), x N-L It refers to the historical data at the NLth moment, and N is the number of historical data.

[0096] In some embodiments, when the number of types of historical data is not 1, the dimension of the Hankel matrix is ​​(m*L+m*n)×(NL-n+1), where m is the number of types of historical data and n is the system order of the robotic arm.

[0097] The above formula (1) can be regarded as the convolution and expression of the discrete LTI system in the time domain. The meaning of the formula can be understood as follows: if there is any output after multiplying α with the Hankel matrix, it must be a certain trajectory of the LTI system

[0098] The above formula implicitly expresses the system model through a period of historical data of the LTI system, replacing the M in MPC, that is, the model, with historical data.

[0099] In some embodiments, the above step 120 can be implemented as at least one of the following steps 121 to 123 (not shown in the figure).

[0100] Step 121, based on the historical data pairs at N moments, construct the constraint function and cost function of the robotic arm. The constraint function is used to describe the constraints that the data must meet, and the cost function is used to represent the difference between the predicted data and the balance data. The balance data includes the input data and output data when the robotic arm and the first object are in an ideal balance state. The predicted data includes predicted input data and predicted output data. The predicted output data is obtained based on the prediction of the historical output data.

[0101] In some embodiments, the constraint function is used to describe the real-world conditions that the robotic arm must meet. In some embodiments, the cost function, which can also be viewed as a loss function, is used to describe the difference between the predicted data and the equilibrium data.

[0102] In some embodiments, an ideal equilibrium state refers to a state in which the robotic arm does not move when its input data is zero, that is, the robotic arm's output data does not change relative to the output data at the previous moment. Taking the example of a first object moving on a forearm of a robotic arm, the ideal equilibrium state can be one in which the forearm of the robotic arm is parallel to a horizontal plane, the first object is located at the midpoint of the forearm, and the plane formed by the center of mass of the first object and the midline of the robotic arm is perpendicular to the horizontal plane.

[0103] In some embodiments, step 121 may be implemented as at least one of the following steps a to c (not shown in the figure).

[0104] Step a: determining a first matrix and a second matrix, wherein the first matrix includes historical input data from historical data pairs at N moments, and the second matrix includes historical output data from historical data pairs at N moments.

[0105] In some embodiments, historical input data in the historical data pairs at N moments are determined as elements of a first matrix, and historical output data in the historical data pairs at N moments are determined as elements of a second matrix.

[0106] In some embodiments, the historical input data from the i-1th to the NL-n+i-1th moments among N moments are determined as the i-th row of the first matrix; the historical output data from the i-1th to the NL-n+i-1th moments among N moments are determined as the i-th row of the second matrix; where i is a positive integer.

[0107] In some embodiments, the first matrix is ​​H in the above formula (1). L (u d ), the second matrix is ​​H in the above formula (1) L (y d ).

[0108] Step b: constructing a constraint function of the manipulator according to the first n rows in the first matrix and the first n rows in the second matrix, where n is the system order of the manipulator and n is a positive integer.

[0109] In some embodiments, since the constraint function is used to describe the constraints imposed by real conditions on the robotic arm, the constraint function may include one or more.

[0110] In some embodiments, the constraint function includes a first constraint function, a second constraint function, a third constraint function, and a fourth constraint function.

[0111] In some embodiments, a first constraint function is constructed based on slack variables, where the slack variables are used to describe measurement noise, and the first constraint function is used to reduce interference of the measurement noise on the predicted input data.

[0112] In some embodiments, due to the presence of measurement noise in real-world scenarios, slack variables are introduced. The purpose of introducing slack variables is to ensure that the Hankel matrix equation (Formula (1)) can still hold true when y with measurement noise is substituted into it (ideally, y in the Hankel matrix equation is noise-free).

[0113] In some embodiments, the first constraint function is expressed as follows:

[0114] Among them, σ(t) refers to the slack variable, is the predicted input data, Refers to the predicted output data, α(t) refers to the unit impulse response, H L+n (u d ) refers to the Hankel matrix composed of historical input data at N moments, It refers to the Hankel matrix composed of historical output data at N moments.

[0115] In some embodiments, a second constraint function is constructed based on the first n rows in the first matrix and the first n rows in the second matrix, and the second constraint function is used to describe the historical data pairs at the first n moments.

[0116] In some embodiments, the second constraint function is used to express the following meaning: the -n to -1 historical input data of the LTI system at a certain time t Historical output data It is the first n historical data pairs of the LTI system at the current moment, which can be understood as different descriptions of the same LTI trajectory of length n.

[0117] In some embodiments, the second constraint function is expressed as follows:

[0118] in, Refers to the -n to -1 predicted input data at time t, Refers to the historical output data from -n to -1 at time t, u [t-n,t-1] Refers to the previous n historical input data at the current time t, Refers to the previous n historical output data at the current time t.

[0119] In some embodiments, a third constraint function is constructed based on the first n rows in the first matrix, the first n rows in the second matrix, and the balance data. The third constraint function indicates that the robotic arm approaches an ideal balance state over time.

[0120] In some embodiments, the third constraint function is a constraint corresponding to a quadratic programming problem in a Data Driven MPC controller. The equality constraint here is referred to as a terminal equality constraint. A terminal equality constraint is typically used to constrain the final value of an optimization variable during the optimization process to be equal to a specific target value. For example, in a control problem, this constraint is used to achieve a specific target state. For example, in the embodiment of the present application, this constraint is used to achieve an ideal equilibrium state for a robotic arm.

[0121] In some embodiments, the third constraint function is expressed as follows:

[0122] in, It refers to the predicted input data of the LTI system with a length of n from t+Ln to t+L-1. Refers to the LTI system output data with a length of n from t+Ln to t+L-1. It refers to the equilibrium point of the LTI system at n moments, where s represents the equilibrium point and n represents the equilibrium point written as a column vector of length n.

[0123] Since the future trajectory from time t+Ln to time t+L-1 is unknown, it can only be predicted based on the properties of the LTI system and historical data on sustainability incentives. Therefore, the control process updates the trajectories of the n points preceding the current moment in real time to update this equality constraint. The "terminal" in the terminal equality constraint can be understood as the final state the system will reach: the equilibrium point. This terminal equality constraint can be expressed in the corresponding form based on the Hankel matrix equation for the LTI system: the Hankel matrix multiplied by α. Thus, the terminal equality constraint is converted into an equation with respect to α.

[0124] In some embodiments, the above-mentioned equilibrium point refers to an ideal equilibrium state.

[0125] In some embodiments, the ideal equilibrium state is the same at each moment in time.

[0126] In some embodiments, a fourth constraint function is constructed based on the slack variable, the first boundary point and the unit impulse response, the first boundary point refers to the upper limit value of the slack variable, and the fourth constraint function is used to constrain the convergence of the process of predicting the input data.

[0127] In some embodiments, the fourth constraint function is expressed as follows:

[0128] Among them, σ k (t) refers to the slack variable, It refers to the upper limit of the slack variable, that is, the first boundary point, and α(t) refers to the unit impulse response.

[0129] In some embodiments, when λ σ When σ is chosen to be large enough, the fourth constraint function can be absolutely satisfied because the regularization term of σ is σ The penalty becomes very small. Therefore, this inequality constraint is not considered in the quadratic programming problem. σ Refers to the penalty weight for the slack variable.

[0130] Step c: constructing a cost function of the robotic arm based on the last L rows in the first matrix and the last L rows in the second matrix, where L is the predicted step length of the robotic arm and L is a positive integer.

[0131] In some embodiments, a first norm is constructed based on the last L rows in the first matrix and the last L rows in the second matrix; wherein the first norm includes: the second-order square norm of the historical input data at the last L moments of N moments, and the second-order square norm of the historical output data at the last L moments of N moments; a first penalty term is constructed based on the first penalty weight, the first boundary point and the first regularization term, the first regularization term is a regularization penalty for the unit impulse response, the first penalty weight refers to the penalty weight for the first regularization term, the first boundary point refers to the upper limit value of the slack variable, and the slack variable is used to describe measurement noise; a second penalty term is constructed based on the second penalty weight and the second regularization term, the second regularization term is a regularization penalty for the slack variable, and the second penalty weight refers to the penalty weight for the second regularization term; a cost function is constructed based on the first norm, the first penalty term and the second penalty term.

[0132] In some embodiments, historical data pairs of the robotic arm at the first n moments are used to predict input data and output data of the robotic arm at the next L moments.

[0133] In some embodiments, the cost function is shown in the following formula (7):

[0134] in, It refers to the square of the second-order norm between the input data and the output data and their corresponding equilibrium points, R refers to the penalty weight matrix of the square of the second-order norm between the input data and its equilibrium point, and Q refers to the penalty weight matrix of the square of the second-order norm between the output data and its equilibrium point.

[0135] It refers to the regularization penalty for α, which is intended to prevent overfitting in the process of solving the quadratic programming problem and reduce the error of the equation composed of the Hankel matrix. α It represents the penalty weight of the regular term. It refers to the upper limit of the slack variable.

[0136] is the penalty for the regularization of the slack variable σ. And λ σ represents the penalty weight for the regularization term. Slack variables are often used in optimization problems to convert inequality constraints into equality constraints, expanding the feasible solution domain and making the optimization problem easier to solve. Finally, the data sequences u and y in the cost function are replaced by the Hankel matrix equation, which implicitly represents the LTI system model. (Here, the last L rows of the Hankel matrix are taken to ensure the correct matrix dimensions on both sides of the equation.) This converts the cost function into an equation related to α.

[0137] Step 122 : Calculate the unit impulse response of the robotic arm according to the constraint function and the cost function. The unit impulse response is used to characterize the feedback of the robotic arm to the predicted input data.

[0138] In some embodiments, both the cost function and the constraint function are converted into functions related to α, and the entire quadratic programming problem can be used to find an α that minimizes the cost function and satisfies the equality constraint.

[0139] Step 123 : Determine predicted input data of the robotic arm according to the unit impulse response.

[0140] In some embodiments, predicted input data of the robotic arm is determined based on the unit impulse response and historical input data pairs at N moments.

[0141] In some embodiments, a third matrix is ​​determined based on historical input data pairs at N moments, where the third matrix includes a first matrix and a second matrix, where the first matrix includes historical input data in the historical data pairs at N moments, and the second matrix includes historical output data in the historical data pairs at N moments; based on the unit impulse response and the third matrix, the predicted input data of the robotic arm is calculated.

[0142] In some embodiments, the third matrix is ​​the Hankel matrix on the left side of the Hankel matrix equation shown in formula (1), and the predicted input data is the Hankel matrix on the right side of the equation.

[0143] Through this method, the task of balancing the first object on the manipulator is treated as a discrete LTI system. The unit impulse response of the LTI system is obtained, and the predicted input data for the manipulator is predicted based on this unit impulse response, making the solution simpler. This process takes into account real-world constraints, ensuring that the predicted input data meets these constraints. Furthermore, slack variables are used to describe the impact of measurement noise on the output data, accounting for noise interference and providing improved adaptability and robustness.

[0144] For example, FIG4 illustrates a workflow diagram of a DD MPC controller provided by one embodiment of the present application. 1. Collect a system history trajectory of length N; 2. Construct a Hankel matrix from the history trajectory; 3. Substitute the Hankel matrix (decompose the first n rows and last L columns of the Hankel matrix, the first n rows are used to form the terminal equality constraint, and the last L columns are used to form the Hessien and Gradient matrices in the quadratic programming problem), the Reference, and the RQ penalty weight matrix into the cost function and the terminal equality constraint to form a quadratic programming problem regarding α; 4. Use a quadratic programming solver (e.g., qpOASES) to solve for α that minimizes the cost function and satisfies the equality constraint; 5. Substitute α back into the Hankel matrix equation to determine the corresponding input and output trajectory and extract the predicted input data u; 6. Send the predicted input data u to the robotic arm for execution; 7. Take the first n input and output trajectory points at the current moment to update the terminal equality constraint as feedback; 8. Set t = t + 1 and repeat step 4 until the system reaches equilibrium.

[0145] In some embodiments, the type of input data and the type of output data of the robotic arm can be derived based on the dynamic model, or the type of input data and the type of output data of the robotic arm can be derived without being based on the dynamic model. The following embodiments of the present application provide exemplary explanations of this.

[0146] 1. Using dynamic model

[0147] In some embodiments, a dynamic equation of the robotic arm is constructed based on the motion parameters of the first object and the motion parameters of the robotic arm.

[0148] The motion parameter includes at least one parameter that affects kinetic energy or potential energy. The dynamic equation includes an underactuated equation and a driven equation. The underactuated equation refers to an equation with an input torque of 0, and the driven equation refers to an equation with an input torque not equal to 0.

[0149] In some embodiments, the motion parameter includes at least one of the following: the length of the robotic arm, the size of the first object, the mass of the robotic arm, the mass of the first object, the moment of inertia of the robotic arm, and the moment of inertia of the first object.

[0150] In some embodiments, the types of motion parameters of the first object and the types of motion parameters of the robotic arm may be the same or different.

[0151] In some embodiments, a state space equation of the robotic arm is constructed based on the under-actuated equation and the third matrix, and the state space equation is used to describe the motion state of the robotic arm; and the predicted input data is calculated based on the state space equation and the drive equation.

[0152] In some embodiments, a first state-space equation of the robotic arm is constructed according to the third matrix, and the first state-space equation is processed according to the under-actuated equation to obtain the state-space equation.

[0153] In some embodiments, an input torque is calculated based on the state space equation and the drive equation, and the input torque is used to control the joint rotation of the robotic arm; the input torque is determined as the predicted input data.

[0154] In some embodiments, an initial dynamic equation of the robotic arm is constructed based on the motion parameters of the bottle and the motion parameters of the robotic arm; the initial dynamic equation is subjected to partial feedback linearization processing to obtain a dynamic equation, and the partial feedback linearization processing refers to approximate linearization processing of the initial dynamic equation.

[0155] In some embodiments, the partial feedback linearization process refers to initializing the parameters in the initial dynamic equation to parameters that meet the linearization requirements. For example, when θ approaches 0, sinθ is initialized to θ.

[0156] For example, with respect to a scenario in which the forearm shell of a robotic arm realizes balancing a bottle, the process of performing dynamic modeling on the robotic arm abstracted based on the scenario is described as follows.

[0157] First, the robotic arm is in the world coordinate system. Define the extension of the forearm as the positive y-axis, the direction perpendicular to the y-axis and parallel to the ground as the x-axis. When standing with the robotic arm facing the same direction, the right hand is in the positive x-axis. The z-axis is the direction opposite to gravity, pointing vertically upward and perpendicular to the ground. This creates a spatial rectangular coordinate system. To control the robot's balancing of the bottle, a two-dimensional model can be created on the YOZ plane.

[0158] A simplified diagram of the two-dimensional model in the YOZ plane is shown in Figure 5. The upper circle 510 represents the cross-section of the bottle, the lower rectangle 520 represents the side of the robot's forearm, and the circle 530 above rectangle 520 represents the axis of rotation of the forearm (e.g., the elbow joint). The first problem is considered as a homogeneous rigid body, and the bottle's position is evaluated using its center of mass. The bottle is approximately perpendicular to the forearm.

[0159] The physical quantities (motion parameters) used in the two-dimensional model and their positive directions are defined as follows: The horizontal distance from the center of mass of the bottle to the center of the axis of the robot arm along the side of the robot arm is s. The torque that rotates the robot arm axis is τ, with the positive direction of the torque being counterclockwise. The corresponding angle of rotation of the forearm relative to the world coordinate system is θ. The length of the side of the forearm is l a , the radius of the bottle is r b , the mass of the forearm is m a , the mass of the bottle is m b, the moment of inertia of the forearm is I a , the bottle's moment of inertia is I b .

[0160] For example, the physical quantities used in the two-dimensional model are defined as follows:

[0161] Forearm length: l a =0.27m

[0162] Bottle radius: r b =0.0325m

[0163] Arm mass: m a =2.48kg

[0164] Bottle mass: m b =0.24kg

[0165] Moment of inertia of forearm: I a =0.03

[0166] The moment of inertia of the bottle: I b =2.535e-04

[0167] The square of the bottle's linear velocity can be expressed as:

[0168] The angular velocity of the bottle can be expressed as:

[0169] According to the Euler-Lagrange equation, the kinetic energy and potential energy of all rigid bodies in the system need to be solved separately. The sum of the kinetic energy of all rigid bodies in the system is:

[0170] The sum of the potential energies of all rigid bodies in the system is:

[0171] The partial derivative of kinetic energy with respect to state s (state s is also the position s of the bottle on the forearm) and θ in its generalized coordinates:

[0172] The partial derivative of the potential energy with respect to the states s and θ in its expanded coordinate system is:

[0173] Calculating the above results using the Euler-Lagrange equations reveals that the simplified dynamic model for the system on the YOZ plane can be expressed as the following equation. The first row represents the dynamic equation for the s degree of freedom (the s degree of freedom also refers to the bottle's position s on the arm). Due to underactuation in this direction, the actual input torque on the right side of the equation is 0. The second row represents the dynamic equation for the arm's rotation angle θ. The actuation for this degree of freedom is the motor torque τ, so the right side of the second row's equation is τ.

[0174] Generally, the underactuated part of the dynamic model (that is, the equation with zero on the right side of the equation) is introduced into the state-space equation so that the state-space equation describing the system takes into account the dynamics of the system.

[0175] According to the input and output of the dynamic model, the input data u of the manipulator is set as the rotational angular acceleration of the rotary joint The output data y is set to the angle θ and angular velocity of the rotating joint and the position s and velocity of the bottle on the arm When written as a state-space equation, the first three lines represent simple differential relations between state variables, while the last line of the state-space equations consists of the underactuated portion of the dynamics. The equilibrium point (ideal equilibrium state) ref is assumed to be zero (and its corresponding units) for all angular velocities and accelerations, zero for all joint rotation angles (0 rad in the derived simplified dynamics model, corresponding to 0.465 rad in the real robotic arm), and the bottle's equilibrium point s on the arm is the midpoint of the forearm (0.135 m in the derived simplified dynamics model, corresponding to 22 in the real robotic arm (here 22 refers to the 22nd column in the tactile sensor array on the robotic arm)). Typically, if the robotic arm motor can be controlled using torque, the dynamic system model is subjected to partial feedback linearization (PFLO). This results in a linearized system model and an expression for the torque τ and the new control variable v introduced by the PFLO. The former is used to design the controller, while the latter is used to convert the controller output v (that is, the new control variable introduced by the partial feedback linearization process) into a torque τ, which is then sent to the robot arm. The entire control process is shown in Figure 6.

[0176] In the embodiment of the present application, directly As the new control quantity v, the partial feedback linearization process is as follows: 1. When θ is near 0 and the change is very small, sinθ is approximately linearized to θ; 2. When θ is near 0 and the change is very small, Approximately linearized to 0.

[0177] After partial feedback linearization, the dynamic underactuated equation is introduced into the state space equation. The state space equation is as follows:

[0178] The dynamic driving equation is a new control quantity v (here is ) between:

[0179] In the above content, the input data and output data of the robot arm are determined according to the dynamic model. At this time, the input data of the robot arm is the rotational angular acceleration of the rotary joint. The output data is the angle θ and angular velocity of the rotating joint and the position s and velocity of the bottle on the arm

[0180] It should be noted that g in the above formula is the acceleration due to gravity.

[0181] In some embodiments, when a dynamic model is adopted, the rotational angular velocity of the joint of the robotic arm is calculated based on the unit impulse response and the third matrix; the rotational angular velocity is determined as the predicted input data of the robotic arm at the current time t.

[0182] 2. No kinetic model

[0183] When the system input and output are not selected according to the dynamic model, the input data is set to the rotation angle θ of the revolute joint, and the output is set to the position of the first object on the arm and the integral of the error with respect to time ∫sdt, as well as the position s of the first object on the arm and the velocity of the first object on the forearm. The balance point (ideal equilibrium state) ref is set as follows: the balance points of all (angular) velocities are 0 (and the corresponding units), and the balance point of the joint rotation angle is 0 rad (the derived dynamic simplified model is 0 rad, and the corresponding real robotic arm is 0.465 rad), and the balance point position s of the bottle on the arm is the midpoint of the forearm (the derived dynamic simplified model is 0.135 m, and the corresponding real robotic arm is 22 (here 22 refers to the 22nd column in the tactile sensor array arranged on the robotic arm)).

[0184] θ represents the pitch angle of the robot's forearm rotating around the y-axis. represents the angular velocity of the manipulator's forearm in the pitch direction around the y-axis, τ represents the control torque applied in the pitch direction around the y-axis. s represents the offset between the center of mass of the first object and the center of gravity of the manipulator's forearm in the y-direction. Represents the offset speed of the center of mass of the bottle and the center of mass of the robot arm in the y direction.

[0185] In some embodiments, due to hardware limitations, it is impossible to accurately collect and send acceleration signals to the robotic arm. Therefore, a method can be used that does not select system inputs and outputs based on the dynamic model.

[0186] In some embodiments, for the current moment t, historical data pairs of multiple consecutive historical moments before the current moment t are obtained; the historical data pairs of multiple consecutive historical moments are uniformly sampled to obtain historical data pairs of N moments.

[0187] For example, we collect historical data pairs (including historical input and output data) with a length of N = 1000. Since the resulting Hankel matrix is ​​too large when N is too large, it significantly increases the computational burden and time of the computer. Therefore, we use interval sampling, taking one point from the input and output data every four times. This compresses the system's historical data of length 1000 to 200 points, which are then input into the DDMPC controller. This can effectively reduce the computational burden and time of the computer. We also set the length of the prediction horizon to 50. We then adjust all penalty weights involved in the DDMPC.

[0188] Finally, the first object is placed on the forearm of the robotic arm and interference is applied to the first object. The DDMPC will output the rotation angle θ of the revolute joint and control the robotic arm to complete the task of balancing the first object on the robotic arm.

[0189] This application also provides an exemplary embodiment for how to obtain historical output data at N moments.

[0190] In some embodiments, a visual perception method can be used to recognize the posture of the first object. However, this method may result in a linear error of 1 to 2 centimeters and an angular error of 5 to 10 degrees. Furthermore, its operation time is relatively long, at approximately 100 ms, or 10 Hz.

[0191] In some embodiments, an engineered visual solution can be employed, employing lightweight data image processing to determine the x-axis position of the first object's center of mass. Specific implementations vary, and this application is not intended to limit these approaches. For example, if the first object is a bottle, cluster analysis can be performed based on the color differences between the bottle and surrounding objects to determine the bottle's geometric center. This is then used to calibrate the bottle's geometric center with the center of mass. For another example, if the first object is a bottle, feature recognition can be performed on the bottle to determine its position in the image, thereby determining its geometric center. This is then used to calibrate the bottle's geometric center with the center of mass. Because this method utilizes lightweight computation, it operates quickly, typically requiring only about 10 ms per operation, or nearly 100 Hz. Its linear error is also relatively manageable, typically around 1 cm, and its angular error is around 5 degrees. However, a disadvantage of this approach is that the error in the depth direction of the camera, i.e., the y-axis of the bottle's world coordinate system, is relatively large. For information regarding the coordinate system (e.g., the x- and y-axis directions) discussed above, please refer to the coordinate system described in the dynamic model embodiment above, and this application will not elaborate further here.

[0192] This error is precisely one of the advantages of tactile sensors. Although tactile sensors cannot accurately measure the x-axis position of the first object, only the absolute position of the contact point between the first object and the robotic arm relative to the robotic arm, they are very accurate in measuring the y-axis position of the contact point between the first object and the robotic arm. Therefore, the advantages of both vision and touch can be used to estimate the state of the first object.

[0193] In some embodiments, a visual-tactile fusion solution can be employed. For example, a tactile sensor provides the bottle's y-position and pitch gesture. Simultaneously, lightweight visual image processing and comparison with prior photo data determine the bottle's x-center of mass. This solution has a time budget of approximately 10 milliseconds, or 100 Hz, with a linear error of approximately 1 cm and an angular error of approximately 5 degrees. This approach is similar to an engineered vision solution, but overcomes the drawback of these solutions, which often lack accurate y-center of mass measurement.

[0194] For example, as shown in Figure 7 , the engineered vision solution can obtain a relatively accurate pose of the first object 710 in the x-direction, while the visual-tactile fusion solution can obtain a relatively accurate pose of the first object 710 in the y-direction. Combining the two solutions, the accurate center of mass position of the first object 710 can be obtained. Table 1 shows the required cycles and corresponding errors for the three solutions.

[0195] Table 1: Three methods for determining the pose of the first object

[0196] Taking the above-mentioned vision-tactile fusion solution as an example, obtaining N historical data pairs of historical moments may include at least one of the following steps 1 to 7.

[0197] Step 1: For the j-th moment among N moments, obtain the historical input data of the j-th moment, where j is a positive integer less than or equal to N.

[0198] Step 2: Determine the pose of the robot arm after movement based on the historical input data at the jth moment.

[0199] Step 3: Acquire a first position of the first object on the robotic arm based on the tactile sensor.

[0200] Step 4: Acquire the image of the first object at the jth moment.

[0201] Step 5: Acquire a second position of the first object on the robotic arm based on the image of the first object.

[0202] Step 6: Determine the center of mass position of the first object based on the first position and the second position.

[0203] Step 7: Determine the historical output data of the robotic arm at the jth moment according to the posture of the robotic arm after movement based on the historical input data at the jth moment and the center of mass position of the first object.

[0204] In some embodiments, the first position and the second position are expressed in the form of coordinates (x, y), the coordinates corresponding to the first position are (x1, y1), and the coordinates corresponding to the second position are (x2, y2).

[0205] Based on the introduction of the visual-tactile fusion solution in the above content, the first position is relatively accurate in the y direction, and the second position is relatively accurate in the x direction, so the center of mass position of the first object can be determined as (x2, y1).

[0206] This application does not limit the method for obtaining the second position of the first object on the robotic arm based on the image of the first object in step 5. For example, a feature recognition method can be used to determine the position of the first object in the image of the first object, and then determine the geometric center of the first object. The second position is determined based on the geometric center of the first object.

[0207] In some embodiments, the types of data included in the historical output data may be determined by whether a dynamic model is used. For example, if a dynamic model is used, the historical output data may include the center of mass position of the first object, the velocity of the first object, the angle of the rotary joint of the manipulator, the angular velocity of the rotary joint of the manipulator, and the angular acceleration of the rotary joint of the manipulator. If a dynamic model is not used, the historical output data may include the center of mass position of the first object, the velocity of the first object, and the time integral of the center of mass position of the first object and the error.

[0208] Through the above method, a relatively accurate position of the first object on the robotic arm can be obtained, providing strong support for the balancing task of the first object on the robotic arm.

[0209] The overall control architecture for implementing the above functions is shown in Figure 8. The ultimate goal is to control the balancing task of the first object on the robotic arm by inputting commands to the robotic arm's joint motors. Each of the robotic arm's joint motors is equipped with joint encoders that provide feedback on the rotation angle, angular velocity, and current of each joint motor. This information can be used to estimate the robotic arm's state. Tactile sensors are also installed on the robotic arm's fingers, palm, and certain links. Based on this tactile information, the position and posture of the bottle obtained from the state estimation is combined with a two-dimensional or three-dimensional system dynamics model between the first object and the forearm. This system dynamics model can be two-dimensional or three-dimensional. The inputs and outputs of the robotic arm can be determined based on the system dynamics model. Therefore, the data required by the Data-Driven MPC controller can be collected based on the inputs and outputs of the robotic arm determined by the dynamics model. The Data-Driven MPC controller is then designed based on the collected historical data. When the first object is placed on the robotic arm and a disturbance is applied to it, the Data-Driven MPC outputs the corresponding predicted input data to complete the balancing task of the first object on the robotic arm. Depending on the robot arm, these predicted input data can be variables in joint space or Cartesian space. If the predicted input data is a variable in Cartesian space, the joint angle of each joint must be calculated using inverse kinematics. Here, as time changes, the controller outputs a sequence of the robot arm's end-point posture or the center-of-mass posture of the connecting rod. Corresponding to a series of inverse kinematics solutions, the resulting result is a sequence of angular velocities and joint angular velocities of each joint of the robot arm. Sending this sequence of joint angles and joint angular velocities to the robot arm can control the end-point posture of the robot arm or the position and posture of the center of mass of a specific connecting rod. If the predicted input data is a variable in joint space, such as joint angles, angular velocities, or angular accelerations, it can be directly sent to the robot to complete the task of balancing the first object on the robot arm.

[0210] In the embodiments of this application, a real-machine experiment was conducted using a bottle as the first object. In the real-machine experiment, the bottle could roll back and forth on the forearm of the robotic arm, and its motion state changed as the absolute position between the forearm of the robotic arm and the ground changed. The bottle's rolling on the forearm could also be achieved by controlling the motors in the robotic arm's joints. Throughout the entire process, the bottle remained balanced and did not fall to the ground while moving on the forearm of the robotic arm. The real-machine experiment demonstrated the action sequence described in the embodiments of this application and demonstrated the effectiveness and stability of the control architecture and controller described in the embodiments of this application.

[0211] The embodiments of the present application also provide relevant information about the robotic arm hardware, and the control methods of the robotic arm involved in the above embodiments can all be implemented on the robotic arm.

[0212] As shown in Figure 9, a humanoid 7-DOF robotic arm is shown. The elbow and wrist control motors are located in the hollow space of the third shoulder joint. The elbow and wrist cables are driven by a belt driven by a motor at the shoulder joint, which transmits the belt to a pulley. The pulley then controls the elbow and wrist movements via a belt drive. The 3-DOF shoulder joint structure is shown in Figure 10. The number of joints in a robotic mechanism capable of independent movement is called its degrees of freedom, or DOF for short. The control method currently used in industrial robots treats each joint on the robotic arm as a separate servo mechanism, with each axis corresponding to a servo. Each servo is controlled via a bus and coordinated by a controller.

[0213] The low-inertia differential shoulder joint structure used in the seven-degree-of-freedom robotic arm utilizes a differential cable drive mechanism at the shoulder. This not only reduces the weight of the mechanism by allowing the motor module to be positioned at the rear, but also enables torque superposition in certain situations. The shoulder joint's third degree of freedom utilizes a pair of large and small pulleys, driven by a cable, further improving transmission precision and reducing weight. Finally, the drive modules for the wrist and elbow joints are positioned at the rear of the shoulder joint arm module, minimizing the weight of the entire robotic arm. All of these structures are easily modularized, simplifying the manufacturing process.

[0214] As shown in Figure 11, this is an overall view of the shoulder joint. It can be seen that the shoulder joint is mainly divided into three modules, namely the differential mechanism module 1, the cross-roller bearing rotation module 2, and the upper arm end drive module 3. Among them, the cross-roller rotation module 2 is the intermediate module connecting the differential mechanism 1 and the upper arm end drive module 3. These three modules are introduced in detail below.

[0215] Figure 12 is the front view and side view of the differential mechanism. The main parts are marked one by one in Figure 12. The parts with complex shapes and the connection methods will be introduced one by one below. It can be seen from Figure 12 that the entire differential mechanism consists of a rotary encoder 1.1, a differential mechanism fixing seat 1.2, a large reel 1.3, a motor protective cover 1.4, a differential mechanism inner ring reel shaft 1.5, a motor 1.6, a differential mechanism outer ring reel shaft 1.7, a motor seat 1.8, a cross-roller bearing outer ring end cover 1.9, a cross-roller inner ring fixing seat 1.10, a differential mechanism small reel 1.11, a connecting block 1.12, a connecting shaft 1.13, a rotary encoder 1.14, a differential mechanism large reel 1.15, a bearing end cover 1.16, a small reel 1.17, a wire rope 1.18, a bearing end cover 1.19, and a large arm connecting seat 1.20. The specific composition and operation principle of the entire differential mechanism are introduced below. The operation principle of the entire differential mechanism is similar to that of the differential mechanism of three bevel gears. Figure 13 shows the operation principle of the differential mechanism. It can be seen that Figure 13a is the main body of the entire differential mechanism, including the large reel 1.3, the differential mechanism inner ring reel shaft 1.5, the differential mechanism outer ring reel shaft 1.7, the differential mechanism small reel 1.11, the differential mechanism large reel 1.15 and the steel wire wrapped thereon. The differential mechanism small reel 1.11 and the differential mechanism large reel 1.15 are fixed to the boom connecting seat 1.20 with screws. Figure 13b shows the cooperation between the inner ring spool shaft 1.5 of the differential mechanism and the small spool 1.11 of the differential mechanism, realizing the cooperation of a gear pair of the differential mechanism. The drive is through the large spool 1.3 and its matching small spool 1.17 connected to the inner ring spool shaft 1.5 of the differential mechanism with screws. Figure 3c shows another gear pair composed of the outer ring spool shaft 1.7 of the differential mechanism and the large spool 1.15 of the differential mechanism. It can be seen that the inner ring spool shaft 1.5 of the differential mechanism and the outer ring spool shaft 1.7 of the differential mechanism are connected in the form of a sleeve shaft. The specific structure will be introduced later. The transmission shaft 1.21 in Figure 13a is connected to the connecting block 1.12 and is responsible for transmitting the rotation of the entire differential mechanism to the rotary encoder 1.1.

[0216] As shown in Figure 14, there is a cross-sectional view of the differential mechanism. Since the small pulley 1.11 and the large pulley 1.15 of the differential mechanism are connected to the upper arm connecting seat 1.20 by screws, their connection method is not described in Figure 14. It can be seen that in Figure 14, the outer ring of the rotary encoder 1.1 is fixed to the connecting pressure plate 1.22 by a nut, and the inner ring of the rotary encoder 1.1 is clamped by the nut and the shoulder of the transmission shaft 1.21, so that the inner ring and the transmission shaft 1.21 rotate together, and the clamping plate 1.23 and the connecting pressure plate 1.22 press the outer ring of the deep groove ball bearing 1.25 (because all the bearings in Figure 14 are deep groove ball bearings, only one number is used to represent them). The connection between the differential mechanism inner ring spool shaft 1.5 and the differential mechanism outer ring spool shaft 1.7 is similar to the above connection. There are two sets of deep groove ball bearings 1.25 at the upper and lower ends of the differential mechanism outer ring spool shaft 1.7. The inner ring of the bearing is fixed by the shoulder of the differential mechanism inner ring spool shaft 1.5 and the retaining ring 1.24, and the outer ring is fixed by the shoulder of the differential mechanism outer ring spool shaft 1.7 and the bearing pressure plate 1.29. The connection between the differential mechanism inner ring spool shaft 1.5 and the cross roller inner ring fixed seat 1.10 and the connecting block 1.12 is not much different from the above connection. They are all achieved by pressing the inner and outer rings of the bearings together, so they will not be described in detail. The connecting shaft 1.13 and the connecting seat 1.12 are fixed and limited by a key 1.28 and screws. In Figure 14, crossed roller bearing 1.27 is clamped in place by crossed roller inner race retaining seat 1.10 and crossed roller bearing inner race cover 1.26, while the outer race is clamped in place by differential mechanism retaining seat 1.2 and crossed roller bearing outer race end cover 1.9. The connection between boom connecting seat 1.20 and connecting shaft 1.13 also uses deep groove ball bearings, secured by compressing the inner and outer races.

[0217] As shown in Figure 15, this is a view of the boom drive module. Due to the complex structure of this part, it is divided into two parts for introduction: one is the shoulder joint large and small wire wheel drive module, and the other is the wrist joint and elbow joint motor drive module. The shoulder joint large and small wire wheel drive module mainly includes a motor protection shell 2.1, a motor 2.2, a motor fixing seat 2.3, a small wire wheel 2.4, a large wire wheel 2.5, an upper cover plate 2.6 for fixing the outer ring of a cross roller bearing, a lower cover plate 2.7 for fixing the inner ring of a cross roller bearing, and a lower cover plate 2.8 for fixing the outer ring of a cross roller bearing. In fact, the structure of this module is not much different from that of the differential mechanism above. The large wire wheel 2.5 is arranged tangentially with the small wire wheel 2.4. The small wire wheel 2.4 is connected to the motor 2.2 to form a drive source. The motor 2.2 is fixed on the boom connection seat 1.20. The rotation module is also composed of a crossed roller bearing 2.22. The lower end of the large pulley 2.5 and the lower cover plate 2.7 that fixes the inner ring of the crossed roller bearing clamp the inner ring of the crossed roller bearing. The upper cover plate 2.6 that fixes the outer ring of the crossed roller bearing and the lower cover plate 2.8 that fixes the outer ring of the crossed roller bearing clamp the outer ring of the crossed roller bearing. At the same time, the upper cover plate 2.6 that fixes the outer ring of the crossed roller bearing is fixed to the upper arm connecting seat 1.20 with screws. The entire rotation mechanism is completed. Now let's take a look at the wrist joint and elbow joint motor drive modules.

[0218] As shown in Figure 16, this is the wrist joint and elbow joint motor drive module. This part is a symmetrical structure, so only half of it needs to be introduced. This module mainly includes the drive motor fixing seat 2.9, drive motor 2.10, retaining spring 2.11, bearing end cover 2.12, pulley shaft fixing seat 2.13, deep groove ball bearing 2.14, small pulley 1 2.15, pulley bearing 2.16, stud 2.17, screw 2.18, synchronous pulley shaft 1 2.19, small pulley 2 2.20, synchronous belt 2.21, cross roller bearing 2.22, intermediate shaft 1 2.23, pulley cover 2.24, locking nut 2.25, synchronous pulley shaft 2 2.26, and retaining bearing 2.27. It can be seen that the motor fixing seat 2.9 and the pulley shaft fixing seat 2.13 are both connected to the cross roller bearing inner ring fixed lower cover plate 2.7 in Figure 5, and rotate with the large pulley 2.5. The drive motor 2.10 is fixed to the motor fixing seat 2.9 by screws. The drive motor 2.10 is connected to the small pulley 1 2.15 and the small pulley 2 2.20, and then the motion is transmitted to the pulley shaft below through the synchronous belt 2.21. The synchronous belt 2.21 is tensioned by the combination of pulley bearing 2.16 and stud 2.17. Synchronous pulley shaft 1 (2.19) and synchronous pulley shaft 2 (2.26) are connected in the same manner as synchronous pulley shaft holder (2.13). They are secured by two rib bearings (2.27) and the ribs of synchronous pulley shaft holder (2.13). The synchronous pulleys are integrally formed with the shafts. To prevent the synchronous belt (2.21) from being dragged out, pulley covers (2.24) are added to each side of the pulleys and secured to them with lock nuts (2.25) and screws (2.18). The intermediate shaft connection is a sleeve shaft structure, which is explained separately below.

[0219] As shown in Figure 17, intermediate shaft 1 (2.23) and intermediate shaft 2 (2.30) are connected by two deep-groove ball bearings (2.28), enabling relative rotation between the two shafts. The inner rings of these bearings are locked by retaining springs (2.31) and the shoulder of intermediate shaft 2 (2.30), while the outer rings are locked by the collar of intermediate shaft 1 (2.23) and the pulley cover (2.24). Deep-groove ball bearings (2.29) are connected to the outside of both shafts. The inner rings of these bearings are locked by the shoulder and retaining springs, while the outer rings are locked by the pulley shaft holder (2.13) and the bearing end cap (2.12). The overall structure is compact and provides high transmission accuracy.

[0220] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0221] Please refer to Figure 18, which shows a block diagram of a robotic arm control device provided by one embodiment of the present application. This device has the functions of implementing the aforementioned exemplary method for robotic arm control. These functions can be implemented by hardware or by hardware executing corresponding software. This device can be the robotic arm described above, or it can be installed on a robotic arm. This device 1800 may include: an acquisition module 1810, a determination module 1820, and a control module 1830.

[0222] Acquisition module 1810 is used to obtain historical data pairs at N moments, where the historical data pairs include historical input data and historical output data. The historical input data is used to control the movement of the robotic arm so that the first object maintains a balanced state on the robotic arm. The historical output data is used to characterize the posture of the robotic arm and the first object after the movement of the robotic arm is controlled based on the historical input data. N is a positive integer.

[0223] The determination module 1820 is configured to determine the predicted input data of the robotic arm at the current time t based on the historical data pairs at the N time moments.

[0224] The control module 1830 is configured to control the movement of the robotic arm based on the predicted input data, so that the first object maintains balance during the movement of the robotic arm.

[0225] In some embodiments, as shown in FIG19 , the determination module 1820 includes a construction unit 1821 , a calculation unit 1822 and a determination unit 1823 .

[0226] Construction unit 1821 is used to construct the constraint function and cost function of the robotic arm based on the historical data pairs at the N moments, the constraint function is used to describe the constraint conditions that the data must meet, and the cost function is used to represent the difference between the predicted data and the balance data. The balance data includes the input data and output data of the robotic arm and the first object when they are in an ideal balance state. The predicted data includes the predicted input data and predicted output data, and the predicted output data is predicted based on the historical output data.

[0227] The calculation unit 1822 is used to calculate the unit impulse response of the robotic arm according to the constraint function and the cost function, where the unit impulse response is used to represent the feedback of the robotic arm with respect to the predicted input data.

[0228] The determination unit 1823 is configured to determine the predicted input data of the robotic arm at a current time t according to the unit impulse response.

[0229] In some embodiments, the construction unit 1821 is used to determine a first matrix and a second matrix, wherein the first matrix includes historical input data in the historical data pairs at the N moments, and the second matrix includes historical output data in the historical data pairs at the N moments; based on the first n rows in the first matrix and the first n rows in the second matrix, the constraint function of the robotic arm is constructed, where n is the system order of the robotic arm, and n is a positive integer; based on the last L rows in the first matrix and the last L rows in the second matrix, the cost function of the robotic arm is constructed, where L is the prediction step size of the robotic arm, and L is a positive integer; wherein the historical data pairs at the first n moments of the robotic arm are used to predict the input data and output data of the last L moments of the robotic arm.

[0230] In some embodiments, the construction unit 1821 is used to determine the historical input data from the i-1th to the NL-n+i-1th moments among the N moments as the i-th row of the first matrix; and to determine the historical output data from the i-1th to the NL-n+i-1th moments among the N moments as the i-th row of the second matrix; wherein i is a positive integer.

[0231] In some embodiments, the constraint function includes a first constraint function, a second constraint function, a third constraint function and a fourth constraint function; the construction unit 1821 is used to construct the first constraint function based on the slack variable, the slack variable is used to describe the measurement noise, and the first constraint function is used to reduce the interference of the measurement noise on the predicted input data; the second constraint function is constructed based on the first n rows in the first matrix and the first n rows in the second matrix, and the second constraint function is used to describe the historical data pairs at the first n moments; the third constraint function is constructed based on the first n rows in the first matrix, the first n rows in the second matrix and the balance data, and the third constraint function indicates that the robotic arm approaches the ideal equilibrium state over time; the fourth constraint function is constructed based on the slack variable, the first boundary point and the unit impulse response, the first boundary point refers to the upper limit value of the slack variable, and the fourth constraint function is used to constrain the convergence of the process of predicting the predicted input data.

[0232] In some embodiments, the construction unit 1821 is used to construct a first norm based on the last L rows in the first matrix and the last L rows in the second matrix; wherein the first norm includes: the second-order square norm of the historical input data of the last L moments of the N moments, and the second-order square norm of the historical output data of the last L moments of the N moments; construct a first penalty term based on a first penalty weight, a first boundary point and a first regularization term, the first regularization term is a regularization penalty for the unit impulse response, the first penalty weight refers to the penalty weight for the first regularization term, the first boundary point refers to the upper limit value of the slack variable, and the slack variable is used to describe measurement noise; construct a second penalty term based on a second penalty weight and a second regularization term, the second regularization term is a regularization penalty for the slack variable, and the second penalty weight refers to the penalty weight for the second regularization term; construct the cost function based on the first norm, the first penalty term and the second penalty term.

[0233] In some embodiments, the determination unit 1823 is used to determine a third matrix based on the historical input data pairs at the N moments, the third matrix including a first matrix and a second matrix, the first matrix including the historical input data in the historical data pairs at the N moments, and the second matrix including the historical output data in the historical data pairs at the N moments; and calculate the predicted input data of the robotic arm at the current moment t based on the unit impulse response and the third matrix.

[0234] In some embodiments, the determination unit 1823 is used to construct the dynamic equation of the robotic arm based on the motion parameters of the first object and the motion parameters of the robotic arm, the motion parameters include at least one parameter affecting kinetic energy or potential energy, and the dynamic equation includes an under-actuated equation and a driven equation, the under-actuated equation refers to an equation with an input torque of 0, and the driven equation refers to an equation with an input torque not equal to 0; based on the under-actuated equation and the third matrix, a state space equation of the robotic arm is constructed, and the state space equation is used to describe the motion state of the robotic arm; based on the state space equation and the driven equation, the predicted input data of the robotic arm at the current time t is calculated; wherein the motion parameters include at least one of the following: the length of the robotic arm, the specifications of the first object, the mass of the robotic arm, the mass of the first object, the moment of inertia of the robotic arm, and the moment of inertia of the first object.

[0235] In some embodiments, the determination unit 1823 is used to calculate the input torque based on the state space equation and the drive equation, and the input torque is used to control the joint rotation of the robotic arm; and the input torque is determined as the predicted input data of the robotic arm at the current time t.

[0236] In some embodiments, the determination unit 1823 is used to construct an initial dynamic equation of the robotic arm based on the motion parameters of the first object and the motion parameters of the robotic arm; and perform partial feedback linearization processing on the initial dynamic equation to obtain the dynamic equation, and the partial feedback linearization processing refers to approximate linearization processing on the initial dynamic equation.

[0237] In some embodiments, the determination unit 1823 is used to calculate the rotational angular velocity of the joint of the robotic arm based on the unit impulse response and the third matrix; and determine the rotational angular velocity as the input data of the robotic arm at the current time t.

[0238] In some embodiments, as shown in FIG. 19 , the apparatus 1800 further includes an obtaining module 1840 .

[0239] Obtaining module 1840 is used to obtain, for a current moment t, historical data pairs of multiple consecutive historical moments before the current moment t; and uniformly sample the historical data pairs of the multiple consecutive historical moments to obtain the historical data pairs of the N moments.

[0240] In some embodiments, the acquisition module 1810 is used to obtain the historical input data of the j-th moment among N moments, where j is a positive integer less than or equal to N; determine the posture of the robotic arm after movement based on the historical input data at the j-th moment; obtain the first position of the first object on the robotic arm based on the tactile sensor; obtain the image of the first object at the j-th moment; obtain the second position of the first object on the robotic arm based on the image of the first object; determine the center of mass position of the first object based on the first position and the second position; determine the historical output data of the robotic arm at the j-th moment based on the posture of the robotic arm after movement based on the historical input data at the j-th moment and the center of mass position of the first object.

[0241] The technical solution provided in the embodiments of the present application predicts the input data of the robotic arm by acquiring historical data pairs at multiple time points, obtaining predicted input data, and controlling the movement of the robotic arm based on the predicted input data to complete the task of balancing a first object on the robotic arm. Predicting the future system state of the robotic arm based on historical data does not rely solely on the feedback variables of the robotic arm system, thereby achieving faster control response.

[0242] It should be noted that the apparatus provided in the above embodiments, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0243] Figure 20 shows a block diagram of a mechanical arm provided by an exemplary embodiment of the present application. The mechanical arm 2000 may be the mechanical arm described above.

[0244] Typically, the robotic arm 2000 includes a processor 2001 and a memory 2002 .

[0245] The processor 2001 may include one or more processing cores, such as a 4-core processor, a 20-core processor, etc. The processor 2001 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), or PLA (Programmable Logic Array). The processor 2001 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 2001 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 2001 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0246] The memory 2002 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 2002 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 2002 stores a computer program, which is loaded and executed by the processor 2001 to implement the robot arm control method provided by each of the above method embodiments.

[0247] In an exemplary embodiment, a computer-readable storage medium is further provided, wherein a computer program is stored in the storage medium. When the computer program is executed by a processor, the computer program implements the above-mentioned robot arm control method.

[0248] Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives), or an optical disk, etc. Among them, the random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0249] In an exemplary embodiment, a computer program product is further provided, the computer program product including a computer program stored in a computer-readable storage medium. A processor of a robot reads the computer program from the computer-readable storage medium and executes the computer program, causing the robot to perform the above-mentioned robot arm control method.

[0250] It should be understood that the term "plurality" used herein refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0251] The above description is merely an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for controlling a robot arm, the method being executed by a computer device, the method comprising: Acquire historical data pairs at N moments, the historical data pairs including historical input data and historical output data, the historical input data being used to control the movement of the robotic arm so that the first object maintains a balanced state on the robotic arm, and the historical output data being used to characterize the postures of the robotic arm and the first object after the movement of the robotic arm is controlled based on the historical input data, and N is a positive integer; Determine the predicted input data of the robot arm at the current time t according to the historical data pairs at the N times; Based on the predicted input data, the movement of the robotic arm is controlled, and the first object maintains balance during the movement of the robotic arm.

2. The method according to claim 1, wherein determining the predicted input data of the robot arm at the current time t based on the historical data pairs at the N time moments comprises: According to the historical data pairs at the N moments, a constraint function and a cost function of the robot arm are constructed, wherein the constraint function is used to describe the constraint conditions that the data must satisfy, and the cost function is used to represent the difference between the predicted data and the balance data, wherein the balance data includes the input data and the output data of the robot arm and the first object when they are in an ideal balance state, and the predicted data includes the predicted input data and the predicted output data, wherein the predicted output data is obtained by prediction based on the historical output data; Calculating a unit impulse response of the robotic arm according to the constraint function and the cost function, wherein the unit impulse response is used to characterize feedback of the robotic arm with respect to the predicted input data; According to the unit impulse response, the predicted input data of the robot arm at the current time t is determined.

3. The method according to claim 2, wherein constructing the constraint function and cost function of the robot arm according to the historical data pairs at the N moments comprises: Determine a first matrix and a second matrix, wherein the first matrix includes historical input data in the historical data pairs at the N moments, and the second matrix includes historical output data in the historical data pairs at the N moments; Constructing a constraint function of the manipulator according to the first n rows in the first matrix and the first n rows in the second matrix, where n is the system order of the manipulator and n is a positive integer; Constructing a cost function of the robotic arm according to the last L rows in the first matrix and the last L rows in the second matrix, where L is a predicted step length of the robotic arm and L is a positive integer; The historical data pairs of the first n moments of the robotic arm are used to predict the input data and output data of the last L moments of the robotic arm.

4. The method according to claim 3, wherein determining the first matrix and the second matrix comprises: Determine the historical input data from the i-1th to the NL-n+i-1th moments among the N moments as the i-th row of the first matrix; Determine the historical output data from the i-1th to the NL-n+i-1th moments among the N moments as the i-th row of the second matrix; Wherein, i is a positive integer.

5. The method according to claim 3, wherein the constraint function comprises a first constraint function, a second constraint function, a third constraint function and a fourth constraint function; The step of constructing the constraint function of the robot arm according to the first n rows in the first matrix and the first n rows in the second matrix comprises: constructing the first constraint function according to the slack variables, wherein the slack variables are used to describe measurement noise, and the first constraint function is used to reduce interference of the measurement noise on the prediction input data; Constructing the second constraint function according to the first n rows in the first matrix and the first n rows in the second matrix, wherein the second constraint function is used to describe the historical data pairs at the first n moments; constructing the third constraint function according to the first n rows in the first matrix, the first n rows in the second matrix and the balance data, wherein the third constraint function indicates that the manipulator approaches the ideal balance state over time; The fourth constraint function is constructed according to the slack variable, the first boundary point and the unit impulse response, wherein the first boundary point refers to the upper limit value of the slack variable, and the fourth constraint function is used to constrain the convergence of the process of predicting the prediction input data.

6. The method according to claim 3, wherein constructing the cost function of the robot arm according to the last L rows in the first matrix and the last L rows in the second matrix comprises: Constructing a first norm according to the last L rows in the first matrix and the last L rows in the second matrix; wherein the first norm includes: the second-order square norm of the historical input data at the last L moments of the N moments, and the second-order square norm of the historical output data at the last L moments of the N moments; Constructing a first penalty term according to a first penalty weight, a first boundary point and a first regularization term, wherein the first regularization term is a regularization penalty for the unit impulse response, the first penalty weight refers to a penalty weight for the first regularization term, the first boundary point refers to an upper limit value of a slack variable, and the slack variable is used to describe measurement noise; Constructing a second penalty term according to a second penalty weight and a second regularization term, wherein the second regularization term is a regularization penalty for the slack variable, and the second penalty weight refers to a penalty weight for the second regularization term; The cost function is constructed according to the first norm, the first penalty term and the second penalty term.

7. The method according to claim 2, wherein determining the predicted input data of the robot arm at the current time t according to the unit impulse response comprises: Determine a third matrix according to the historical input data pairs at the N moments, the third matrix comprising a first matrix and a second matrix, the first matrix comprising the historical input data in the historical data pairs at the N moments, and the second matrix comprising the historical output data in the historical data pairs at the N moments; Based on the unit impulse response and the third matrix, the predicted input data of the robotic arm at the current time t is calculated.

8. The method according to claim 7, wherein the step of calculating the predicted input data of the robot arm at the current time t according to the unit impulse response and the third matrix comprises: Constructing a dynamic equation of the robotic arm according to the motion parameters of the first object and the motion parameters of the robotic arm, wherein the motion parameters include at least one parameter affecting kinetic energy or potential energy, and the dynamic equation includes an underactuated equation and a driven equation, wherein the underactuated equation refers to an equation in which the input torque is 0, and the driven equation refers to an equation in which the input torque is not 0; Constructing a state space equation of the robotic arm according to the underactuated equation and the third matrix, wherein the state space equation is used to describe the motion state of the robotic arm; Calculate the predicted input data of the robot arm at the current time t according to the state space equation and the drive equation; Wherein, the motion parameters include at least one of the following: the length of the robotic arm, the specifications of the first object, the mass of the robotic arm, the mass of the first object, the moment of inertia of the robotic arm, and the moment of inertia of the first object.

9. The method according to claim 8, wherein the step of calculating the predicted input data of the manipulator at the current time t according to the state space equation and the drive equation comprises: Calculating an input torque according to the state space equation and the drive equation, wherein the input torque is used to control the joint rotation of the robotic arm; The input torque is determined as the predicted input data of the robot arm at the current time t.

10. The method according to claim 8, wherein constructing the dynamic equation of the robotic arm according to the motion parameters of the first object and the motion parameters of the robotic arm comprises: constructing an initial dynamics equation of the robotic arm according to the motion parameters of the first object and the motion parameters of the robotic arm; The initial kinetic equation is subjected to partial feedback linearization processing to obtain the kinetic equation, wherein the partial feedback linearization processing refers to approximate linearization processing of the initial kinetic equation.

11. The method according to claim 7, wherein the step of calculating the predicted input data of the robot arm at the current time t according to the unit impulse response and the third matrix comprises: Calculating the rotational angular velocity of the joint of the robotic arm according to the unit impulse response and the third matrix; The rotation angular velocity is determined as the predicted input data of the robot arm at the current time t.

12. The method according to claim 1, further comprising: For a current moment t, obtaining historical data pairs of multiple consecutive historical moments before the current moment t; The historical data pairs of the multiple consecutive historical moments are uniformly sampled to obtain the historical data pairs of the N moments.

13. The method according to claim 1, wherein obtaining historical data pairs at N moments comprises: For the j-th moment among N moments, obtaining the historical input data of the j-th moment, where j is a positive integer less than or equal to N; Determine the position and posture of the robotic arm after movement based on the historical input data at the j-th moment; Based on the tactile sensor, obtaining a first position of the first object on the robotic arm; Acquire the image of the first object at the j-th moment; acquiring a second position of the first object on the robotic arm according to the image of the first object; Determining a center of mass position of the first object according to the first position and the second position; The historical output data of the robotic arm at the jth moment is determined according to the posture of the robotic arm after movement based on the historical input data at the jth moment and the center of mass position of the first object.

14. A robot arm control device, the device comprising: an acquisition module, configured to acquire historical data pairs at N moments, wherein the historical data pairs include historical input data and historical output data, wherein the historical input data is used to control the movement of the robotic arm so that the first object maintains a balanced state on the robotic arm, and the historical output data is used to characterize the posture of the robotic arm and the first object after the movement of the robotic arm is controlled based on the historical input data, and N is a positive integer; A determination module, used to determine the predicted input data of the robot arm at the current time t according to the historical data pairs at the N time moments; A control module is used to control the movement of the robotic arm based on the predicted input data, so that the first object maintains balance during the movement of the robotic arm.

15. A robotic arm, comprising a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the robotic arm control method as described in any one of claims 1 to 13.

16. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, wherein the computer program is loaded and executed by a processor to implement the robot arm control method according to any one of claims 1 to 13.

17. A computer program product, comprising a computer program, wherein the computer program is stored in a computer-readable storage medium, and a processor reads and executes the computer program from the computer-readable storage medium to implement the robot arm control method according to any one of claims 1 to 13.

Images