Robot, method of operating the same, apparatus, storage medium, program product

Abstract

Embodiments of the present application provide a robot and an operating method, device, storage medium and program product thereof. The method comprises: determining, based on a joint dynamics model of the robot, a whole-body Jacobian matrix and at least one torque limit condition, a priority, a task parameter and a constraint parameter of a plurality of sub-tasks of a hierarchical quadratic programming task, with joint torque of a whole-body joint as an optimization variable; solving, in order from high to low of the priority of the sub-tasks, an optimal joint torque of each sub-task layer by layer by using a cumulative null space of a previous layer sub-task; and driving the whole-body joint of the robot based on the optimal joint torque finally solved to control the operation of the robot. The method realizes a balance between strict priority control and whole-body torque limitation, guarantees that a low-priority task will not interfere with a high-priority task by means of hierarchical solving of the cumulative null space, and significantly improves stability and predictability when multiple tasks are parallel.

Description

Technical Field

[0001] This application relates to the field of robotics, and more particularly to robots and their operating methods, devices, storage media, and program products. Background Technology

[0002] As robots are increasingly used in manufacturing, service, and human-robot collaboration, they often need to meet multiple operational requirements simultaneously. These requirements are often coupled or even conflicting, and are influenced by multiple factors, making torque level control of the robot's joints a highly challenging engineering task.

[0003] Based on this, embodiments of this application provide robots and their operating methods, devices, storage media, and program products to improve related technologies. Summary of the Invention

[0004] The purpose of this application is to provide a robot and its operating method, device, and storage medium to achieve torque control of all joints of the robot, support the processing of multiple tasks according to a specified priority, and be applicable to complex robot operation scenarios.

[0005] The objective of this application embodiment is achieved using the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a robot operation method, the method comprising: determining the priorities, task parameters, and constraint parameters of multiple subtasks of a hierarchical quadratic programming task based on the robot's joint dynamics model, whole-body Jacobian matrix, and at least one torque constraint, using the joint torques of the whole-body joints as optimization variables; solving for the optimal joint torque of each subtask layer by layer according to the order of priority of the subtasks from high to low, using the cumulative null space of the previous layer subtasks; the priority of the previous layer subtask is one level higher than the priority of the current layer subtask; and driving the whole-body joints of the robot based on the finally solved optimal joint torque to control the operation of the robot.

[0007] In some embodiments, the cumulative null space of the k-th subtask is determined based on the robot's inertia matrix, the task Jacobian matrix of the k-th subtask, and the cumulative null space of the (k-1)-th subtask, where k is an integer greater than 1.

[0008] In some embodiments, the process of determining the cumulative null space of the k-th subtask includes: projecting the cumulative null space of the (k-1)-th subtask onto the null space of the k-th subtask to obtain the cumulative null space of the k-th subtask; wherein, the null space of the k-th subtask is determined based on the task Jacobian matrix and the dynamic consistency transformation operator of the k-th subtask, the dynamic consistency transformation operator of the k-th subtask is constructed based on the robot's inertia matrix and the projected Jacobian matrix of the k-th subtask, and the projected Jacobian matrix of the k-th subtask is obtained by projecting the task Jacobian matrix of the k-th subtask onto the cumulative null space of the (k-1)-th subtask.

[0009] In some embodiments, the plurality of subtasks includes a highest priority operational space impedance task, the task parameters of which are determined based on the whole-body Jacobian matrix and its transpose, the inverse of the robot’s equivalent inertia matrix, and the desired end force; wherein the desired end force is calculated by a Cartesian impedance controller.

[0010] In some embodiments, the plurality of subtasks further include a desired joint torque task with a priority one level lower than the operational space impedance task, wherein the task parameters of the desired joint torque task are determined based on the null space of the operational space impedance task and the desired joint torque of the robot; wherein the desired joint torque is calculated by a null space controller.

[0011] In some embodiments, the at least one torque constraint includes a first type of torque constraint, which indicates an upper limit and a lower limit of joint torque. The constraint parameters include a first type of constraint parameters corresponding to the first type of torque constraint. The process of determining the first type of constraint parameters includes: constructing a first type of linear inequality based on the joint torque representation based on the first type of torque constraint and the joint dynamic compensation amount, so as to determine the first type of constraint parameters.

[0012] In some embodiments, the at least one torque constraint further includes one or more second-type torque constraint conditions, which are used to indicate joint limit parameters, environmental collision constraint parameters, or self-collision constraint parameters. The constraint parameters further include second-type constraint parameters corresponding to the second-type torque constraint conditions. The process of determining the second-type constraint parameters includes: constructing a second-order impedance inequality corresponding to the second-type torque constraint conditions, and transforming the second-order impedance inequality into a second-type linear inequality based on the joint torque expression based on the joint dynamics model, so as to determine the second-type constraint parameters.

[0013] In some embodiments, the first type of constraint parameters are applied to all subtasks, and the second type of constraint parameters are applied to subtasks other than the highest priority subtask.

[0014] In some embodiments, the joint dynamics model and / or the whole-body Jacobian matrix are updated based on sensor data of the robot, the sensor data including at least one of joint angles, joint velocities, end-effector forces, and end-effector torques.

[0015] Secondly, embodiments of this application provide a robot operation device, the device comprising: a task planning module, configured to determine the priority, task parameters, and constraint parameters of multiple subtasks of a hierarchical quadratic programming task based on the robot's joint dynamics model, whole-body Jacobian matrix, and at least one torque constraint, using the joint torques of the whole-body joints as optimization variables; a task solving module, configured to solve for the optimal joint torque of each subtask layer by layer according to the priority of the subtasks from high to low, using the cumulative null space of the previous layer subtasks; the priority of the previous layer subtask is one level higher than the priority of the current layer subtask; and an operation control module, configured to drive the whole-body joints of the robot based on the finally solved optimal joint torque, so as to control the operation of the robot.

[0016] Thirdly, embodiments of this application provide a robot, the robot including a control module, the control module being used to execute any of the methods described above.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0018] Fifthly, embodiments of this application provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the steps of any of the above methods.

[0019] This application provides a robot and its operating method, device, storage medium, and program product, realizing torque control of the robot's entire joints, supporting the processing of multiple tasks according to specified priorities, and suitable for complex robot operation scenarios. For example, using joint torque as the optimization variable, the overall goal is first broken down into several sub-tasks (such as end-effector force / pose tracking, posture and energy consumption preference, limit avoidance / collision avoidance, etc.) according to the operational requirements, and these sub-tasks are assigned strict priorities from high to low. Then, a hierarchical quadratic programming model is constructed. In the highest priority layer, the joint torque solution satisfying the main task is obtained, while at least one torque constraint is applied to ensure physical feasibility; the cumulative null space of this layer is calculated to characterize the unoccupied degrees of freedom. In the next priority layer, the current sub-task is optimized only within the aforementioned cumulative null space, so that the lower-level solution does not affect the higher-level effect. Furthermore, constraint parameters are uniformly or layered across all layers according to a preset strategy. The above process iterates layer by layer according to priority until all sub-tasks are solved, and the final optimal joint torque is used to drive the robot's entire joints to perform corresponding operations. This method achieves a balance between strict priority control and overall torque constraints. On the one hand, by leveraging hierarchical solutions with accumulated null spaces, low-priority tasks are guaranteed not to interfere with high-priority tasks, significantly improving the stability and predictability of multi-task parallelism. On the other hand, safety requirements such as actuator saturation, joint constraints, and (self)collisions are unified into torque domain constraints, ensuring that the solution results are inherently executable and satisfy hardware and environmental safety boundaries. This method has good adaptability to redundant degrees of freedom, automatically adjusting the posture to avoid risks without sacrificing the performance of the main task, and supports rapid reuse and scenario expansion by adding or deleting subtasks, improving the practicality and implementation efficiency of the relevant control framework. Attached Figure Description

[0020] The embodiments of this application are further described below with reference to the accompanying drawings and specific implementation details.

[0021] Figure 1 This is a flowchart illustrating a robot operation method provided in an embodiment of this application.

[0022] Figure 2 This is a structural block diagram of a robot operating device provided in an embodiment of this application.

[0023] Figure 3 This is a structural block diagram of a robot provided in an embodiment of this application.

[0024] Figure 4 This is a structural block diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the embodiments of this application.

[0026] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0027] Currently, most control designs for robot manipulation are based on position-level or velocity-level approaches, and the decoupled control of whole-body operations lacks multi-task-level decoupling, with limited constraints. Related torque control methods use analytical solutions based on null space mapping. When a robot performs multiple conflicting tasks (such as position tracking and force control), these methods struggle to achieve strict task priority ordering through simple control laws, often leading to performance degradation in critical tasks. Hardware constraints such as joint torque limits are difficult to integrate directly into the control framework, including self-collision and environmental collision constraints. Furthermore, relying on matrix pseudo-inverses or inverse kinematics calculations increases the computational burden of real-time control, especially in redundant robots. In scenarios with unknown contact forces, balancing the needs of rigid control and flexible interaction is difficult, affecting operational stability. Therefore, this application's embodiments mainly combine a quadratic programming task-layering framework with various torque control methods and constraints to improve the torque manipulation control effect of humanoid or multi-armed robots.

[0028] Some related solutions use analytical methods based on geometric pseudo-inverses and null spaces to quickly superimpose tasks. However, this approach struggles to handle constraints such as torque saturation and collisions, and can only guarantee "geometric non-interference," while dynamics still interfere with each other. Other related solutions shift towards velocity-level hierarchical QP (using joint velocities as variables) to unify multiple tasks and constraints. This method still requires additional steps for actuator torque and dynamically related safety boundaries, resulting in complex coupling. Still other related solutions use analytical manipulation space impedance to directly map to torque, but once multiple tasks and constraints are superimposed, the choice between pseudo-inverses and weighted summaries becomes fragile.

[0029] This application uses joint torque as an optimization variable and establishes a hierarchical quadratic programming (QP) framework to model the torque control problem as a hierarchical quadratic programming task. The hierarchical quadratic programming can include multiple subtasks with different priorities. High-priority subtasks can be hard-isolated from low-priority subtasks by accumulating null spaces. Furthermore, safety and hardware constraints can be uniformly converted into linear inequalities in the torque domain, ensuring that solutions at each level are executable and deployable. To improve numerical stability, the objective of the highest-priority subtask can be selected in a torque domain form without pseudo-inverses, which both fits the desired force generated by the operating space impedance and eliminates the need for explicit pseudo-inverse calculation. The embodiments of this application will be described in detail below.

[0030] See Figure 1 , Figure 1 This is a flowchart illustrating a robot operation method provided in an embodiment of this application.

[0031] In order to improve the relevant technology, this application provides a robot operation method, which includes steps S101 to S103.

[0032] Step S101: Based on the robot's joint dynamics model, whole-body Jacobian matrix, and at least one torque constraint, using the joint torques of the whole-body joints as optimization variables, determine the priority, task parameters, and constraint parameters of multiple sub-tasks of the hierarchical quadratic programming task.

[0033] Step S102: According to the order of priority of the subtasks from high to low, use the cumulative null space of the previous layer subtasks to solve the optimal joint torque of each subtask layer by layer; the priority of the previous layer subtask is one level higher than the priority of the current layer subtask.

[0034] Step S103: Based on the optimal joint torque obtained in the final solution, drive all joints of the robot to control the operation of the robot.

[0035] In some embodiments, the above method can be executed in a whole body controller.

[0036] The above embodiments do not limit the method of driving all joints of the robot. As an example, the optimal joint torque can be sent to each joint actuator to perform the corresponding joint control operation.

[0037] The above embodiments do not limit the joint dynamics model of the robot. As an example, the joint dynamics model can be as follows.

[0038]

[0039] Where q represents the joint position (e.g., the generalized joint position). For joint velocities (e.g., generalized joint velocities), Let M(q) be the joint acceleration (e.g., generalized joint acceleration), and M(q) be the robot's inertia matrix. Let g(q) be the Coriolis force term in the joint space, g(q) be the gravity term in the joint space, and τ be the joint torque. ext The external torque applied to the robot is not the joint torque τ actively applied by the robot's joint motors.

[0040] The joint dynamics model can be converted into an operation space dynamics model, for example, as shown below.

[0041]

[0042] Where Λ=(JM ―1 J T ) ―1 , where represents the inertial force term in the operational space. J is the whole-body Jacobian matrix, corresponding to the Jacobian matrix of all joints in the robot, used to map joint velocities to end-effector velocities. M ―1 Let J be the inverse matrix of M(q). T Let J be the transpose of J. x represents the generalized position of the end point in the operation space (e.g., 3D position or 6D pose), H is the Coriolis / centrifugal force term in the operation space, G is the gravity term in the operation space (gravitational equivalent generalized force), and F is the generalized force in the operation space. ext It refers to the generalized force exerted by external forces on the operational space.

[0043] For a non-redundant degree-of-freedom robot, the end-effector force F and joint torque τ in its operational space have the following relationship.

[0044]

[0045] in, The dynamically consistent Jacobian matrix under the inertial metric is labeled with the symbol ^ to identify the inertial metric. In the non-redundant case, it degenerates into J. for The transpose of the matrix is ​​indicated by the superscript T.

[0046] For a robot with redundant degrees of freedom, as an example, if the operand space is expanded to two tasks, where F1 is the Cartesian space desired force of the high-priority task and τ2 is the desired joint torque of the low-priority task, let F... 1|2 and Let F represent the force and acceleration generated by the low-priority task in the high-priority task, respectively. Decoupling can be achieved by mapping the second task into the cumulative null space N1 of the first task, thus making F...1|2 =0.

[0047]

[0048] Where I is the identity matrix, and J1 is the task Jacobian matrix of the first-level subtask. This is the dynamic consistency transformation operator for the first-level subtasks. yes The transpose of .

[0049] To extend the above solution process to QP solution for more sub-tasks, and to ensure that the sub-tasks are decoupled and do not interfere with each other, the embodiments of this application may define the dynamic consistency condition as follows.

[0050] J i (q)M(q) ―1 N j (q)=0(i <j)

[0051] Among them, J i (q) is the task Jacobian matrix associated with the i-th subtask, where i is a positive integer. N j (q) represents the cumulative null space corresponding to the j-th subtask, where j is a positive integer. The dynamic consistency condition is used to determine the cumulative null space N. j (q) Whether it is dynamically consistent, ensuring that the current layer subtask will not generate acceleration in the operation space of higher priority subtasks, so that the layer subtasks do not interfere with each other during the transition from transient to steady state.

[0052] The above embodiments do not limit the method of determining the cumulative null space of the k-th sub-task. In some embodiments, the cumulative null space of the k-th sub-task can be determined based on the robot's inertia matrix, the task Jacobian matrix of the k-th sub-task, and the cumulative null space of the (k-1)-th sub-task, where k is an integer greater than 1.

[0053] In some embodiments, the process of determining the cumulative null space of the k-th layer sub-task may include: projecting the cumulative null space of the (k-1)-th layer sub-task onto the null space of the k-th layer sub-task to obtain the cumulative null space of the k-th layer sub-task. The null space of the k-th layer sub-task is determined based on the task Jacobian matrix and the dynamic consistency transformation operator of the k-th layer sub-task. The dynamic consistency transformation operator of the k-th layer sub-task is constructed based on the robot's inertia matrix and the projected Jacobian matrix of the k-th layer sub-task. The projected Jacobian matrix of the k-th layer sub-task is obtained by projecting the task Jacobian matrix of the k-th layer sub-task onto the cumulative null space of the (k-1)-th layer sub-task.

[0054] For the k-th subtask, define the projection Jacobian matrix. The dynamic uniform transformation operator for the k-th layer is constructed based on the inertia matrix M(q) and the projected Jacobian matrix as shown below.

[0055]

[0056] The null space of the corresponding k-th layer can be represented as follows.

[0057]

[0058] Furthermore, the cumulative null space N of the k-th layer k The cumulative null space N of the (k-1)th layer k―1 The following recursive relationship is satisfied, thus satisfying the above dynamic consistency condition.

[0059] N k =N k― 1P k

[0060] As an example, assuming N0 = I, then the dynamic consistent transition operator of the first layer... It can be written as: It can be used to characterize the conversion relationship between operating space torque (also known as end torque) and joint torque.

[0061] It is important to note that the Cartesian space (also known as the operational space or task space) of an end effector refers to the space in robotics that uses a Cartesian coordinate system to describe the position and orientation of the robot's end effector. It is typically used to specify the specific task and motion path that the robot needs to complete. This differs from joint space, which uses the angles of the robot's joints to describe its state. Cartesian space directly reflects the position in the real world, such as the position and orientation of the robot's end effector in three-dimensional space.

[0062] This application uses a hierarchical QP method based on whole-body control to derive joint torques. Specifically, multiple subtasks are arranged in descending order of priority to form multi-level subtasks, which are solved through a series of cascaded quadratic programming (QP) problems. Unlike other methods, hierarchical QP utilizes the accumulated null space of high-priority subtasks to solve for the remaining available degrees of freedom, thereby reducing the number of variables and improving computational efficiency in each optimization level. The mathematical derivation of the hierarchical QP solution method is explained below.

[0063] First, optimize the objective and constraints. The k-th level subtask (i.e., the k-th priority subtask) is formalized as a constrained least squares problem. The optimization objective and constraints of the k-th level subtask are defined in the full variable space as follows.

[0064]

[0065] Here, arg is an abbreviation for argument (i.e., the independent variable). That is to make A k x k ―b k x when the minimum value is reached k The value of x. Assuming this problem is used to solve for joint torques, then x... k =τ. In the above formula, the optimization parameters include task parameters and constraint parameters. Task parameters may include, for example, A. k and b k Constraint parameters may include, for example, C. k d m,k and d M,k .

[0066] Based on the cumulative null space N of the (k-1)th layer k―1 By projecting, we obtain the following dimension-reduced expression.

[0067]

[0068]

[0069] in,

[0070]

[0071] Task parameter A k and b k and constraint parameter C k d m,k and d M,k It can be considered as the original parameters. These are the equivalent projection parameters of the corresponding parameters in the cumulative null space.

[0072] At this point, the goal of the optimization problem is to minimize the projected task error (through...). (Associating the results of the first k-1 layers). The constraint of this optimization problem is to limit the projected variables. Within the scope of the task, physical or task feasibility is guaranteed. The purpose of this step is to project the k-th layer subtask onto the unconstrained degrees of freedom of the first k-1 layers (i.e., the cumulative null space of the k-1 layers), ensuring that the solution of the high-priority task is not violated.

[0073] Next, we perform the recursive calculation of the cumulative null space. The cumulative null space of the (k-1)th layer is the accumulation of the constraints of the first (k-1)th layers. In geometric / Euclidean metric notation, the recursive relationship can be shown below.

[0074] N k-1 =N k-2 *Null(Ak-1 N k-2 )

[0075] Z k―1 =Null(A k―1 N k―2 ) is the null space basis for the (k-1)th layer task. The geometric / Euclidean metric notation is essentially the same as the dynamic consistent notation mentioned above, the difference being that the two notations use different metrics, the former using Euclidean and the latter using inertial weighting.

[0076] Assuming the initial layer (k=0) satisfies N0=I, x0=0, then N1=Null(A1)*N0, which is the identity matrix (all degrees of freedom are available when there are no constraints). N1 is the cumulative null space of the first layer constraint A1. Subsequent layers of constraints are added one by one, compressing the available degrees of freedom. The cumulative null space can be obtained through QR decomposition in this process.

[0077] To achieve the synthesis of the final solution, after optimization at the k-th layer, the solutions x from the first k-1 layers can be combined. k―1 Optimization results in the cumulative null space of the k-th layer The updated solution is obtained as the solution for the k-th layer, for example, as shown below.

[0078]

[0079] In the above embodiment, firstly, based on the robot's joint dynamics model and the whole-body Jacobian, and combined with at least one torque constraint, multiple sub-tasks of a hierarchical QP are constructed with joint torque as the optimization variable, and the priority, task parameters, and constraint parameters of each layer are determined. Secondly, the optimal joint torque is solved layer by layer according to priority from high to low, using the cumulative null space of the previous layer, so that low-priority tasks are optimized only within the remaining degrees of freedom without interfering with the upper layers. Thirdly, torque saturation, joint constraints, environment / self-collision, etc., are unified into torque domain inequalities, which take effect in all layers. Finally, the optimal joint torque obtained in the last solution drives the whole-body joints to complete the operation. Through this link, the robot achieves a balance between strict priority, unified torque constraints, and real-time executableness, improving the key problems of non-strict priority, difficulty in unifying constraints, unstable values, and non-executability in related schemes. It is suitable for diverse application scenarios of humanoid robots or dual-arm systems for whole-body force control and human-machine interaction.

[0080] In some embodiments, the plurality of subtasks may include the highest priority operational space impedance task, the task parameters of which may be determined based on the whole-body Jacobian matrix and its transpose, the inverse of the robot’s equivalent inertia matrix, and the desired end force; wherein the desired end force may be calculated by a Cartesian impedance controller.

[0081] In some embodiments, the plurality of subtasks may further include a desired joint torque task with a priority one level lower than the operational space impedance task. The task parameters of the desired joint torque task may be determined based on the null space of the operational space impedance task and the desired joint torque of the robot. The desired joint torque may be calculated by a null space controller.

[0082] For example, by combining the conversion relationship between the end effector torque and joint torque in the robot's operating space and the dynamic consistency condition, the following optimization objective can be established for torque control within the operating space.

[0083]

[0084] The existing control framework relies on the calculation of the pseudo-inverse of the Jacobian matrix, which increases the computational burden of real-time control, especially in redundant robots. This optimization objective function (which can be simply referred to as the optimization objective) does not require the calculation of the pseudo-inverse of the Jacobian matrix, thus saving computational resources.

[0085] For example, in order to achieve the task of manipulating space impedance and the task of desired joint torque (also known as the zero space task), a Cartesian impedance controller and a zero space controller can be added to the whole body controller.

[0086] One example of a Cartesian impedance controller is shown below.

[0087]

[0088] Λ d =Λ=QQ T

[0089] K d =QK x0 Q T

[0090]

[0091] Among them, F d For the expected force of the operating space, Λ d Let K be the desired inertia matrix. d Let D be the desired stiffness matrix. d This represents the desired damping matrix. The subscript "d" indicates the expected value of the parameter, and the superscript symbol indicates the first derivative. Denotes the second derivative. Q is the operation space quality normalization factor / square root factor, and K is the second derivative. x0 As the reference stiffness matrix, Here is the damping ratio matrix. It is the square root of the reference stiffness matrix.

[0092] In addition, the zero-space controller can be as follows.

[0093]

[0094] Where, τ p For reference joint torque (or zero-space preferred torque), K p Let q be the joint space stiffness (proportional) gain matrix. d For reference / desired joint position, D p This is the joint space damping (differential) gain matrix. For this type of joint task, the Jacobian matrix can be set as the identity matrix.

[0095] The primary task (e.g., the operational space impedance task) is derived from the desired force F above through the solution process of the first-level subtask in the hierarchical QP, resulting in joint moments. The zero-space controller provides a reference joint moment τ for attitude / energy preference. p It is executed only within the cumulative null space of the first-level subtask and does not interfere with the achievement of the goal of the first-level subtask.

[0096] Furthermore, singularity avoidance controllers can be added to the whole-body controller, and the above embodiments are not limited to this. Moreover, priority management can be performed according to various subtasks, with low-priority tasks being solved in the cumulative null space of high-priority tasks, ensuring that low-priority tasks do not affect the execution results of high-priority tasks.

[0097] In the aforementioned whole-body controller, various constraints can be added to obtain torque values ​​that meet user needs. In some embodiments, the at least one torque constraint may include a first type of torque constraint, which indicates an upper limit and a lower limit of joint torque. The constraint parameters include first-type constraint parameters corresponding to the first type of torque constraint. The process of determining the first-type constraint parameters includes: constructing a first-type linear inequality based on the joint torque representation based on the first-type torque constraint and the joint dynamics compensation amount to determine the first-type constraint parameters.

[0098] For example, a first-type torque constraint could be as follows.

[0099] τ min ≤τ≤τ max

[0100] Where, τ min τ max These are the lower limit and upper limit of joint torque, respectively.

[0101] Based on the first type of torque constraint and joint dynamics compensation, a first type of linear inequality based on the joint torque representation is constructed, for example as shown below.

[0102]

[0103] in, This is the joint dynamics compensation amount.

[0104] In practical applications, the first type of constraint parameters can be determined by constructing a one-sided form of Cτ≤d based on the first type of linear inequality, or by retaining the explicit upper and lower limits (e.g., l≤τ≤u). This will not be elaborated further here.

[0105] In addition to the first type of torque constraint conditions mentioned above, the embodiments of this application may also add physical constraint conditions such as joint limit constraints (corresponding to joint limit parameters), environmental collision constraints (corresponding to environmental collision constraint parameters, such as collision distance), and self-collision constraints (corresponding to self-collision constraint parameters). Unlike related solutions, the embodiments of this application can convert these physical constraint conditions into second type of torque constraint conditions through impedance form and dynamic constraints.

[0106] In some embodiments, the at least one torque constraint may further include one or more second-type torque constraint conditions, which are used to indicate joint limit parameters, environmental collision constraint parameters, or self-collision constraint parameters. The constraint parameters further include second-type constraint parameters corresponding to the second-type torque constraint conditions. The process of determining the second-type constraint parameters may include: constructing a corresponding second-order impedance inequality based on the second-type torque constraint conditions; and transforming the second-order impedance inequality into a second-type linear inequality based on the joint torque expression using the joint dynamics model, thereby determining the second-type constraint parameters.

[0107] In the above embodiments, the second type of torque constraint is derived from geometric / safety physical constraints (joint limit, environmental collision, self-collision, etc.). For example, a second-order impedance type inequality (which can also be equivalent to a second-order control barrier) is first set on the constraint variable, and then combined with robot joint dynamics to map it into a linear inequality about joint torque, which serves as a hard or soft constraint of QP.

[0108] It's important to clarify that hard constraints are those that must be strictly satisfied. That is, these constraints cannot be violated in the solution to a problem. Hard constraints are typically used to define the validity or legality of a solution; any valid solution must satisfy these hard constraints. Soft constraints, on the other hand, are those constraints that can be satisfied as much as possible but are not mandatory. Unlike hard constraints, soft constraints allow for some degree of violation. The goal of soft constraints is to make the solution as close as possible to the ideal state without violating the hard constraints. The solution will try to satisfy soft constraints, but some compromises are permissible.

[0109] Joint limit parameters, for example, indicate the safety range and margin of a single joint, and may also include parameters such as the upper limit of joint velocity. Environmental collision constraint parameters, for example, indicate the minimum safe distance from external objects (walls, table edges, human bodies, etc.), and the corresponding distance function. Self-collision constraint parameters, for example, indicate the minimum safe distance between different parts of the robot, the corresponding distance function, and the gradient. The second-order impedance inequality (also known as the second-order barrier / impedance safety constraint) can be illustrated as follows.

[0110]

[0111] Where z is the safety variable z(q) (e.g., distance margin or margin from the limit). For the damping ratio, ω is the convergence rate, ω>0. The physical meaning of this second-order impedance inequality can be seen as making the system exponentially maintain / recover to the safe set z≥0 in the manner of second-order impedance.

[0112] In the above embodiments, the second type of constraint parameters can be obtained online by using second-order impedance inequalities and dynamic models based on given physical constraint parameters (limits, minimum distances, etc.), and applied to each layer of QP solution in the form of linear inequalities in the joint moment domain. Thus, without introducing complex nonlinear constraints, geometric / safety constraints are unified into time-varying linear constraints (which can be hard or soft constraints), balancing real-time performance with strict priority given to safety and executability within the layered framework.

[0113] The above embodiments do not limit the subtasks to which the first type of constraint parameters and the second type of constraint parameters apply constraints. In some embodiments, the first type of constraint parameters and the second type of constraint parameters can be applied to all subtasks.

[0114] Alternatively, in other embodiments, the first type of constraint parameters can be applied to all subtasks, and the second type of constraint parameters can be applied to subtasks other than the highest priority subtask.

[0115] In the hierarchical QP framework, the highest priority subtask (e.g., the operational space impedance task) is first solved in the torque domain. Using joint torques as optimization variables and applying only Type I constraints defined by Type I constraint parameters to ensure physical feasibility, the most joint torques corresponding to the first-level subtasks are obtained. Next, lower priority subtasks are solved, i.e., within the remaining degrees of freedom, both Type I and Type II constraints are applied, where Type II constraints are defined by Type II constraint parameters. Since lower-level optimization is confined to the cumulative null space of the previous priority level, and the highest priority level does not introduce Type II constraints, lower levels only utilize the remaining degrees of freedom to satisfy geometric / safety requirements without interfering with the effects of higher-level tasks. Type I constraints apply uniformly across all levels, ensuring that solutions at any level do not exceed the bounds. This achieves a balance between the rigid reachability of the main task and the flexible reconciliation of safety constraints. Decoupling Type II constraints from the highest priority level significantly reduces the risk of infeasibility at the top level and the number of numerical conditions, improving real-time performance and solution stability. Meanwhile, the second type of constraint operates in the null space, enabling the system to automatically adjust redundant postures to avoid limits and collisions without sacrificing the main task (such as force / trajectory / contact stability), demonstrating strict priority and safe convergence. A unified first type of torque constraint runs through all layers, ensuring physical executableness and actuator safety. The second type of constraint, through impedance form and dynamic mapping, is transformed into a time-varying linear inequality, facilitating seamless integration with QP solvers and supporting the on-demand introduction of relaxation and weights. This achieves provable safety, degradable task execution, and high-frequency real-time control, suitable for humanoid / dual-arm full-body force control and human-machine interaction scenarios.

[0116] In some embodiments, the joint dynamics model and / or the whole-body Jacobian matrix are updated based on sensor data of the robot, the sensor data including at least one of joint angles, joint velocities, end-effector forces, and end-effector torques.

[0117] As an example, relevant parameters can be updated once per control cycle (e.g., 0.5–2 ms, corresponding to 500–2000 Hz), and a warm-start QP is employed to ensure real-time performance. This periodic parameter update ensures that the task mapping and null space projection remain consistent with the instantaneous attitude / velocity, reducing priority leakage and tracking deviations caused by model lag. The real-time update mechanism improves the accuracy of impedance parameter matching and force compensation, resulting in higher trajectory / force tracking accuracy, stronger interaction compliance, and more stable whole-body control, while facilitating consistent control performance across different platforms and load variations.

[0118] The above embodiments utilize a hierarchical QP optimization approach for torque control within the robot's manifold. The designed whole-body controller supports the addition of various sub-task controllers and differentiates between high and low priorities, ensuring that low-priority tasks do not affect high-priority tasks. The process of multi-task solution using hierarchical OP is derived in detail above. Through the rational design of the optimization objective, the calculation of the Cartesian space impedance task controller is simplified. The manifold impedance task controller applies dynamic consistency conditions to adjust the Jacobian matrix mapping relationship, ensuring that sub-tasks at different levels do not generate acceleration in the manifold of higher-priority sub-tasks, thus preventing interference between hierarchical sub-tasks during the transition from transient to steady state. This whole-body controller supports the addition of constraints such as joint torque limits, joint limits, singularity avoidance, environmental collisions, and self-collisions, exhibiting good scalability and a wide range of applications.

[0119] See Figure 2 , Figure 2 This is a structural block diagram of a robot operating device provided in an embodiment of this application.

[0120] This application also provides a robot operating device, which includes a task planning module, a task solving module, and an operation control module.

[0121] The task planning module is used to determine the priority, task parameters, and constraint parameters of multiple sub-tasks of the hierarchical quadratic programming task based on the robot's joint dynamics model, whole-body Jacobian matrix, and at least one torque constraint, with the joint torques of the whole-body joints as optimization variables.

[0122] The task solving module is used to solve the optimal joint torque of each subtask layer by layer, according to the priority of the subtasks from high to low, using the cumulative null space of the previous layer subtasks; the priority of the previous layer subtask is one level higher than the priority of the current layer subtask.

[0123] The operation control module is used to drive all the joints of the robot based on the optimal joint torque obtained at the last solution, so as to control the operation of the robot.

[0124] See Figure 3 , Figure 3 This is a structural block diagram of a robot provided in an embodiment of this application.

[0125] This application also provides a robot, which includes a control module for executing any of the methods described above.

[0126] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0127] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0128] The computer program product may be a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the computer program product of the embodiments of this application is not limited thereto, and the computer program product may be any combination of one or more computer-readable media.

[0129] See Figure 4 , Figure 4 This is a structural block diagram of a computer device provided in an embodiment of this application.

[0130] This application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above methods.

[0131] The embodiments of this application do not limit the computer device, which may be, for example, a local computer device, a cloud computer device, a distributed computer device, etc.

[0132] The computer device may include: a memory 110, a processor 120, and a communication interface 130. The memory 110, the processor 120, and the communication interface 130 are connected through internal connection paths.

[0133] The memory 110 is used to store computer programs, which in some implementations may include code for implementing the methods of the embodiments of this application.

[0134] The processor 120 executes the computer program stored in the memory 110 to control the communication interface 130 to receive input data and information, and output operation results and other data. In some implementations, when the solutions of the embodiments of this application are implemented by software or firmware, the computer program used to implement the solutions of the embodiments of this application can be stored in the processor 120 and executed by the processor 120.

[0135] The memory 110 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM). It should be noted that the memory 110 described herein is intended to include, but is not limited to, any memory of these and other suitable types. As an example, the memory 110 includes random access memory (RAM), cache memory, and read-only memory (ROM). The memory 110 stores a computer program that can be executed by processor 120, causing processor 120 to implement the steps of any of the methods described above.

[0136] The processor 120 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor 120 can be any conventional processor.

[0137] In implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware in the processor 120 or by instructions in software form. The method disclosed in the embodiments of this application can be directly implemented by the hardware processor, or by a combination of hardware and software modules in the processor 120. The software modules can be located in mature storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in the memory 110, and the processor 120 reads the information in the memory 110 and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are not provided here.

[0138] In some implementations, in addition to the hardware units described above, computer devices may also include software modules, such as operating systems, basic input / output systems (BIOS), and application software.

[0139] An operating system is used to manage the hardware and / or software resources of a computer device; it is the kernel and foundation of the computer. The operating system handles fundamental tasks such as managing and configuring memory, determining the priority of system resource allocation, controlling input and output devices, operating the network, and managing the file system. To facilitate user operation, most operating systems provide a user interface for interaction with the system.

[0140] The BIOS is used to perform hardware initialization during the power-on boot phase and to provide runtime services for the operating system and applications. In some implementations, the BIOS can also monitor and display processor temperature and execute temperature protection strategies.

[0141] Application software, also known as an application program, can be understood as software written for a specific user application purpose, and is one of the main categories of computer software. For example, application software can be a program used to achieve purposes such as power control and temperature management.

[0142] It is understood that the specific examples in this application are only intended to help those skilled in the art better understand the implementation of this application, and are not intended to limit the scope of protection of this application.

[0143] It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.

[0144] It is understood that the various implementation methods described in this application can be implemented individually or in combination, and this application does not limit them.

[0145] Unless otherwise stated, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0146] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0147] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the embodiments described above can be referred to the corresponding processes in other embodiments, and will not be repeated here.

[0148] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0149] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the technical solution in this application, depending on actual needs.

[0150] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0151] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, essentially, or the part that contributes to related technologies, or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A robot operation method, characterized in that, The method includes: Based on the robot's joint dynamics model, whole-body Jacobian matrix, and at least one torque constraint, the priorities, task parameters, and constraint parameters of multiple sub-tasks of the hierarchical quadratic programming task are determined using the joint torques of the whole-body joints as optimization variables. According to the order of priority of the subtasks from high to low, the optimal joint torque of each subtask is solved layer by layer using the cumulative null space of the previous layer subtasks; the priority of the previous layer subtask is one level higher than the priority of the current layer subtask. Based on the optimal joint torque obtained in the final solution, the joints of the robot are driven to control the operation of the robot. Wherein, the at least one torque limiting condition includes a first type of torque limiting condition, the first type of torque limiting condition is used to indicate the upper limit of joint torque and the lower limit of joint torque, and the constraint parameters include a first type of constraint parameters corresponding to the first type of torque limiting condition; The at least one torque limiting condition further includes one or more second-type torque limiting conditions, which are used to indicate joint limit parameters, environmental collision constraint parameters, or self-collision constraint parameters. The constraint parameters further include second-type constraint parameters corresponding to the second-type torque limiting conditions. The process of determining the second type of constraint parameters includes: constructing the corresponding second-order impedance inequality based on the second type of torque constraint conditions, and transforming the second-order impedance inequality into a second type of linear inequality based on the joint torque expression based on the joint dynamics model, so as to determine the second type of constraint parameters; The first type of constraint parameters applies to all subtasks, while the second type of constraint parameters applies to all subtasks except the one with the highest priority.

2. The robot operation method according to claim 1, characterized in that, The cumulative null space of the k-th subtask is determined based on the robot's inertia matrix, the task Jacobian matrix of the k-th subtask, and the cumulative null space of the (k-1)-th subtask, where k is an integer greater than 1.

3. The robot operation method according to claim 2, characterized in that, The process of determining the cumulative null space of the k-th subtask includes: projecting the cumulative null space of the (k-1)-th subtask onto the null space of the k-th subtask to obtain the cumulative null space of the k-th subtask. The null space of the k-th sub-task is determined based on the task Jacobian matrix and dynamic consistency transformation operator of the k-th sub-task. The dynamic consistency transformation operator of the k-th sub-task is constructed based on the robot's inertia matrix and the projected Jacobian matrix of the k-th sub-task. The projected Jacobian matrix of the k-th sub-task is obtained by projecting the task Jacobian matrix of the k-th sub-task onto the cumulative null space of the (k-1)-th sub-task.

4. The robot operation method according to claim 1, characterized in that, The multiple subtasks include the highest priority operational space impedance task, whose task parameters are determined based on the whole-body Jacobian matrix and its transpose, the inverse of the robot's equivalent inertia matrix, and the desired end force. The desired end force is calculated using a Cartesian impedance controller.

5. The robot operation method according to claim 4, characterized in that, The multiple subtasks also include a desired joint torque task with a priority one level lower than the operational space impedance task. The task parameters of the desired joint torque task are determined based on the null space of the operational space impedance task and the desired joint torque of the robot. The desired joint torque is calculated using a zero-space controller.

6. The robot operation method according to claim 1, characterized in that, The process of determining the first type of constraint parameters includes: constructing a first type of linear inequality based on the joint torque expression based on the first type of torque constraint conditions and joint dynamic compensation amount, so as to determine the first type of constraint parameters.

7. The robot operation method according to any one of claims 1 to 6, characterized in that, The joint dynamics model and / or the whole-body Jacobian matrix are updated based on sensor data of the robot, including at least one of joint angle, joint velocity, end effector force, and end effector torque.

8. A robot, characterized in that, The robot includes a control module for performing the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

Images