Robot control method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a robot control method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps that in response to a task instruction, an expected pose of the tail end of a mechanical arm of the robot is obtained; taking an optimization target of driving a chassis of the robot to move towards a preset reference parking pose while driving the tail end of the mechanical arm to move towards the expected pose, and determining an intermediate state point sequence of the robot; planning a chassis moving path of the robot according to chassis poses in the intermediate state point sequence; and aiming at each chassis path point on the chassis moving path, searching a collision-free path section for enabling the tail end of the mechanical arm to move to a mechanical arm joint pose corresponding to the current chassis path point, and obtaining a target whole-body motion path for the robot to move to an expected pose. The efficiency and success rate of robot control can be improved.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a robot control method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of robotics technology, wheeled manipulators, which possess mobility and maneuverability, have gradually become a research and application hotspot. These robots typically consist of a mobile chassis and one or more robotic arms mounted on the chassis, aiming to integrate the long-distance movement capabilities of mobile robots with the dexterous maneuvering capabilities of robotic arms, thereby achieving end-to-end automated operations in scenarios such as warehousing and logistics, indoor services, and security inspection.

[0003] Traditional wheelbarrow robot task planning typically employs a sequential, separate planning architecture. In this architecture, the planning process is divided into two sequential stages: First, the planning system, based on the task objective, plans a path for the mobile chassis from its starting point to a pre-defined work point (i.e., a stopping point), and controls the chassis to move to that point and come to a complete stop. Then, while the chassis remains stationary, a separate collision-free trajectory is planned for the robotic arm from its initial state to the target work state (e.g., a grasping pose). This traditional planning process is cumbersome and inefficient, and in narrow or complex working environments, it is highly susceptible to task failure due to the inability to find a suitable stopping point.

[0004] Therefore, a robot control solution that can improve the success rate and efficiency of operations is needed. Summary of the Invention

[0005] Therefore, it is necessary to provide a robot control method, device, computer equipment, computer-readable storage medium, and computer program product that can improve the success rate and efficiency of operations, in response to the above-mentioned technical problems.

[0006] In a first aspect, this application provides a robot control method, the method comprising:

[0007] In response to task instructions, obtain the desired pose for the robot's end effector.

[0008] With the optimization objective of simultaneously driving the robot's end effector to move towards the desired pose and driving the robot's chassis to move towards a preset reference parking pose, a sequence of intermediate state points of the robot is determined; each intermediate state point represents a full-body state of the robot; the full-body state includes the chassis pose and the robot's joint pose.

[0009] Based on the chassis pose in the intermediate state point sequence, plan the chassis movement path of the robot;

[0010] For each chassis path point on the chassis movement path, search for a collision-free path segment that allows the end effector of the robotic arm to move to the joint pose of the robotic arm corresponding to the current chassis path point.

[0011] By combining the collision-free path segments found at each chassis path point, the target full-body motion path for the robot to move to the desired pose is obtained.

[0012] In some embodiments, determining the robot's intermediate state point sequence with the optimization objective of simultaneously driving the robot arm's end effector to move towards the desired pose and driving the robot's chassis to move towards a preset reference parking pose includes:

[0013] Based on the optimization objective, a differential inverse kinematics model corresponding to the robot is constructed; wherein, the differential inverse kinematics model is used to solve a set of joint motion velocities of the chassis and the robotic arm, such that the first error between the current pose of the robotic arm end and the desired pose and the second error between the current pose of the chassis and the reference parking pose tend to zero.

[0014] The differential inverse kinematics model is transformed into a quadratic programming model; wherein, the objective function of the quadratic programming model includes minimizing a weighted sum; the weighted sum is obtained by weighting and summing the first velocity residual obtained from the first error mapping and the second velocity residual obtained from the second error mapping; the first weight corresponding to the first velocity residual is greater than the second weight corresponding to the second velocity residual;

[0015] Solve the quadratic programming model to obtain the joint motion velocity;

[0016] The robot's overall state is updated based on the integral of the joint motion velocity, and the intermediate state point sequence is generated iteratively.

[0017] In some embodiments, the objective function of the quadratic programming model further includes a regularization term, which is used to induce the joint motion velocity to tend to drive the robot's joint pose to a preset reference joint pose.

[0018] In some embodiments, searching for a collision-free path segment that allows the robotic arm end effector to move to the joint pose corresponding to the current chassis path point for each chassis path point on the chassis movement path includes:

[0019] For each of the chassis path points, the collision-free path is searched forward and backward, starting from the current joint pose of the robotic arm and the joint pose of the robotic arm corresponding to the current chassis path point, respectively.

[0020] When the collision-free paths obtained from the forward search and backward search are connected in the joint pose space of the robotic arm, the total path obtained is taken as the collision-free path segment corresponding to the current chassis path point.

[0021] In some embodiments, after searching for a collision-free path segment that moves the robotic arm end effector to the joint pose corresponding to the current chassis path point for each chassis path point on the chassis movement path, the method further includes:

[0022] For the chassis path point for which no collision-free path segment was found, obtain the first full-body state corresponding to the chassis path point for which the search failed and the second full-body state corresponding to the chassis path point for which the most recent collision-free path segment was successfully searched.

[0023] Using the first full-body state as the starting state and the second full-body state as the target state, the smooth trajectory of the coordinated motion of the chassis and the robotic arm from the starting state to the target state is solved to obtain the collision-free path segment corresponding to the chassis path point where the search failed.

[0024] In some embodiments, the objective of solving the smooth trajectory includes at least one of the following: a trajectory tracking term to ensure trajectory continuity, an inequality constraint term to achieve obstacle avoidance and physical limitation, and a control energy term to improve trajectory smoothness.

[0025] Secondly, this application also provides a robot control device, the device comprising:

[0026] The acquisition module is used to acquire the desired pose of the robot's end effector in response to task instructions;

[0027] The determination module is used to determine the sequence of intermediate state points of the robot with the optimization objective of simultaneously driving the robot arm end effector to move towards the desired pose and driving the robot chassis to move towards a preset reference parking pose; each intermediate state point represents a whole-body state of the robot; the whole-body state includes the chassis pose and the robot arm joint pose.

[0028] The planning module is used to plan the chassis movement path of the robot based on the chassis pose in the intermediate state point sequence.

[0029] The search module is used to search for collision-free path segments that allow the end effector of the robotic arm to move to the joint pose of the robotic arm corresponding to the current chassis path point for each chassis path point on the chassis moving path.

[0030] The combination module is used to combine the collision-free path segments searched at each chassis path point to obtain the target whole-body motion path for the robot to move to the desired pose.

[0031] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps included in any of the foregoing robot control method embodiments.

[0032] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps included in any of the aforementioned robot control method embodiments.

[0033] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps included in any of the aforementioned robot control method embodiments.

[0034] The aforementioned robot control method, apparatus, computer equipment, computer-readable storage medium, and computer program product, in response to task instructions, acquire the desired pose of the robot's end effector; with the optimization objective of simultaneously driving the robot's chassis to move towards a preset reference parking pose while driving the end effector to move towards the desired pose, determine a sequence of intermediate state points for the robot; each intermediate state point represents a full-body state of the robot; the full-body state includes the chassis pose and the robot's joint pose; based on the chassis pose in the sequence of intermediate state points, plan the chassis movement path of the robot; for each chassis path point on the chassis movement path, search for a collision-free path segment that allows the end effector to move to the robot's joint pose corresponding to the current chassis path point; combine all the collision-free path segments searched for at each chassis path point to obtain the target full-body motion path for the robot to move to the desired pose. This invention generates a sequence of intermediate state points that are strongly correlated with the states of the chassis and the robotic arm by constructing a whole-body collaborative optimization target. This reduces the dependence on the accuracy of the chassis's preset docking points. Furthermore, hierarchical planning is used to efficiently verify and refine the sequence of intermediate state points, enabling the robot to achieve synchronous preparation for movement and operation, which significantly improves task execution efficiency. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating a robot control method in one embodiment;

[0037] Figure 2 This is a schematic diagram of robotic arm path planning in a robot control method in one embodiment;

[0038] Figure 3 This is a flowchart illustrating the robot control method in another embodiment;

[0039] Figure 4 This is a schematic diagram of the architecture of a robot control system on which another embodiment of the robot control method is based;

[0040] Figure 5 This is a schematic diagram of the path planning results in a robot control method in another embodiment;

[0041] Figure 6 This is a structural block diagram of a robot control device in one embodiment;

[0042] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0044] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0045] Before describing the embodiments of the present invention, the related technologies and their problems will be explained:

[0046] Currently, commercially available robots primarily rely on navigation capabilities to perform tasks such as cleaning and delivery. However, due to technological limitations, purely mobile robots require assistance and coordination to complete tasks. For example, delivery robots need human assistance to place items into and retrieve them from the robot's delivery compartment; robots need IoT modifications to open and close gates. Therefore, more and more robots are being equipped with robotic arms to increase their operational capabilities, thus enabling them to complete tasks fully end-to-end. However, the addition of robotic arms significantly increases the dimensionality of the robot's motion space. For instance, a typical chassis has only two degrees of freedom, controlled by linear velocity and angle. But an arm has at least six degrees of freedom, and dual arms can have more than a dozen. Obstacle avoidance planning in high-dimensional spaces presents a significant challenge. Traditional solutions, to avoid planning problems in high-dimensional spaces, typically plan the robotic arm and chassis movements separately—that is, plan the chassis movement first, and then plan the arm movement separately after the chassis stops. The problems with this approach include: First, because of the separate planning, the robot's execution efficiency is lower. For example, if a robot needs to move from a distant location to a table to get a glass of water... Using the current planning method, the robot needs to navigate to the vicinity of the table, stop, and then plan the robotic arm's movements, resulting in a longer trajectory path. Second, planning the robotic arm and chassis separately reduces the robot's overall flexibility, making it more prone to unsolvable situations in narrow areas, leading to planning failures. Third, in planning the chassis and robotic arm separately, the chassis's stopping point often needs to be given a priori. If the priori information is inaccurate, the robot may need to adjust the chassis position multiple times before successfully completing the task, further reducing task execution efficiency.

[0047] In summary, current sequential separation planning methods suffer from low robot execution efficiency due to the fragmented planning of the chassis and robotic arm. The robot cannot begin operation until the chassis has completely come to a stop, failing to achieve "pre-adjustment" during movement and resulting in a long overall task cycle. Secondly, this method relies on prior settings for the chassis docking point. The rationality of this preset point directly determines the success of subsequent robotic arm planning. Inappropriate docking point selection—for example, too far from the target making it unreachable, or too close or at an unfavorable angle causing collisions—will lead to planning failure. In such cases, the system often requires manual intervention or higher-level logic to re-designate the docking point and replan.

[0048] Therefore, a robot control solution with higher efficiency and success rate is needed.

[0049] The robot control method provided in this application can be applied to robots, which can be wheeled robots, such as delivery robots with robotic arms and chassis.

[0050] In one exemplary embodiment, such as Figure 1 As shown, a robot control method is provided. Taking the application of this method to a wheeled robot as an example, the method includes the following steps:

[0051] Step 202: In response to the task instruction, obtain the desired pose for the robot's end effector.

[0052] In this context, the task instruction refers to the external command that triggers the robot to perform a specific operation. The desired pose represents the target spatial position and orientation that the end effector of the robotic arm (such as the robot's dexterous hand) needs to achieve in the world coordinate system. The desired pose can be directly contained in the task instruction or obtained through analysis of the task instruction by a relevant perception system. For example, the task instruction could be "grab a water glass on the table." Correspondingly, the desired pose is the three-dimensional coordinates (X, Y, Z) and grasping direction (e.g., the orientation represented by quaternions) of a grasping point on the handle of the water glass, as identified by the vision system. By responding to the task instruction, the target of the robot's subsequent motion planning task is established. It can be understood that when the robot is close to the object being manipulated, the desired pose can be obtained through visual analysis. That is, when the robot approaches the target object, the collaborative planning of the robot's chassis and robotic arm in subsequent steps is initiated. However, if the robot is too far from the object, the robot can first move its chassis to quickly approach the object, thereby improving the robot's task processing efficiency while saving motion planning resources.

[0053] Step 204: With the optimization objective of simultaneously driving the robot's end effector to move toward the desired pose and driving the robot's chassis to move toward a preset reference parking pose, determine the robot's intermediate state point sequence; each intermediate state point represents a full-body state of the robot; the full-body state includes the chassis pose and the robot's joint pose.

[0054] The reference parking pose represents the preset, roughly expected final position and orientation of the chassis. The intermediate state sequence consists of multiple intermediate state points. Each intermediate state point corresponds to a full-body state of the robot. The full-body state simultaneously includes the chassis pose (e.g., planar coordinates X, Y and orientation angle θ) and the poses of all joints of the robotic arm at that moment / position. This sequence depicts an idealized motion trajectory that the robot should follow from the initial state to the completion of the task.

[0055] Unlike related technologies that require manually and isolated pre-specifying a precise chassis docking point and completely separate the motion planning tasks of the chassis and robotic arm, this invention unifies the robotic arm task (reaching the desired pose) and the chassis task (approaching the reference parking pose) into a single mathematical problem. The optimization objective includes two parallel and potentially competing sub-objectives: the first sub-objective is to drive the robotic arm end effector to approach the desired pose; the second sub-objective is to drive the chassis to approach a preset reference parking pose (e.g., an ideal working position one meter away from the target table, directly facing the table). This achieves a one-time, collaborative generation of a rough draft of the robot's full-body motion (i.e., a sequence of intermediate state points). Specifically, each point in the intermediate state point sequence is not isolated; a smooth correlation is established between consecutive points through the optimization process, jointly describing an ideal trajectory from the starting point to the ending point, where the chassis and robotic arm states evolve synchronously. This provides a series of strongly correlated path points for subsequent planning, thereby improving the feasibility and synergy of subsequent planning.

[0056] Step 206: Based on the chassis pose in the intermediate state point sequence, plan the chassis movement path of the robot.

[0057] In this process, all chassis poses are extracted from the intermediate state point sequence generated in the preceding steps. Using a preset path planning algorithm (such as A* algorithm, Dijkstra's algorithm, etc.), a collision-free, passable, and continuous chassis motion path is planned in the environment map. This chassis motion path allows the chassis to sequentially pass through or approach these extracted chassis poses. The series of discrete points included in the chassis movement path are the chassis path points, thereby transforming the chassis portion of the aforementioned motion draft into a specific path that the robot can safely execute in the physical environment. Optionally, in this embodiment of the invention, the chassis path planning can employ the Hybrid A* algorithm.

[0058] Step 208: For each chassis path point on the chassis movement path, search for a collision-free path segment that allows the end effector of the robotic arm to move to the joint pose of the robotic arm corresponding to the current chassis path point.

[0059] The current chassis path point represents the target state of the paired robotic arm, calculated collaboratively with the chassis state at that path point. A collision-free path segment refers to a short, collision-free trajectory from the robotic arm's current joint pose to a target joint pose that matches the chassis pose, provided the chassis is fixed at the current path point pose. The robotic arm's path planning can employ either the RRT (Rapidly-exploring Random Tree Star) or RRT*-CONNNET algorithm; this embodiment of the invention does not limit the specific algorithms used.

[0060] The search for the robotic arm's state is based on the pairing state between the chassis and the robotic arm executed at intermediate state points. This ensures that each motion target of the robotic arm is not arbitrarily set, but is matched with the chassis position after global optimization. This increases the probability of successful planning of the robotic arm at each path point and avoids the problems of the robotic arm "not being able to reach" or "colliding" due to improper selection of docking points in related technologies.

[0061] Specifically, for each chassis path point on the chassis movement path obtained in the aforementioned steps, the target robotic arm joint pose corresponding to that path point is read. Under the constraint of this chassis pose, a collision-free path is searched in the candidate joint pose space of the robotic arm, moving from the current joint pose of the robotic arm to the aforementioned target joint pose. This path is a collision-free path segment. This ensures that at each key point along the chassis path, the robotic arm can safely adjust from one form to the next. The candidate joint pose space refers to the data space composed of all achievable joint poses of the robotic arm. Each specific form (configuration) of the robotic arm corresponds to a state point in this space. In this embodiment of the invention, path search and collision detection are performed in this space.

[0062] Step 210: Combine the collision-free path segments found at each chassis path point to obtain the target full-body motion path for the robot to move to the desired pose.

[0063] Among them, the collision-free path segments searched at each chassis path point can be combined according to time sequence or spatial topological relationship to obtain the target whole-body motion path of the robot moving to the desired pose.

[0064] For example, such as Figure 2As shown, the chassis has path points 0, 1, 2, 3, 4…N. For chassis path point 1, a robot arm motion path segment within the chassis motion segment 0-1 is searched; that is, a short, collision-free trajectory of the robot arm moving from its current state at 0 to the target state at 1. Similarly, for chassis path point 2, a robot arm motion path segment within the chassis motion segment 1-2 is searched; that is, a short, collision-free trajectory of the robot arm moving from its current state at 1 to the target state at 2, and so on. When collision-free path segments are successfully found for all chassis path points, these scattered path segments are aligned and combined with the chassis movement path in time or space, ultimately outputting a complete target full-body motion path that includes both the chassis trajectory and the robot arm trajectory. The robot controller can then execute this path, achieving fully automated operation from the starting point to the completion of the grasping task. By executing the target's full-body motion path, the chassis and robotic arm can achieve smooth and coordinated movement, ultimately reaching the desired pose accurately and safely. If any point fails to be planned, it means that the robot cannot safely complete the posture switch at that point, and the entire path is not feasible.

[0065] Compared to related technologies that first plan the chassis movement to a fixed point and stabilize before separately planning the robotic arm's motion, this invention breaks down the temporal and planning isolation between the chassis and robotic arm movements when planning the robot's whole-body motion path. By setting dual optimization objectives—chassis pose optimization and robotic arm pose optimization—whole-body collaborative optimization of the robot is achieved, avoiding robotic arm planning failures caused by improper chassis docking point selection and solving the problem of excessive reliance on prior positions in related technologies. Furthermore, this invention employs a strategy of first generating a collaborative draft and then refining it through layered verification, offering greater flexibility than purely separate planning in complex scenarios. Through collaborative optimization, the final chassis pose and even the pose throughout the entire movement process are automatically adjusted to adapt to the robotic arm's operational needs, significantly reducing the planning failure rate in narrow or complex scenarios.

[0066] In some embodiments, determining the robot's intermediate state point sequence with the optimization objective of simultaneously driving the robot arm's end effector to move towards the desired pose and driving the robot's chassis to move towards a preset reference parking pose includes:

[0067] Based on the optimization objective, a differential inverse kinematics model corresponding to the robot is constructed; wherein, the differential inverse kinematics model is used to solve a set of joint motion velocities of the chassis and the robotic arm, such that the first error between the current pose of the robotic arm end and the desired pose and the second error between the current pose of the chassis and the reference parking pose tend to zero.

[0068] The differential inverse kinematics model is transformed into a quadratic programming model; wherein, the objective function of the quadratic programming model includes minimizing a weighted sum; the weighted sum is obtained by weighting and summing the first velocity residual obtained from the first error mapping and the second velocity residual obtained from the second error mapping; the first weight corresponding to the first velocity residual is greater than the second weight corresponding to the second velocity residual;

[0069] Solve the objective function corresponding to the quadratic programming model to obtain the joint motion velocity;

[0070] The robot's overall state is updated based on the integral of the joint motion velocity, and the intermediate state point sequence is generated iteratively.

[0071] The differential inverse kinematics model refers to a mathematical model that treats the robot chassis and robotic arm as an integrated system, and inversely calculates the required motion velocities of each joint (including virtual joints of the chassis) from the desired end effector motion (pose change). Compared to related technologies that solve the motion of the chassis and robotic arm independently and sequentially at the position level, this invention constructs a unified differential inverse kinematics model at the velocity level, which can naturally describe and plan the continuous and synchronous motion of the chassis and robotic arm, providing a theoretical framework for achieving collaborative operation of "moving and operating simultaneously," thereby overcoming the temporal isolation problem in related technologies.

[0072] Specifically, the construction process of the differential inverse kinematics model can include: firstly, calculating the first error between the current pose and the desired pose of the robotic arm's end effector, and the second error between the current pose of the chassis and the reference parking pose. Then, to drive these errors to converge, a joint motion velocity vector can be solved (this vector simultaneously includes the linear velocity and angular velocity of the chassis, and the angular velocities of each joint of the robotic arm). The relationship between this velocity and the error can be determined by the robot's kinematic equations. Specifically, the desired rate of change of error and the required joint velocity are linked through the Jacobian matrix, i.e., J·v = -α·e, where J is the Jacobian matrix, v is the joint motion velocity to be determined, e is the synthesized error vector, and α is a positive definite gain matrix. Therefore, constructing the differential inverse kinematics model can establish the mathematical relationship of the joint motion velocity v of the robotic arm, providing theoretical support and computational basis for the subsequent solution of the robotic arm's motion trajectory, enabling a search in the velocity space along the direction of -α·e, thereby driving the error to decay exponentially to zero.

[0073] Quadratic programming is a mathematical optimization model suitable for minimizing quadratic objective functions with linear constraints. Considering that the differential inverse kinematics equation J · v = -α · e is usually overdetermined or underdetermined, and physical constraints such as joint velocity limits need to be taken into account, direct solution often fails to yield a feasible or optimal solution. Therefore, in this embodiment of the invention, it is transformed into a quadratic programming model, thereby robustly solving for the feasible velocity command that best matches the desired motion trend under strict physical constraints.

[0074] Specifically, the degree to which the aforementioned differential inverse kinematics relation J · v + α · e ≈ 0 is achieved can be used as the optimization objective. For example, the first velocity residual is defined as J_arm · v + α_arm · e_arm, which characterizes the degree of dissatisfaction of the robotic arm task at the velocity level; the second velocity residual is defined as J_base · v + α_base · e_base, which characterizes the degree of dissatisfaction of the chassis task at the velocity level. The objective function of the quadratic programming model is to minimize the weighted sum of squares of these two residuals: ||W_arm · (J_arm · v + α_arm · e_arm)||² + ||W_base · (J_base · v + α_base · e_base)||².

[0075] In the actual solution process, by assigning a higher weight to the first velocity residual (i.e., the norm of W_arm is greater than the norm of W_base), the priority of the tasks is established during the optimization process, meaning that the robotic arm task has a higher priority. This ensures that when the precise operation of the robotic arm and the precise docking of the chassis cannot be simultaneously achieved, the solver prioritizes ensuring the robotic arm's end effector accurately reaches the desired pose, while allowing the chassis position to be compromised and adjusted within a certain range. This weighting strategy prevents the system from rigidly pursuing docking at a preset precise point, thus greatly reducing the dependence on the accuracy of the preset chassis reference parking pose and significantly improving the success rate of solving inverse kinematics problems under complex configurations.

[0076] The first and second velocity residuals can be obtained by mapping the first and second errors in the pose space to the velocity space using the Jacobian matrix in robot kinematics. In solving the quadratic programming model, the optimal joint motion velocity can be found by minimizing this weighted sum. Specifically, the solution process for this quadratic programming model can be iterative. In each iteration, based on the Jacobian matrix J(q) and error e(q) calculated from the robot's current state q, the constrained optimization problem is solved, and the optimal joint motion velocity command v* at the current moment is output.

[0077] Updating the robot's overall state based on the integral of the joint motion velocity refers to the numerical integration process of calculating the robot's overall state from joint motion velocities based on kinematic principles. Specifically, the solved joint motion velocity v* is multiplied by a preset control period Δt to obtain the state increment Δq = v* · Δt. This state increment is then updated to the robot's overall state, i.e., q_new = q_old + Δq. This new overall state q_new is recorded as an intermediate state point. An intermediate state point represents a robot's chassis pose and robotic arm joint pose.

[0078] It can be understood that the combined motion velocity v* is a multi-dimensional vector (containing different types of velocities from the chassis and the robotic arm). After multiplying the velocities of different dimensions with the control period Δt, the resulting state increment Δq corresponds to parameters with different physical meanings. For example, multiplying the chassis translational velocity in the X direction by the control period Δt yields the chassis position increment in the X direction, which is used to update the chassis's coordinates on the X-axis of the world coordinate system. Multiplying the robotic arm forearm joint rotation velocity by the control period Δt yields the forearm joint angle increment, which is used to update the rotation angle of the robotic arm forearm joint.

[0079] By repeatedly executing the above steps (i.e., the "build-transform-solve-integrate" loop), the continuous motion of the robot from the initial state to the target state can be iteratively simulated, thereby generating the entire sequence of intermediate state points. This sequence not only contains path information but also records the state pairs of how the chassis and the robotic arm should cooperate with each other at each step under global cooperative optimization, providing high-quality and strongly correlated input for subsequent hierarchical planning.

[0080] In this embodiment of the invention, a higher weight is assigned to the first velocity residual (corresponding to the robotic arm pose optimization task) than to the second velocity residual (corresponding to the chassis pose optimization task). Therefore, when the two tasks cannot be perfectly satisfied simultaneously, priority is given to ensuring the robotic arm's end effector accurately reaches the desired pose, while the chassis position is adaptively adjusted as a compromise. This reduces the dependence on the accuracy of the preset chassis reference parking pose and significantly improves the success rate of inverse kinematics solution.

[0081] In some embodiments, the objective function of the quadratic programming model further includes a regularization term, which is used to induce the joint motion velocity to tend to drive the robot's joint pose to a preset reference joint pose.

[0082] The regularization term is an additional term introduced into the objective function of the optimization model. It guides the optimization solution towards a more desirable or stable region, avoiding non-compliant or undesirable solutions. Specifically, in this embodiment of the invention, considering the possibility of singular configurations (i.e., postures where the robot loses its ability to move in a certain direction) or self-collision regions when solving the inverse kinematics model of a high-degree-of-freedom wheeled robot, without guidance, a joint motion velocity might be output that drives the robot to quickly enter such dangerous or failed regions. By introducing a carefully designed regularization term, soft constraints can be applied to the solution space, actively avoiding these problems, thereby significantly improving the safety and reliability of the generated trajectory.

[0083] Regular terms can include γ || v - k_p (q_ref - q) ||².

[0084] Here, γ is the weight coefficient of the regularization term, which is used to control the influence of this term in the overall objective function.

[0085] q_ref is a preset reference joint pose that represents a desired and safe full-body robot configuration, such as the intermediate state of a robotic arm with "elbow retracted and compact posture", and also includes the pose of the chassis.

[0086] q is the robot's current full-body joint pose.

[0087] k_p is a positive proportionality coefficient.

[0088] The deviation between the current solved velocity *v* and a desired velocity *k_p*(q_ref - q) is penalized by a regularization term. This desired velocity can be a proportional controller that drives the robot's state *q* towards the reference state *q_ref*. By introducing the regularization term, when solving for the joint motion velocity *v*, not only is the primary pose tracking task (i.e., minimizing the velocity residual) required, but a velocity is also preferred that allows the robot's overall posture to gradually return to a safe and stable reference state *q_ref*. This is equivalent to injecting a continuous "regression force" or "damping" into the optimization process, effectively preventing the robot from falling into unusual configurations or joint limitations, and ensuring the kinematic feasibility of the planned solution.

[0089] This invention, by introducing the regularization term into the quadratic programming model that optimizes the inverse solution throughout the entire robot body, adds stability assurance to the high-dimensional programming problem. This allows the robot to spontaneously avoid undesirable configurations while pursuing the task objective, generating motion trajectories that meet both task requirements and high safety, thereby improving the overall robustness of the wheelbarrow robot's operation.

[0090] In some embodiments, searching for a collision-free path segment that allows the robotic arm end effector to move to the joint pose corresponding to the current chassis path point for each chassis path point on the chassis movement path includes:

[0091] For each of the chassis path points, the collision-free path is searched forward and backward, starting from the current joint pose of the robotic arm and the joint pose of the robotic arm corresponding to the current chassis path point, respectively.

[0092] When the collision-free paths obtained from the forward search and backward search are connected in the joint pose space of the robotic arm, the total path obtained is taken as the collision-free path segment corresponding to the current chassis path point.

[0093] Simultaneous forward and backward search is a bidirectional search strategy used to initiate searches from both the starting point and the target point in the search space, moving towards each other until they meet somewhere in between. Considering that in the high-dimensional joint pose space of a robotic arm, the exploration range of traditional unidirectional search (e.g., searching only from the starting point to the target point) increases exponentially with the path length, leading to excessively long planning times and difficulty meeting real-time requirements. In this embodiment of the invention, to balance computational efficiency and success rate, a bidirectional search strategy is used to significantly reduce the state space that needs to be explored by narrowing the search range from both ends, thereby significantly improving the search speed. This makes it possible to quickly verify the robotic arm path for a large number of path points in global planning.

[0094] Specifically, this search process may include the following steps:

[0095] Initialize two search trees: In the joint pose space of the robotic arm, one search tree (called the forward search tree) starts growing with the current joint pose of the robotic arm as the root node; the other search tree (called the backward search tree) starts growing with the target robotic arm joint pose corresponding to the current chassis path point as the root node.

[0096] Alternating Expansion and Connection Attempts: During the search process, the forward search tree and the backward search tree are expanded alternately. Each time an expansion occurs, a node is randomly selected from one tree, and a new node is expanded in a random direction. After expanding the new node, the distance between that new node and all nodes in the other tree is immediately checked. If two nodes (one from the forward tree and one from the backward tree) are very close and a collision will not occur when connecting them, the two trees are considered successfully connected.

[0097] Path extraction: Once two trees are connected, the algorithm starts from the point of connection, traces forward to the root node of the forward tree (current state) and backward to the root node of the backward tree (target state), and splices these two paths together to form a complete collision-free path segment from the current state to the target state.

[0098] In summary, the embodiments of the present invention effectively solve the problem of long planning time in high-dimensional space by adopting a bidirectional search strategy to complete the collision-free path planning of the robotic arm.

[0099] In some embodiments, after searching for a collision-free path segment that moves the robotic arm end effector to the joint pose corresponding to the current chassis path point for each chassis path point on the chassis movement path, the method further includes:

[0100] For the chassis path point for which no collision-free path segment was found, obtain the first full-body state corresponding to the chassis path point for which the search failed and the second full-body state corresponding to the chassis path point for which the most recent collision-free path segment was successfully searched.

[0101] Using the first full-body state as the starting state and the second full-body state as the target state, the smooth trajectory of the coordinated motion of the chassis and the robotic arm from the starting state to the target state is solved to obtain the collision-free path segment corresponding to the chassis path point where the search failed.

[0102] One issue is that the hierarchical planning approach, which first fixes the chassis path and then plans the robotic arm, may reduce the feasible space, causing some previously solvable situations to become unsolvable. This is because the hierarchical planning process abandons the possibility of creating feasible motion space for the robotic arm by fine-tuning the chassis pose. Therefore, this embodiment of the invention introduces a local replanning strategy. In order to liberate the chassis's degrees of freedom again in local areas where global planning fails, feasible paths are found through the coordinated motion of the chassis and the robotic arm.

[0103] Specifically, such as Figure 3 As shown, suppose the upper-level planner needs to plan a full-body path for the robot from point A to point E. During the global hierarchical planning phase, collision-free paths (i.e., feasible solutions) were successfully found for the robot arm in segments AB and CD, but planning failed in segments BC and DE. These segments consist of a series of continuous chassis path points; if the robot arm planning fails at any of these path points, it signifies that the planning for that "segment" has failed. At this point, local full-body planning is triggered, using segments BC and DE as processing units. The local replanning process may include the following steps:

[0104] 1. Determine the Local Planning Interval: Referring to the previous example, for the failed BC segment, extract the overall state corresponding to point B as the starting state and the overall state corresponding to point C as the target state. For the DE segment, extract the state at point D as the starting state and the state at point E as the target state. In practice, the local planning interval process can be initiated by detecting planning failures. That is, when it is detected that no collision-free path segment for the robotic arm can be found at a certain chassis path point P_fail, local replanning is triggered. Obtain the first overall state (denoted as S_fail) corresponding to the failed point P_fail. Simultaneously, backtrack along the chassis path to find the most recent successfully planned chassis path point P_prev for the robotic arm path segment and obtain its corresponding second overall state (denoted as S_prev). These S_prev and S_fail define a local planning interval, with S_prev as the starting state and S_fail as the target state. This is the specific operation of "extracting the starting point of the failed trajectory segment".

[0105] 2. Constructing and Solving the Local Whole-Body Planning Problem: Within this local planning interval, unlike the fixed chassis path in the aforementioned global planning strategy, the chassis path is no longer fixed. Instead, the degrees of freedom of both the chassis and the robotic arm are simultaneously used as optimization variables to construct a local whole-body planning problem. Specifically, this optimization problem aims to directly solve for the smooth, collision-free motion trajectory of the robot's entire body from S_prev to S_fail. The optimization objective of this problem can be constructed by comprehensively considering multiple factors such as trajectory smoothness, connection with the global path, collision-free nature, and energy efficiency. By solving this problem through an efficient numerical optimization algorithm, a locally optimized trajectory on the interval [S_prev, S_fail] is finally obtained. This locally optimized trajectory is the result of the cooperative motion of the chassis and the robotic arm.

[0106] 3. Trajectory stitching: The locally optimized trajectory obtained from the previous steps is seamlessly stitched with the successful path segments in the global planning (the parts before S_prev and after S_fail), thereby filling the gaps left by the global planning and forming a complete and feasible target whole-body motion path.

[0107] This invention effectively overcomes the limitations of hierarchical planning strategies by introducing a local whole-body planner triggered at points where global planning fails. By employing collaborative optimization—which requires greater computational resources but has stronger search capabilities—in key areas, the poses of the chassis and robotic arm are dynamically adjusted. This allows for the discovery of feasible and safe paths even when global planning proves unsolvable, significantly improving the overall success rate and robustness of the wheeled robot in complex and confined environments.

[0108] In some embodiments, the objective of solving the smooth trajectory includes at least one of the following: a trajectory tracking term to ensure trajectory continuity, an inequality constraint term to achieve obstacle avoidance and physical limitation, and a control energy term to improve trajectory smoothness.

[0109] To efficiently solve this high-dimensional nonlinear problem, this embodiment employs a Nonlinear Model Predictive Control (NMPC) framework to model it. The mathematical form of this optimization problem can be expressed as:

[0110] Where x represents the robot's overall state (including the chassis and robotic arm), and u represents the control input. The optimization objective function L(x, u) must include at least a trajectory tracking term, an inequality constraint term, and a control energy term.

[0111] The trajectory tracking item (C_ee_tracking) ensures that the starting point state and the entire trajectory can be connected, and that the local trajectory can be smoothly connected to the globally planned S_prev state and approach the S_fail state.

[0112] Inequality constraint term (∑B_i): It includes boundary constraints and collision constraints, which are used to ensure that the solved trajectory satisfies joint limits and velocity limits, and that the robot does not collide with the environment during the entire motion process.

[0113] Control energy term (u^TR u): It is used to ensure optimal energy or shortest path, thereby obtaining a smooth and efficient motion trajectory.

[0114] This NMPC problem is solved using an efficient numerical optimization algorithm (such as the SLQ-MPC algorithm), ultimately yielding a locally optimized trajectory over the interval [S_prev, S_fail]. This trajectory is the result of the coordinated motion between the chassis and the robotic arm.

[0115] Figure 4 A schematic diagram of the robot control system architecture based on another embodiment of the robot control method of the present invention is shown. This system can integrate environmental perception devices such as RGBD sensors and LiDAR in hardware, and the robot's main processor executes the calculations of the following modules. These modules work collaboratively to achieve efficient and robust whole-body motion planning. For example... Figure 4 As shown, the robot control system may include the following modules:

[0116] Robot model loading module:

[0117] During initialization, a standardized file describing the robot's own model needs to be loaded, which may include:

[0118] URDF: Unified Robot Description Format. This document defines the physical structure of a robot, specifically including physical properties such as the linkage structure of the chassis and robotic arm, joint types, mass, and inertia tensor. It provides the most basic model basis for kinematic and dynamic calculations.

[0119] SRDF: Semantic Robot Description Format. This document provides a semantic-level supplementary definition to the URDF model, specifically specifying which links and joints constitute a planning group. For example... Figure 3 As shown, the robot is divided into a robotic arm group and a chassis group according to the SRDF file, and a default collision-disabled matrix is ​​configured for subsequent collision detection to avoid the robot performing self-collision checks and improve planning efficiency.

[0120] Environmental perception and modeling module:

[0121] Raw data of the surrounding environment is acquired using RGBD sensors and LiDAR. This data is used to construct and update a local 3D cost map in real time. This map typically covers a certain area behind the chassis (e.g., 1 meter) to a larger area in front (e.g., 3 meters), and is updated at a high frequency (e.g., >10Hz). The occupancy map information in the cost map provides the basis for subsequent obstacle avoidance planning.

[0122] Whole-body optimization inverse kinematics module:

[0123] The input is the target pose to be achieved (i.e., the desired pose of the robotic arm's end effector). This module treats the chassis and robotic arm as a unified motion system, generating a series of intermediate state points (including path points and timestamps) by solving an optimization problem. These state points simultaneously contain the target poses of both the chassis and the robotic arm, representing a preliminary motion draft that considers global coordination.

[0124] Global hierarchical planner: Responsible for efficient planning in open environments. It receives chassis and robotic arm targets from the whole-body optimization inverse kinematics module.

[0125] In the chassis group, search algorithms such as Hybrid A* can be used to plan a collision-free chassis movement path based on the environment map. The chassis movement path includes multiple chassis path points and the timestamps corresponding to the chassis path points.

[0126] In the robotic arm assembly, parallel algorithms such as RRT*-CONNECT can be used to independently plan the joint trajectory of the robotic arm to reach its corresponding target state for each point on the chassis path.

[0127] If both planning is successful, the system will align the chassis path with the robotic arm trajectory to form a joint trajectory plan, which will be directly issued to the robot as the output trajectory (i.e., the whole-body trajectory) for execution.

[0128] Local-to-whole planner: This is used to handle global planning failures in confined spaces. When the global hierarchical planner reports a solution failure, the local-to-whole planner is triggered. It receives cached trajectory information from the global planner and extracts the start and end states of the failed segment. Then, it uses incremental ESDF for precise distance and collision queries and constructs a nonlinear model predictive control (NMPC) problem for solution. This optimization problem simultaneously considers the degrees of freedom of the chassis and the robotic arm, ultimately generating a cooperative motion trajectory within the failed region to complete the final overall path. The final planning result can be shown as follows: Figure 5 As shown. Among them, Figure 5 The light gray circular dots represent various chassis path points along the chassis's movement path. Figure 5 The dark gray blocks in the diagram represent the alternative solution space for the robotic arm's motion path.

[0129] This invention, through the organic combination of the aforementioned modules, forms a complete planning closed loop of "generation-verification-assurance". The system first generates a cooperative motion draft through whole-body optimization inverse kinematics, then efficiently implements it in open scenarios using a global hierarchical planner, and finally overcomes the planning challenges in narrow areas using a local whole-body planner, thus balancing planning success rate and computational efficiency. The following describes the entire process from task reception to final path execution using a specific application example, demonstrating how this invention achieves efficient and robust whole-body motion planning through the synergy of global hierarchical planning and local whole-body planning in both open and narrow scenarios. Specifically, it includes the following:

[0130] 1. Task reception and initialization:

[0131] Suppose a wheeled robot equipped with a mobile chassis and a six-DOF robotic arm receives a task instruction in a warehouse environment to "grab the box on the right side of the third shelf". The robot uses its onboard vision sensors to identify the spatial coordinates and orientation of a preset gripping point on the target box, and analyzes this to obtain the desired pose of the robotic arm's end effector. Simultaneously, the system presets a reference parking pose, for example, a rough position approximately 1.5 meters in front of the shelf, directly facing the target box.

[0132] 2. Generate a draft of coordinated motion:

[0133] The robot control system initiates the whole-body optimization inverse kinematics module. This module aims to simultaneously drive the robotic arm's end effector to approach the desired pose and the chassis to approach the reference parking pose. It constructs a differential inverse kinematics model and transforms it into a quadratic programming model for solution. The first velocity residual corresponding to the robotic arm task is assigned a higher weight than the second velocity residual corresponding to the chassis task, ensuring priority is given to operational accuracy. Through iterative solving and integration, a series of intermediate state points are generated. Each intermediate state point contains a whole-body state, representing the specific chassis pose and robotic arm joint pose at that moment. This sequence constitutes a rough draft of a cooperative motion from the initial state to task completion, but with a strong correlation between the chassis and robotic arm states.

[0134] 3. Global hierarchical planning and verification:

[0135] Subsequently, the system enters the global hierarchical planner. This includes: Chassis path planning: extracting all chassis poses from the intermediate state point sequence, and using algorithms such as Hybrid A*, the chassis team plans a collision-free, passable, continuous chassis movement path in the environmental cost map. This path consists of a series of discrete chassis path points.

[0136] Robotic arm path search: For each chassis path point on the chassis movement path, the robotic arm assembly reads the pose of the target robotic arm joint paired with that point. With the chassis fixed at that pose, algorithms such as bidirectional RRT*-Connect are used to search for a collision-free path segment in the robotic arm's joint pose space, from the robotic arm state corresponding to the previous path point to the current target state. This process employs a strategy of simultaneous forward and backward searches to improve search efficiency.

[0137] 4. Successful execution in open scenarios (global solution):

[0138] Considering that in most open areas there are few environmental obstacles and ample space for the robotic arm to move, the above-mentioned search for collision-free path segments by the robotic arm is successful for each chassis waypoint. The combination module aligns and combines all successful collision-free path segments with the chassis movement path in space and time to obtain a complete and feasible full-body motion path from the starting point to the grasping point. The robot controller executes this path, achieving smooth coordination between chassis movement and robotic arm operation, and efficiently completing the task. This corresponds to the situation where "in open scenes, the robotic arm can generally solve the problem globally."

[0139] 5. Handling confined spaces and local replanning:

[0140] When the chassis path traverses complex areas such as narrow passages or gaps between shelves, after fixing the chassis pose, the robotic arm may be unable to find a collision-free path segment due to space constraints, resulting in search failures at one or more consecutive chassis path points. This triggers local planning: At this point, the global hierarchical planner passes the planning failure information to the local full-body planner. This planner obtains the first full-body state (denoted as S_fail) corresponding to the chassis path point where the search failed, and backtracks along the chassis path to find the second full-body state (denoted as S_prev) corresponding to the most recent successfully searched chassis path point.

[0141] The local optimization problem is constructed as follows: Starting with S_prev and targeting S_fail, a local whole-body planner solves for a smooth trajectory of coordinated motion between the chassis and the robotic arm from the initial state to the target state. In this process, the chassis path is no longer fixed; instead, the degrees of freedom of both the chassis and the robotic arm are simultaneously used as optimization variables. The objective of solving this smooth trajectory integrates trajectory tracking (ensuring continuity with the global path), inequality constraints (achieving obstacle avoidance and physical limits), and control energy (improving trajectory smoothness).

[0142] Trajectory Repair and Combination: The obtained local cooperative trajectories fill in the gaps in the global planning in failed sections (such as...). Figure 3 The blank spaces in segments BC or DE are then filled. Finally, the previously globally planned path segments are seamlessly spliced ​​with this locally optimized trajectory to form a complete and feasible target whole-body motion path.

[0143] This embodiment first generates a collaborative draft: by constructing an optimization problem that integrates the robotic arm operation task and the chassis movement task, a sequence of intermediate state points with strong correlation between the chassis and robotic arm states is generated in one go, breaking the fragmentation of traditional sequential separate planning and reducing the dependence on the accuracy of preset chassis docking points. Next, hierarchical verification and refinement are performed: based on the collaborative draft, efficient global hierarchical planning (first planning the chassis path, then verifying the robotic arm movement at each path point) is adopted to implement the specific path, which can quickly obtain feasible solutions in open scenarios. Finally, local robustness is guaranteed: when global hierarchical planning fails due to the fixed chassis in narrow and complex scenarios, the system can intelligently trigger local full-body planning. By re-liberating the chassis degrees of freedom within the failed segment and solving the collaborative smooth trajectory of the chassis and robotic arm, the pose is dynamically adjusted to find feasible space, thus successfully overcoming the bottleneck of local planning.

[0144] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0145] Based on the same inventive concept, this application also provides a robot control device for implementing the robot control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more robot control device embodiments provided below can be found in the limitations of the robot control method described above, and will not be repeated here.

[0146] In one exemplary embodiment, such as Figure 6 As shown, a robot control device 300 is provided, the device comprising:

[0147] The acquisition module 302 is used to acquire the desired pose of the robot's robotic arm end effector in response to a task instruction.

[0148] The determination module 304 is used to determine the sequence of intermediate state points of the robot with the optimization goal of driving the robot's chassis to move towards a preset reference parking pose while driving the end effector of the robotic arm to move towards the desired pose; each intermediate state point represents a whole-body state of the robot; the whole-body state includes the chassis pose and the joint pose of the robotic arm.

[0149] Planning module 306 is used to plan the chassis movement path of the robot based on the chassis pose in the intermediate state point sequence;

[0150] Search module 308 is used to search for collision-free path segments that allow the end effector of the robotic arm to move to the joint pose of the robotic arm corresponding to the current chassis path point for each chassis path point on the chassis moving path.

[0151] The combination module 310 is used to combine the collision-free path segments searched at each chassis path point to obtain the target whole-body motion path for the robot to move to the desired pose.

[0152] Each module in the aforementioned robot control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0153] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a robot control method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0154] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0155] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps included in any of the foregoing robot control method embodiments.

[0156] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps included in any of the foregoing robot control method embodiments.

[0157] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps included in any of the foregoing robot control method embodiments.

[0158] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0159] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0160] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0161] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A robot control method characterized by, The method comprises: in response to a task instruction, acquiring a desired pose of an end of a robot arm; determining a sequence of intermediate state points of the robot, with the optimization objective of driving the end of the robot arm to move to the desired pose while driving a chassis of the robot to move to a preset reference parking pose; each intermediate state point representing a full-body state of the robot; the full-body state comprising a chassis pose and a robot arm joint pose; planning a chassis movement path of the robot according to the chassis poses in the sequence of intermediate state points; for each chassis path point on the chassis movement path, searching for a collision-free path segment that moves the end of the robot arm to the robot arm joint pose corresponding to the current chassis path point; combining the collision-free path segments searched for each chassis path point to obtain a target full-body movement path of the robot to the desired pose.

2. The method of claim 1, wherein, The method comprises: constructing a differential inverse kinematics model corresponding to the robot according to the optimization objective; wherein the differential inverse kinematics model is used to solve a set of joint movement velocities of the chassis and the robot arm, so that a first error between a current pose of the end of the robot arm and the desired pose and a second error between a current pose of the chassis and the reference parking pose tend to zero; transforming the differential inverse kinematics model into a quadratic programming model; wherein an objective function of the quadratic programming model comprises a weighted sum; the weighted sum is obtained by weighting and summing a first velocity residual obtained by mapping the first error and a second velocity residual obtained by mapping the second error; a first weight corresponding to the first velocity residual is greater than a second weight corresponding to the second velocity residual; solving the quadratic programming model to obtain the joint movement velocities; integrating and updating the full-body state of the robot according to the joint movement velocities to iteratively generate the sequence of intermediate state points.

3. The method of claim 2, wherein, The objective function of the quadratic programming model further comprises a regularization term, which is used to promote the joint movement velocities to tend to drive the joint pose of the robot to a preset reference joint pose.

4. The method of claim 1, wherein, The method comprises: for each chassis path point, searching forward and backward from the current joint pose of the robot arm and the robot arm joint pose corresponding to the current chassis path point respectively to obtain the collision-free path; when the collision-free paths obtained by forward searching and backward searching are connected in the joint pose space of the robot arm, taking the total path obtained by connection as the collision-free path segment corresponding to the current chassis path point.

5. The method of claim 1, wherein, After searching for a collision-free path segment that makes the mechanical arm end move to the mechanical arm joint pose corresponding to the current chassis path point for each chassis path point on the chassis movement path, the method further comprises: For the chassis path point for which the collision-free path segment is not searched, acquiring a first full-body state corresponding to the chassis path point for which the search fails and a second full-body state corresponding to a chassis path point for which the search for the last collision-free path segment is successful; Taking the first full-body state as a starting state and the second full-body state as a target state, solving a smooth trajectory of the collaborative motion of the chassis and the mechanical arm between the starting state and the target state to obtain the collision-free path segment corresponding to the chassis path point for which the search fails.

6. The method of claim 5, wherein, The solving target of the smooth trajectory includes at least one of a trajectory tracking term for ensuring trajectory continuity, an inequality constraint term for realizing obstacle avoidance and physical limiting, and a control energy term for improving trajectory smoothness.

7. A robot control device characterized by comprising: The device comprises: An acquisition module configured to acquire, in response to a task instruction, a desired pose of a mechanical arm end of a robot; A determination module configured to determine a sequence of intermediate state points of the robot, with driving the chassis of the robot to move to a preset reference parking pose as an optimization target while driving the mechanical arm end to move to the desired pose; each intermediate state point represents a full-body state of the robot; the full-body state includes a chassis pose and a mechanical arm joint pose; A planning module configured to plan a chassis movement path of the robot according to the chassis poses in the sequence of intermediate state points; A search module configured to search for a collision-free path segment that makes the mechanical arm end move to the mechanical arm joint pose corresponding to the current chassis path point for each chassis path point on the chassis movement path; A combination module configured to combine the collision-free path segments searched for all chassis path points to obtain a target full-body motion path of the robot moving to the desired pose.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

Images