Four-wheel foot manipulator hierarchical control method and system for end effector tracking

Abstract

The present disclosure provides a four-wheel foot manipulator end effector tracking-oriented hierarchical control method and system, relating to the technical field of manipulator system control, comprising: defining the control problem of the four-wheel foot manipulator system as a long-distance flexible six-dimensional end effector tracking task, decoupling the overall task and adopting a two-stage hierarchical strategy, the two-stage hierarchical strategy comprising a low-level strategy and a high-level strategy, the low-level strategy learning a movement strategy and an operation strategy respectively, the high-level strategy performing unified whole-body control through coordination of the movement strategy and the operation strategy, introducing a proximity area reward and a movement-operation balance mechanism to guide the robot to efficiently approach the target, and realizing smooth phase transition in the movement-operation process, and combining a mask-based conditional strategy to enable a single strategy to flexibly switch between regular end effector tracking and posture conditional end effector tracking. The present disclosure significantly improves the success rate of open and posture-constrained movement and operation tasks.

Description

Technical Field

[0001] This disclosure relates to the field of robotic arm system control technology, specifically to a hierarchical control method and system for a four-wheeled legged robotic arm oriented towards end effector tracking. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] Wheeled-quadrupedal-manipulator (WQM) systems possess both efficient and stable mobility and strong adaptability to complex terrain, making them highly promising for mobile manipulation tasks. In practical daily applications, these robots not only need to perform routine grasping in open spaces but also handle constrained scenarios, such as operating under tables, deep within shelves, or facing high targets. In these scenarios, the robot must also meet explicit body posture constraints. The former only requires efficient and stable end effector pose tracking, while the latter requires active control of the body posture. Therefore, practical control algorithms should be able to handle both types of situations simultaneously and switch flexibly between them.

[0004] However, most existing methods focus primarily on conventional end-effector (EE) tracking, neglecting the importance of actively adjusting the robot's posture. Currently, long-range flexible six-dimensional end-effector tracking is a practically valuable but extremely challenging problem, requiring robots to simultaneously achieve stable and efficient long-range target approach, smooth motion-operation transitions, accurate six-dimensional end-effector pose tracking, and flexible switching between conventional end-effector tracking and posture-conditional end-effector tracking.

[0005] However, existing long-range flexible six-dimensional end effector tracking methods still have the following limitations: (1) Some methods require the assumption that the base is fixed during pose tracking. When the target is far away from the robot, additional teleoperation is still required to move the base to the vicinity of the target, so they are not suitable for long-distance movement operations.

[0006] (2) The existing design uses a hierarchical framework to achieve long-distance grasping. However, because the movement and operation rewards are designed separately, the transitions between stages are still discontinuous, resulting in insufficiently smooth generated motion. Alternatively, only a single feasible solution is provided for each six-dimensional pose command, which limits control flexibility and practical usability. Summary of the Invention

[0007] To address the aforementioned issues, this disclosure proposes a hierarchical control method and system for a four-wheeled legged robotic arm oriented towards end effector tracking. It proposes a hierarchical framework RoboTrio that employs a two-stage training strategy. The lower layer acquires strong motion and manipulation capabilities, while the higher layer achieves unified whole-body control. An adaptive reward integration mechanism is introduced to improve target approach efficiency and enhance the smoothness of stage transitions. Combined with conditional commands, RoboTrio can achieve long-distance flexible six-dimensional end effector tracking under a unified strategy.

[0008] According to some embodiments, the present disclosure adopts the following technical solutions: A hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking includes: Construct a kinematic model of a four-wheeled robotic arm system; The control problem of the four-wheeled robotic arm system is defined as a long-distance flexible six-dimensional end effector tracking task. The overall task is decoupled and a two-stage hierarchical strategy is adopted to achieve coordinated control and mode switching of long-distance flexible six-dimensional end effector tracking. The two-stage hierarchical strategy includes a low-level strategy and a high-level strategy. The low-level strategy learns the movement strategy and the operation strategy respectively. The high-level strategy coordinates the movement strategy and the operation strategy to achieve unified whole-body control. It introduces a proximity area reward and a movement-operation balance mechanism to guide the robot to approach the target efficiently and achieve a smooth stage transition during the movement-operation process. Combined with a mask-based conditional strategy, a single strategy can flexibly switch between conventional end effector tracking and posture-conditional end effector tracking.

[0009] According to some embodiments, the present disclosure adopts the following technical solutions: A hierarchical control system for a four-wheeled robotic arm oriented towards end-effector tracking includes: The initialization module is used to build the kinematic model of the four-wheeled robotic arm system; The hierarchical control module is used to define the control problem of the four-wheeled robotic arm system as a long-distance flexible six-dimensional end effector tracking task, decouple the overall task, and adopt a two-stage hierarchical strategy to achieve coordinated control and mode switching of long-distance flexible six-dimensional end effector tracking. The two-stage hierarchical strategy includes a low-level strategy and a high-level strategy. The low-level strategy learns the movement strategy and the operation strategy respectively. The high-level strategy coordinates the movement strategy and the operation strategy to achieve unified whole-body control. It introduces a proximity area reward and a movement-operation balance mechanism to guide the robot to approach the target efficiently and achieve a smooth stage transition during the movement-operation process. Combined with a mask-based conditional strategy, a single strategy can flexibly switch between conventional end effector tracking and posture-conditional end effector tracking.

[0010] According to some embodiments, the present disclosure adopts the following technical solutions: A computer program product includes a computer program that, when executed by a processor, implements the hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking.

[0011] According to some embodiments, the present disclosure adopts the following technical solutions: A non-transitory computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking.

[0012] According to some embodiments, the present disclosure adopts the following technical solutions: An electronic device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform the hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking.

[0013] Compared with the prior art, the beneficial effects of this disclosure are as follows: This paper discloses a hierarchical control method for a four-wheeled legged robotic arm with end effector tracking, proposing RoboTrio, a hierarchical framework employing a two-stage training strategy. First, low-level policies are trained separately to achieve strong motion and manipulation capabilities; then, high-level policies are learned to achieve unified whole-body control. Furthermore, an adaptive reward integration mechanism is introduced to improve target approach efficiency and enhance the smoothness of stage transitions. Combined with conditional commands, RoboTrio can achieve long-range, flexible, six-dimensional end effector tracking under a unified strategy. Simulation results demonstrate that RoboTrio can effectively complete this task. Real-world experiments further show that this capability significantly improves the success rate of open-ended and posture-constrained motion manipulation tasks. In challenging real-world tasks, RoboTrio achieves a success rate exceeding 60%, significantly outperforming baseline methods.

[0014] This disclosure presents a hierarchical control method for a four-wheeled quadrupedal robotic arm designed for end-effector tracking, addressing the challenge of long-range, flexible six-dimensional end-effector tracking in quadrupedal robotic arm systems. This method enables the robot to accurately and efficiently track six-dimensional end-effector targets over long distances and flexibly adjust its posture according to different task requirements, thereby improving operability and practical usability during mobile operations.

[0015] This disclosure presents a hierarchical control method for a four-wheeled robotic arm with end-effector tracking. It employs a hierarchical framework combining a two-stage training scheme to decouple this complex task and simplify the policy learning process. Adaptive reward integration improves target approach efficiency and smooths the movement-operation transition. Combined with conditional commands, these designs enable a unified policy to flexibly handle both conventional end-effector tracking and attitude-conditional end-effector tracking.

[0016] This disclosure presents a hierarchical control method for a four-wheeled robotic arm focused on end-effector tracking. Extensive simulations and real-world experiments were conducted to verify the effectiveness of RoboTrio in long-range, flexible, six-dimensional end-effector tracking tasks. Experimental results show that this method significantly improves performance in open environments and challenging tasks with explicit body posture requirements, consistently outperforming baseline methods. Attached Figure Description

[0017] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0018] Figure 1 This is a schematic diagram of the hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking, according to an embodiment of the present disclosure. Figure 2 For proximity area rewards in this embodiment of the disclosure; Figure 3 The present disclosure provides an embodiment of a robot system and control device. Detailed Implementation

[0019] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0020] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0021] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0022] Example 1 One embodiment of this disclosure provides a hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking, the method steps including: Step 1: Construct a kinematic model of the four-wheeled robotic arm system; Step 2: Define the control problem of the four-wheeled robotic arm system as a long-distance flexible six-dimensional end effector tracking task. Decouple the overall task and adopt a two-stage hierarchical strategy to achieve coordinated control and mode switching of long-distance flexible six-dimensional end effector tracking. The two-stage hierarchical strategy includes a low-level strategy and a high-level strategy. The low-level strategy learns the movement strategy and the operation strategy respectively. The high-level strategy coordinates the movement strategy and the operation strategy to achieve unified whole-body control. It introduces a proximity area reward and a movement-operation balance mechanism to guide the robot to approach the target efficiently and achieve a smooth stage transition during the movement-operation process. Combined with a mask-based conditional strategy, a single strategy can flexibly switch between conventional end effector tracking and posture-conditional end effector tracking.

[0023] As one embodiment, this disclosure presents a hierarchical control method for a four-wheeled quadruped robotic arm oriented towards end-effector tracking. This method addresses the practical whole-body control strategy for a wheeled quadruped robotic arm system, aiming to achieve stable and efficient target approach over long distances, smooth transition from movement to manipulation, accurate tracking of the six-dimensional end-effector pose at long distances, and further adjustment of the robot's posture when task constraints require. This unified control problem is defined as a long-distance flexible six-dimensional end-effector tracking task. RoboTrio decomposes the complex task into simpler sub-tasks and employs a two-stage strategy. The first stage learns low-level movement and manipulation strategies separately, while the second stage learns high-level strategies to coordinate the two, achieving unified whole-body control. Furthermore, an adaptive reward integration module is introduced to improve strategy efficiency and flexibility. This module consists of three parts: a proximity area reward for efficient target approach, a movement-manipulation balance term for smooth movement-manipulation transitions, and conditional rewards for different control modes. Combined with mask-based conditional instructions, these designs enable the strategy to flexibly switch between posture-conditional end-effector tracking and conventional end-effector tracking. Therefore, the RoboTrio disclosed herein can achieve long-range six-dimensional end-effector pose tracking and flexibly adjust the body posture when needed. The specific implementation process is as follows: Step 1: Construct a kinematic model of the four-wheeled robotic arm system; The robotic system consists of a 16-DOF Unitree B2W wheeled robot, a 7-DOF torque-controlled Unitree Z1 robotic arm equipped with grippers, and a RoboSense Helios-5515 LiDAR. All control strategies run on the B2W's onboard computer. The entire system is powered by the B2W's built-in battery, while the Z1 can also be powered by a custom battery pack.

[0024] To accurately track the target's six-dimensional end effector pose in the world coordinate system, the base pose is first estimated using FAST-LIO. Then, the target pose is transformed from the world coordinate system to the base coordinate system using the corresponding transformation matrix. The current end effector pose in the base coordinate system is calculated using the transformation matrix between the robot coordinate system and the base coordinate system, combined with forward kinematics. During operation, a human operator wearing a VR device uses the left and right joysticks to specify the target pose and robot attitude commands in the world coordinate system.

[0025] Step 2: Define the control problem of the four-wheeled robotic arm system as a long-distance flexible six-dimensional end effector tracking task, decouple the overall task, and adopt a two-stage hierarchical strategy to achieve coordinated control and mode switching of long-distance flexible six-dimensional end effector tracking; This disclosure presents the RoboTrio framework, such as Figure 2 As shown, RoboTrio, as a hierarchical control framework, adopts a two-stage strategy scheme, including a low-level strategy and a high-level strategy, and performs two-stage training. This allows the low-level strategy to focus on improving motion control and operational performance, while the high-level strategy focuses on system coupling and coordination optimization. Furthermore, an Adaptive Reward Integration mechanism is proposed, consisting of Approach RegionReward (ARR), Loco-Manipulation Balance (LMB), and Conditional Reward. This mechanism guides the wheeled manipulator system (WQM) to efficiently approach the target and achieves smooth stage transitions during motion-manipulation. Combined with conditional commands, they together constitute a conditional strategy for flexible 6D end effector tracking. The specific control content is as follows: (a) Low-level strategies To achieve coordinated whole-body control in WQM, this disclosure trains a flexible movement strategy and an operation strategy, enabling accurate tracking of the end effector pose while avoiding collisions. The low-level strategy learns the movement and operation strategies respectively. Specifically: (1) Motion strategy: The goal of the movement strategy is to track the target instruction:

[0026] in, The lower edge of the base coordinate system shaft and The desired linear velocity of the axis, For the desired yaw rate, For the desired body height, and These are the desired roll and pitch angles of the aircraft, respectively. (Superscript) This represents variables related to the organism. The inputs to the movement strategy include the target command and the estimated linear velocity. and historical observation data from the past 15 time steps. Historical observation data includes the base angular velocity. Gravity projection Non-wheel joint position Joint velocity and the action at the previous moment . It is obtained by an estimator trained with MSE loss based on stacked observation history.

[0027] Because wheel joints are better suited for speed control, the current movement Desired positions classified as non-wheel joints Desired speed of wheel joint These outputs are then converted into joint torques via a proportional-derivative controller:

[0028] Among them, superscript and These represent non-wheel joints and wheel joints, respectively. This is the action scaling factor. and For PD gain, This is the default joint position. and The joint encoder at time 10:00 to 11:00 respectively The measured joint position and velocity.

[0029] (2) Operation strategy: The goal of the operation strategy is to track the six-dimensional pose command:

[0030] in, This indicates the desired end effector position in the Cartesian coordinate system of the base. This represents the desired Euler angles expressed in the base coordinate system. (Superscript) This represents variables related to the robotic arm. The inputs to the operational strategy include target commands and observation data. The observation data includes joint positions. Joint velocity Actions at the previous moment Position error between the target position and the current end effector position and the rotation vector from the current pose to the target pose. Current robotic arm movement This indicates the desired positions of the motors at the joints of the six robotic arms.

[0031] (II) High-level strategy The goal of the high-level mobility-operation strategy is to accomplish flexible six-dimensional end effector tracking tasks by sending predefined instructions to the low-level strategy. The high-level instructions are defined as follows:

[0032] The first six elements specify the target six-dimensional end effector pose in the base coordinate system, which is consistent with the operation strategy instructions. Indicates the aircraft's altitude. and These represent the roll and pitch commands, respectively. It is a binary mask used to indicate whether the body attitude command is enabled.

[0033] High-level policy inputs include instructions and high-level policy observation data. Including gravity projection Joint positions other than wheel joints Joint velocity The actions of the top leadership at the last moment Position error between the target position and the current end effector position Rotation vector from the current pose to the target pose The target end effector and the predefined head reference point in the base coordinate system x and y Position error in the axial direction and the target end effector relative to the head reference point in the base coordinate system xy Relative heading angle in the plane High-level strategy output The instructions are divided into two parts: the operation strategy and the movement strategy.

[0034] in, and Instructions serving as operational and movement strategies, respectively. and The overall control process is as follows: Figure 2 As shown, by simultaneously predicting the target instructions of two lower-level policies, the higher-level policy achieves coupled control of WQM.

[0035] As one embodiment, the two-stage hierarchical strategy of this disclosure employs two-stage training. First, the movement-operation task is decoupled into two sub-tasks: movement and operation. Then, a high-level strategy is trained to achieve full-body control of the system. To improve target approach efficiency, this disclosure proposes ARR, which provides explicit velocity guidance, enabling WQM to approach the target quickly and efficiently. Furthermore, since the movement-operation task has different policy objectives at different stages, this disclosure designs a balancing mechanism to adjust the weights of various rewards at different stages. Tables 1-3 summarize the key reward functions of all strategies and provide the stage-specific rewards for the movement-operation strategy. , , static arm belongs to award, wheelcontact, keep wheel speed symmetry belongs to award, , , Action smoothing belongs to award.

[0036] Table 1 Key Reward Functions of Motivation Strategies

[0037] Table 2 Key Reward Functions of the Operational Strategy

[0038] Table 3 Key Reward Functions for Motion Operation Strategies

[0039] (1) Low-level strategy training In the first phase, this disclosure learns low-level strategies for solving movement and manipulation tasks, laying the foundation for full-body control.

[0040] Motion Strategy: To achieve flexible body movement, the motion strategy needs to track not only the base velocity commands but also the body attitude commands. To prioritize velocity tracking over attitude tracking, the velocity tracking reward is weighted as the attitude tracking reward. in, and The rewards are for tracking the linear velocity and angular velocity of the base, respectively. As a reward for high-speed tracking of the aircraft, For the roll tracking reward, Rewards for aircraft pitch tracking.

[0041] To further improve the command tracking accuracy of the robot base, this disclosure estimates the linear velocity based on the robot's state history, thereby improving the linear velocity command tracking effect. This disclosure also introduces a command tracking error penalty to further enhance the tracking performance of body posture commands. During training, to ensure that the movement strategy can adapt to changes in the body's center of mass and inertia while maintaining stability, this disclosure fixes all robotic arm joints in the default configuration (0, 1.57, ...). 1.57,0,0,0).

[0042] Operational Strategy: The goal of the operational strategy is to achieve accurate end-effector pose tracking while avoiding collisions. To eliminate the influence of changes in aircraft attitude, this disclosure samples the target position in a spherical coordinate system independent of aircraft attitude and converts it to Cartesian coordinates in the base coordinate system as input. To improve the end-effector attitude tracking accuracy, this disclosure follows... The Euler angles are sampled sequentially by rotation and used as input. This sampling scheme can filter out a large number of infeasible poses and make the command distribution more uniform, thus benefiting policy learning. This disclosure uses Euclidean distance for position. and quaternion-based attitude angular distance The end effector pose tracking reward is calculated and defined as follows: Among them, weight Used to balance the priority between position and attitude This represents the quaternion inner product. During training, the body joints are fixed in the default standing configuration. An exponential shaping function is further employed to improve the tracking accuracy of the end effector.

[0043] (2) High-level strategy training The goal of the second stage is to achieve target end effector tracking through whole-body control. In this stage, all joints of the robot are activated.

[0044] Motion-Operation Strategy: To support long-distance motion, the target pose is sampled in a Cartesian coordinate system independent of the robot's body pose. This disclosure further introduces a proximity region reward to generate a distance-aware reference velocity, thereby achieving efficient target approach.

[0045] To better estimate the relative pose between the target and the robot, this disclosure defines a head reference point located at (0.68,0,0) m in the base coordinate system and uses this reference point in all calculations. Based on the relationship between the target position and this reference point, this disclosure calculates the position in the base coordinate system... shaft and Distance scaling factor along the axis: in, , as well as , These represent the target and head reference points in the base coordinate system, respectively. shaft and Coordinates along the axis. Define the boundary between the stopping region and the deceleration region. Define the boundary between the deceleration region and the high-speed region.

[0046] Because WQM can move forward and backward efficiently, when the absolute heading error is close to... At this time, the amplitude of the angular velocity should increase. The heading factor is calculated as follows: The relative heading angle is defined as follows: To encourage faster course adjustments under larger angular errors, when At that time, this disclosure will further Saturate to 1. This is equivalent to... The maximum angle factor is assigned at that time.

[0047] Similarly, base coordinate system and The direction of motion along the axis and the direction of yaw rotation are determined by the relative position of the target and the reference point: Since the same direction of steering corresponds to opposite signs of yaw speed in forward and backward motion, therefore Need to be multiplied by Accordingly, the reference speed is defined as follows: in, Indicates the previous time step The amplitude. When At the same time, when there is a large heading error, forward motion is suppressed to encourage the robot to complete heading alignment before forward translation. This indicates the distance between the target pose and the head reference point: also, and Activated in different distance regions: when hour, Used for long-distance course correction; otherwise... Used for close-range lateral correction.

[0048] Motion-Operation Balance: Motion operations emphasize different objectives at different stages. When the target is far away, the strategy should primarily focus on moving the base towards the target, rather than end effector pose tracking, to avoid unnecessary energy consumption. Once the target enters the robot's workspace, the strategy should shift its focus to full-body control for end effector pose tracking, while simultaneously adjusting the base to a suitable operating position.

[0049] To balance the above objectives, this disclosure proposes a movement-operation balancing mechanism. Specifically, based on the activation phase of different reward items, they are divided into movement rewards, operation rewards, and basic rewards, as shown in Table 1. Subsequently, this disclosure introduces a distance system. To indicate the effective stage of different reward items: in, The target and head reference points in the base coordinate system Piecewise linear function of Euclidean distance in the plane: Therefore, the reward composition is continuously adjusted with distance, enabling the strategy to smoothly switch its behavior between different stages.

[0050] Furthermore, based on a low-level policy with strong mobility and maneuverability, and a high-level policy for conventional end-effector tracking, this disclosure further considers the problem of flexible six-dimensional end-effector tracking. In this task, when attitude commands are activated, the task transforms into attitude-conditional end-effector tracking, meaning the policy tracks the target end-effector pose while simultaneously tracking the given body attitude as much as possible; when attitude commands are not activated, the task degenerates into conventional end-effector tracking. A simple approach is to directly input attitude commands into the policy and train the policy using tracking rewards. However, in conventional tasks, this can lead to a mismatch between the input commands and the reward targets. Therefore, a mechanism is needed to train the high-level policy to determine whether to enable attitude tracking and to ensure that the policy input aligns with the corresponding reward tracking targets.

[0051] To achieve unified modeling, this disclosure proposes a mask-based conditional policy (MCP). This policy introduces a binary mask into the input. Used to indicate the attitude of the aircraft Whether it is activated. Accordingly, this disclosure designs conditional instructions and conditional rewards for attitude tracking. Conditional Instructions Defined as in, These represent the default aircraft height, roll angle, and pitch angle, respectively. The conditional bonus for aircraft attitude is calculated as follows:

[0052] For conditional commands, a binary mask switches the attitude target between a default attitude and a commanded attitude, ensuring that the attitude-related inputs remain consistent with the actual tracked target across different control modes. This design reduces the effective command space, enabling the unified strategy to handle both conventional end-effector tracking and attitude-conditional end-effector tracking simultaneously. Consistently, the conditional reward is also based on the same conditional command definition and multiplied by... The coefficients are used to ensure consistency between policy inputs and reward targets under different mask settings, while maintaining the end effector tracking priority.

[0053] Simulation Experiment All control policies were trained using PPO in Isaac Gym with 4096 parallel environments. Course learning and domain randomization were employed to improve policy performance. Each policy in the framework was trained for 10,000 iterations. All networks were multilayer perceptrons (MLPs) with ELU activation functions in the hidden layers. All training was performed on a workstation equipped with Ubuntu 20.04, an Intel i9-13900K CPU, and an NVIDIA RTX 4090 GPU.

[0054] The robotic system used in the experiment consisted of a 16-DOF Unitree B2W wheeled robot, a 7-DOF torque-controlled Unitree Z1 robotic arm equipped with grippers, and a RoboSense Helios-5515 LiDAR. All control strategies ran on the B2W's onboard computer. The entire system was powered by the B2W's built-in battery, while the Z1 could also be powered by a custom battery pack.

[0055] To accurately track the target six-dimensional end effector pose in the world coordinate system, the body base pose is first estimated using FAST-LIO

[29] . Subsequently, the target pose is transformed from the world coordinate system to the base coordinate system using the corresponding transformation matrix. The current pose of the end effector in the base coordinate system is calculated by combining the transformation matrix between the manipulator coordinate system and the base coordinate system with forward kinematics. During operation, the human operator wearing VR equipment specifies the target pose and body posture commands in the world coordinate system using the left and right joysticks respectively.

[0056] In mobile operations, performance is typically evaluated using the task success rate, defined as the ratio of successful trials to the total number of trials. In the experiments disclosed herein, a trial is considered a failure if any of the following conditions are not met: (1) The robotic arm or robot body collides.

[0057] (2) The object being grabbed accidentally falls during the movement operation.

[0058] (3) Failed to place the grabbed object to the target location.

[0059] In addition, the following indicators are used to evaluate the proposed method: (1) End effector position tracking error: ; (2) End effector attitude tracking error: ; (3) Survival rate: The proportion of experiments that do not fail when a random external force of 150-250 N is applied for 3 seconds; (4) Solvability: The ratio of the number of collision-free samples to the total number of samples; (5) Power: The time average of the sum of the absolute mechanical power of all joints; (6) Acceleration: The time average of the sum of the absolute accelerations of all joints; (7) Working height range: The target height range corresponding to when the tracking error is lower than a given threshold; (8) Time: The time required from the issuance of the instruction to the end effector reaching the error threshold and remaining stable; (9) Altitude error: The absolute error between the height of the aircraft and the height of the target; (10) Roll error: The absolute error between the roll angle of the aircraft and the target roll angle; (11) Pitch error: The absolute error between the pitch angle of the aircraft and the pitch angle of the target.

[0060] Except for the task success rate, all metrics are calculated after excluding failed trials.

[0061] Experimental results The effectiveness of RoboTrio was verified through simulation and hardware experiments, focusing on the following aspects: 1) high-precision tracking of the end effector for long-range targets; 2) smooth transition between movement and manipulation; 3) motion efficiency during target approach; 4) tracking accuracy of the end effector and body attitude commands in attitude condition tracking mode; and 5) improved success rate of movement manipulation tasks. Ablation experiments further validated the effectiveness of the proposed learning framework.

[0062] Comprehensive performance evaluation To evaluate the effectiveness of the proposed hierarchical control framework, ARR, and LMB, this disclosure compares RoboTrio variants with different policy network settings, reward functions, and reward balancing schemes. The results are shown in Table 4. In each set of experiments, this disclosure uniformly samples 1000 target six-dimensional poses and performs task evaluation in two ranges: in close-range scenarios, the target position is... and The distance from the robot along the axis is no more than 1 meter; in long-distance scenarios, the target location is... and The distance from the robot along the axis is 3 to 5 meters.

[0063] Baseline methods: The baseline in ablation experiments includes the following variations: (1) Two Policy: The lower-level controller contains only the motion policy. The higher-level policy has two output heads, one for controlling the robotic arm joints and the other for sending commands to the motion policy.

[0064] (2) Unfrozen: When training high-level policies, the parameters of low-level policies remain trainable.

[0065] (3) AR: In high-level policy training, Approach Reward (AR) is used instead of ARR.

[0066] (4) w / o LMB: In high-level policy training, the optimal fixed weights are used to fuse movement rewards and action rewards, i.e. .

[0067] Results: The Two Policy baseline exhibits significant end-effector pose tracking errors due to increased complexity in high-level policy learning. The Unfrozen baseline performs poorly in terms of survival rate because joint fine-tuning of movement and manipulation policies during high-level training biases the policy towards the primary end-effector tracking target, thus weakening movement stability and disturbance resistance. The AR baseline requires more time and energy to complete tasks because its continuous adjustment of heading via lateral velocity and yaw rate results in a longer path. Consequently, its execution time is approximately 35% longer than RoboTrio. The w / o LMB baseline exhibits robot jitter and large joint accelerations, indicating an inability to achieve smooth transitions between movement and manipulation.

[0068] In comparison, RoboTrio achieved superior performance across almost all metrics. These results demonstrate that the hierarchical framework effectively decouples tasks, allowing low-level policies to focus on their respective subtasks; simultaneously, LMB enables smooth system coupling across different task phases, thus supporting stable whole-body control. The two-stage training scheme reduces optimization difficulty while preserving low-level capabilities. Furthermore, ARR fully leverages the motion characteristics of WQM, significantly reducing joint mechanical power and task execution time.

[0069] Table 4 Ablation experimental results of different baseline methods at the near and far ranges.

[0070] Task efficiency analysis. This paper studies the impact of ARR (Automatic Radiation Response) on robot motion and trajectory optimization. The robot starts from the origin, and the base coordinate system... and Align the axes with the world coordinate system and move them to the four target positions respectively: 2.32, 3,0.8), ( 2.32,3,0.8), (3.68, (3,0.8) and (3.68,3,0.8).

[0071] The methods used in this disclosure include: (1) ARR (w / o (saturation): High-level policies are trained using ARR, but when Time will no longer Set to 1.

[0072] (2) ARR (w / o conditional) The high-level strategy is trained using ARR, but according to calculate.

[0073] As a result, AR adjusted the robot's heading during movement to keep its head always facing the target. However, this strategy did not fully utilize the robot's tracking capabilities. The limited axis speed results in longer motion paths, higher power consumption, and poorer performance when manipulating objects located behind the robot.

[0074] ARR (w / o conditional The method aligns the head with the target when it is in front of the target and the tail with the target when it is behind the target. However, the method still gradually adjusts the heading during movement, causing the robot to take steps and rotate its wheels as it approaches the target.

[0075] ARR w / o Saturation and ARR use the same heading selection rules, but both first perform a stationary repositioning of the aircraft, and then primarily rely on wheeled motion. Axis velocity tracking approaches the target. This significantly reduces step time and path length, thereby substantially reducing overall drive cost. Compared to ARR, ARR w / o Saturation rotates at a slower speed, thus requiring a longer path to complete the task.

[0076] Tracking accuracy in attitude condition end effector tracking This paper evaluates the impact of a mask-based conditional policy on the performance of RoboTrio in a posture-conditional end effector tracking task. In the first experiment, this disclosure assesses the ability of each baseline to track body posture commands during the movement phase. The desired six-dimensional pose in the base coordinate system is fixed at (4.0,0,0.3,0,0,0), so only the posture commands change over time. The robot tracks a sequence of posture commands, each command held for 3 seconds.

[0077] In the second experiment, this disclosure evaluates the tracking performance of different baselines for end effector pose commands and body attitude commands during the operational phase. Specifically, the target pose space is... The area is divided into 12 height levels, with a sampling interval of 0.1m. At each height level, this disclosure... 100 target poses are uniformly sampled within the range. For each target pose, 10 body posture commands are further randomly sampled, so each height level contains 1000 test samples.

[0078] Baseline method: (1) Raw Input: The strategy uses raw instructions. Conduct training.

[0079] (2) MCP: The high-level policy is trained using MCP, which uses conditional instructions and conditional rewards.

[0080] Ultimately, MCP was able to quickly and accurately track all target attitude commands within 2 seconds. In contrast, Raw Input failed to learn any meaningful machine attitude command tracking behavior.

[0081] Raw Input failed to learn based on The control mode switching behavior of the other approach also fails to effectively track body attitude commands during operation. In contrast, the MCP achieves excellent body attitude tracking at the cost of only a slight increase in end-effector tracking error. Although the attitude tracking error is significantly greater than during the movement phase due to end-effector tracking constraints, it is close to the achievable limit under these conditions. The strategy performs best when the target height is between 0.4 m and 0.7 m above the ground. Beyond this range, the tracking performance of all commands deteriorates significantly due to physical limitations such as the body's working height and the reach of the robotic arm.

[0082] These results demonstrate that explicit pattern-aware conditionation is crucial when learning flexible six-dimensional end-effector tracking in a single policy. By aligning attitude-dependent inputs and rewards with the currently active tracking target, MCP enables a reliable switch between conventional end-effector tracking and attitude-conditional end-effector tracking.

[0083] Real-world environment experiment In real-world experiments, this disclosure designs two types of tasks to verify the effectiveness of the proposed method in practical scenarios and its ability to handle challenging tasks. This disclosure deploys Floating Base+IK, Roboduet, Deep Whole-Body Control (DBC), and the proposed method to evaluate the impact of whole-body control, long-range tracking, six-dimensional end-effector tracking, and flexible end-effector tracking on motion manipulation tasks. For each method, 100 trials are conducted for each independent task.

[0084] Open-world tasks This disclosure presents a set of grasping experiments at different target height ranges. In other methods, the end effector pose command is specified via a single joystick on a VR device. For the method of this disclosure, depending on the control mode, a single joystick can be used to specify only the end effector pose command, or two joysticks can be used to specify the end effector pose command and the body pose command respectively. Due to the lack of full-body control, Floating Base+IK is prone to self-collisions and collisions with the environment, resulting in poor performance. In contrast, other methods can passively adapt to the body pose while tracking the target pose, thus significantly improving the success rate. RoboTrio further supports active body pose adjustment during the grasping process, enabling the robot to reach a more suitable configuration and achieve the highest success rate across all height ranges. This advantage is even more pronounced for target heights above 60cm, with a success rate of approximately 80%.

[0085] Posture-constrained challenging tasks To further evaluate the performance of the method disclosed herein in challenging mobile manipulation tasks with body posture constraints, four sets of tasks were designed. In the first set of tasks, the robot operates under a table with a workspace height of less than 40 cm. In the second and third sets of tasks, objects are placed on shelves with a height of 110 cm and a horizontal depth of 25 cm from the edge of the shelf, respectively. cm and 40 cm In the fourth task, the object is placed at an extreme height of 130 cm to 140 cm.

[0086] In the first set of tasks, other methods could not actively adjust the robot's height, making collisions with the tabletop highly likely, with a success rate typically below 10%. In contrast, RoboTrio could actively lower its height, significantly reducing the risk of collisions and expanding the robot's workspace, increasing the success rate by over 50%. In the second and third sets of tasks, other methods could not actively adjust the robot's attitude, limiting the effective workspace of the robot's end effector and increasing the probability of collisions with shelves. In contrast, RoboTrio could adjust its height and pitch angle, allowing the robot to maintain a near-horizontal attitude when grasping objects at heights. This made it easier for the robot to reach inside the shelves, increasing the success rate by over 30% compared to other methods. In the fourth set of tasks, other methods struggled to reach such extreme heights, and their relatively fixed end effector attitude further increased the probability of collisions with shelves. In contrast, RoboTrio raised its body to near the maximum height allowed by the hardware while maintaining a near-horizontal attitude, significantly expanding the robot's vertical workspace and greatly improving the success rate.

[0087] Despite the high flexibility of the disclosed method, collisions with surrounding structures such as tables and shelves remain a major cause of failure in challenging tasks. Overall, RoboTrio can flexibly switch between different control modes and significantly improves the success rate of movement operations through active body posture adaptation.

[0088] Example 2 One embodiment of this disclosure provides a hierarchical control system for a four-wheeled robotic arm oriented towards end effector tracking, comprising: The initialization module is used to build the kinematic model of the four-wheeled robotic arm system; The hierarchical control module is used to define the control problem of the four-wheeled robotic arm system as a long-distance flexible six-dimensional end effector tracking task, decouple the overall task, and adopt a two-stage hierarchical strategy to achieve coordinated control and mode switching of long-distance flexible six-dimensional end effector tracking. The two-stage hierarchical strategy includes a low-level strategy and a high-level strategy. The low-level strategy learns the movement strategy and the operation strategy respectively. The high-level strategy coordinates the movement strategy and the operation strategy to achieve unified whole-body control. It introduces a proximity area reward and a movement-operation balance mechanism to guide the robot to approach the target efficiently and achieve a smooth stage transition during the movement-operation process. Combined with a mask-based conditional strategy, a single strategy can flexibly switch between conventional end effector tracking and posture-conditional end effector tracking.

[0089] Example 3 One embodiment of this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking.

[0090] Example 4 One embodiment of this disclosure provides a non-transitory computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking.

[0091] Example 5 One embodiment of this disclosure provides an electronic device, including a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to execute the hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking.

[0092] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0093] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0094] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking, characterized in that, include: Construct a kinematic model of a four-wheeled robotic arm system; The control problem of the four-wheeled robotic arm system is defined as a long-distance flexible six-dimensional end effector tracking task. The overall task is decoupled and a two-stage hierarchical strategy is adopted to achieve coordinated control and mode switching of long-distance flexible six-dimensional end effector tracking. The two-stage hierarchical strategy includes a low-level strategy and a high-level strategy. The low-level strategy learns a movement strategy and an operation strategy, respectively. The goal of the movement strategy is to track the target command. The input of the movement strategy includes the target command, the estimated linear velocity, and the historical observation data of the past set time steps. The historical observation data includes the base angular velocity, gravity projection, non-wheel joint position, joint velocity, and the action at the previous moment. The action at the current moment is divided into the expected position of the non-wheel joint and the expected velocity of the wheel joint. Then, the final output is converted into joint torque through a proportional-derivative controller. The high-level strategy achieves unified whole-body control by coordinating movement and operational strategies. Its goal is to complete flexible six-dimensional end effector tracking tasks by sending predefined commands to the low-level strategy. The high-level strategy input includes commands and observation data, which includes gravity projection, joint positions (excluding wheel joints), joint velocities, the previous high-level action, the positional error between the target position and the current end effector position, the rotation vector from the current posture to the target posture, and the coordinates of the target end effector and the predefined head reference point in the base coordinate system. x and y Position error in the axial direction, and the position error of the target end effector relative to the head reference point in the base coordinate system. xy The relative heading angle in the plane; the output of the higher-level strategy is divided into two parts, namely the instructions of the operation strategy and the movement strategy. The higher-level strategy achieves the coupled control of the wheeled quadruped robot system by simultaneously predicting the target instructions of the movement strategy and the operation strategy in the lower-level strategy. Introducing proximity zone rewards and a movement-operation balance mechanism to guide the robot toward the target, including: Define a head reference point in the base coordinate system and use this reference point in all calculations. Calculate the base coordinate system based on the relationship between the target position and this reference point. x shaft and y Distance scaling factor along the axis; Calculate the heading factor and relative heading angle, and calculate the base coordinate system. x and y The direction of motion along the axis and the direction of yaw rotation are determined by the relative position between the target and the reference point, generating the reference velocity. It achieves a smooth phase transition during the movement-operation process, and combined with a mask-based conditional strategy, it enables a single strategy to flexibly switch between conventional end effector tracking and attitude-conditional end effector tracking.

2. The hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking as described in claim 1, characterized in that, The goal of the operation strategy is to track six-dimensional pose commands. The inputs to the operation strategy include target commands and observation data. The observation data includes joint positions, joint velocities, previous motion, positional error between the target position and the current end effector position, and rotation vector from the current posture to the target posture. The current robotic arm motion is represented by the desired positions of the six robotic arm joint motors.

3. The hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking as described in claim 1, characterized in that, The movement-operation task emphasizes different objectives at different stages. When the target is far away, the strategy focuses on moving the base towards the target rather than end effector pose tracking. When the target enters the robot arm's workspace, the strategy shifts its focus to whole-body control for end effector pose tracking, while adjusting the base to a suitable operating position. To balance the objectives of the movement-operation task, a movement-operation balancing mechanism is proposed: based on the activation phase of different reward items, they are divided into movement rewards, operation rewards, and basic rewards. Then, a distance coefficient is introduced to indicate the effective phase of different reward items. Therefore, the reward composition is continuously adjusted with distance, enabling the strategy to smoothly switch its behavior between different phases.

4. A hierarchical control system for a four-wheeled robotic arm oriented towards end-effector tracking, characterized in that, Specifically, the hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking as described in any one of claims 1-3 includes: The initialization module is used to build the kinematic model of the four-wheeled robotic arm system; The hierarchical control module is used to define the control problem of the four-wheeled robotic arm system as a long-distance flexible six-dimensional end effector tracking task, decouple the overall task, and adopt a two-stage hierarchical strategy to achieve coordinated control and mode switching of long-distance flexible six-dimensional end effector tracking. The two-stage hierarchical strategy includes a low-level strategy and a high-level strategy. The low-level strategy learns the movement strategy and the operation strategy respectively. The high-level strategy coordinates the movement strategy and the operation strategy to achieve unified whole-body control. It introduces a proximity area reward and a movement-operation balance mechanism to guide the robot to approach the target and achieves a smooth phase transition during the movement-operation process. Combined with a mask-based conditional strategy, a single strategy can flexibly switch between conventional end effector tracking and posture-conditional end effector tracking.

5. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking as described in any one of claims 1-3.

6. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking as described in any one of claims 1-3.

7. An electronic device, characterized in that, include: The device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform the hierarchical control method for a four-wheeled robotic arm oriented towards end effector tracking as described in any one of claims 1-3.

Images