Robot control method, device, equipment, storage medium and product

Abstract

Embodiments of the present application provide a robot control method, device, equipment, storage medium and product, which are applied to the technical field of robot control. The real-time pose of a target object in an environment is obtained; based on the real-time pose, a target task execution track corresponding to a task instruction is generated; in the process of controlling the robot to execute the task according to the target task execution track, the actual contact force of the robot end effector is obtained in real time, and based on the actual contact force, the expected pose and the expected contact force of the track point, the robot end motion information is calculated; the control instruction for controlling the robot is generated based on the robot end motion information; the control instruction is verified in real time, and the issuance of the control instruction is controlled according to the verification result, which can accurately plan the track while accurately regulating the contact force of the robot execution action, thereby effectively adapting to the household service operation scene.

Description

Technical Field

[0001] This application belongs to the field of robot control technology, and in particular relates to a robot control method, device, equipment, storage medium and product. Background Technology

[0002] With the rapid development of artificial intelligence and robotics, humanoid robots are increasingly being used in the home service sector, covering a variety of tasks such as moving items, cleaning, and companionship. Currently, the mainstream control methods for humanoid robots mainly rely on visual recognition for trajectory planning. Subsequently, based on this trajectory, the robot's joints are driven to move, achieving precise tracking of the end effector's position and posture, and realizing basic grasping, moving, and manipulating functions.

[0003] However, existing robot control methods are prone to damaging household items due to improper control of the force applied, pose safety hazards when in contact with the human body, and are severely unsuitable for tasks requiring fine force perception (such as wiping windows, mopping floors, and organizing delicate items), making it difficult to meet the complex and ever-changing needs of household service tasks. Summary of the Invention

[0004] This application provides a robot control method, device, equipment, storage medium, and product that can accurately control the contact force of the robot's actions while precisely planning the trajectory, thereby effectively adapting to complex and ever-changing home service work scenarios.

[0005] In a first aspect, embodiments of this application provide a robot control method, the method comprising: Obtain the real-time pose of the target object in the environment; Based on the real-time pose, the target task execution trajectory corresponding to the task instruction is generated; wherein, the target task execution trajectory includes at least one trajectory point, and each trajectory point corresponds to a desired pose and a desired contact force; During the process of controlling the robot to perform a task according to the target task trajectory, the actual contact force of the robot's end effector is acquired in real time, and the robot's end motion information is calculated based on the actual contact force, the expected pose of the trajectory point and the expected contact force. Control commands for controlling the robot are generated based on the robot's end-effector motion information; The control commands are verified in real time, and the issuance of control commands is controlled based on the verification results. In one feasible implementation, based on real-time pose, the target task execution trajectory corresponding to the task instruction is generated, including: Obtain the task template corresponding to the task instruction. The task template includes at least one key trajectory point, and each key trajectory point is constructed based on a dynamic parent coordinate system. The dynamic parent coordinate system is dynamically adjusted based on the real-time state of the target object. Based on the transformation relationship between the dynamic parent coordinate system and the world coordinate system, the poses of key trajectory points in the task template are converted into the desired poses in the world coordinate system to generate the target task execution trajectory.

[0006] In one feasible implementation, the target object includes at least one of the following: a target object, an execution tool end configured at the robot end, or a preset virtual object within the task space.

[0007] In one feasible implementation, the robot end effector motion information is calculated based on the actual contact force, the desired pose of the trajectory point, and the desired contact force, including: Calculate the contact force deviation based on the actual contact force and the expected contact force; Based on the admittance control equation, combined with the desired pose of the current trajectory point, the real-time pose of the current trajectory point, and the contact force deviation, the end-effector motion correction is calculated. The initial end-effector motion information is calculated based on the expected pose of the current trajectory point and the expected pose of the next trajectory point. Then, the initial end-effector motion information is corrected by combining the end-effector motion correction amount to obtain the end-effector motion information.

[0008] In one feasible implementation, control commands for controlling the robot are generated based on the end-effector motion information, including: Based on the end-effector motion information, a kinematic constraint model is constructed with the goal of realizing the end-effector motion information, and solving the kinematic constraint model is taken as the first priority control task. Based on the coefficient matrix corresponding to the kinematic constraint model, a null space projection operator is constructed. Using the null space projection operator as a constraint, the second priority control task is solved, and the solution is used as the robot control command; wherein, the second priority control task includes: solving at least one of the singularity avoidance constraint model and the joint limit constraint model.

[0009] In one feasible implementation, before solving the second priority control task with the null space projection operator as a constraint, the method further includes: Solve the first priority control task to obtain the initial control instructions; When the initial control command causes the motion state of a robot joint to exceed its preset physical limit threshold, the robot joint that exceeds the physical limit threshold is identified as a saturated joint. The coefficient matrix corresponding to the kinematic constraint model is updated based on the saturated joint, and the null space projection operator is reconstructed using the updated coefficient matrix.

[0010] Secondly, embodiments of this application provide a robot control device, the device comprising: The acquisition module is used to acquire the real-time pose of the target object in the environment; The first generation module is used to generate the target task execution trajectory corresponding to the task instruction based on the real-time pose; wherein the target task execution trajectory includes at least one trajectory point, and each trajectory point corresponds to a desired pose and a desired contact force; The calculation module is used to acquire the actual contact force of the robot's end effector in real time during the process of controlling the robot to perform tasks according to the target task trajectory, and to calculate the robot's end effector motion information based on the actual contact force, the expected pose of the trajectory point and the expected contact force. The second generation module is used to generate control commands for controlling the robot based on the robot's end-effector motion information. The verification module is used to verify the control commands in real time and control the issuance of control commands based on the verification results.

[0011] Thirdly, embodiments of this application provide an electronic device, the device including: a processor, and a memory storing computer program instructions; A robot control method in which the processor reads and executes computer program instructions to achieve any one of the first aspects.

[0012] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement a robot control method as described in any of the first aspects.

[0013] Fifthly, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform a robot control method as described in the first aspect.

[0014] This application provides a robot control method, device, equipment, storage medium, and product that can acquire the real-time pose of a target object in the environment, generate a target task execution trajectory corresponding to the task instruction based on the real-time pose, and, during the actual control of the robot to execute the task according to the target task execution trajectory, integrate the actual contact force of the robot's end effector. Based on the actual contact force, the expected pose of the trajectory point, and the expected contact force, the robot's end effector motion information is calculated. This enables coordinated control of the end effector pose and contact force, real-time compensation for deviations caused by the flexible deformation of the robotic arm, environmental interference, and slight displacement of the target object, and precise adjustment of the end effector's motion speed, posture, and contact pressure. This fundamentally avoids the problem of work failure caused by improper operation force, meets the requirements of fine force perception tasks, and finally, the control instructions generated based on the end effector motion information can drive the robot to accurately track the target task execution trajectory. While ensuring the accuracy of the end effector motion, the expected contact force is stably maintained, ensuring that the robot can safely complete fine tasks in complex dynamic scenarios such as home services. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating a robot control method according to a first embodiment of this application is shown; Figure 2 A flowchart illustrating a robot control method according to a second embodiment of this application is shown; Figure 3 A flowchart illustrating a robot control method according to a third embodiment of this application is shown; Figure 4 A flowchart illustrating a robot control method according to a fourth embodiment of this application is shown; Figure 5 The computational unit of a robot control method according to this application is shown; Figure 6 A schematic diagram of the robot control device structure provided in an embodiment of this application is shown; Figure 7 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0017] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0018] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0019] It should be noted that the acquisition, storage, use, and processing of data in this application embodiment all comply with the relevant provisions of national laws and regulations.

[0020] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0021] Currently, in the field of robot control, existing methods mainly rely on visual recognition technology for motion trajectory planning. Position control drives the robot to move along a preset trajectory, thereby achieving basic functions such as grasping and movement. However, in the home service sector, robots need to frequently interact physically with various household items and people. The objects handled include fragile and soft items made of different materials. Existing robot control methods can lead to problems such as damage to household items due to improper control of the force applied, safety hazards when in contact with people, and insufficient adaptability for tasks requiring fine force perception, as they can only execute actions along fixed trajectories.

[0022] To address the problems of existing technologies, embodiments of this application provide a robot control method, apparatus, device, storage medium, and product. By integrating the actual contact force of the robot's end effector, it can achieve coordinated control of the end effector's pose and contact force, precisely adjust the end effector's motion speed, posture, and contact pressure, fundamentally avoiding the problem of work failure caused by improper operation force, meeting the requirements of fine force perception tasks, and finally, the control commands generated based on the end effector motion information can drive the robot to accurately track the target task execution trajectory, ensuring the end effector motion accuracy while stably maintaining the desired contact force, and ensuring that the robot can safely complete fine tasks in complex dynamic scenarios such as home services.

[0023] The following section first introduces a robot control method provided in the embodiments of this application.

[0024] Figure 1 A flowchart illustrating a robot control method according to a first embodiment of this application is shown. Figure 1 As shown, the method may include the following steps: S101: Obtain the real-time pose of the target object in the environment.

[0025] In this embodiment, the target object refers to a specific item or scene element that the robot needs to manipulate, such as a table to be wiped, a cup to be grasped, or books to be organized. Real-time pose refers to the 6-dimensional pose of the target object in the robot's global world coordinate system, including three-dimensional position (e.g., X, Y, Z axes, describing spatial position) and three-dimensional orientation (e.g., represented by rotation angles around the X, Y, Z axes, such as Euler angles or quaternions). This pose is dynamically updated. The perception processing layer can acquire the real-time pose of the target object in the environment. The following illustrates a specific implementation method for acquiring the real-time pose: First, to construct a unified environment model that combines global coverage with high-resolution local details, enabling the acquisition of the real-time pose of target objects, the robot employs a layered vision system consisting of a head-mounted main camera, a chest-mounted auxiliary camera, and dual-wrist close-range cameras (high-precision operation areas). This system effectively avoids occlusion and blind spots from a single viewpoint through complementary "far-mid-near" fields of view, and achieves higher spatiotemporal resolution in local operation areas where interaction is imminent. For example, when the robotic arm's end effector approaches a cup to be grasped, the wrist close-range camera can perform higher-resolution local imaging of the cup and its handle area, providing finer geometric details in space and enabling higher-frequency pose tracking and updates in time. Furthermore, based on the high-resolution local imaging, the surface normal vectors (such as the horizontal orientation of a tabletop) and curvature (such as the arc of a plate's edge) of objects within the area can be accurately calculated, providing refined geometric data for the end effector's posture adjustment and force control. Among them, the camera can be an RGBD (Red, Green, Blue, Depth) depth camera, which can simultaneously acquire color images and depth information of the environment, thereby providing comprehensive and accurate basic data for environment modeling, pose calculation and subsequent trajectory planning.

[0026] To accurately fuse images captured by cameras from different perspectives and construct a unified environment model, markers with known geometric features, such as a checkerboard or AprilTag Visual Fiducial System, can be placed in front of the robot. This allows each camera to capture multiple sets of images from different perspectives. Then, the Oriented FAST and Rotated BRIEF (ORB) algorithm is used to extract stable sparse features such as corners and edges from each camera's images. The Random Sample Consensus (RANSAC) algorithm is then used to filter out valid interior points from noisy and occluded feature data, eliminating outliers caused by lens distortion and environmental interference to ensure robust feature matching. Finally, the Bundle Adjustment (BA) optimization algorithm iteratively adjusts the intrinsic parameters (focal length, principal point coordinates, distortion coefficients, etc.) and extrinsic parameters (camera position and orientation in the world coordinate system) of each camera. By minimizing the reprojection error of the same feature point in the multi-view images and the consistency deviation of the depth data, unified and stable camera intrinsic and extrinsic parameters are obtained.

[0027] In another example, because the mounting positions of the head, chest, and wrist cameras dynamically change during robot operation, and issues such as assembly errors and joint flexibility deformation in the robotic arm cause camera parameters to change, the camera parameters can be dynamically adjusted according to the operation scenario and perception requirements during robot execution. To ensure a balance between 3D modeling accuracy and computational efficiency, the parameter adjustment strategy and triggering timing can be dynamically changed for cameras mounted in different locations. For example, for cameras mounted in relatively fixed positions (such as the head and chest), their extrinsic parameters change slowly; therefore, low-frequency, event-based periodic calibration can be used. Triggering timings include: during task breaks after long periods of continuous operation, and when system self-checks detect that the consistency of multi-view data fusion remains below a threshold, parameter correction can be performed. For wrist cameras that move continuously with the robotic arm, their extrinsic parameters are significantly affected by joint motion and flexible deformation, and the changes are real-time and high-frequency. Therefore, high-frequency dynamic compensation synchronized with the motion control cycle is adopted. In each control cycle, the theoretical extrinsic parameters of the wrist camera are first calculated in real time through the robot's forward kinematics, and online fine-tuning or local window BA optimization is performed based on the reference markers in the current field of view to compensate for the deviation caused by flexible deformation and vibration in real time, and ensure the accuracy of local perception at the moment of operation.

[0028] In another example, to eliminate temporal discrepancies in data acquisition from different cameras, hardware timestamps and interpolation algorithms can be used to synchronize the data, ensuring that images and depth information from cameras at different locations have a unified time reference. Building upon this, to balance the real-time requirements of large-scale environmental understanding and local operations, a multi-frequency processing pipeline is constructed: computationally intensive tasks such as global mapping and semantic understanding are run on low-frequency threads, while close-range target tracking and pose refinement, involving fine-grained operations, are run on high-frequency threads.

[0029] After completing the multi-camera data synchronization and multi-frequency processing pipeline, the image data acquired from each viewpoint needs to be preprocessed to improve the accuracy of environment modeling and pose calculation. Specifically, each image data channel is first subjected to denoising, smoothing, and hole filling preprocessing in sequence. For example, Gaussian filtering is used to eliminate environmental noise interference, median filtering is used to optimize the surface smoothness of the depth map, and linear interpolation algorithms are used to fill missing holes in the image data to ensure the integrity and reliability of the image data. Subsequently, the data is processed in parallel by geometric reconstruction and semantic analysis channels to achieve a unified understanding of the environment and accurate localization of target objects.

[0030] The geometry reconstruction channel is responsible for constructing an accurate 3D geometric model of the environment. In one example, truncated least squares estimation with semidefinite programming relaxation (TEASER++) can be used to perform fast and robust initial relative pose estimation of multi-view point clouds, followed by fine alignment using the Iterative Closest Point (ICP) algorithm.

[0031] The semantic analysis channel is responsible for understanding the semantic composition of the environment and identifying actionable targets. This channel uses the YOLOv8-seg model (YOLOv8 instance segmentation model) to perform instance segmentation on red, green, and blue (RGB) images (where YOLOv8 (You Only Look Once Version 8) is an object detection algorithm), identifying movable object categories such as "cup" and "table". To distinguish between the parallax caused by the movement of real moving objects and the robot's own motion, a lightweight optical flow estimation algorithm (Pyramidal, Warping, and CostVolume Network, PWC-net) is used, combined with self-motion compensation, to detect real moving regions in the scene and generate dynamic masks to prevent moving objects from being incorrectly integrated into the static background reconstruction.

[0032] Based on the above parallel processing flow, the dynamic mask generated by the semantic analysis channel can be used to filter the geometrically aligned multi-view point cloud, and the static background point cloud can be incrementally fused into the global environment model, thereby achieving robust environment modeling with separation of dynamic and static elements.

[0033] In one example, the global environment model can be incrementally reconstructed using a hash-truncated signed distance field (TSDF) data structure. TSDF divides the space into a voxel grid, with each voxel storing the signed distance from its location to the nearest object surface (negative inside the object, positive outside), and only updating values ​​within the distance truncation threshold. This allows for efficient fusion of point clouds from multiple frames, generating a smooth and complete geometric representation of the scene. Simultaneously, a corresponding Euclidean signed distance field (ESDF) can be derived in real-time from this TSDF model. ESDF stores the precise Euclidean distance to the nearest surface and its gradient information in each voxel, explicitly expressing spatial occupancy, freedom, and distance information. This provides efficient collision lookup and obstacle avoidance support for the robot's subsequent whole-body path planning and navigation. Among them, TSDF and ESDF are two data structures that represent spatial occupancy, and they have different performance focuses: TSDF has higher computational efficiency in reconstruction and update process, and is suitable for high-frequency real-time scene fusion; while ESDF can provide accurate distance field information, and is more suitable for navigation and motion planning tasks that require strict distance constraints.

[0034] Based on the construction of a global environment model and the identification of operable targets, the precise pose estimation, tracking and evaluation of the target objects are further performed to provide dynamic reference information that meets the needs of fine operation.

[0035] For each operable target object identified by the semantic analysis channel, an initial pose estimate relative to the camera is obtained based on the RGB image using a pose estimation algorithm, such as the CosyPose 6D pose estimation algorithm. To improve the robustness and accuracy of the pose estimation, the initial pose is subjected to reprojection consistency screening by combining observation data from multiple view cameras, eliminating abnormal estimation results caused by occlusion or misidentification. The screened initial pose serves as the starting point for the refinement process. On the local high-precision point cloud corresponding to the target object, an iterative nearest-point algorithm is executed for refinement, thereby outputting a higher-precision 6D pose. This refinement can be based on the ICP algorithm.

[0036] This application does not calculate the pose all at once, but rather performs continuous pose tracking and uncertainty assessment for each operable target object, providing a precise dynamic reference benchmark for generating the target mission trajectory. The uncertainty comprehensively reflects the confidence level of the pose estimation, and can be determined based on factors such as ICP fitting residuals, consistency of multi-view observations, and sensor noise models.

[0037] To ensure the reliability of robot control commands, an active perception closed-loop mechanism is integrated. When the pose uncertainty of a target object exceeds a preset threshold (e.g., tracking failure due to brief complete occlusion), a multi-view re-observation or a wrist camera is actively triggered to perform a scanning motion along a specific trajectory to reacquire clear visual image data, thereby reducing uncertainty and restoring high-confidence pose tracking. For transient dynamic voxels detected by algorithms such as optical flow, a time decay strategy is employed to reduce their impact over time, thus avoiding continuous interference with the static environment model.

[0038] Finally, the perception processing layer can output: a global environment model, including TSDF / ESDF maps and their semantic labels; a list of operable objects, in which each object entry contains its precise 6D pose, real-time pose uncertainty, and semantic attributes (such as "cup" or "fragile"); and local high-precision geometric patches, providing local enhanced geometric information, including details such as surface normals and curvature, for the areas to be interacted with.

[0039] S102: Based on the real-time pose, generate the target task execution trajectory corresponding to the task instruction; wherein, the target task execution trajectory includes at least one trajectory point, and each trajectory point corresponds to a desired pose and a desired contact force.

[0040] In this embodiment, the target task execution trajectory is a sequence of trajectory points generated by the robot after receiving and parsing the task instructions, based on the real-time pose of the target object, to guide the end effector to complete the entire work process. Each trajectory point not only defines the expected pose (position and attitude) that the robot's end effector should reach at a certain moment, but also specifies the expected contact force (amplitude and direction of the force) that should be maintained with the environment or target object at that point. This ensures that the robot accurately reaches the work position, adapts to the real-time spatial pose of the target object, and avoids damage to household items due to excessive force or failure of cleaning, grasping, and other operations due to insufficient force.

[0041] For example, for the task of "wiping the table", the target task execution trajectory can be planned as follows: the trajectory starts at point A (the rag touches the edge of the table), passes through points B, C, ... (along an S-shaped path on the table), and ends at point N (leaving the table). The desired pose of trajectory point A is to make the center of the rag lightly touch a point on the edge of the table in a vertical posture, and the desired contact force is a positive pressure with a preset threshold applied in a direction perpendicular to the table. The desired poses of subsequent points are updated along the preset path, while the desired contact force can always maintain the same vertical pressure to ensure the wiping effect.

[0042] S103: During the process of controlling the robot to perform the task according to the target task trajectory, the actual contact force of the robot's end effector is acquired in real time, and the robot's end motion information is calculated based on the actual contact force, the expected pose of the trajectory point and the expected contact force.

[0043] In this embodiment, the real-world working environment presents numerous uncertainties, such as uneven surfaces, unknown or changing surface friction coefficients, etc. This causes the actual interaction force between the robot's end effector and the environment to deviate from the pre-set expected value. Therefore, it is necessary to acquire the actual contact force of the end effector in real time and dynamically adjust its motion state accordingly to achieve safe and adaptive physical interaction. End effector motion information refers to the instructions that drive the robot's end effector to move in Cartesian space, typically represented in the form of velocity or acceleration. The robot's end effector motion information can be calculated in real time based on the control execution layer.

[0044] In one example, initial end-effector motion information (such as desired velocity) can be calculated by trajectory interpolation and differentiation based on the desired pose of the current target point in the trajectory. Subsequently, based on the deviation between the actual contact force and the desired contact force, a motion correction amount is calculated in real time and superimposed on the initial motion information to obtain the final end-effector motion information with fused force feedback sent to the underlying controller.

[0045] In one example, by integrating six-dimensional force sensors at the wrist joints of both robot arms, the forces and torques of the robot's end effector in three directions can be measured in real time, thus providing accurate actual contact force.

[0046] S104: Generate control commands for controlling the robot based on the robot's end effector motion information.

[0047] In this embodiment, robot end-effector motion information refers to the instructions described in Cartesian space (task space) used to drive the robot's end effector motion, typically the end-effector's velocity, acceleration, or displacement increment. Robot control instructions, on the other hand, are the low-level physical instructions ultimately issued to the servo drives of each robot joint, typically manifested as joint position, velocity, or torque / current setpoints. Since the robot drivers directly receive and respond to instructions in the joint space, it is also necessary to convert the end-effector motion information in the task space into control instructions in the joint space through inverse kinematics or dynamic mapping to drive the robot entity to perform actions.

[0048] S105: Perform real-time verification of control commands and control the issuance of control commands based on the verification results.

[0049] In this embodiment of the application, since the home service environment is unstructured, highly dynamic, and often coexists with humans, in order to ensure that all actions of the robot are absolutely safe and reliable and to avoid dangerous commands caused by perception errors, planning deviations, or control calculation abnormalities, it is also necessary to perform real-time verification of the generated control commands at the final stage of command execution.

[0050] In one example, the validation rules include at least one of hard limits for each joint position and an instantaneous speed limit. For instance, after each set of speed commands is generated, it checks whether any joints have exceeded their limits or whether the command speed exceeds the threshold. If so, the command issuance is immediately blocked, and an emergency stop or controlled deceleration command is sent, while simultaneously triggering error logging. The log includes a timestamp, task identifier, violating joint number, current and expected values, and thresholds, and supports persistent storage of error logs. The rule set can be extended to limits for parameters such as acceleration, force / torque, and temperature according to task requirements, and the thresholds can be dynamically adjusted, thereby improving the safety and adaptability of system operation.

[0051] In this embodiment, to mitigate the problem of robots failing to meet fine force perception tasks due to improper control of operational force in existing technologies, this application acquires the real-time pose of the target object in the environment. Based on the real-time pose, it generates the target task execution trajectory corresponding to the task instruction. Furthermore, during the actual control of the robot to execute the task according to the target task execution trajectory, by fusing the actual contact force of the robot's end effector, and calculating the robot's end effector motion information based on the actual contact force, the expected pose of the trajectory point, and the expected contact force, it can achieve coordinated control of the end effector pose and contact force. This allows for real-time compensation for deviations caused by the flexible deformation of the robotic arm, environmental interference, and minor displacement of the target object, and precise adjustment of the end effector's motion speed, posture, and contact pressure. This reduces the problem of task failure due to improper operational force, meets the requirements of fine force perception tasks, and ultimately enables the control instructions generated based on the end effector motion information to drive the robot to accurately track the target task execution trajectory. While ensuring the accuracy of the end effector motion, it stably maintains the expected contact force, ensuring that the robot can safely complete fine tasks in complex dynamic scenarios such as home services.

[0052] Figure 2 A flowchart illustrating a robot control method according to a second embodiment of this application is shown. In the above... Figure 1 Based on the illustrated embodiments, as Figure 2 As shown, one specific implementation of step S102 is as follows: S201: Obtain the task template corresponding to the task instruction. The task template includes at least one key trajectory point, and each key trajectory point is constructed based on a dynamic parent coordinate system. The dynamic parent coordinate system is dynamically adjusted based on the real-time state of the target object.

[0053] In this embodiment, the task template is a predefined trajectory point for a standardized operation. The trajectory point in the task template does not record absolute world coordinates, but rather the relative position with respect to a dynamic parent coordinate system. Therefore, by decoupling the logic of "how to operate" from the scenario information of "where to operate", the reusability of the task template can be guaranteed, and the repetitive design of task templates for different scenarios of the same type of operation can be avoided.

[0054] Key trajectory points can be stored using a relative representation related to the target object. For example, the key trajectory points in the task template record the parent coordinate system identifier, relative pose parameters, and the direction and magnitude of the expected active output force. The relative pose parameters and the direction of the force are established with reference to the parent coordinate system.

[0055] In one example, the target object includes at least one of the following: a target object, an execution tool end configured at the robot end, or a pre-defined virtual object within the task space.

[0056] In this embodiment, the target object, i.e., the carrier of the dynamic parent coordinate system, includes at least one of the following: a target object, an execution tool end configured on the robot's end effector, and a preset virtual object within the task space. The target object can be a physical entity to be manipulated, such as a cup or a door handle; the execution tool end configured on the robot's end effector can be the tool itself mounted on the robot's wrist, such as a clamped screwdriver or welding torch; and the preset virtual object within the task space can be a virtual coordinate system or path point defined manually in space.

[0057] Therefore, the parent coordinate system can be flexibly pointed to any of the above-mentioned carriers, thereby realizing the planning of different types of task assignments: When the parent coordinate system points to the target object (such as a cup), the relative trajectory in the task template will automatically update its absolute path in the world coordinate system as the cup moves in real time. This allows the robot to grasp a cup that has been moved unexpectedly, enabling real-time tracking of dynamic targets.

[0058] When the parent coordinate system points to the execution tool end (such as the nozzle of a spray gun) configured at the robot's end effector, the trajectory describes the movement path of the tool end effector itself. For example, for a template "spraying an S-shaped pattern," the trajectory points are the relative paths that the spray gun nozzle should take relative to its own initial coordinate system. This ensures that regardless of the robot's posture when holding the spray gun, the tool end effector's work trajectory can be executed with absolute precision, making it suitable for high-precision tasks such as painting and applying glue.

[0059] When the parent coordinate system points to a pre-defined virtual object within the task space (such as a virtual point in the center of a room), the trajectory describes the movement of the robot's end effector relative to that fixed virtual coordinate system. This is suitable for absolute space operations independent of specific objects, such as performing demonstrative actions at a fixed location or conducting inspections and patrols around a virtual center.

[0060] S202: Based on the transformation relationship between the dynamic parent coordinate system and the world coordinate system, the poses of the key trajectory points in the task template are converted into the desired poses in the world coordinate system to generate the target task execution trajectory.

[0061] In this embodiment of the application, during the robot's execution, the planning and decision layer can receive the real-time pose of the target object in the world coordinate system obtained by the perception and processing layer. Then, for each key trajectory point defined in the task template, the recorded pose can be converted into the expected pose in the world coordinate system through coordinate transformation, thereby obtaining the absolute expected pose of the trajectory point.

[0062] In one example, because the state (position and orientation) of the target object bound to its dynamic parent coordinate system may constantly change due to external intervention or its own movement during the robot's execution of instructions, i.e., as it moves along the target task's trajectory, the perception processing layer needs to continuously acquire the real-time pose of the target object and update the coordinate transformation relationship in real time, thereby continuously recalculating the absolute expected pose of subsequent points in the trajectory. This ensures that even if the target object moves or rotates during the operation, the robot's execution trajectory can adapt and adjust in real time, maintaining the correct operational logic.

[0063] Similarly, since force is a vector, its direction of action is also coordinate system dependent. Therefore, the desired contact force defined in the task template is also described based on the dynamic parent coordinate system. When the attitude of the parent coordinate system changes, the direction vector of the desired force will be synchronously rotated and transformed according to the latest attitude of the parent coordinate system to ensure that the force applied in the world coordinate system always maintains the preset and correct geometric relationship with the surface of the target object (such as always being perpendicular to the surface).

[0064] This feature enables the direct generalization of the same task template across scenarios where the target object's position and orientation can change arbitrarily, through parent coordinate system binding. For example, a "wiping" template defined in the "desktop" parent coordinate system can quickly generate a world coordinate system wiping trajectory suitable for the new location, regardless of where the table is moved in the room, simply by obtaining the new pose of the desktop, without any manual reprogramming or online replanning.

[0065] In a specific example, to obtain smooth and continuous execution instructions, after obtaining discrete key trajectory points in the world coordinate system, the key trajectory points are interpolated at constant distances or time intervals (e.g., using linear or polynomial interpolation algorithms) to generate a dense sequence of trajectory points. For example, dozens of intermediate points are inserted between two key points. This densification process facilitates accurate velocity planning and acceleration smoothing in subsequent control layers and ensures consistency between the force control direction and the desired contact posture throughout the entire trajectory.

[0066] Furthermore, this embodiment introduces an intelligent state management mechanism to improve execution robustness. A single "current target trajectory point" is defined on the generated target task execution trajectory and assigned four states: start, pause, continue, and stop. The "current target trajectory point" changes continuously over time along the target task execution trajectory.

[0067] When a preset pause event is detected (such as actual contact force far exceeding the expected threshold, trajectory tracking pose deviation exceeding limits, or a decrease in perception confidence), the system immediately sets the current point state to "pause," freezing trajectory progression. The perception confidence is the pose uncertainty acquired by the perception processing layer. The robot's end effector will maintain its current position and actively adjust the output force to a safe holding value to avoid damage caused by continuous resistance. When the continuation conditions are met (such as the removal of abnormal external force, pose error returning to within the threshold, or a high-confidence pose being reacquired through active perception), the state switches to "continue," automatically resuming the remaining trajectory execution from the current point. The stop state is used for safe termination upon task completion or in the event of an unrecoverable anomaly. This operational mechanism allows for localized adjustments to task execution based on real-time events such as external forces or perception uncertainties, without requiring global replanning, ensuring both efficiency and robustness of the robot's operation in dynamic, unstructured environments.

[0068] In this embodiment, by acquiring the real-time state of the target object and defining a parent coordinate system based on the real-time state, the key trajectory points in the task template only need to record the relative pose and relative expected force, without binding to the absolute position of a fixed world coordinate system. This allows the task template to directly adapt to various dynamic change scenarios such as position shift, posture flip, and spatial displacement of the target object. This reduces the problem in the prior art where trajectory planning is based on a fixed world coordinate system, and the original trajectory becomes invalid after the target object's pose changes, requiring trajectory planning to be redone. Therefore, the task template can be generalized and reused across scenarios, improving the efficiency of trajectory planning.

[0069] Figure 3 A flowchart illustrating a robot control method according to a third embodiment of this application is shown. In the above... Figure 1 Based on the illustrated embodiments, as Figure 3As shown, one specific implementation of step S103 is as follows: S301: Calculate the contact force deviation based on the actual contact force and the expected contact force.

[0070] In this embodiment, the contact force deviation is the core parameter for achieving force control, used to quantify the degree of deviation between the actual contact state of the end effector and the preset operation requirements.

[0071] In one example, the contact force deviation can be calculated at each trajectory point to improve the accuracy and real-time response of contact force control. This ensures that the actual contact force at each trajectory point can accurately approach the expected contact force throughout the entire process of the end effector moving along the target task execution trajectory. This effectively avoids the accumulation of deviations between adjacent trajectory points, thereby preventing job failures caused by excessive local contact force deviations (such as missed wiping, unstable gripping, and scratches on the object surface).

[0072] In another example, to balance operational efficiency with the rational allocation of computing resources and avoid unnecessary computational waste caused by point-by-point calculation of contact force deviation, the calculation frequency and timing of contact force deviation can be flexibly adjusted based on the semantic attributes of the target object. For example, for target objects made of fragile and soft materials such as ceramics and glass, a point-by-point calculation mode can be adopted to precisely control the range of contact force fluctuations; for target objects made of hard and impact-resistant materials such as solid wood and metal, the calculation frequency can be appropriately reduced to balance efficiency and accuracy. Alternatively, the calculation strategy can be divided according to the complexity of the task: for core tasks such as grasping and wiping, point-by-point contact force deviation calculation can be forcibly performed on key trajectory points (such as clamping contact points, trajectory points of key wiping areas, and trajectory points of object edge contact) to ensure the force control accuracy of core operations; for non-core tasks such as obstacle avoidance transitions and long-distance transportation, the contact force deviation can be calculated every few trajectory points (such as every 3-5 trajectory points) to reduce the computational load without affecting the quality of the operation.

[0073] S302: Based on the admittance control equation, combined with the desired pose of the current trajectory point, the real-time pose of the current trajectory point, and the contact force deviation, the end motion correction is calculated.

[0074] In this embodiment, admittance control, by simulating a second-order mass-spring-damped system, can map force deviations into motion adjustments. Therefore, based on the admittance control equation and combined with the actual contact force, the external environmental forces can be mapped into motion information in the Cartesian coordinate system at the end point. The end-point motion correction can be a velocity correction or an acceleration correction.

[0075] The admittance governing equation is: (1) Where M is the virtual inertia matrix, B is the damping matrix, and K is the stiffness matrix. The desired pose of the current trajectory point. This represents the expected acceleration of the current trajectory point. This represents the expected velocity of the current trajectory point. This represents the real-time pose of the current trajectory point. This is used to measure external force, specifically the actual contact force read by a six-dimensional force sensor. In practical control, the contact force deviation can also be used directly.

[0076] In one example, since the compliance characteristics of the end effector in different directions and rotational degrees of freedom can be controlled by adjusting the diagonal elements of each parameter matrix: reducing the stiffness K value allows the end effector to make reasonable concessions at contact, increasing the damping B value suppresses oscillations, and adjusting the inertia M value affects the dynamic response speed, in order to optimize the interaction performance to adapt to diverse requirements and avoid robot control failure, the key parameters in the admittance control equation can be dynamically adjusted based on the semantic information of the task and the real-time interaction state. In other words, the virtual inertia matrix, damping matrix, and stiffness matrix are not fixed, but constitute a dynamically configurable set of impedance parameters.

[0077] For example, preset parameter templates can be invoked based on task semantic information (such as "wiping fragile surfaces" or "tightening bottle caps"). For the "wiping fragile surfaces" task, a very low stiffness K value and a moderate damping B value can be used in the direction perpendicular to the surface, allowing the robot end effector to easily "yield" when encountering protrusions, avoiding scratching or crushing the surface; a higher damping B value is used in the tangential wiping direction to suppress slippage oscillations and ensure smooth wiping. For the "tightening bottle caps" task, a higher stiffness K value can be set in the rotational degree of freedom around the axis to ensure rotation into place, while moderate stiffness and damping are used in the axial direction to maintain stable downward pressure.

[0078] Alternatively, online fine-tuning can be performed based on the real-time interactive status. For example, when an abnormally large increase in the rate of change of the contact force between the end and the environment is detected, the stiffness K can be reduced and the damping B can be increased to quickly absorb the impact energy and avoid generating destructive peak forces; once the contact is determined to be stable, the original parameters can be restored.

[0079] S303: Calculate the initial end-effector motion information based on the expected pose of the current trajectory point and the expected pose of the next trajectory point, and then correct the initial end-effector motion information by combining the end-effector motion correction amount to obtain the end-effector motion information.

[0080] In this embodiment, the end-effector motion information can be the motion command of the robot's end-effector in Cartesian space, specifically represented as an end-effector velocity command or an end-effector acceleration command. Correspondingly, the end-effector motion correction calculated from the admittance control equation must maintain a consistent form. When the end-effector motion information is set to an end-effector velocity command, the end-effector motion correction is a velocity correction, which can be directly obtained by integrating the admittance control equation once or by designing a velocity-type admittance controller. When the end-effector motion information is set to an end-effector acceleration command, the end-effector motion correction is an acceleration correction, which can be directly obtained by discretizing the admittance control equation.

[0081] In the embodiments of this application, the admittance control equation can establish a clear dynamic mapping relationship between the end effector motion and the contact force. Therefore, by calculating the contact force deviation, the force perception information can be converted into specific motion correction instructions in real time, thereby dynamically correcting the initial motion trajectory that does not take into account the actual interaction state. This enables the robot's end effector to strictly follow the task target while having the ability to safely, compliantly and adaptively interact with complex environments.

[0082] Figure 4 A flowchart illustrating a robot control method according to a fourth embodiment of this application is shown. In the above... Figure 1 Based on the illustrated embodiments, as Figure 4 As shown, one specific implementation of step S104 is as follows: S401: Based on the end-effector motion information, construct a kinematic constraint model with the goal of realizing the end-effector motion information, and take solving the kinematic constraint model as the first priority control task.

[0083] In this embodiment, the control commands for driving the robot are obtained by solving a series of optimization tasks with strict priority levels. The core of this approach is to classify the tasks, ensuring that the conditions of high-priority tasks are met first, while low-priority tasks are only allowed to execute if their execution does not conflict with any high-priority tasks. Each priority level (e.g., level k) consists of an equality constraint and an inequality constraint, expressed mathematically as follows: (2) in, Indicates the priority level of task K; This represents the coefficient matrix corresponding to the k-th layer task, reflecting the linear relationship between variables; It is the variable to be determined; This represents a constant vector, indicating the target value of the task at this layer. This represents the constraint matrix in the inequality; Indicates the lower limit of the constraint condition; This indicates the upper limit of the constraint condition.

[0084] Since end-effector motion information directly determines whether a robot can accurately track the target task trajectory and ensure the effectiveness of force and position coordinated control, it is a core control objective supporting the execution of precision tasks. Therefore, tracking and realizing end-effector motion information is processed as a first-priority task. Let's take a typical scenario where the end-effector motion information is represented as the desired end-effector velocity as an example: In this case, priority level K=1, and the coefficient matrix... This refers to the Jacobian matrix of the robot arm corresponding to the end-effector velocity tracking task layer, used to establish a linear mapping relationship between joint space control variables and end-effector Cartesian space motion velocity; variables to be determined The velocity value mentioned above is after end-effector motion correction. In other embodiments of this application, the physical form of end-effector motion information is not limited to velocity, but can also be extended to kinematic parameters such as desired end-effector displacement and desired end-effector acceleration, depending on the control requirements of the actual operation.

[0085] S402: Construct the null space projection operator based on the coefficient matrix corresponding to the kinematic constraint model.

[0086] In this embodiment of the application, after the construction of the first priority kinematic constraint model, in order to achieve hierarchical and orderly solution of multiple tasks, ensure that the solution and execution of low priority control tasks will not destroy the high priority control tasks that have been satisfied, and make full use of the redundant motion capability of the redundant degree of freedom manipulator to achieve other control objectives, this step constructs a null space projection operator based on the coefficient matrix corresponding to the kinematic constraint model, as a constraint on the solution space between tasks of different priorities.

[0087] The core function of the null space projection operator is to strictly limit the solutions of joint control commands for subsequent secondary control tasks to the null space of the Jacobian matrix corresponding to the completed previous priority task. This ensures that secondary tasks can only utilize redundant degrees of freedom that do not affect the implementation of higher priority tasks for optimization. The calculation formula for the null space projection operator is as follows: (3) in, This represents the null space projection operator corresponding to the k-th priority level; This represents the null space projection operator corresponding to the first k-1 priority levels, which forms the prerequisite basis for constructing the operator at the current level. , representing the coefficient matrix after pre-projection transformation, its physical meaning is that it represents the coefficient matrix of the k-th layer task. First map to the front Within the joint null space of Level 1 tasks; yes Moore-Penrose pseudo-inverse (Moore-Penrose generalized inverse). This indicates the conjugate transpose. Represents the identity matrix.

[0088] It should be noted that the core mechanism of the null space projection operator is based on the mathematical properties of its recursive construction and the nested nature of the null space. A joint null space for the first k-1 priority tasks is defined, and this subspace consists of all tasks that satisfy... The vector is composed of , where i represents the priority level index, meaning that any movement command within this space will not break the previous priority level. Equality constraints for level 1 tasks. When constructing... At that time, first through Project the Jacobian matrix of the k-th task onto the joint null space of the first k-1 priority tasks, and then use... Construct a new projection operator, which in The constraint direction of the k-th task is further removed within the defined subspace. Due to the idempotency and range-preserving properties of the projection operator, any vector passing through... After the action, it will inevitably remain Within the range of values, this is mathematically guaranteed. For all This ensures that the null space constraint of the first k-1 tasks is strictly maintained.

[0089] S403: Solve the second priority control task using the null space projection operator as a constraint, and use the solution result as the robot control command; wherein, the second priority control task includes: solving at least one of the singularity avoidance constraint model and the joint limit constraint model.

[0090] In this embodiment, the implementation of multiple constraints is ensured through the recursive construction of the projection operators and the hierarchical decomposition of the null space. Each null space projection operator... New constraints are added incrementally based on the constraints of the previous level. Utilizing the least-squares property of the Moore-Penrose pseudoinverse, the current level task is solved optimally (e.g., by minimizing the norm) while satisfying all high-priority constraints. This recursive structure enables the simultaneous handling of multiple constraints such as position constraints, velocity constraints, joint limitations, and singularity avoidance, all processed strictly according to priority.

[0091] In one example, the hierarchical control framework of this application supports flexibly setting different priority levels of constraints at different physical levels, such as position constraints, velocity constraints, and acceleration constraints, according to actual task requirements. For example, when performing assembly tasks requiring high-precision positioning, the end-effector position constraint can be set to high priority to ensure operational accuracy; while in wiping tasks that emphasize smooth motion and require suppression of jitter, the end-effector acceleration constraint can be placed in a higher priority to optimize dynamic performance. At the same time, other constraints (joint limits, singularity avoidance) are set as secondary priority constraints.

[0092] For example, when end-velocity tracking is the core task, it is set to the highest priority. Then the first-level projection matrix... Jacobian matrix directly targeting end-velocity tasks The structure is designed so that its null space contains all redundant degrees of freedom that do not affect end-effector velocity tracking. All subsequent low-priority tasks must find solutions within this constrained null space, thus ensuring that no matter what additional tasks the redundant robot performs, it will not interfere with end-effector velocity tracking, achieving absolute priority for the core task.

[0093] In this embodiment, a hierarchical solution framework is constructed, with the kinematic constraint model that achieves end-effector motion information as the core objective as the first priority control task. This prioritizes ensuring the core requirement of end-effector trajectory tracking. Based on the coefficient matrix (i.e., the Jacobian matrix of the robot arm) corresponding to the kinematic constraint model, a null space projection operator is constructed. The solution of the second priority control task (solving at least one of the singularity avoidance constraint model and joint limit constraint model) is strictly limited to the Jacobian null space of the first priority control task. This ensures that no matter what additional constraint tasks the redundant robot performs, it will not interfere with the accurate realization of the end-effector motion information, thus ensuring the priority satisfaction of the core task.

[0094] In one example, to ensure that the hierarchical solution process always occurs within the physically feasible domain of the robot joints and to avoid potential violations of inequality constraints during the hierarchical solution process, before solving the second priority control task using the null space projection operator as constraints, the process includes dynamically handling constraint violations using the saturation concept. Key constraints are converted into equality constraints and included in the saturation set. When redundancy is exhausted, the task is adjusted using a scaling factor. Specifically: Solve the first priority control task to obtain the initial control instructions; In this embodiment of the application, the first priority control task is first solved to obtain the initial control command. The initial control command is the low-level command that directly drives the servo motors of each joint of the robot. It is usually expressed as a control quantity in the joint space, such as joint speed command, joint position increment command or joint torque command.

[0095] In one example, when the end-effector motion information in the first priority control task is velocity information, the kinematic equation can be solved to obtain the joint velocity vector. However, to ensure that the above joint velocity vector can be adapted to the standard control interface of the robot's underlying servo drive, while taking into account the continuous smoothness of joint motion, position positioning accuracy, and drive output safety, and to avoid problems such as motion overshoot, cumulative deviation, and vibration shock caused by directly driving with only velocity commands, the joint velocity information needs to be further converted into initial control commands that can be directly executed at the underlying level. That is, the velocity components corresponding to each joint in the vector are respectively calculated and converted into two types of core control quantities: joint position increment and joint torque, which together constitute a complete initial control command.

[0096] When the initial control command causes the motion state of a robot joint to exceed its preset physical limit threshold, the robot joint that exceeds the physical limit threshold is identified as a saturated joint. In this embodiment, a saturated joint refers to a joint that has reached its motion capability boundary and can no longer move in the original command direction. Its motion state must be limited to the boundary value to prevent hardware damage or control failure.

[0097] In one example, a motion state quantity exceeding its preset physical limit threshold means that any one of the joint velocity, joint position increment, or joint torque exceeds its preset physical limit threshold.

[0098] The coefficient matrix corresponding to the kinematic constraint model is updated based on the saturated joint, and the null space projection operator is reconstructed using the updated coefficient matrix.

[0099] In this embodiment, the mathematical implementation of saturation processing involves removing joints that have reached their limits from the optimization variables and dynamically reconstructing the null space projection operator by modifying the corresponding Jacobian matrix structure. Specifically, for a saturated joint, its corresponding Jacobian matrix column is set to zero or excluded from the calculation, thus fixing the joint's motion to its limit value. This dynamic constraint processing ensures that the solution process always occurs within the feasible region, avoiding violations of inequality constraints while maintaining the mathematical consistency and convergence of the hierarchical solution framework. The Jacobian matrix is ​​the coefficient matrix corresponding to the kinematic constraint model.

[0100] In one example, when multi-level motion constraints exist, such as a second-priority control task that solves a singularity avoidance constraint model, the solution process and the handling of saturated joints will form a recursive dynamic interaction. Specifically, when solving the second-level task (singularity avoidance) based on the null projection operator of the first-priority task, it will simultaneously determine whether the joint motion increments obtained by the optimization solution in the current layer will cause the motion state quantities of some joints (including marked saturated joints and other joints) to exceed their physical limit thresholds. If so, these joints will be added or maintained as saturated joints.

[0101] For example, after solving the first-priority end-effector velocity tracking problem, joint A is marked as saturated because its position is close to the positive limit. In the second-priority singularity avoidance problem, the algorithm calculates a joint velocity increment in the null space that moves the robot arm away from the singular configuration. However, this increment may cause the velocity of joint B to exceed the maximum value. At this point, joint B is also marked as saturated, and the coefficient matrix is ​​updated to simultaneously fix the position of joint A and the velocity of joint B. Then, in the new null space where both joints A and B are constrained, the singularity avoidance task is recalculated, resulting in a feasible solution that avoids singularities without causing any joint out-of-bounds errors.

[0102] Specifically, the hierarchical controller of this application outputs the joint velocity solution of the k-th priority layer, which can be mathematically expanded into the sum of multiple vector components with clear physical meaning. Through the expansion of the joint velocity solution by superimposing multiple components, the coordinated satisfaction of multiple constraints in hierarchical priority control is realized from a mathematical perspective.

[0103] (4) The velocity solution of the joint is divided into multiple velocity components, each with a different function, collectively achieving the satisfaction of constraints and the attainment of the optimization objective. This expression is the core element for simultaneously satisfying equality and inequality constraints at each level, and the specific meaning of each component is as follows: : Represents the solution of the current layer (the kth layer).

[0104] : indicates the solution of the previous priority level (k-1 level), reflecting that the solution of low priority tasks depends on the results of high priority tasks, ensuring that high priority tasks are satisfied first.

[0105] : Indicates the component handling saturation constraints. Other components may violate inequality constraints while satisfying equality constraints or optimization objectives. Used to dynamically adjust its own value to compensate for deviations, ultimately ensuring It can satisfy all inequality constraints.

[0106] : Represents the equality constraint of the current layer, expressed as The Moore-Penrose pseudo-inverse projection is used to project the solution into the null space of the next higher layer (layer k-1), thus ensuring that the solution of the current layer task does not interfere with the upper layer task; where, This represents the coefficient matrix corresponding to the task in the current layer (the k-th layer); This represents the null space projection operator corresponding to the previous layer (k-1). This represents the constant vector corresponding to the current layer (the kth layer).

[0107] : Represents the compensation term for the current layer, expressed as follows This term is to compensate for specific deviation terms in the current level's equality constraints. Upper-level solution and saturation constraint components Violation of the equality constraints of the current layer after its introduction. Quantitative correction of deviations ensures that the results of equality constraints remain within the range of the upper-level null space, avoiding interference with the already calculated results at the upper level. Among these, This represents the coefficient matrix corresponding to the task in the current layer (the k-th layer); This represents the null space projection operator corresponding to the previous layer (k-1). This represents the inherent deviation term of the equality constraint at the k-th level; This represents the solution from the previous priority level (level k-1); This indicates the component that handles saturation constraints.

[0108] : Represents the redundancy space optimization term, expressed as: The design goal is to optimize the auxiliary performance of the current layer using reference vectors without violating hard constraints (equality and inequality constraints), fully utilizing redundant degrees of freedom while ensuring the feasibility of the solution. This represents the null space projection operator corresponding to the k-th priority level; This represents the reference velocity vector for redundancy optimization at the k-th layer.

[0109] In this embodiment, to avoid violations of the physical limits of robot joints during the solution process, i.e., the initial control command driving the joint movement causes its state variables to exceed the preset physical limit threshold, thereby leading to problems such as damage to the joint's mechanical structure and deviation of the work trajectory, when the initial control command causes the joint to exceed the physical limit threshold and become a saturated joint, the motion state of the saturated joint is limited to the preset physical limit threshold range by identifying the saturated joint, updating the coefficient matrix corresponding to the kinematic constraint model, and reconstructing the null space projection operator. The motion solution of the unsaturated joint is limited to the feasible region corresponding to the updated null space projection operator, thereby avoiding mechanical failures and work failures caused by joint over-limits. This enables the robot to operate stably in complex constraint environments, meeting the core task requirement of accurate tracking of end-effector motion information while ensuring the safety of the robot system.

[0110] Figure 5 The illustration shows a computational unit for a robot control method according to this application. This unit processes multimodal perception data from visual and force sensors, performing operations such as visual information processing, intelligent keyframe planning, trajectory interpolation generation, admittance control calculation, and real-time motion control command generation. The computational unit uses a seven-DOF dual-arm humanoid robot as its execution platform. Each arm has seven degrees of freedom, enabling flexible position control and posture adjustment in three-dimensional space, providing sufficient motion redundancy for complex tasks. Six-dimensional force sensors are integrated at the wrist joints of both arms to measure the forces and torques of the robot's end effectors in three directions in real time, providing accurate force feedback information to the system. RGBD depth cameras are configured on the robot's head, wrists, and chest as primary visual sensors. These cameras can simultaneously acquire color image information and depth information of the scene, enabling three-dimensional perception and recognition of the working environment and target objects, providing necessary spatial geometric information for subsequent keyframe planning and trajectory generation.

[0111] The computing unit consists of a low-speed real-time computing module 51 and a high-speed real-time computing module 52, further forming a dual-speed collaborative mechanism. The low-speed real-time computing module 51 includes a perception processing layer 511 and a planning and decision-making layer 512, while the high-speed real-time computing module 52 includes a control execution layer 521 and a security monitoring layer 522.

[0112] The low-speed real-time computing module 51 focuses on the preprocessing, fusion, and global keyframe and basic trajectory generation of high-dimensional and large-volume perception data. It has a long processing link, high algorithm complexity, and a refresh cycle in the second range, providing stable support for upper-layer planning and global consistency. Among them, the perception processing layer 511 is used to integrate multi-camera perspective data to build an accurate and comprehensive 3D environment model; the planning and decision layer 512 is used to construct the target motion trajectory containing pose and expected force information, and to advance the movement of the target point on the trajectory according to the time change.

[0113] The high-speed real-time computing module 52 closely follows the execution phase, performing fine-grained online corrections to the generated trajectory based on real-time force feedback and end-effector interaction status. Admittance adjustment achieves compliant and smooth interaction control, with a cycle time on the order of milliseconds to tens of milliseconds, thus maintaining responsiveness and trajectory smoothness in dynamic environments or human-machine collaborative scenarios. Specifically: the control execution layer 521 combines outer admittance control with inner layer hierarchical optimization speed control, enabling safe physical interaction with the environment while tracking the trajectory; the safety monitoring layer 522 verifies upcoming commands in real-time based on configurable safety rules and provides log management, thereby improving system operational safety. It can synchronously verify each control command issued, rapidly determining performance based on multi-dimensional constraints such as position, velocity, acceleration, joint limits, force thresholds, and potential collision risks. When a deviation from the safe domain is detected, degradation strategies such as amplitude limiting, deceleration, freezing, or safe exit are triggered, ensuring the system possesses both the decision-making stability of the global task layer and the high-speed interactive adaptability and operational reliability of the execution layer.

[0114] Based on the robot control method provided in the above embodiments, this application also provides a specific implementation of a robot control device. Please refer to the following embodiments.

[0115] First see Figure 6 , Figure 6 The diagram shows a schematic of the structure of a robot control device provided in an embodiment of this application. The robot control device 600 provided in this embodiment includes: an acquisition module 601, a first generation module 602, a calculation module 603, a second generation module 604, and a verification module 605.

[0116] The acquisition module 601 is used to acquire the real-time pose of the target object in the environment; The first generation module 602 is used to generate the target task execution trajectory corresponding to the task instruction based on the real-time pose; wherein the target task execution trajectory includes at least one trajectory point, and each trajectory point corresponds to a desired pose and a desired contact force; The calculation module 603 is used to acquire the actual contact force of the robot end effector in real time during the process of controlling the robot to perform a task according to the target task trajectory, and to calculate the robot end motion information based on the actual contact force, the expected pose of the trajectory point and the expected contact force. The second generation module 604 is used to generate control commands for controlling the robot based on the robot end effector motion information. The verification module 605 is used to verify the control commands in real time and control the issuance of control commands based on the verification results.

[0117] In one example, the first generation module 602 includes: The acquisition submodule is used to acquire the task template corresponding to the task instruction. The task template includes at least one key trajectory point, and each key trajectory point is constructed based on a dynamic parent coordinate system. The dynamic parent coordinate system is dynamically adjusted based on the real-time state of the target object. The transformation submodule is used to convert the poses of key trajectory points in the task template into the desired poses in the world coordinate system based on the transformation relationship between the dynamic parent coordinate system and the world coordinate system, so as to generate the target task execution trajectory.

[0118] In one example, the calculation module 603 includes: The first calculation submodule is used to calculate the contact force deviation based on the actual contact force and the expected contact force. The second calculation submodule is used to calculate the end motion correction based on the admittance control equation, combined with the expected pose of the current trajectory point, the real-time pose of the current trajectory point, and the contact force deviation. The correction submodule is used to calculate the initial end-effector motion information based on the expected pose of the current trajectory point and the expected pose of the next trajectory point, and then correct the initial end-effector motion information by combining the end-effector motion correction amount to obtain the end-effector motion information.

[0119] In one example, the second generation module 604 includes: The construction submodule is used to build a kinematic constraint model based on the end-effector motion information with the goal of realizing the end-effector motion information, and solving the kinematic constraint model is the first priority control task. The construction submodule is used to construct the null space projection operator based on the coefficient matrix corresponding to the kinematic constraint model; The solver submodule is used to solve the second priority control task with the null projection operator as a constraint, and to use the solution result as the robot control command; wherein, the second priority control task includes: solving at least one of the singularity avoidance constraint model and the joint limit constraint model.

[0120] In one example, the second generation module 604 includes: The solver submodule is used to solve the first priority control task and obtain the initial control instructions. The determination submodule is used to determine the robot joint that exceeds the physical limit threshold as a saturated joint when the initial control command causes the motion state of the robot joint to exceed its preset physical limit threshold. The construction submodule is also used to update the coefficient matrix corresponding to the kinematic constraint model based on the saturated joints, and to reconstruct the null space projection operator using the updated coefficient matrix.

[0121] Figure 7 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0122] The electronic device may include a processor 701 and a memory 702 storing computer program instructions.

[0123] Specifically, the processor 701 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0124] Memory 702 may include mass storage for data or instructions. For example, and not limitingly, memory 702 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. In one instance, memory 702 may include removable or non-removable (or fixed) media, or memory 702 may be a non-volatile solid-state memory.

[0125] In one instance, memory 702 may be read-only memory (ROM). In one instance, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0126] Memory 702 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described in the method according to one aspect of this disclosure.

[0127] The processor 701 implements a robot control method in the above-described embodiment by reading and executing computer program instructions stored in the memory 702.

[0128] In one example, the electronic device may also include a communication interface 707 and a bus 704. Wherein, as... Figure 7 As shown, the processor 701, memory 702, and communication interface 707 are connected through bus 704 and complete communication with each other.

[0129] The communication interface 707 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0130] Bus 704 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 704 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0131] The robot control method described in the above embodiments can be implemented using a computer-readable storage medium provided in this application. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the robot control methods described in the above embodiments.

[0132] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the robot control methods described in the above embodiments.

[0133] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0134] The flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products of this disclosure have been described above with reference to embodiments thereof. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowcharts and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0135] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A robot control method, characterized in that, include: Obtain the real-time pose of the target object in the environment; Based on the real-time pose, a target task execution trajectory corresponding to the task instruction is generated; wherein, the target task execution trajectory includes at least one trajectory point, and each trajectory point corresponds to a desired pose and a desired contact force; During the process of controlling the robot to perform the task according to the target task trajectory, the actual contact force of the robot end effector is acquired in real time, and the robot end motion information is calculated based on the actual contact force, the expected pose of the trajectory point and the expected contact force. Control commands for controlling the robot are generated based on the robot's end-effector motion information; The control commands are verified in real time, and the issuance of the control commands is controlled based on the verification results.

2. The method according to claim 1, characterized in that, The step of generating the target task execution trajectory corresponding to the task instruction based on the real-time pose includes: Obtain a task template corresponding to the task instruction. The task template includes at least one key trajectory point, and each key trajectory point is constructed based on a dynamic parent coordinate system. The dynamic parent coordinate system is dynamically adjusted based on the real-time state of the target object. Based on the transformation relationship between the dynamic parent coordinate system and the world coordinate system, the poses of the key trajectory points in the task template are converted into the desired poses in the world coordinate system to generate the target task execution trajectory.

3. The method according to claim 2, characterized in that, The target object includes at least one of the following: a target object, an execution tool end configured at the end of a robot, or a preset virtual object within the task space.

4. The method according to claim 1, characterized in that, The calculation of robot end effector motion information based on the actual contact force, the desired pose of the trajectory point, and the desired contact force includes: Calculate the contact force deviation based on the actual contact force and the expected contact force; Based on the admittance control equation, combined with the desired pose of the current trajectory point, the real-time pose of the current trajectory point, and the contact force deviation, the end-effector motion correction is calculated. The initial end-effector motion information is calculated based on the expected pose of the current trajectory point and the expected pose of the next trajectory point. The initial end-effector motion information is then corrected by combining the end-effector motion correction amount to obtain the final end-effector motion information.

5. The method according to claim 1, characterized in that, The generation of control commands for controlling the robot based on the end effector motion information includes: Based on the end-effector motion information, a kinematic constraint model is constructed with the goal of realizing the end-effector motion information, and solving the kinematic constraint model is taken as the first priority control task. Based on the coefficient matrix corresponding to the kinematic constraint model, a null space projection operator is constructed. Using the null space projection operator as a constraint, a second priority control task is solved, and the solution result is used as the control command for the robot; wherein, the second priority control task includes: solving at least one of a singularity avoidance constraint model and a joint limit constraint model.

6. The method according to claim 5, characterized in that, Before solving the second priority control task using the null space projection operator as a constraint, the following steps are also included: Solve the first priority control task to obtain the initial control command; When the initial control command causes the motion state of a robot joint to exceed its preset physical limit threshold, the robot joint that exceeds the physical limit threshold is identified as a saturated joint. The coefficient matrix corresponding to the kinematic constraint model is updated based on the saturated joint, and the null space projection operator is reconstructed using the updated coefficient matrix.

7. A robot control device, characterized in that, The device includes: The acquisition module is used to acquire the real-time pose of the target object in the environment; The first generation module is used to generate a target task execution trajectory corresponding to the task instruction based on the real-time pose; wherein the target task execution trajectory includes at least one trajectory point, and each trajectory point corresponds to a desired pose and a desired contact force; The calculation module is used to acquire the actual contact force of the robot's end effector in real time during the process of controlling the robot to perform a task according to the target task execution trajectory, and to calculate the robot's end effector motion information based on the actual contact force, the expected pose of the trajectory point and the expected contact force. The second generation module is used to generate control commands for controlling the robot based on the robot end effector motion information. The verification module is used to verify the control commands in real time and control the issuance of the control commands based on the verification results.

8. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement a robot control method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement a robot control method as described in any one of claims 1-6.

10. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device causes the electronic device to perform a robot control method as described in any one of claims 1-6.

Images