Robot learning data processing method and device based on human behavior data

Abstract

The embodiment of the invention provides a robot learning data processing method and device based on human behavior data, and the method comprises the steps: converting multi-source heterogeneous human behavior data into a unified intermediate behavior representation which describes a human operation intention through a key point pose; based on the kinematic model and the kinetic model of the target robot body, the intermediate behavior representation is mapped into a robot action sequence, and the robot action sequence meets the joint limit, moment constraint and contact stability requirements of the target robot body. By abstracting human behavior data into intermediate behavior representation irrelevant to a specific acquisition mode, unified processing of multi-source heterogeneous data is realized, and bidirectional mapping from human intention to robot action is realized based on kinematics and dynamics models. Through decoupling data acquisition, representation and mapping processes, the problems of data splitting, domain difference and insufficient generalization ability caused by body difference between human behavior data and a robot body in the prior art are solved.

Description

Technical Field

[0001] This application relates to the fields of data processing and robot control, and in particular to a robot learning data processing method and apparatus based on human behavior data. Background Technology

[0002] In scenarios such as industrial automation, home service robots, medical assistive devices, and human-robot collaborative systems, robots need to learn human behavioral patterns to complete complex tasks. These scenarios all require robots to have the ability to extract operational intentions from human behavioral data, understand task logic, and generate executable actions.

[0003] Humans and robots differ fundamentally in their physical structures. For example, humans possess flexible, multi-degree-of-freedom hands and complex joint topologies, while robots typically employ rigid robotic arms and fixed degrees of freedom. This results in significant domain differences when directly reusing human behavioral data. Furthermore, existing solutions struggle to support efficient learning and deployment of robots across different bodily structures for large-scale, cross-task human behavioral data collection and reuse. Summary of the Invention

[0004] This application provides a robot learning data processing method and apparatus based on human behavior data to solve the problems of data fragmentation, domain gaps and insufficient generalization ability caused by individual differences.

[0005] In a first aspect, embodiments of this application provide a robot learning method based on human behavior data, the method comprising:

[0006] Acquire multi-source, heterogeneous human behavior data;

[0007] Multi-source heterogeneous human behavior data is transformed into a unified intermediate behavior representation, which describes human operational intentions using key point poses.

[0008] Based on the kinematic and dynamic models of the target robot body, intermediate behaviors are mapped to robot action sequences, which satisfy the joint limits, torque constraints, and contact stability requirements of the target robot body.

[0009] In one possible implementation, transforming multi-source heterogeneous human behavior data into a unified intermediate behavior representation includes:

[0010] Determine the target processing path based on the source type of human behavior data;

[0011] Human behavior data is processed based on the target processing path to obtain intermediate behavior representations.

[0012] In one possible implementation, the source type includes indirect acquisition type; processing human behavior data based on the target processing path to obtain intermediate behavior representation includes:

[0013] Based on the configuration file of the robot or its simulation model, determine the robot's key points and the key points related to human operating intentions;

[0014] Map the robot's key points to Cartesian space to obtain the robot's pose in the Cartesian coordinate system;

[0015] Based on the key points related to human operational intentions, solve the pose sequence of the first human hand key points;

[0016] The intermediate behavior representation is determined based on the pose of the robot's key points in the Cartesian coordinate system and the pose sequence of the first human hand key points.

[0017] In one possible implementation, the source type includes direct acquisition, and the human behavior data includes video sequences; the human behavior data is processed based on the target processing path to obtain intermediate behavior representations, including:

[0018] Identify the pose of key points of the human upper limbs and hands in video sequences;

[0019] Perform at least one of the following processing steps on the pose of key points of the human upper limbs and hands: coordinate system alignment, scale normalization, and temporal alignment, to obtain the corresponding intermediate behavior representation.

[0020] In one possible implementation, the source type includes directly acquired types, and the human behavior data includes data collected by handheld devices; the human behavior data is processed based on the target processing path to obtain an intermediate behavior representation, including:

[0021] Acquire wrist poses related to operational behaviors from data collected by handheld devices, as well as gripper signals from handheld devices;

[0022] The gripper signals are mapped to a second sequence of key points in the human hand pose.

[0023] Based on the wrist pose related to the operation and the second human hand key point pose sequence, the intermediate behavior representation is determined.

[0024] In one possible implementation, based on the kinematic and dynamic models of the target robot body, intermediate behavior representations are mapped to robot action sequences, including:

[0025] Transform the key operational intentions in the intermediate behavior representation into target constraints in the task space of the target robot;

[0026] At each moment in the intermediate behavior representation, based on the target constraints and structural constraints in the task space, inverse kinematics solution is performed to generate a sequence of joint angles that satisfy the joint limits; the structural constraints are determined based on the kinematic model.

[0027] The execution model predictive control optimization jointly considers dynamics, contact and actuator capabilities within a finite time domain to generate robot motion sequences that satisfy torque constraints and contact stability.

[0028] In one possible implementation, the method further includes:

[0029] The robot's motion sequence is input into a simulation environment for pre-verification to detect whether there are joint limits, torque constraints, or collision risks.

[0030] If a constraint violation is detected, a rollback strategy is triggered, including adjusting the magnitude of the action, switching to the feasible solution of the previous time step, or adjusting the weight priority.

[0031] In one possible implementation, the method further includes:

[0032] The robot's action sequence is input into the robot learning system to train human strategies for intermediate behavior representations;

[0033] The trained human strategy is adapted to the target robot body to generate executable control signals.

[0034] Secondly, embodiments of this application provide a robot learning data processing apparatus based on human behavior data, comprising:

[0035] The data acquisition module is used to acquire multi-source heterogeneous human behavior data;

[0036] The data representation and specification module is used to transform multi-source heterogeneous human behavior data into a unified intermediate behavior representation, which describes human operational intentions using key point poses.

[0037] The cross-local mapping module is used to map intermediate behavior representations into robot action sequences based on the kinematic and dynamic models of the target robot body. The robot action sequences satisfy the joint limits, torque constraints, and contact stability requirements of the target robot body.

[0038] Thirdly, embodiments of this application provide a computer device, including: a memory and a processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0039] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0040] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0041] This application provides a robot learning data processing method and apparatus based on human behavior data. The method includes: converting multi-source heterogeneous human behavior data into a unified intermediate behavior representation, wherein the intermediate behavior representation describes human operational intentions using keypoint poses; and mapping the intermediate behavior representation into a robot action sequence based on the kinematic and dynamic models of the target robot body, wherein the robot action sequence satisfies the joint limits, torque constraints, and contact stability requirements of the target robot body. By abstracting human behavior data into an intermediate behavior representation independent of specific acquisition methods, unified processing of multi-source heterogeneous data is achieved, and a bidirectional mapping from human intentions to robot actions is realized based on kinematic and dynamic models. By decoupling the data acquisition, representation, and mapping processes, the method solves the problems of data fragmentation, domain differences, and insufficient generalization ability caused by the specific differences between human behavior data and the robot body in the prior art, providing a scalable technical framework for general robot learning. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0043] Figure 1 A flowchart illustrating the robot learning data processing method based on human behavior data provided in this application. Figure 1 ;

[0044] Figure 2 A flowchart illustrating the robot learning data processing method based on human behavior data provided in this application. Figure 2 ;

[0045] Figure 3 A flowchart illustrating the robot learning data processing method based on human behavior data provided in this application. Figure 3 ;

[0046] Figure 4 A schematic diagram of the robot learning data processing device based on human behavior data provided in this application;

[0047] Figure 5A schematic diagram of the structure of the computer device provided in this application.

[0048] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0049] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0050] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0051] First, let's explain the terms that appear in this article:

[0052] Embodied difference, also known as ontological difference, refers to the differences in a robot's physical form, materials, sensors, actuators, and other hardware characteristics that fundamentally determine its ability to perceive the world, interact with the environment, and ultimately learn and make decisions.

[0053] Domain gap refers to the inconsistency in data distribution and feature representation between the source domain (such as synthetic data, dataset A) and the target domain (such as real data, dataset B).

[0054] In scenarios such as industrial automation, home service robots, medical assistive devices, and human-robot collaborative systems, robots need to learn human behavior patterns to complete complex tasks. For example, in industrial settings, robotic arms need to mimic the assembly movements of human workers to perform precision assembly; in home services, service robots need to learn how humans operate appliances or move items; in the medical field, surgical robots need to analyze the surgeon's trajectories to assist in performing minimally invasive surgeries. These scenarios all require robots to have the ability to extract operational intentions from human behavioral data, understand task logic, and generate executable actions.

[0055] Existing robot learning systems primarily rely on the following data sources in practical applications:

[0056] (1) Interactive data generated in the simulation environment

[0057] This type of data can be generated at a low cost and on a large scale. However, due to the significant differences between the simulation environment and the real physical world in terms of dynamic characteristics, contact behavior, perceived noise, and environmental complexity, the models trained based on simulation data suffer from obvious performance degradation in the real world.

[0058] (2) Data acquisition of direct teleoperation of robots in real environment

[0059] This method typically involves a teleoperator directly controlling the target robot to complete tasks and collect data. The resulting data is of high quality and highly compatible with the target robot. However, this type of data acquisition is costly and inefficient, and the data is usually strongly bound to a specific robot body, making it difficult to reuse on robot platforms with different structures or degrees of freedom, resulting in significant domain differences.

[0060] (3) Internet-based video data

[0061] This method can acquire large-scale data at extremely low cost, but video data usually only contains visual information, lacks clear motion, force or control signals, and has a significant gap with the robot's execution space, with the greatest domain difference.

[0062] (4) Teaching data based on handheld devices or exoskeleton systems

[0063] This method collects human operational data through handheld or wearable devices. The collection cost and data quality fall between those of the methods mentioned above, but the data still depends on the specific collection device and mapping method, limiting its versatility.

[0064] This demonstrates that data from different sources (such as teleoperation data, video data, and simulation data) differ significantly in structure, semantic hierarchy, and physical characteristics, lacking a unified intermediate representation. Furthermore, the differences between humans and robots in the number of degrees of freedom, joint topology, and motion constraints make it difficult to directly use human behavioral data for robot learning. In addition, existing technologies tightly bind data to specific tasks or the robot itself, hindering cross-task and cross-platform reuse and limiting the construction of a general-purpose robot learning foundation model.

[0065] To address the aforementioned technical problems, this application provides a robot learning data processing method based on human behavior data. Using the human embodied space as a unified intermediate baseline, it constructs a robot learning system with cross-ontology mapping and data closure. By abstracting human behavior data into an intermediate behavior representation independent of specific acquisition methods, it achieves unified processing of multi-source heterogeneous data (simulation, teleoperation, video, exoskeleton, etc.) and realizes bidirectional mapping from human intentions to robot actions based on kinematic and dynamic models. This technical concept solves the problems of data fragmentation, domain differences, and insufficient generalization ability caused by embodied differences in existing technologies by decoupling the data acquisition, representation, and mapping processes, providing a scalable technical framework for general robot learning.

[0066] The robot learning data processing method based on human behavior data provided in this application is applicable to scenarios such as industrial automation, home services, medical assistance, and human-robot collaboration. For example, in industrial scenarios, robotic arms need to learn assembly operations through human instruction; in home services, service robots need to understand how humans operate home appliances; and in the medical field, surgical robots need to replicate the operating trajectories of surgeons.

[0067] In one embodiment, such as Figure 1 As shown, a robot learning data processing method based on human behavior data is described, and the method includes:

[0068] Step 101: Obtain multi-source heterogeneous human behavior data;

[0069] Step 102: Convert multi-source heterogeneous human behavior data into a unified intermediate behavior representation, which describes human operational intentions using key point poses.

[0070] Step 103: Based on the kinematic and dynamic models of the target robot body, the intermediate behavior representation is mapped into a robot action sequence. The robot action sequence satisfies the joint limits, torque constraints, and contact stability requirements of the target robot body.

[0071] Multi-source heterogeneous human behavior data refers to human operational behavior data acquired through different methods such as teleoperation, video acquisition, and exoskeleton systems, which differ in structure, semantic hierarchy, and physical characteristics. For example, teleoperation data includes the robot's end effector trajectory, video data includes sequences of human key points, and exoskeleton data includes upper limb pose information. It should be noted that, regardless of the data acquisition device or method used, the multi-source heterogeneous human behavior data in the embodiments of this application essentially represents the human's operational intentions and their temporal changes during task completion, rather than the control signals or execution results of a specific robot.

[0072] Human behavior data is recorded using a standardized keypoint pose format, where each keypoint corresponds to a human structure or a functional part directly related to human operation, and its position and / or orientation in space characterize the motion state of that part. In one embodiment, the human behavior data includes at least the following:

[0073] (1) Postural data of key points of the human upper limb:

[0074] Key points include at least one or more upper limb key points directly related to the operational behavior, such as the shoulder, elbow, and wrist, used to describe the postural changes and coarse-grained operational intentions of humans during the performance of operational tasks. In this embodiment, the data acquisition and representation process focuses on the human upper limbs and operation-related parts. For lower limb key points with a lower correlation to the operational behavior, their recording is optional or performed at a lower resolution.

[0075] (2) Fine-grained key point pose data of the hand:

[0076] To characterize fine motor skills such as grasping, pinching, and twisting, human behavioral data further includes hand keypoint pose data. Hand keypoints include 5 to 25 finger keypoints, and the pose of each keypoint contains at least spatial position and / or orientation information, used to describe the fine motor states of the human hand during interaction with objects or tools at high resolution.

[0077] (3) Key point representation of the tool, external object, and end effector behavior of the controlled robot:

[0078] In scenarios where humans use tools, manipulate external objects, or remotely control robot end effectors, this embodiment does not directly use the pose or key points of the tool, object, or robot end effector itself as data representation. Instead, it uniformly converts their motion behavior into changes in the pose of key points of the human hand to reflect the human's true operational intent in the corresponding operational context. This ensures that operational behaviors from different entities are uniformly represented and reused within the same human hand key point space.

[0079] (4) Spatial pose data:

[0080] To support scenarios where humans perform tasks while moving, human behavior data also includes the overall pose of the human body or human reference coordinate system in space, used to describe changes in the spatial position and orientation of humans when performing tasks.

[0081] The aforementioned human behavior data can be collected through teleoperation devices, exoskeleton systems, motion capture systems, visual perception systems, or combinations thereof. Depending on the data source, human operational intentions can be obtained through direct data collection or by inferring the behavior performed by the controlled robot and converting it into human operational intentions. By defining acquisition methods for multi-source heterogeneous data, such as teleoperation, motion capture, video, and exoskeletons, data coverage and diversity are ensured.

[0082] Intermediate behavior representation is a data representation form that is independent of the specific data acquisition method but consistent with human anatomy, describing human operational intentions through keypoint pose sequences. Examples include hand keypoint pose sequences (such as fingertips and palms) or full-body keypoint pose sequences (such as shoulders and waist). Human behavioral data from different sources are processed uniformly and mapped onto the human embodied space.

[0083] A robot motion sequence refers to an executable sequence of actions generated by the target robot body under physical constraints, including joint angles, end-effector pose, or control signals. Examples include the trajectory sequence of a robotic arm's end-effector tracking the pose of key points on its hand, or the joint angle sequence of a humanoid robot's whole-body movement.

[0084] Specifically, human behavior data is acquired through multiple sources, including teleoperation, video capture, and exoskeleton systems. These data differ significantly in structure and semantics due to their diverse sources. This heterogeneous data is then converted into an intermediate behavior representation centered on human keypoint pose sequences, eliminating the differences in data sources. Finally, based on the kinematic and dynamic models of the target robot, the intermediate behavior representation is mapped to a sequence of robot actions, ensuring that the actions meet the robot's joint limits, torque constraints, and contact stability requirements.

[0085] The method provided in the above embodiments solves the cross-ontology mapping problem between multi-source heterogeneous human behavior data and robot bodies by introducing intermediate behavior representations. First, the intermediate behavior representation abstracts human operational intentions using keypoint pose sequences, eliminating the structural and semantic fragmentation of multi-source data such as teleoperation data and video data, providing a unified baseline for subsequent mapping. Second, the mapping mechanism based on kinematic and dynamic models ensures that the generated robot action sequences are physically executable, avoiding the problem of "geometrically reachable but dynamically infeasible" action generation. For example, in grasping tasks, the intermediate representation retains the hand's keypoint pose information, and the mapping module combines torque constraints to optimize the gripper's opening and closing degree, thereby preventing damage to the object due to excessive gripping. Through these technical means, this method significantly improves the reusability of human behavior data across different robot bodies and enhances the physical feasibility of robot actions and the task success rate.

[0086] In one embodiment, converting multi-source heterogeneous human behavior data into a unified intermediate behavior representation includes:

[0087] Determine the target processing path based on the source type of human behavior data;

[0088] Human behavior data is processed based on the target processing path to obtain intermediate behavior representations.

[0089] A processing path refers to a customized data cleaning, parsing, and transformation pipeline tailored to the characteristics of a specific data source, used to transform human behavioral data into an intermediate behavioral representation. The processing path bridges the gap between the data source and the "unified human embodied space." Since human behavioral data from different sources varies in distance from real human embodied behavior, different processing paths are required.

[0090] In one embodiment, such as Figure 2 As shown, the source types include indirect acquisition types; human behavior data is processed based on the target processing path to obtain intermediate behavior representations, including:

[0091] Step 201: Based on the configuration file of the robot or its simulation model, determine the robot's key points and the key points related to human operating intentions.

[0092] Step 202: Map the robot key points to Cartesian space to obtain the pose of the robot key points in the Cartesian coordinate system;

[0093] Step 203: Solve the pose sequence of the first human hand key points based on the key points related to the human operation intention;

[0094] Step 204: Determine the intermediate behavior representation based on the pose of the robot's key points in the Cartesian coordinate system and the pose sequence of the first human hand key points.

[0095] The configuration file is not a single file, but a structured set of parameters that guides the entire data transformation process. The configuration file defines "what to focus on" and "how to transform." In one embodiment, the configuration file includes:

[0096] Keypoint definition configuration: Specifies which joints or links to select from the robot's kinematic model, including but not limited to joints, link ends, or end effector-related parts. For example: ["joint1", "joint2", ..., "flange", "tool_center_point"].

[0097] Redirection rule configuration: Defines the correspondence between the robot end effector (such as a gripper or welding torch) and the key point model of the human hand. It may also include constraints, such as the range of motion of human joints and prior knowledge of hand posture.

[0098] Normalization parameter configuration: This includes parameters required for coordinate system alignment, scale normalization, and temporal alignment. For example, coordinate system alignment defines the origin and orientation of the world coordinate system; for instance, the origin is the center of the workbench, with the Z-axis pointing upwards. Scale normalization: Scaling factor. Due to the significant size difference between robots and humans, the robot's motion scale (e.g., meters) needs to be scaled to a range similar to the human motion scale (e.g., within a unit sphere) for easier model processing. Temporal alignment: This may require downsampling or interpolation to ensure all data sequences have the same frame rate and length.

[0099] The process of converting indirectly acquired human behavior data into intermediate behavior representations based on configuration files involves the following steps: First, based on the kinematic description file of the robot or its simulation model, the robot keypoints and keypoints related to human operational intentions defined in the configuration file are determined. Then, forward kinematic calculations are performed on the robot keypoints, mapping them from joint space to a unified Cartesian coordinate system. For robot end effector-related behaviors, through independent redirection algorithms and configurations, the end effector behaviors are converted into corresponding human hand keypoint pose changes, i.e., the first human hand keypoint pose sequence, to represent the human's true operational intention in that operational context. The processed robot keypoint pose sequence and the first human hand keypoint pose sequence are then uniformly represented as intermediate behavior representations in human embodied space.

[0100] Taking a simulation of a robot using a gripper to pick up building blocks as an example, let's connect the entire process:

[0101] Input: Robot simulation log, containing the angles of all joints in each frame [θ1, θ2, ..., θ6,gripper_aperture].

[0102] Step 1: Determining the key points of the robot

[0103] Read the configuration file and determine the key points that need to be calculated: ["base", "shoulder", "elbow", "wrist", "flange", "tool_center_point"].

[0104] Step 2: Forward Kinematic Mapping

[0105] For each frame of data, perform forward kinematics calculations for each of the above key points.

[0106] Input: Joint angles [θ1, θ2, ...] + robot kinematics model.

[0107] Output: The pose of each keypoint in Cartesian coordinates. For example, the pose of "tool_center_point" is (x_tcp, y_tcp, z_tcp, rot_x, rot_y, rot_z).

[0108] Step 3: Key Point Selection and Standardization

[0109] Filtering: Depending on the configuration file, only the points most important to the crawling intent, such as "wrist" and "tool_center_point", may be retained.

[0110] (1) Coordinate system alignment: Convert the coordinates of all points to a coordinate system with the “workbench center” as the origin.

[0111] (2) Scale normalization: Divide all coordinate values ​​by a constant (such as the robot's arm span) to scale their range to around [-1, 1].

[0112] (3) Timing alignment: Ensure that all demo sequences have the same length (e.g., fixed to 100 frames by interpolation).

[0113] Step 4: End-effector redirection processing

[0114] Input: The pose sequence of "tool_center_point" obtained in the previous step.

[0115] The redirection algorithm and configuration are invoked. The algorithm considers: "To make a person's hand reach the same position and orientation as this gripper, and perform a similar 'grasping' action, what should the person's wrist and fingers be positioned?" It solves for a set of natural human hand postures through inverse kinematics, namely the first sequence of human hand keypoint poses.

[0116] Output: First human hand keypoint pose sequence, a virtual human hand pose sequence of 21 keypoints.

[0117] Step 5: Generation of intermediate behavior representations

[0118] Input: Normalized robot keypoints from step 3 (e.g., coordinates of "wrist"). Coordinates of 21 keypoints for the virtual human hand from step 4.

[0119] These two sets of points are concatenated together according to time frames. Assuming there are 3 robot points and 21 human points, each frame contains 24 points. These are organized into a standard data structure. For example, a multidimensional array of shape [100, 24, 6] (100 frames, 24 points, each point has 6 data dimensions: x, y, z, rx, ry, rz). This data structure is the "intermediate behavior representation in human embodied space".

[0120] The method provided in the above embodiments ingeniously "translates" human behavioral data into a learnable form rich in biomotor intelligence.

[0121] In one embodiment, the source type includes direct acquisition type, and the human behavior data includes video sequences; the human behavior data is processed based on the target processing path to obtain an intermediate behavior representation, including:

[0122] Identify the pose of key points of the human upper limbs and hands in video sequences;

[0123] Perform at least one of the following processing steps on the pose of key points of the human upper limbs and hands: coordinate system alignment, scale normalization, and temporal alignment, to obtain the corresponding intermediate behavior representation.

[0124] The video sequence is human behavior data acquired through video capture. For human behavior data in video format, key points are extracted from the video sequence using a human key point recognition algorithm to obtain the poses of key points of the human upper limbs and hands. The selected poses of key points of the human upper limbs and hands are then subjected to coordinate system alignment, scale normalization, and temporal alignment to obtain an intermediate behavior representation mapped to human embodied space.

[0125] In one embodiment, the source type includes direct acquisition type, and the human behavior data includes data collected by handheld devices; the human behavior data is processed based on the target processing path to obtain an intermediate behavior representation, including:

[0126] Acquire wrist poses related to operational behaviors from data collected by handheld devices, as well as gripper signals from handheld devices;

[0127] The gripper signals are mapped to a second sequence of key points in the human hand pose.

[0128] Based on the wrist pose related to the operation and the second human hand key point pose sequence, the intermediate behavior representation is determined.

[0129] For data collected from handheld devices, only wrist pose information directly related to human operation is retained. Simultaneously, the gripper opening degree or equivalent gripper signal provided by the device is mapped to changes in human hand keypoint pose, resulting in a second human hand keypoint pose sequence. This step maps simple opening degree to complex human hand posture. The wrist poses related to the operation and the second human hand keypoint pose sequence are then used to construct an intermediate behavior representation.

[0130] In one embodiment, the source type includes direct acquisition type, and the human behavior data includes data collected by exoskeleton devices; for data collected by exoskeleton devices, the pose information related to the human upper limbs and hands is obtained based on the application interface provided by the exoskeleton device; the data is directly converted into the corresponding human key point pose sequence through a predefined interface mapping relationship to generate a highly complete representation of human embodied behavior.

[0131] By employing specialized mapping strategies for different data sources (such as forward kinematics mapping, keypoint recognition, and interface mapping), teleoperation data, video data, and exoskeleton data are converted into human hand keypoint pose sequences. Subsequently, a unified intermediate behavior representation is generated through coordinate alignment (e.g., unifying data from different coordinate systems to the human reference coordinate system), scale normalization (e.g., unifying data from different resolutions to a standardized space), and time sequence alignment (e.g., aligning timestamps from multi-source data). The entire process ensures the consistency of multi-source data in semantics and physical properties, providing standardized input for subsequent cross-ontology mapping.

[0132] In one embodiment, such as Figure 3 As shown, based on the kinematic and dynamic models of the target robot body, intermediate behavior representations are mapped to robot action sequences, including:

[0133] Step 301: Transform the key operational intent in the intermediate behavior representation into target constraints in the task space of the target robot;

[0134] Step 302: At each moment in the intermediate behavior representation, based on the target constraints and structural constraints in the task space, perform inverse kinematics solution to generate a sequence of joint angles that satisfy the joint limits; the structural constraints are determined based on the kinematic model.

[0135] Step 303: Perform model predictive control optimization, jointly considering dynamics, contact and actuator capabilities within a finite time domain, to generate a robot motion sequence that satisfies torque constraints and contact stability.

[0136] Based on intermediate behavior representations and combined with the kinematic and dynamic models of the target robot, as well as control and safety constraints, human behavior data is converted into action representations that the target robot can execute or learn. This embodiment ensures cross-body action consistency while satisfying the robot's own joint limits, velocity / acceleration limits, torque / drive capability, contact and stability constraints, and collision constraints, thus achieving practical executableness.

[0137] Among them, inverse kinematics solution: a mathematical method for inferring joint angles from the end-effector pose. Model predictive control optimization: a finite-time domain optimization control method based on a dynamic model.

[0138] Joint angle sequences are generated through inverse kinematics solutions, and model predictive control optimization further ensures the dynamic feasibility of the motion. For example, inverse kinematics solutions ensure that joint angles do not exceed physical limits, while model predictive control optimization ensures that the motion satisfies torque constraints and contact stability. This two-stage process balances geometric accessibility and dynamic feasibility.

[0139] In one embodiment, the mapping process includes:

[0140] 1) Input and configuration: Read the target robot's structural description file and configuration parameters, including but not limited to URDF (Unified Robot Description Format), joint constraints, safety constraints, self-collision / environment collision model, and task-related key point correspondence configuration file.

[0141] The configuration file is used to define: the correspondence between human embodied key points / intents and robot critical links or end effectors; the set of key points to be kept / ignored; priority and weight settings; and the set of constraints and tolerances.

[0142] 2) Kinematic mapping: From human embodied space to robotic task space target

[0143] Key operational intentions in the human embodied space, such as hand keypoint pose sequences, wrist / elbow / shoulder keypoint poses, and overall pose, are converted into target constraints in the robot's task space, for example:

[0144] End effector pose tracking;

[0145] Hand / gripper opening and closing mapping;

[0146] Critical link attitude or relative geometric relationship constraints;

[0147] Soft constraints on contact points / contact normals.

[0148] 3) Feasible attitude solution: Constrained inverse kinematics

[0149] Within each time moment or time window, the robot joint space solution is obtained based on the target constraints and robot structural constraints of the aforementioned task space, forming candidate trajectories. This inverse kinematics solution employs a constrained numerical optimization approach, with constraints including at least:

[0150] Joint angle and velocity limits;

[0151] Avoidance of self-collision and environmental collision;

[0152] Zero-space optimization of redundant degrees of freedom enhances attitude naturalness and controllability.

[0153] 4) Dynamic Executability Guarantee: (MPC) + Constraint Optimization

[0154] Model Predictive Control (MPC) considers dynamics, contact, and actuator capabilities jointly within a finite time domain to ensure the output motion meets the robot's realistic executable requirements. MPC optimization objectives may include: tracking inverse kinematics output or task space reference, minimizing torque / energy consumption or motion jitter, and penalizing constraint violations. MPC constraints may include: dynamic consistency, torque limits; contact / balance constraints; collision avoidance and joint limit constraints.

[0155] Furthermore, by jointly optimizing inverse kinematics and model predictive control, the physical feasibility of mapped actions and the success rate of tasks are significantly improved. For example, in grasping tasks, inverse kinematics solutions ensure reasonable gripper angles, while model predictive control optimization prevents the grippers from tightening too much and damaging the object. This step effectively solves the problem of generating actions that are "geometrically reachable but dynamically infeasible," enhancing system safety and task robustness.

[0156] In one embodiment, the method further includes:

[0157] The robot's motion sequence is input into a simulation environment for pre-verification to detect whether there are joint limits, torque constraints, or collision risks.

[0158] If a constraint violation is detected, a rollback strategy is triggered, including adjusting the magnitude of the action, switching to the feasible solution of the previous time step, or adjusting the weight priority.

[0159] Pre-verification in a simulation environment refers to testing the feasibility and safety of actions through physical simulation (such as MuJoCo) before actual deployment. For example, in MuJoCo, the trajectory of the robotic arm's end effector can be simulated to check whether it exceeds the joint angle limits.

[0160] The rollback strategy is a degradation handling strategy implemented when a constraint violation is detected. If the gripper torque exceeds the limit, the range of motion is scaled back to a safe range.

[0161] The physical feasibility of the mapped actions is ensured through pre-verification in a simulation environment. For example, joint limits, torque constraints, and collision detection are performed on the generated robot action sequence. If a violation is found (such as joint angles exceeding the range), a backoff strategy is triggered, including but not limited to: downgrading to a more conservative task objective, adjusting the action amplitude, switching to the feasible solution of the previous time step, and adjusting the weights or priorities.

[0162] Furthermore, by employing simulation pre-verification and a backoff strategy, the safety and reliability of mapped actions were significantly improved. For example, in mobile operation tasks, simulation pre-verification can detect collision risks with the environment in advance, while the backoff strategy can dynamically adjust the action trajectory to avoid collisions. This step effectively avoids robot execution failures caused by constraint violations, enhancing the system's robustness and task success rate.

[0163] In one embodiment, the method further includes:

[0164] The robot's action sequence is input into the robot learning system to train human strategies for intermediate behavior representations;

[0165] The trained human strategy is adapted to the target robot to generate executable control signals.

[0166] Human policy refers to a policy model that uses intermediate behaviors as inputs and outputs to characterize human operational intentions. Examples include policy networks trained based on reinforcement learning or imitation learning.

[0167] The processed human behavior data or intermediate behavior representations are input into the robot learning system to support efficient learning and deployment based on human behavior data. Unlike existing technologies that train complete control strategies for each robot body separately, this embodiment learns a human policy only for a unified intermediate behavior representation during the training phase. The human policy uses the intermediate behavior representation in the human embodied space as input and output to characterize the human's operational intentions and temporal decision-making patterns during task completion.

[0168] During the training phase, human behavior data from different sources, but uniformly mapped to the human embodied space, are used to train human strategies, supporting learning methods including but not limited to: imitation learning based on human teaching data, reinforcement learning or policy guidance based on human behavior data, and training of visual-language-action models or multimodal models with human behavior sequences as the core supervision signal.

[0169] During the deployment phase, human strategies are not directly executed as the final control strategy for the target robot, but are adapted to the specific robot body through at least one of the following methods:

[0170] (1) Combining the aforementioned cross-entity mapping module, the actions or intentions output by human strategies in the human embodied space are mapped into robot actions that conform to the kinematic and dynamic constraints of the target robot.

[0171] (2) Based on a small amount of robot-specific data, learn a lightweight action alignment network to align the human policy output to the action space of the target robot.

[0172] By decoupling "human policy training" from "robot ontology adaptation" in the above manner, the training cost of robot learning in various embodied scenarios is significantly reduced, and the domain differences between different robot ontologs are effectively mitigated. This allows the same human policy to be reused and deployed on multiple robot platforms. For example, the same human policy can be adapted to a dual-arm robotic arm or a humanoid robot without the need for separate training for each ontology. Furthermore, this step achieves a complete closed loop from data acquisition to model deployment, providing a scalable technical framework for general robot learning.

[0173] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0174] Based on the same inventive concept, this application also provides a robot learning data processing device based on human behavior data. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more robot learning data processing device embodiments based on human behavior data provided below can be found in the limitations of the robot learning data processing method based on human behavior data above, and will not be repeated here.

[0175] In one embodiment, such as Figure 4 As shown, the robot learning data processing device based on human behavior data includes a data acquisition module 401, a data representation and normalization module 402, and a cross-local mapping module 403, wherein:

[0176] Data acquisition module 401 is used to acquire multi-source heterogeneous human behavior data;

[0177] The data representation and specification module 402 is used to convert multi-source heterogeneous human behavior data into a unified intermediate behavior representation, which describes human operational intentions using key point poses.

[0178] The cross-local mapping module 403 is used to map intermediate behavior representations into robot action sequences based on the kinematic and dynamic models of the target robot body. The robot action sequences satisfy the joint limits, torque constraints, and contact stability requirements of the target robot body.

[0179] In one possible implementation, the data representation and specification module 402 is specifically used for:

[0180] Determine the target processing path based on the source type of human behavior data;

[0181] Human behavior data is processed based on the target processing path to obtain intermediate behavior representations.

[0182] In one possible implementation, the data representation and specification module 402 is specifically used for:

[0183] Based on the configuration file of the robot or its simulation model, determine the robot's key points and the key points related to human operating intentions;

[0184] Map the robot's key points to Cartesian space to obtain the robot's pose in the Cartesian coordinate system;

[0185] Based on the key points related to human operational intentions, solve the pose sequence of the first human hand key points;

[0186] The intermediate behavior representation is determined based on the pose of the robot's key points in the Cartesian coordinate system and the pose sequence of the first human hand key points.

[0187] In one possible implementation, the data representation and specification module 402 is specifically used for:

[0188] Identify the pose of key points of the human upper limbs and hands in video sequences;

[0189] Perform at least one of the following processing steps on the pose of key points of the human upper limbs and hands: coordinate system alignment, scale normalization, and temporal alignment, to obtain the corresponding intermediate behavior representation.

[0190] In one possible implementation, the data representation and specification module 402 is specifically used for:

[0191] Acquire wrist poses related to operational behaviors from data collected by handheld devices, as well as gripper signals from handheld devices;

[0192] The gripper signals are mapped to a second sequence of key points in the human hand pose.

[0193] Based on the wrist pose related to the operation and the second human hand key point pose sequence, the intermediate behavior representation is determined.

[0194] In one possible implementation, the cross-local mapping module 403 is specifically used for:

[0195] Transform the key operational intentions in the intermediate behavior representation into target constraints in the task space of the target robot;

[0196] At each moment in the intermediate behavior representation, based on the target constraints and structural constraints in the task space, inverse kinematics solution is performed to generate a sequence of joint angles that satisfy the joint limits; the structural constraints are determined based on the kinematic model.

[0197] The execution model predictive control optimization jointly considers dynamics, contact and actuator capabilities within a finite time domain to generate robot motion sequences that satisfy torque constraints and contact stability.

[0198] In one possible implementation, the cross-local mapping module 403 is specifically used for:

[0199] The robot's motion sequence is input into a simulation environment for pre-verification to detect whether there are joint limits, torque constraints, or collision risks.

[0200] If a constraint violation is detected, a rollback strategy is triggered, including adjusting the magnitude of the action, switching to the feasible solution of the previous time step, or adjusting the weight priority.

[0201] In one possible implementation, the device further includes a robot learning utilization module for:

[0202] The robot's action sequence is input into the robot learning system to train human strategies for intermediate behavior representations;

[0203] The trained human strategy is adapted to the target robot to generate executable control signals.

[0204] The modules in the aforementioned robot learning data processing device based on human behavior data can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0205] Figure 5 A schematic diagram of the structure of the computer device provided in this application. Figure 5 As shown, the computer device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus 504.

[0206] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0207] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0208] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0209] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0210] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0211] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0212] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0213] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0214] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0215] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0216] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0217] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0218] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0219] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0220] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A robot learning data processing method based on human behavior data, characterized in that, The method includes: Acquire multi-source, heterogeneous human behavior data; The multi-source heterogeneous human behavior data is converted into a unified intermediate behavior representation, which describes the human operation intention in terms of key point poses. Based on the kinematic and dynamic models of the target robot body, the intermediate behavior is represented and mapped to a robot action sequence, which satisfies the joint limits, torque constraints and contact stability requirements of the target robot body.

2. The method according to claim 1, characterized in that, The process of converting the multi-source heterogeneous human behavior data into a unified intermediate behavior representation includes: Determine the target processing path based on the source type of human behavior data; The human behavior data is processed based on the target processing path to obtain intermediate behavior representations.

3. The method according to claim 2, characterized in that, The source type includes indirect acquisition type; the processing of the human behavior data based on the target processing path to obtain intermediate behavior representation includes: Based on the configuration file of the robot or its simulation model, determine the robot's key points and the key points related to human operating intentions; Map the robot key points to Cartesian space to obtain the pose of the robot key points in the Cartesian coordinate system. Based on the key points related to human operational intentions, solve the pose sequence of the first human hand key points; The intermediate behavior representation is determined based on the pose of the robot's key points in the Cartesian coordinate system and the pose sequence of the first human hand's key points.

4. The method according to claim 2, characterized in that, The source type includes direct acquisition type, and the human behavior data includes video sequences; the processing of the human behavior data based on the target processing path to obtain intermediate behavior representation includes: Identify the poses of key points of the human upper limbs and hands in the video sequence; The pose of the key points of the upper limbs and hands of the human body is processed by at least one of the following: coordinate system alignment, scale normalization, and temporal alignment, to obtain the corresponding intermediate behavior representation.

5. The method according to claim 2, characterized in that, The source type includes direct acquisition type, and the human behavior data includes data collected by handheld devices; the processing of the human behavior data based on the target processing path to obtain intermediate behavior representation includes: The wrist pose related to the operation behavior and the gripper signal of the handheld device are obtained from the data collected by the handheld device. The gripper signal is mapped to a second human hand key point pose sequence; Based on the wrist pose associated with the operation and the second human hand key point pose sequence, an intermediate behavior representation is determined.

6. The method according to claim 1, characterized in that, The kinematic and dynamic models based on the target robot body map the intermediate behavior representations into robot action sequences, including: The key operational intentions in the intermediate behavior representation are transformed into target constraints in the task space of the target robot. At each moment in the intermediate behavior representation, based on the target constraints and structural constraints of the task space, inverse kinematics solution is performed to generate a sequence of joint angles that satisfy the joint limits; the structural constraints are determined based on the kinematic model. The execution model predictive control optimization jointly considers dynamics, contact and actuator capabilities within a finite time domain to generate robot motion sequences that satisfy torque constraints and contact stability.

7. The method according to claim 6, characterized in that, The method further includes: The robot motion sequence is input into a simulation environment for pre-verification to detect whether there are joint limits, torque constraints or collision risks. If a constraint violation is detected, a rollback strategy is triggered, including adjusting the magnitude of the action, switching to the feasible solution of the previous time step, or adjusting the weight priority.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: The robot action sequence is input into the robot learning system to train a human strategy for the intermediate behavior representation; The trained human strategy is adapted to the target robot to generate executable control signals.

9. A robot learning data processing device based on human behavior data, characterized in that, The device includes: The data acquisition module is used to acquire multi-source heterogeneous human behavior data; The data representation and standardization module is used to convert the multi-source heterogeneous human behavior data into a unified intermediate behavior representation, which describes the human operation intention in terms of key point poses. The cross-local mapping module is used to map the intermediate behavior representation into a robot action sequence based on the kinematic and dynamic models of the target robot body. The robot action sequence satisfies the joint limits, torque constraints, and contact stability requirements of the target robot body.

10. A computer device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 8.

Images