Cross-robot action reorientation method, apparatus, dispatch server, and medium

Abstract

This application relates to the field of embodied robot training technology, and discloses a cross-robot motion redirection method, device, scheduling server, and medium. The method includes: receiving a redirection task request submitted by a user, containing a source motion file identifier and a target robot model; generating a mapping configuration file based on the target robot model; obtaining a source joint angle sequence organized frame by frame based on a general human standard skeleton; allocating the task to a local computing resource pool or a cloud GPU computing resource pool based on a comparison of computational load and a preset threshold; converting the source joint angle sequence into a target robot joint angle sequence frame by frame using the allocated computing resources; and sending the target robot joint angle sequence and a 3D simulation model to the user terminal for rendering a preview animation through a browser simulation engine. This application can improve the versatility of motion redirection, balance processing efficiency and resource costs, and facilitate users' intuitive verification of the redirection effect.

Description

Technical Field

[0001] This application relates to the field of embodied robot training, and more particularly to a method, apparatus, scheduling server, and medium for cross-robot motion redirection. Background Technology

[0002] Motion retargeting refers to the process of converting source motion data into a sequence of joint commands that can be executed by a target robot. Source motion data typically originates from a motion capture system and is stored as a sequence of rotational angles of human skeletal joints. Different robots vary significantly in the number of joints, topology, skeletal proportions, and range of motion, making it difficult to directly apply the same motion data to different robot models.

[0003] Existing methods typically involve writing dedicated mapping and conversion code for specific robot models. When new robot models need to be supported, the corresponding code must be redeveloped and debugged, resulting in poor versatility. Furthermore, after generating the target motion, existing methods often require deploying the motion data to a real robot for playback to verify the effect, making the debugging process inconvenient. Summary of the Invention

[0004] Therefore, it is necessary to address the technical problems of insufficient versatility, high computational cost, and inconvenient verification and debugging in existing action redirection methods.

[0005] Firstly, a cross-robot motion redirection method is provided, the method comprising: Receive a redirection task request submitted by a user, the redirection task request including the source action file identifier and the target robot model; Based on the target robot model, obtain the real-time generated mapping configuration file, which defines the joint mapping rules and motion constraints from the general human standard skeleton to the target robot; Obtain the source motion file corresponding to the source motion file identifier, wherein the source motion file includes a source joint angle sequence organized frame by frame based on the general human standard skeleton; Determine whether the computational load corresponding to the redirection task request is lower than a preset threshold. If so, allocate the redirection task to the local computing resource pool; otherwise, allocate it to the GPU computing resource pool in the cloud. Using the allocated computing resources, for each frame of the source motion file, the source joint angle sequence is converted into the target robot joint angle sequence according to the joint mapping rules and motion constraints; The target robot's joint angle sequence and 3D simulation model are sent to the user terminal to generate a preview animation through the browser's simulation engine.

[0006] Secondly, a cross-robot motion redirection device is provided, the device comprising: The receiving module is used to receive a redirection task request submitted by the user, wherein the redirection task request includes the source action file identifier and the target robot model; The acquisition module is used to acquire a real-time generated mapping configuration file based on the target robot model. The mapping configuration file defines the joint mapping rules and motion constraints from a general human standard skeleton to the target robot. The acquisition module is further configured to acquire the source action file corresponding to the source action file identifier, wherein the source action file includes a source joint angle sequence organized frame by frame based on the general human standard skeleton; The allocation module is used to determine whether the computational load corresponding to the redirection task request is lower than a preset threshold. If so, the redirection task is allocated to the local computing resource pool; otherwise, it is allocated to the GPU computing resource pool in the cloud. The conversion module is used to utilize the allocated computing resources to convert the source joint angle sequence into the target robot joint angle sequence for each frame of the source motion file according to the joint mapping rules and motion constraints. The sending module is used to send the joint angle sequence of the target robot and the three-dimensional simulation model of the target robot to the user terminal so that a preview animation can be generated by rendering through the browser simulation engine.

[0007] Thirdly, a scheduling server is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described cross-robot motion redirection method.

[0008] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described cross-robot motion redirection method.

[0009] The beneficial effects of this application are: This application defines joint mapping rules and motion constraints from a general human standard skeleton to a target robot through a configurable mapping configuration file. When it is necessary to support a new robot model, the corresponding mapping configuration file can be automatically generated without modifying the core conversion code. This solves the problem of insufficient universality of existing methods and helps to reduce the development and maintenance costs of multi-model adaptation.

[0010] This application determines whether the computational load corresponding to the redirected task request is lower than a preset threshold, and then assigns tasks with lower computational load to the local computing resource pool for execution, while assigning tasks with higher computational load to the cloud GPU computing resource pool for execution. This allows lightweight tasks to be processed quickly and locally, avoiding cloud call overhead, and batch tasks to be processed faster by utilizing the parallel computing capabilities of the cloud. This solves the problem of high computational overhead in existing methods and balances processing efficiency and resource cost.

[0011] After generating the joint angle sequence of the target robot, this application sends it along with the 3D simulation model of the target robot to the user terminal. The user terminal then uses the browser simulation engine to drive the 3D model rendering to generate a preview animation. This allows users to intuitively verify the redirection effect without deploying motion data to a real robot, solving the problem of inconvenient verification and debugging in existing methods and reducing debugging costs and safety risks. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] in: Figure 1 This is a schematic diagram of an application scenario provided in one embodiment; Figure 2 Here is a flowchart of a cross-robot motion redirection method in one embodiment; Figure 3 This is a structural block diagram of a cross-robot motion redirection device in one embodiment; Figure 4 This is a schematic diagram of a computer storage medium in one embodiment; Figure 5 This is a block diagram of the scheduling server in one embodiment. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] like Figure 1As shown, the cross-robot motion redirection system provided in this application includes three entities interconnected via a network: user terminal 1, scheduling server 2, and cloud resource server 3.

[0016] User terminal 1 is the entry device for users to initiate redirection tasks and view preview animations. User terminal 1 can be a desktop computer, laptop, tablet, or smartphone. User terminal 1 runs browser software, which embeds a browser simulation engine based on WebAssembly technology. Users access the front-end interactive interface provided by scheduling server 2 through the browser, select the source motion file and target robot model in the interface, and submit a redirection task request. After the redirection task is completed, user terminal 1 receives the target robot joint angle sequence and 3D simulation model returned by scheduling server 2 through the browser. The browser simulation engine drives the 3D simulation model to render frame by frame, generating a preview animation for users to view and confirm.

[0017] Scheduling server 2 is the core device for executing the cross-robot motion redirection method described in this application. Scheduling server 2 can be a single physical server, a server cluster consisting of multiple physical servers, or a virtual server instance deployed in the cloud. Scheduling server 2 includes a processor, memory, a network interface, and persistent storage. The memory is used to temporarily store source motion file data, mapping configuration file data, source joint angle sequences, and target robot joint angle sequences during runtime. The network interface is used for data communication with user terminal 1 and cloud resource server 3. The persistent storage device is used to store the mapping configuration file library, source motion file library, and 3D simulation model library. When scheduling server 2 needs to obtain the mapping configuration file, it executes a configuration generation script or configuration parsing service. The specific process includes: first, reading the robot unified description format file corresponding to the target robot model, and extracting the original joint information from it. Subsequently, using a set of connection rules between the source joints of a universal human standard skeleton and different robot joints, the joint name correspondence and coordinate system transformation parameters applicable to this task are calculated in real time. The raw motion range data read from the robot description file is formatted into standard motion range constraints. Finally, all information is packaged into a complete mapping configuration file that can be directly used in subsequent conversion steps. The source motion file is retrieved from the source motion file library based on the source motion file identifier. Then, based on the comparison between the computational load and a preset threshold, the redirection task is assigned to either the local computing resource pool or the cloud resource server 3 for execution. When the computational load is less than the preset threshold, the scheduling server 2 assigns the redirection task to a computing node in its managed local computing resource pool for execution. When the computational load is greater than or equal to the preset threshold, the scheduling server 2 sends the redirection task to the cloud resource server 3 for execution via a dedicated network channel.

[0018] Cloud Resource Server 3 is a remote computing facility providing elastic parallel computing capabilities. Cloud Resource Server 3 can be a GPU container cluster consisting of multiple GPU container instances, deployed on the infrastructure of a public cloud service provider. Cloud Resource Server 3 establishes a connection with Scheduling Server 2 via the Internet or a dedicated network. Cloud Resource Server 3 receives a redirection task data packet from Scheduling Server 2, which contains the source joint angle sequence, joint mapping rules from the mapping configuration file, and motion constraints. Cloud Resource Server 3 assigns the redirection task to an idle GPU container instance in the cluster. This GPU container instance utilizes the parallel computing power of the GPU to perform coordinate system transformation, angle limiting, and velocity limiting processing on each frame of the source motion file, quickly generating the target robot joint angle sequence. After processing, the GPU container instance returns the target robot joint angle sequence to Scheduling Server 2 via Cloud Resource Server 3.

[0019] The present application will now be described in detail through specific embodiments.

[0020] Please see Figure 2 As shown, Figure 2 A flowchart illustrating the cross-robot motion redirection method provided in this application embodiment includes the following steps: S1. Receive the redirection task request submitted by the user. The redirection task request includes the source action file identifier and the target robot model.

[0021] The scheduling server's processor receives redirection task requests from user terminals via a network interface. This redirection task request is a structured data packet carrying at least two key parameters. The first parameter is the source motion file identifier, which uniquely identifies a piece of human motion data to be migrated; this identifier can be a file storage path, a file hash value, or a unique number in the database. The second parameter is the target robot model, which specifies the specific robot to which the motion data will be migrated; this model can be a product model string or a unique name from the robot's configuration file.

[0022] For example, a user selects a dance motion file with the ID BVH_00385 from the motion library via a web interface, and selects a humanoid robot with the model number Humanoid_V2 from the robot model drop-down menu. The browser encapsulates this information into a redirection task request and sends it to the scheduling server over the network. Upon receiving the request, the scheduling server's processor parses the source motion file identifier as BVH_00385 and the target robot model as Humanoid_V2, and temporarily stores these two parameters in memory for use in subsequent steps.

[0023] S2. Based on the target robot model, obtain the real-time generated mapping configuration file. The mapping configuration file defines the joint mapping rules and motion constraints from the general human standard skeleton to the target robot.

[0024] The scheduling server's processor, based on the target robot model parsed from S1, executes a configuration generation script or configuration parsing service when it needs to obtain the mapping configuration file. The specific process includes: first, reading the robot's unified description format file corresponding to the target robot model and extracting the original joint information; then, using a set of connection rules between the source joints of a universal human standard skeleton and different robot joints, calculating in real time the joint name correspondence and coordinate system transformation parameters applicable to this task; formatting the original motion range data read from the robot description file into standard motion range constraints; and finally, packaging all information into a complete mapping configuration file that can be directly used in subsequent conversion steps. The core content of the mapping configuration file is the definition of joint mapping rules and motion constraints from a universal human standard skeleton to the specific target robot model. The universal human standard skeleton is a predefined, robot-independent human skeleton template containing the hierarchical relationships of key joints such as the hip, knee, ankle, shoulder, elbow, and wrist joints. The joint mapping rules specify which joint name on the target robot corresponds to each source joint name on the universal human standard skeleton. Motion constraints specify the maximum angular range and maximum angular velocity allowed for each joint of the target robot during movement.

[0025] For example, the scheduling server's processor parses the joint information of the generic human skeleton and Humanoid_V2 in real time, generates a mapping configuration file `humanoid_v2_mapping.json` using connection rules, and loads its contents into memory. In this configuration file, the joint mapping rules section records a mapping relationship that maps the RightShoulder joint of the generic human skeleton to the joint_1 joint of the target robot, and the RightElbow joint to the joint_2 joint. The motion constraints section records that the angle limit of the joint_1 joint is -180 degrees to 180 degrees, and the velocity limit is 120 degrees per second; the angle limit of the joint_2 joint is -135 degrees to 135 degrees, and the velocity limit is 90 degrees per second.

[0026] S3. Obtain the source motion file corresponding to the source motion file identifier. The source motion file includes a source joint angle sequence organized frame by frame based on a general human standard skeleton.

[0027] The scheduling server's processor retrieves and reads the corresponding source motion file from the motion data repository based on the source motion file identifier parsed in S1. The motion data repository can be a distributed object storage system or a file directory on the scheduling server's local disk. The processor loads all the data of the source motion file into memory for parsing. The source motion file contains a sequence of source joint angles based on a standard human skeleton, and this angle sequence is organized frame by frame; that is, each frame records the rotation angle value of each joint in the standard human skeleton at that moment. The time interval between adjacent frames is fixed, thus forming a complete piece of motion data.

[0028] For example, the scheduling server's processor downloads the corresponding motion file, dance_routine.bvh, from the object storage system based on the source motion file identifier BVH_00385. This file is stored in binary or text format. After parsing the file, the processor extracts the source joint angle sequence organized by frames. The parsing result shows that the motion file contains 500 frames of data, with a frame interval of 0.01 seconds and a total duration of 5 seconds. Each frame records the Euler angle rotation values ​​of source joints such as the RightShoulder and RightElbow joints in the universal human standard skeletal coordinate system.

[0029] S4. Determine whether the computational load corresponding to the redirected task request is lower than the preset threshold. If so, allocate the redirected task to the local computing resource pool; otherwise, allocate it to the GPU computing resource pool in the cloud.

[0030] In this process, after obtaining the source action file, the processor of the scheduling server first calculates the computational load of the current redirection task. The computational load is quantified by multiplying the total number of frames in the source action file by the number of joints in the target robot. The processor obtains the total number of frames from the parsed source joint angle sequence in S3. The processor obtains the number of joints in the target robot from the joint name mapping relationship in the mapping configuration file loaded in S2. Specifically, the joint name mapping relationship records multiple mapping entries, each corresponding to one joint of the target robot; therefore, the total number of mapping entries is the number of joints in the target robot. The processor multiplies the total number of frames by the number of joints and uses this product as the computational load for the current task. The processor compares this computational load with a preset threshold. The preset threshold is an empirical value, pre-set by the system administrator based on the difference in processing power between the local computing resource pool and the cloud GPU computing resource pool. If the computational load is less than the preset threshold, it indicates that the task's computational scale is small and suitable for local processing; otherwise, it indicates that the task's computational scale is large and suitable for parallel processing in the cloud.

[0031] For example, the source motion file parsed in S3 has a total of 500 frames. The joint name mapping in the mapping configuration file loaded in S2 contains 6 mapping entries, meaning the target robot has 6 joints. The processor multiplies 500 by 6, resulting in a computational cost of 3000. The preset threshold is 5000. The processor compares 3000 with 5000, determines that 3000 is less than 5000, and therefore decides to allocate this redirection task to the local computing resource pool for execution.

[0032] S5. Using the allocated computing resources, for each frame of the source motion file, convert the source joint angle sequence into the target robot joint angle sequence according to the joint mapping rules and motion constraints.

[0033] If S4 determines that the task will be allocated to the local computing resource pool, the processor of the scheduling server will package the source joint angle sequence, the joint mapping rules in the mapping configuration file, and the motion constraints into a single computing job and submit it to a computing node in the local computing resource pool for execution. If S4 determines that the task will be allocated to the cloud GPU computing resource pool, the processor will package the same data and send it to the interface of the cloud GPU container cluster through a dedicated network channel, where it will be received and executed by a GPU container instance in the cloud. The computing node or GPU container instance performs the following specific transformation process: For each frame of the source joint angle sequence, the target joint corresponding to the current source joint is first determined according to the joint name correspondence. Then, the rotation offset in the coordinate system transformation relationship is used to rotate and correct the source joint angle of the current frame, and the angle after rotation is scaled using the bone length scaling ratio to generate the initial target joint angle. Next, the initial target joint angle is restricted within the angle limit included in the motion range constraint. That is, if the initial angle is greater than the maximum allowable angle, the maximum allowable angle is taken; if it is less than the minimum allowable angle, the minimum allowable angle is taken, resulting in the target joint angle after angle limitation. Then, the change in the target joint angle after angle limiting relative to the final target joint angle of the previous frame is calculated, and the absolute value of this change is compared with the velocity limit. If the absolute value of the change exceeds the velocity limit, the change is clamped within the velocity limit, i.e., the velocity limit is used as the actual change, with the same sign as the original change, thus obtaining the final target joint angle of the current frame. After performing the above processing on all frames, the final target joint angles of all frames constitute the target robot joint angle sequence.

[0034] S6. Send the target robot joint angle sequence and the target robot's 3D simulation model to the user terminal so that a preview animation can be generated by rendering through the browser's simulation engine.

[0035] The scheduling server's processor, after completing the conversion and returning the target robot's joint angle sequence on a compute node or GPU container instance within S5, retrieves the corresponding 3D simulation model file from a pre-stored 3D model library. The 3D simulation model is the geometric model of the target robot, containing mesh data for each joint and hierarchical assembly relationships. The processor packages the target robot's joint angle sequence and the 3D simulation model file together and sends them via network protocol to the user terminal that initiated the redirection request. Upon receiving this data, the browser on the user terminal launches its embedded browser simulation engine. Based on the angle values ​​in each frame of the target robot's joint angle sequence, the browser simulation engine drives the rotation of each joint of the 3D simulation model frame by frame, rendering the animation in real time and ultimately outputting a preview animation for the user to view and confirm.

[0036] In one possible implementation, the mapping configuration file is a structured data description file that fully defines all the conversion rules from a general human skeleton to a specific target robot model. The general human skeleton is a predefined, morphology-independent human skeletal template containing key joints such as the hip, left hip, right hip, knee, ankle, shoulder, elbow, and wrist joints. These joints are organized in a hierarchical tree structure, where rotation of a parent joint changes the spatial position of its child joints. The target robot is a robotic entity with any number of joints and topology, such as a quadruped robot, a six-axis robotic arm, a bipedal humanoid robot, or a wheeled hybrid robot. The mapping configuration file establishes a complete and resolvable conversion specification between the source skeleton and the target robot. This mapping configuration file specifically includes the following three parts.

[0037] The first part is the correspondence between joint names.

[0038] The joint name mapping records the correspondence between each source joint name involved in motion retargeting in the general human standard skeleton and a specific joint name in the target robot in the form of key-value pairs. For source joints that do not exist on the target robot, the corresponding entries can be omitted in this mapping, and the conversion algorithm will automatically skip these joints during processing. For redundant degrees of freedom that exist on the target robot but do not have corresponding joints on the source skeleton, the mapping can also be omitted, and the conversion algorithm will keep the default angles of these joints unchanged.

[0039] For example, for a six-axis desktop robotic arm, its joint name mapping might include the following six entries: RightShoulder is mapped to joint_1, RightElbow to joint_2, RightWrist to joint_3, LeftShoulder to joint_4, LeftElbow to joint_5, and LeftWrist to joint_6. The hip and knee joints in the standard human skeleton do not have corresponding physical joints on this robotic arm, therefore they are not included in the joint name mapping. During frame-by-frame processing, the transformation algorithm only performs mapping and transformation operations on the six joints explicitly listed in the joint name mapping.

[0040] The second part is about coordinate system transformation relationships.

[0041] The coordinate system transformation relationship includes two parameters: rotation offset and bone length scaling. The rotation offset defines the fixed rotational correction that needs to be applied from the local coordinate system of the source joint to the local coordinate system of the target joint. This correction aligns the initial orientation of the two joints within their respective skeletons. Since the local coordinate system orientation of each joint in a standard human skeleton follows human anatomy, while the orientation of the target robot's joint coordinate system is defined by its mechanical structure and factory firmware, their initial orientations are typically different. The rotation offset is recorded in the configuration file as Euler angles or quaternions. During calculation, the transformation algorithm applies this rotation offset to the source joint angles, ensuring that the rotated angle values ​​are aligned with the target joint's coordinate system.

[0042] The bone length scaling ratio defines the proportional relationship between the bone lengths of adjacent joints in the source skeleton and the corresponding bone lengths of adjacent joints in the target robot. Due to the dimensional differences between the human skeleton and the robot's mechanical structure, directly using the bone lengths of the source skeleton will result in the end-effector's spatial trajectory deviating from expectations. The bone length scaling ratio is a dimensionless ratio. After performing coordinate transformation on the source joint angles, the transformation algorithm applies this scaling ratio to the displacement components, thereby achieving a proportional adjustment of the end-effector trajectory.

[0043] For example, in the aforementioned six-axis robotic arm, the initial orientation of the local coordinate system of the RightShoulder joint in the standard human skeleton is upward along the Z-axis, while the initial orientation of the local coordinate system of the joint_1 joint in the target robotic arm is horizontal along the Z-axis. A rotational offset of 90 degrees around the X-axis needs to be applied between the two. This rotational offset is recorded as 90 0 0 in Euler angles. The bone length from RightShoulder to RightElbow in the source skeleton is preset to 0.3 meters, while the link length from joint_1 to joint_2 in the target robotic arm is 0.45 meters. Therefore, the bone length scaling ratio is 0.45 divided by 0.3, which equals 1.5. The conversion algorithm first rotates the source angle by 90 degrees to align the coordinate system, and then multiplies the displacement component by 1.5 for scaling.

[0044] The third part is the constraint on the range of motion.

[0045] The range of motion constraints include angular limits and velocity limits for each joint of the target robot. Angular limits define the maximum angle value that the joint is allowed to rotate in both directions, while velocity limits define the maximum angular change that the joint is allowed to rotate per unit time.

[0046] The determination of angular limits is based on the physical structural limits of the target robot joints. These limits are determined by the following factors: the position of the mechanical limit stop, the range of the motor encoder, the allowable torsional range of the cabling, and the mechanical interference boundary between adjacent links. Specifically, when writing the mapping configuration file, technicians first consult the target robot's product specifications to obtain the nominal range of motion for each joint. If the product specifications do not provide this range or the provided values ​​are conservative, the actual mechanical limit angles are obtained through physical measurement. The measurement method involves manually rotating the robot joint to its positive limit position and reading the angle value at that position as the maximum angle limit; then manually rotating it to its opposite limit position and reading the angle value at that position as the minimum angle limit. These two angle values ​​are recorded as angle limit parameters. If the robot joint is a continuous rotary joint, the angle limit can be set to unlimited, indicated by a specific marker in the configuration file.

[0047] The speed limit is determined based on the maximum speed of the target robot's joint drive motor and the reduction ratio of the joint reducer. Specifically, technicians first obtain the rated speed of the drive motor in revolutions per minute (rpm), and then obtain the reduction ratio of the joint reducer. Dividing the rated speed by the reduction ratio yields the rated speed at the joint output, which is then multiplied by 6 to obtain the rated angular velocity in degrees per second (°S). The speed limit is typically set as a safety factor multiple of this rated angular velocity, with the safety factor ranging from 0.8 to 1.0. Furthermore, the speed limit should also consider the end effector mass of the robot mounted on the joint. The larger the load mass, the lower the allowable maximum angular velocity, and the lower the speed limit should be accordingly. The speed limit derating factor corresponding to the load mass can be obtained by consulting the motor load characteristic curve. The final speed limit value is the rated angular velocity multiplied by the safety factor and then multiplied by the load derating factor.

[0048] It should be noted that the angle limit and speed limit can be read from the robot's description file, and then the read parameters are written into the mapping configuration file.

[0049] In one possible embodiment, the source motion file is a BVH format file. BVH is an abbreviation for Biovision Hierarchical Data, a standard file format widely used in the motion capture field, defined by Biovision. BVH format files store data in plain text, allowing those skilled in the art to easily view and modify their contents using a text editor.

[0050] The BVH format file is structurally divided into two main parts: the skeleton definition section and the motion data section. The skeleton definition section, located in the first half of the file, begins with the keyword `HIERARCHY`. This section defines the parent-child relationships between joints in the standard human skeleton through a nested hierarchical structure. The top-level joint is called the root joint, and the root joint can contain multiple child joints, each of which can also contain its own child joints, thus forming a tree-like skeleton hierarchy. Each joint in the skeleton definition section records its joint name, initial offset relative to its parent joint, and the channel information it contains. The channel information indicates the number and order of rotational and translational degrees of freedom recorded for that joint in each frame in the motion data section. The motion data section, located in the second half of the file, begins with the keyword `MOTION`. This section first declares the number of frames for motion data and the time interval between each frame, then records the channel data values ​​of all joints in each frame line by line. Each line corresponds to one frame of data at one moment, and the order of the values ​​within each line strictly matches the order in which the joint channels are declared in the skeleton definition section.

[0051] For example, a BVH format file used for motion retargeting first defines the root joint Hips in its skeleton definition section. This joint has six channels: X position, Y position, Z position, Z rotation, X rotation, and Y rotation. Below Hips are two sub-joints: LeftHip and RightHip, each with three rotation channels. Below LeftHip is LeftKnee, and below RightHip is RightKnee. The spine joint Spine is defined along the spine, followed by Shoulders, and so on up to the wrist and head joints. This skeleton definition section clearly describes the complete hierarchical relationship of the standard human skeleton from root joints to distal joints using nested bracket structures. The motion data section declares a total of 500 frames with a frame interval of 0.033333 seconds, corresponding to a sampling rate of 30 frames per second. The subsequent 500 lines of data each contain the values ​​for all channels declared in the skeleton definition section. For example, the value in the first row and first column represents the X position of the Hips joint in the first frame, the second column represents the Y position, and the subsequent columns correspond to the rotation angle values ​​of each channel of each joint.

[0052] When the processor on the scheduling server executes step S3 to obtain the source action file, it reads the entire text content of the BVH format file, first parsing the skeleton definition section. Based on the nested bracket structure and keywords, the processor identifies the name of each joint, its parent joint pointer, and channel information, constructing a hierarchical tree data structure of the universal human standard skeleton in memory. Specifically, the processor creates a data object for each joint, containing four attributes: joint name, parent joint reference, list of child joints, and list of channels. The processor reads line by line in the nested order of the file, creating a new data object for each joint definition encountered and adding it as a child joint of the current parent joint. When encountering a closing bracket, the processor returns to the previous parent joint. Through this parsing method, the processor completely reconstructs the hierarchical structure of the universal human standard skeleton defined in the BVH file in memory.

[0053] After parsing the skeleton definition, the processor continues to parse the motion data. The processor first reads the frame number and frame interval, then allocates a two-dimensional array to store the source joint angle sequence. The first dimension of this array corresponds to the frame index, and the second dimension corresponds to the channel index. The processor reads the motion data row by row, parsing each channel value in each row into a floating-point number and filling it into the corresponding position in the two-dimensional array according to the channel declaration order. For values ​​declared as rotation channels, the processor stores them directly as angle values, retaining their original degree units without radian conversion to maintain consistency with the angle and velocity limit units in the mapping configuration file. After parsing, the source joint angle sequence exists in memory as a two-dimensional floating-point array, with the number of rows equal to the frame number and the number of columns equal to the total number of joint channels. This array is the input data for the motion conversion operation in step S5.

[0054] For example, in the aforementioned 500-frame, 30-frame-per-second BVH file, after parsing the skeleton definition, the processor constructs a skeleton hierarchy tree containing 23 joints. The parsed motion data portion yields a 500-row, 69-column floating-point array. The 69 columns correspond to the total data for each of the 23 joints' 3 rotation channels. At frame 100, the Z-rotation channel value of the RightShoulder joint is located in row 100, column 31 of this array, with a value of 45.0 degrees.

[0055] The separation of the skeleton hierarchy and motion data in the BVH format file allows the motion redirection method of this application to use the skeleton definition as a reference benchmark for a universal human standard skeleton and the motion data as the input source for the source joint angle sequence. The same BVH file of a universal human standard skeleton can be adapted to different target robot models through different mapping configuration files without any modification to the BVH file itself, achieving complete decoupling of motion data and target robot morphology.

[0056] In one possible implementation, determining whether the computational load corresponding to the redirected task request is lower than a preset threshold includes: S41. Obtain the total number of frames of the source joint angle sequence in the source motion file.

[0057] In step S3, the scheduling server's processor parses the source motion file into a frame-organized sequence of source joint angles, which is stored in memory as a two-dimensional array. In this step, the processor directly reads the length of the first dimension of this two-dimensional array, which represents the total number of frames in the source joint angle sequence. For BVH format source motion files, this total number of frames can also be directly read from the Frames field declared in the motion data section of the BVH file, which is recorded in the metadata area of ​​memory during file parsing.

[0058] For example, in step S3, the processor parses a BVH file containing 500 frames of data, resulting in a two-dimensional array with 500 rows. The processor reads the row value 500 of this array, uses 500 as the total number of frames in the source joint angle sequence, and stores it in an integer variable for use in subsequent steps.

[0059] S42. Obtain the number of joints of the target robot based on the joint name correspondence in the mapping configuration file.

[0060] In step S2, the scheduling server's processor loads the entire mapping configuration file into memory. The joint name mapping in the configuration file is stored as a list of key-value pairs. Each key-value pair records a mapping from a source joint name in a standard human skeleton to a target joint name in the target robot. The processor iterates through this list, counting the total number of key-value pairs, which represents the number of joints in the target robot. The joint count here only includes joints involved in motion redirection, i.e., joints with a clearly established mapping relationship in the joint name mapping. The target robot may have some auxiliary joints or end effector joints that do not participate in motion redirection; these joints do not appear in the joint name mapping and are therefore not included in the joint count. This approach ensures that the computational load only reflects the scale of joints that actually require conversion calculations, avoiding redundant counting of joints that do not participate in the calculations.

[0061] For example, the joint name mapping section of the mapping configuration file contains 6 mapping entries: RightShoulder maps to joint_1, RightElbow maps to joint_2, RightWrist maps to joint_3, LeftShoulder maps to joint_4, LeftElbow maps to joint_5, and LeftWrist maps to joint_6. The processor iterates through these 6 entries, counts the total number of key-value pairs as 6, and stores this value of 6 as the number of joints in the target robot in an integer variable for use in subsequent steps.

[0062] S43. The product of the total number of frames and the number of joints is used as the calculation amount and compared with the preset threshold.

[0063] The scheduling server's processor obtains the total number of frames from step S41 and the number of joints from step S42, multiplies the two to obtain an integer product, which is the computational cost of this redirection task. The processor compares this computational cost with a preset threshold. If the computational cost is less than the preset threshold, the processor determines that this task is a lightweight task and allocates the redirection task to the local computing resource pool for execution. If the computational cost is greater than or equal to the preset threshold, the processor determines that this task is a batch task or a high-precision task and allocates the redirection task to the cloud GPU computing resource pool for execution.

[0064] The method for determining the preset threshold is as follows. The preset threshold is used to define the task scheduling boundary between the local computing resource pool and the cloud GPU computing resource pool, and its value needs to take into account the following two factors.

[0065] The first factor is the relative performance difference between the local compute resource pool and the cloud GPU compute resource pool in motion retargeting computation tasks. This difference can be quantified by performing a set of standardized motion retargeting benchmarks. The benchmark method involves selecting a standard BVH motion file with a fixed number of frames and a standard mapping configuration file with a fixed number of joints. The complete S5 transformation process is then executed on a typical compute node in the local compute resource pool and a standard GPU container instance in the cloud GPU compute resource pool, respectively, and the completion time is recorded. The completion time of the local compute node is recorded as the local time, and the completion time of the cloud GPU container instance is recorded as the cloud time. Dividing the local time by the cloud time yields the cloud speedup ratio. A typical cloud speedup ratio is between 2 and 10, depending on the specific configuration of the local compute node's CPU and the cloud GPU.

[0066] The second factor is the system administrator's expected response time for task processing. If the system administrator expects tasks with lower computational loads to be processed locally in real time to reduce network transmission latency and cloud call overhead, while tasks with higher computational loads are submitted to the cloud for parallel acceleration, then an expected task processing time threshold can be set. This threshold can be set to 100 milliseconds. For tasks with computational loads below a certain threshold, the processing time on the local computing node is within 100 milliseconds, and the latency perceived by the user is extremely low. For tasks with computational loads above this threshold, the local processing time will exceed 100 milliseconds. In this case, although submitting to the cloud for processing increases network transmission time by about 20 to 50 milliseconds, the cloud processing time is significantly reduced, and the overall response time can still be kept within an acceptable range.

[0067] The specific calculation method for the preset threshold is as follows: First, determine the average computation time corresponding to the local computing node processing the joint product of a single frame. In the benchmark test, the product of the total number of frames and the number of joints is used as the benchmark computation amount. Divide the local time consumption by the benchmark computation amount to obtain the time required for local processing of a unit of computation amount. Then, divide the expected task processing time threshold by the time required for that unit of computation amount to obtain the corresponding computation amount threshold.

[0068] For example, the benchmark uses a standard BVH motion file with a total of 1000 frames, and the target robot joints in the standard mapping configuration file are 10. The benchmark computation is 1000 multiplied by 10, which equals 10000. The local compute node takes 200 milliseconds to complete this benchmark, so the time required per unit of computation is 200 divided by 10000, which equals 0.02 milliseconds. The expected task processing time threshold is 100 milliseconds, so the preset threshold is calculated as 100 divided by 0.02, which equals 5000. This means that when the computation is less than 5000, the local processing time is less than 100 milliseconds, and the task is assigned to the local compute resource pool for execution; when the computation is greater than or equal to 5000, the local processing time exceeds 100 milliseconds, and the task is assigned to the cloud GPU compute resource pool for execution to achieve faster processing speed.

[0069] In actual deployment, system administrators can pre-run a benchmark test based on the specific hardware configuration of the local computing resource pool and the cloud GPU computing resource pool, following the method described above. This will calculate a preset threshold suitable for the deployment environment, which will then be written into the scheduling server's configuration parameters. This preset threshold can be re-tested and updated based on subsequent hardware upgrades or resource configuration adjustments.

[0070] In one possible implementation, S5, for each frame of the source motion file, converts the source joint angle sequence into the target robot joint angle sequence according to joint mapping rules and motion constraints, including: S51. Determine the target joint corresponding to the current source joint based on the joint name correspondence.

[0071] The compute node maintains two data tables in memory. The first table contains the current frame data of the source joint angle sequence, recording the rotation angle values ​​of each source joint in the standard human skeleton in the current frame. The second table contains the joint name mapping relationship in the mapping configuration file, which records the mapping from source joint names to target joint names in key-value pairs. The compute node iterates through each mapping entry in the joint name mapping relationship, retrieves the source joint name, and searches for the corresponding angle value in the current frame data of the source joint angle sequence. If a corresponding angle value is found, it is determined that the source joint has valid data in the current frame, and the target joint name corresponding to the source joint name is used as the conversion target. If a mapping entry in the joint name mapping relationship points to a source joint that does not have a corresponding record in the current frame data, the mapping entry is skipped, the angle of the target joint is not updated, and the target joint retains its angle value from the previous frame.

[0072] For example, the compute node reads that the current frame is frame 100. In the source joint angle sequence, this frame records the angle value of RightShoulder as 45.0 degrees and the angle value of RightElbow as 30.0 degrees. The joint name mapping relationship contains a mapping entry that maps RightShoulder to joint_1. The compute node finds the value of RightShoulder as 45.0 degrees in the source joint angle sequence based on this mapping entry, determines that the target joint corresponding to the current source joint RightShoulder is joint_1, and uses this source angle value of 45.0 degrees as the source input angle of joint_1, proceeding to step S52 for processing.

[0073] S52. Transform the source joint angle of the current frame using the rotation offset and bone length scaling ratio in the coordinate system transformation relationship to generate the initial target joint angle.

[0074] The compute node reads the rotation offset and bone length scaling ratio corresponding to the current target joint from the coordinate system transformation relationship in the mapping configuration file. The rotation offset is stored in Euler angles or quaternions. If stored in Euler angles, the rotation offset consists of three angle values, representing the rotation correction around the X, Y, and Z axes, respectively. The compute node performs an angle addition operation on the source joint angle of the current frame and the rotation offset, that is, adds the component of the source joint angle in each axis to the component of the rotation offset in the corresponding axis to obtain the rotated joint angle. Then, the rotated joint angle is multiplied by the bone length scaling ratio to obtain the initial target joint angle. If the bone length scaling ratio is applied to the displacement component but not the rotation component, the displacement component in the rotated joint angle is multiplied by the scaling ratio only, while the rotation component remains unchanged. For joints that only contain rotational degrees of freedom, the bone length scaling ratio is 1.0, that is, no scaling is performed.

[0075] For example, the compute node reads that the rotation offset corresponding to joint_1 is 90 degrees around the Z-axis, and the bone length scaling ratio is 1.0. The current frame's source joint RightShoulder angle is 45 degrees around the Z-axis. The compute node adds the source joint angle of 45 degrees to the rotation offset of 90 degrees, obtaining a rotated angle of 135 degrees. Since the bone length scaling ratio is 1.0, the initial target joint angle remains at 135 degrees. This 135 degrees is the initial target joint angle of joint_1 in the current frame, and the process proceeds to step S53.

[0076] S53. Limit the initial target joint angle within the angle limit included in the range of motion constraint to obtain the target joint angle after angle limit.

[0077] The compute node reads the angle limit corresponding to the current target joint from the motion range constraints of the mapping configuration file. The angle limit includes two boundary values: a minimum angle value and a maximum angle value. The compute node compares the initial target joint angle generated in step S52 with this angle limit. If the initial target joint angle is greater than the maximum angle value, the angle value is corrected to the maximum angle value. If the initial target joint angle is less than the minimum angle value, the angle value is corrected to the minimum angle value. If the initial target joint angle is between the minimum and maximum angle values, the angle value remains unchanged. The angle value obtained after the above comparison and correction is the target joint angle after angle limiting.

[0078] The method for determining the angle limit has been described in detail in the above embodiments. It is based on the physical structural limits of the target robot joint, including the position of the mechanical limit stop, the range of the motor encoder, the allowable torsional range of the cable wiring, and the mechanical interference boundary of adjacent links. It will not be described again here.

[0079] For example, the compute node reads the angle limit for joint_1 as -180 degrees to 180 degrees. The initial target joint angle generated in step S52 is 135 degrees. 135 degrees is greater than -180 degrees but less than 180 degrees, falling within the angle limit range, therefore the target joint angle remains unchanged at 135 degrees after the angle limit is set. If the initial target joint angle calculated in a frame is 190 degrees, exceeding the maximum angle value of 180 degrees, the compute node corrects it to 180 degrees. If the initial target joint angle is -190 degrees, lower than the minimum angle value of -180 degrees, the compute node corrects it to -180 degrees.

[0080] S54. Limit the change in the target joint angle after angle limitation relative to the final target joint angle of the previous frame to within the velocity limit included in the motion range constraint, and obtain the final target joint angle of the current frame.

[0081] The compute node maintains a history state table in memory, which records the final target joint angle value of each target joint in the previous frame. For the first frame of the source motion file, since there is no data from the previous frame, the compute node directly uses the target joint angle after angle limiting as the final target joint angle of that frame and writes it into the history state table for use in the next frame.

[0082] For the second frame and subsequent frames, the compute node reads the final target joint angle value of the current target joint in the previous frame from the historical state table. The compute node subtracts the final target joint angle of the previous frame from the angle-limited target joint angle obtained in step S53 to obtain the angle change. Then, it reads the velocity limit corresponding to the current target joint from the motion range constraints in the mapping configuration file. The velocity limit is a scalar value representing the maximum allowed angle change of the joint between adjacent frames. The absolute value of this velocity limit is the maximum allowed change per frame. The compute node compares the absolute value of the angle change with the velocity limit. If the absolute value of the angle change is less than or equal to the velocity limit, the angle change remains unchanged. If the absolute value of the angle change is greater than the velocity limit, the angle change is clamped to the velocity limit value, and the sign of the clamped angle change remains consistent with the sign of the original angle change. The final target joint angle is equal to the final target joint angle of the previous frame plus the clamped angle change.

[0083] The method for determining the speed limit has been described in detail in the above embodiments. It is based on the product of the rated speed of the joint drive motor, the reduction ratio, the safety factor, and the load derating factor. For a source motion file with a fixed frame interval, the actual usage value of the speed limit in step S54 is the speed limit value recorded in the configuration file multiplied by the frame interval to obtain the maximum allowable angle change per frame.

[0084] For example, the compute node reads the velocity limit for joint_1 as 259.2 degrees per second. The frame interval of the source motion file is 0.01 seconds, therefore the maximum allowed angle change per frame is 259.2 multiplied by 0.01, which equals 2.592 degrees. The final target joint angle in the previous frame, frame 99, is 130.0 degrees. The target joint angle after angle limiting obtained in step S53 is 135.0 degrees. The compute node calculates the angle change as 135.0 minus 130.0, which equals 5.0 degrees. The absolute value of 5.0 degrees is greater than 2.592 degrees, therefore the angle change is clamped to 2.592 degrees, with a positive sign. The final target joint angle in frame 100 is 130.0 plus 2.592, which equals 132.592 degrees. The compute node writes this 132.592 degrees into the history state table, overwriting the previous frame record for joint_1, for use in the calculation of frame 101. S55. The target robot joint angle sequence is composed of the final target joint angles of all frames. The compute node repeats steps S51 to S54, processing each frame of the source motion file and each mapping entry in the joint name correspondence of the mapping configuration file for the target joint, generating the final target joint angle for each target joint in each frame. The compute node allocates a two-dimensional array in memory to store the target robot joint angle sequence; the first dimension of this array corresponds to the frame index, and the second dimension corresponds to the target joint index. After completing the calculation of all target joints in a frame, the compute node writes the final angle value of each target joint in that frame to the corresponding position in the two-dimensional array. Once all frames of the source motion file have been processed, this two-dimensional array constitutes the target robot joint angle sequence. The compute node returns this sequence to the processor of the scheduling server through an internal communication interface, and the processor executes the operation in step S6.

[0085] For example, the source motion file contains 500 frames, and the target robot has 6 joints. The compute node processes each of the 500 frames one by one, executing steps S51 to S54 for each of the 6 target joints in each frame. After processing, a 500-row, 6-column two-dimensional floating-point array is generated. The value in the 100th row and 1st column is 132.592, which is the final target joint angle of joint_1 in the 100th frame. This array is the target robot joint angle sequence, completely describing a segment of executable motion data on the target robot that satisfies physical constraints.

[0086] Through the orderly execution of the above five sub-steps, this application achieves a complete transformation from the source joint angle sequence to the target robot joint angle sequence. The three layers of processing applied sequentially during the transformation—coordinate system transformation, angle constraint, and velocity limiting—ensure that the generated target motion matches the actual physical characteristics of the target robot in terms of geometric relationships, spatial range, and motion speed. The angle constraint and velocity limiting safety constraints are built into the transformation process, reducing the possibility of generating dangerous motion commands that exceed the robot's physical limits.

[0087] Furthermore, this embodiment provides a detailed explanation of the specific process of transforming the source joint angle using coordinate system transformation relationships to generate the initial target joint angle. This process is executed by the allocated computing resources; the following explanation uses a CPU computing node in the local computing resource pool as an example: S521. Apply a rotation offset to the source joint angle of the current frame to obtain the rotated joint angle. After determining the target joint corresponding to the current source joint in step S51, the computing node reads the rotational offset corresponding to the target joint from the coordinate system transformation relationship in the mapping configuration file. The rotational offset describes the fixed rotational correction that needs to be applied between the local coordinate system of the source joint in the general human standard skeleton and the local coordinate system of the target robot joint. This rotational offset is stored in the mapping configuration file in the form of quaternions or Euler angles.

[0088] If the rotation offset is stored in Euler angles, it contains three angle values, representing the rotation angles around the X, Y, and Z axes, respectively. The compute node adds the source joint angle of the current frame to the rotation offset along the corresponding axes. The source joint angle is also represented in Euler angles, containing three rotation components around the X, Y, and Z axes. The compute node adds the X-axis component of the source joint angle to the X-axis component of the rotation offset, adds the Y-axis component of the source joint angle to the Y-axis component of the rotation offset, and adds the Z-axis component of the source joint angle to the Z-axis component of the rotation offset, obtaining three new angle values. These three angle values ​​constitute the rotated joint angle.

[0089] If the rotation offset is stored in quaternion form, the compute node first converts the source joint angle from Euler angles to quaternion form. The method for converting Euler angles to quaternions is common knowledge in the field. The compute node converts the three components of the Euler angle into their corresponding quaternions in the order of rotation around the Z-axis, then around the Y-axis, and then around the X-axis, and multiplies them to obtain the quaternion representation of the source joint angle. Then, the compute node performs a quaternion multiplication operation between the source joint angle quaternion and the rotation offset quaternion, with the rotation offset quaternion multiplied on the left by the source joint angle quaternion. The result of the quaternion multiplication is still a quaternion, which is the quaternion representation of the rotated joint angle. The compute node then converts this quaternion back to Euler angles to obtain the rotated joint angle.

[0090] For example, the compute node is processing frame 100, the current source joint is RightShoulder, and the corresponding target joint is joint_1. The rotation offset is stored in Euler angles, with three components: 0 degrees around the X-axis, 0 degrees around the Y-axis, and 90 degrees around the Z-axis. The angle values ​​of the current frame's source joint RightShoulder are 0 degrees around the X-axis, 0 degrees around the Y-axis, and 45 degrees around the Z-axis. The compute node adds the corresponding components of the two Euler angles to obtain the rotated joint angles: 0 degrees around the X-axis, 0 degrees around the Y-axis, and 135 degrees around the Z-axis.

[0091] S522. Scale the rotated joint angles according to the bone length scaling ratio to generate the initial target joint angles.

[0092] The compute node reads the bone length scaling ratio corresponding to the current target joint from the coordinate system transformation relationship in the mapping configuration file. The bone length scaling ratio is a dimensionless scalar value, representing the ratio of the link length corresponding to the target robot to the bone length corresponding to the standard human skeleton.

[0093] During the coordinate system transformation in motion redirection, the bone length scaling factor is mainly used to correct the displacement component, not the rotation component. For target joints that only have rotational degrees of freedom, i.e., joints that can only rotate around one or a few axes on the target robot without spatial translation, the bone length scaling factor is set to 1.0. The joint angles after rotation do not need to be scaled and are directly used as the initial target joint angles.

[0094] For a target joint that includes translational degrees of freedom, such as the root joint of a target robot, its pose data may contain positional information. The computation node keeps the rotational component of the rotated joint angle unchanged and multiplies the displacement component by a scaling factor based on the bone length. The scaled displacement component and the original rotational component together constitute the initial target joint angle.

[0095] For example, the compute node reads the bone length scaling factor corresponding to joint_1 as 1.0. joint_1 is a pure rotational joint, without translational degrees of freedom. The rotated joint angle obtained in step S521 is 135 degrees around the Z-axis. Since the scaling factor is 1.0, this angle does not need any adjustment, and the initial target joint angle is 135 degrees around the Z-axis.

[0096] For another example, if the target robot's root joint `base_joint` contains translational degrees of freedom, its corresponding bone length scaling factor is 1.5. The rotated joint angle obtained in step S521 includes position components with X coordinate of 0.2 meters, Y coordinate of 0.0 meters, and Z coordinate of 0.5 meters, and a rotation component of 30 degrees around the Z-axis. The computation node multiplies each value of the position component by 1.5, obtaining X coordinate of 0.3 meters, Y coordinate of 0.0 meters, and Z coordinate of 0.75 meters, while keeping the rotation component unchanged at 30 degrees around the Z-axis. The scaled position and rotation components together constitute the initial target joint angle of the `base_joint` joint.

[0097] The method for determining the bone length scaling ratio is as follows. When writing the mapping configuration file, the technician first obtains the bone length between the parent joint and the source joint in the general human standard skeleton. This bone length can be read from the OFFSET field of each joint in the skeleton definition section of the BVH format source motion file. The OFFSET field records the initial offset of the joint relative to its parent joint in the X, Y, and Z axes. The Euclidean norm of these three offsets is the bone length. Then, the technician consults the product specifications or 3D mechanical design drawings of the target robot to obtain the parent link length of the target joint. Dividing the target link length by the source bone length yields the bone length scaling ratio corresponding to that joint.

[0098] For example, in a standard human skeleton, the parent joint of the RightShoulder joint is Shoulders. The offset of RightShoulder relative to Shoulders is 0.0 meters on the X-axis, -0.3 meters on the Y-axis, and 0.0 meters on the Z-axis, with a calculated Euclidean norm of 0.3 meters. The parent link length of the target robot joint_1 is 0.45 meters. The scaling factor for the bone length is 0.45 divided by 0.3, which equals 1.5.

[0099] By applying the rotation offset in step S521 and scaling the bone length in step S522, the computing node completes the coordinate system transformation of a single source joint angle within a frame, generating an initial target joint angle that matches the target joint. This initial target joint angle then enters the angle limiting and velocity limiting processing flow in steps S53 and S54.

[0100] Furthermore, this embodiment provides a detailed description of the specific process of generating a preview animation through browser emulation engine rendering.

[0101] Step S61: Send the target robot joint angle sequence and 3D simulation model to the user terminal, so that the user terminal can drive the 3D simulation model through the browser simulation engine according to the target robot joint angle sequence and render and generate a preview animation.

[0102] After receiving the target robot joint angle sequence returned by the computing resources, the processor of the scheduling server first retrieves the 3D simulation model file corresponding to the target robot model from a pre-built 3D model library. The 3D model library is stored in a persistent storage device connected to the scheduling server; this storage device can be a local disk array or a cloud-based object storage service. The 3D model library is indexed according to the target robot model, with each model directory containing the 3D simulation model file for that model. The 3D simulation model file records the complete geometric information and joint-level assembly relationships of the target robot. The geometric information includes the triangular mesh data, vertex coordinates, normal directions, and texture map coordinates of each link component surface. The joint-level assembly relationships define the parent-child connection order between joints and the position and direction of each joint's rotation axis. The 3D simulation model file can be in GLB or glTF format, both of which are widely used 3D model exchange formats in the field and can be directly loaded and parsed by browser simulation engines.

[0103] The processor of the scheduling server packages the target robot joint angle sequence and the retrieved 3D simulation model file into a response data packet. This response data packet contains two parts: the first part is the target robot joint angle sequence, organized in JSON array format, where each element of the array corresponds to a frame of data, and the frame data records the final target joint angle value of each target joint in that frame in key-value pairs; the second part is the binary data of the 3D simulation model file. The processor sends this response data packet to the user terminal that initiated this redirection task request via HTTP or WebSocket protocol. The user terminal's IP address and communication port have already been recorded in the scheduling server's memory when the redirection task request is received in step S1.

[0104] After receiving the response data packet, the browser on the user terminal parses out the target robot's joint angle sequence and 3D simulation model file. The browser then loads its embedded browser simulation engine. This browser simulation engine is a lightweight 3D rendering engine implemented based on WebAssembly technology, providing call methods to the outside world in the form of a JavaScript interface. The browser simulation engine first parses the 3D simulation model file and constructs the scene graph data structure of the target robot in the browser's memory. The scene graph is organized in a tree structure, with the root node representing the origin of the 3D scene, and the child nodes corresponding to the various linkage components of the target robot. Each linkage node carries the corresponding triangular mesh data. Each joint node in the scene graph records a rotation transformation matrix, the initial value of which is the default pose of the joint in the model file.

[0105] After constructing the scene graph, the browser simulation engine enters the animation rendering loop. The animation rendering loop progresses frame-by-frame according to the frame intervals of the source motion file, with the frame interval value carried in the metadata of the target robot's joint angle sequence. For each frame, the browser simulation engine reads the angle values ​​of each target joint corresponding to that frame from the target robot's joint angle sequence and sets these angle values ​​to the rotation transformation matrix of the corresponding joint node in the scene graph. After the joint rotates, the browser simulation engine updates the world transformation matrix of each node sequentially from top to bottom along the parent-child hierarchy of the scene graph. The rotation of the parent node causes changes in the spatial position of the child nodes, achieving an attitude update for the entire target robot model. After the attitude update for each frame is completed, the browser simulation engine calls the browser's underlying WebGL graphics interface to perform a rendering, rendering the current frame's 3D scene onto the browser window's canvas elements.

[0106] The browser emulation engine simultaneously launches an off-screen frame capture module during the animation rendering loop. After each frame is rendered, this module reads the pixel data of the current frame from the canvas element and sends the pixel data sequentially to a video encoder. The video encoder can use the browser's built-in MediaRecorder interface, which directly calls the operating system's hardware video encoder to encode consecutive frame pixel data into a video file. The video file encoding format can be H.264, the container can be MP4, and the frame rate must match the source animation file. After all frames are rendered and encoded, the browser emulation engine saves the generated video file as a preview animation that can be played directly in the browser.

[0107] For example, the user terminal is a laptop running the Chrome browser. The scheduling server sends the target robot's joint angle sequence and a GLB format 3D simulation model file to the laptop. After receiving the data, the browser loads the WebAssembly simulation engine. The simulation engine parses the GLB file and constructs a scene diagram of the six-axis robotic arm, including the base link, six rotary joints (joint_1 to joint_6), and eight nodes (end-effector). The target robot's joint angle sequence consists of 500 frames with a frame interval of 0.01 seconds. Starting from frame 1, the simulation engine reads the six angle values ​​of joint_1 to joint_6 frame by frame, sets them to the rotation matrix of the corresponding joint node, updates the posture of the entire robotic arm, calls WebGL to render a frame, and simultaneously sends the pixel data to MediaRecorder. After all 500 frames have been processed, MediaRecorder outputs a 5-second MP4 video file at 30 frames per second. The user can directly click to play the video file in the browser to observe the robotic arm performing the redirected dance movements and confirm the naturalness and physical feasibility of the movements.

[0108] During the rendering and preview animation generation process, a preset threshold needs to be determined. This threshold is used to decide whether to simplify the mesh of the 3D simulation model to ensure rendering performance on the browser side. This threshold is called the rendering face count threshold. The browser performance of the user terminal is affected by the performance of the terminal device's central processing unit, graphics processing unit, and available memory capacity. If the total number of triangles in the 3D simulation model is too high, the browser simulation engine may not be able to reach the original frame rate of the source animation file when rendering frame by frame, resulting in stuttering or dropped frames in the preview animation.

[0109] The method for determining the rendering face count threshold is as follows. The system administrator selects a representative target user terminal device whose hardware configuration should represent the usage scenarios of most users. A set of standardized rendering benchmark tests is performed on this device using a browser emulation engine. The benchmark tests use a standard 3D model with a fixed number of joints, and the number of triangles in this model is set to different orders of magnitude, such as 10,000, 50,000, 100,000, 200,000, and 500,000 faces. For each face count level, the browser emulation engine renders 500 frames continuously at a frame rate of 30 frames per second, and the actual average frame rate is recorded. If the actual average frame rate is greater than or equal to 29 frames per second, the rendering performance is considered satisfactory. If the actual average frame rate is less than 29 frames per second, the rendering performance is considered unsatisfactory. The rendering face count threshold is the number of triangles corresponding to the first time the actual average frame rate falls below 29 frames per second, multiplied by a safety factor of 0.8.

[0110] For example, benchmark results show that the actual average frame rate is 29.5 frames per second when the number of triangles is 200,000, and 22 frames per second when the number of triangles is 500,000. Therefore, the first time the frame rate drops below 29 frames per second is when the number of triangles is 500,000, and the rendering face count threshold is 500,000 multiplied by 0.8, which equals 400,000 faces. This 400,000 faces is the preset rendering face count threshold, which is written into the scheduling server's configuration parameters.

[0111] In the actual execution of step S61, after retrieving the 3D simulation model file, the processor of the scheduling server first reads the total number of triangles in the model. If the total number of triangles exceeds the rendering facet threshold of 400,000, the processor calls a mesh simplification algorithm to reduce the facet count to below 400,000, generating a simplified 3D simulation model. This simplified model, along with the target robot's joint angle sequence, is then sent to the user terminal. If the total number of triangles is less than or equal to 400,000, the processor directly sends the original model to the user terminal. This mechanism ensures smooth rendering and preview animation generation on user terminals with varying performance levels, allowing users to receive consistent visual feedback to confirm the redirection effect.

[0112] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0113] Please see Figure 3 This illustration shows a structural schematic diagram of a cross-robot motion redirection device provided in an exemplary embodiment of this application, hereinafter referred to as device 3. Device 3 can be implemented as all or part of a cross-robot motion redirection device through software, hardware, or a combination of both. Device 3 includes: The receiving module 301 is used to receive a redirection task request submitted by a user, wherein the redirection task request includes the source action file identifier and the target robot model; The acquisition module 302 is used to acquire a real-time generated mapping configuration file according to the target robot model. The mapping configuration file defines the joint mapping rules and motion constraints from the general human standard skeleton to the target robot. The acquisition module 302 is further configured to acquire the source action file corresponding to the source action file identifier, wherein the source action file includes a source joint angle sequence organized frame by frame based on the general human standard skeleton; The allocation module 303 is used to determine whether the computational load corresponding to the redirection task request is lower than a preset threshold. If so, the redirection task is allocated to the local computing resource pool; otherwise, it is allocated to the GPU computing resource pool in the cloud. The conversion module 304 is used to utilize the allocated computing resources to convert the source joint angle sequence into the target robot joint angle sequence for each frame of the source motion file according to the joint mapping rules and motion constraints. The sending module 305 is used to send the joint angle sequence of the target robot and the three-dimensional simulation model of the target robot to the user terminal so as to generate a preview animation through the browser simulation engine.

[0114] For further details regarding the implementation of the above technical solution by each module in the control device of the above-mentioned cross-robot motion redirection device, please refer to the description of the cross-robot motion redirection method provided in the above-mentioned embodiments of the invention, which will not be repeated here.

[0115] It should be noted that the device 3 provided in the above embodiments is only illustrated by the division of the above functional modules when executing the cross-robot motion redirection method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the above functions. In addition, the cross-robot motion redirection device and the cross-robot motion redirection method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0116] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0117] Figure 4 The diagram shown is a schematic of a computer storage medium provided in an embodiment of this application. The computer storage medium can store multiple instructions (i.e., ... Figure 4 (a computer program in the above), the instructions are adapted to be loaded and executed by a processor as described above. Figure 2 The method steps of the illustrated embodiment can be found in the following documentation for detailed execution. Figure 2 The specific details of the illustrated embodiments will not be elaborated here.

[0118] This application also provides a computer program product that stores at least one instruction, which is loaded and executed by the processor to implement the cross-robot motion redirection method as described in the above embodiments.

[0119] Please see Figure 5 The diagram below illustrates the structure of a scheduling server as provided in this embodiment. Figure 5 As shown, the scheduling server 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0120] The communication bus 502 is used to enable communication between these components.

[0121] The user interface 503 may include input / output units such as a display screen, camera, keyboard, and touch screen.

[0122] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface) for communication and data transmission with external devices.

[0123] The processor 501 may include one or more processing cores. The processor 501 connects to various parts of the scheduling server 500 via various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 505, and by calling data stored in the memory 505. Optionally, the processor 501 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 501 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 501.

[0124] The memory 505 may include random access memory (RAM) or read-only memory. Optionally, the memory 505 may include a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 505 may also be at least one storage device located remotely from the aforementioned processor 501. Figure 5 As shown, the memory 505, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and application programs.

[0125] exist Figure 5 In the scheduling server 500 shown, the user interface 503 is mainly used to provide an input interface for users and obtain user input data; while the processor 501 can be used to call the application program stored in the memory 505 and specifically execute, such as Figure 1 The method shown can be referred to for details. Figure 1 As shown, it will not be elaborated further here.

[0126] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0127] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method for redirecting actions across robots, characterized in that, include: Receive a redirection task request submitted by a user, the redirection task request including the source action file identifier and the target robot model; Based on the target robot model, obtain the real-time generated mapping configuration file, which defines the joint mapping rules and motion constraints from the general human standard skeleton to the target robot; Obtain the source motion file corresponding to the source motion file identifier, wherein the source motion file includes a source joint angle sequence organized frame by frame based on the general human standard skeleton; Determine whether the computational load corresponding to the redirection task request is lower than a preset threshold. If so, allocate the redirection task to the local computing resource pool; otherwise, allocate it to the GPU computing resource pool in the cloud. Using the allocated computing resources, for each frame of the source motion file, the source joint angle sequence is converted into the target robot joint angle sequence according to the joint mapping rules and motion constraints; The target robot's joint angle sequence and 3D simulation model are sent to the user terminal to generate a preview animation through the browser's simulation engine.

2. The method according to claim 1, characterized in that, The joint mapping rules and motion constraints include: The joint name mapping is used to map the source joint names of the general human standard skeleton to the target joint names of the target robot; The coordinate system transformation relationship includes the rotation offset from the source joint local coordinate system to the target joint coordinate system and the bone length scaling ratio; Range of motion constraints include angular limits and velocity limits for each target joint.

3. The method according to claim 1, characterized in that, The source motion file is a BVH format file. In the BVH format file, the general human standard skeleton is defined by a hierarchical structure, and the source joint angle sequence is organized frame by frame.

4. The method according to claim 2, characterized in that, The step of determining whether the computational load corresponding to the redirected task request is lower than a preset threshold includes: Obtain the total number of frames of the source joint angle sequence in the source motion file; The number of joints of the target robot is obtained based on the joint name correspondence in the mapping configuration file; The product of the total number of frames and the number of joints is used as the computational quantity and compared with the preset threshold.

5. The method according to claim 2, characterized in that, For each frame of the source motion file, converting the source joint angle sequence into a target robot joint angle sequence according to the joint mapping rules and motion constraints includes: The target joint corresponding to the current source joint is determined based on the joint name correspondence. The source joint angle of the current frame is transformed using the rotation offset and bone length scaling ratio in the coordinate system transformation relationship to generate the initial target joint angle. The initial target joint angle is restricted within the angle limit included in the range of motion constraint to obtain the target joint angle after angle limitation; The change in the target joint angle after the angle limit is set relative to the final target joint angle of the previous frame is restricted within the velocity limit included in the motion range constraint to obtain the final target joint angle of the current frame. The target robot joint angle sequence is composed of the final target joint angles of all frames.

6. The method according to claim 5, characterized in that, The step of transforming the source joint angle of the current frame using the rotation offset and bone length scaling ratio in the coordinate system transformation relationship to generate the initial target joint angle includes: Apply the rotation offset to the source joint angle of the current frame to obtain the rotated joint angle; The joint angles after rotation are scaled according to the bone length scaling ratio to generate the initial target joint angles.

7. The method according to claim 1, characterized in that, The step of generating a preview animation by rendering it using a browser emulation engine includes: The target robot joint angle sequence and the 3D simulation model are sent to the user terminal, so that the user terminal can use the browser simulation engine to drive the 3D simulation model according to the target robot joint angle sequence and render the preview animation.

8. A cross-robot motion redirection device, characterized in that, The device includes: The receiving module is used to receive a redirection task request submitted by the user, wherein the redirection task request includes the source action file identifier and the target robot model; The acquisition module is used to acquire a real-time generated mapping configuration file based on the target robot model. The mapping configuration file defines the joint mapping rules and motion constraints from a general human standard skeleton to the target robot. The acquisition module is further configured to acquire the source action file corresponding to the source action file identifier, wherein the source action file includes a source joint angle sequence organized frame by frame based on the general human standard skeleton; The allocation module is used to determine whether the computational load corresponding to the redirection task request is lower than a preset threshold. If so, the redirection task is allocated to the local computing resource pool; otherwise, it is allocated to the GPU computing resource pool in the cloud. The conversion module is used to utilize the allocated computing resources to convert the source joint angle sequence into the target robot joint angle sequence for each frame of the source motion file according to the joint mapping rules and motion constraints. The sending module is used to send the joint angle sequence of the target robot and the three-dimensional simulation model of the target robot to the user terminal so that a preview animation can be generated by rendering through the browser simulation engine.

9. A scheduling server, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the cross-robot motion redirection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the cross-robot motion redirection method as described in any one of claims 1 to 7.

Images