Data processing method, system and device for motion reorientation and storage medium

Abstract

Embodiments of the present application provide a data processing method, system, device and storage medium for motion retargeting. The method comprises: obtaining source motion data and mapping configuration information representing the association relationship between retargeting markers and source motion nodes; determining intermediate representation description information based on the mapping configuration information, which contains marker declaration information and data organization information of a plurality of retargeting markers; mapping the motion data of the source motion nodes using the mapping configuration information to obtain marker data of each retargeting marker; organizing the marker data based on the data organization information to generate intermediate representation frame data; and providing the intermediate representation description information and the intermediate representation frame data to a motion retargeting solving module, wherein the intermediate representation description information is separated from the intermediate representation frame data and is used to analyze the intermediate representation frame data. By obtaining the intermediate representation with the intermediate representation description information separated from the intermediate representation frame data, the development complexity is reduced and the data transmission efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of teleoperation technology, and in particular to a data processing method, system, device and storage medium for motion redirection. Background Technology

[0002] In humanoid robot motion control and teleoperation, acquiring operator motion data through motion capture equipment and redirecting it to robots of different shapes is an important technical approach to achieve teleoperation of complex tasks.

[0003] Due to significant differences between humans and robots in skeletal structure, joint degrees of freedom, and range of motion, it is necessary to accurately map human movement intentions to heterogeneous robot bodies, while adapting to combinations of various motion capture devices and robots. Dedicated data conversion adapters can be developed for each motion capture device and robot combination to convert motion capture data in specific formats into robot control commands.

[0004] However, this architecture, which tightly couples devices and algorithms, causes integration complexity to increase exponentially with the number of device and robot types, severely limiting the architecture's versatility. Summary of the Invention

[0005] This application provides a data processing method, system, device, and storage medium for motion redirection, in order to solve the technical problem of excessive complexity in the integration and development of data conversion adapters between motion capture devices and robots in related technologies.

[0006] In a first aspect, embodiments of this application provide a data processing method for motion redirection, comprising the following steps.

[0007] Receive source motion data and mapping configuration information. The source motion data includes at least one motion frame, and each motion frame includes node motion data corresponding to multiple source motion nodes. The mapping configuration information is used to characterize the association relationship between multiple redirection markers and one or more of the source motion nodes, as well as the marker generation method corresponding to each redirection marker. Based on the mapping configuration information, intermediate representation description information is determined, which includes marker declaration information and data organization information corresponding to the plurality of redirection marker points; For a target motion frame in the at least one motion frame, based on the mapping configuration information, the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame are mapped to obtain the marker point data corresponding to the redirection marker points associated with at least a portion of the source motion nodes in the target motion frame; According to the data organization method indicated by the data organization information, the marker point data corresponding to the plurality of redirection marker points are organized to generate intermediate representation frame data corresponding to the target motion frame; The intermediate representation description information and the intermediate representation frame data are respectively provided to the robot's motion relocation solution module, so that the motion relocation solution module can parse the intermediate representation frame data based on the intermediate representation description information.

[0008] Secondly, embodiments of this application provide a data processing system for motion retargeting, including a motion capture data interface, a data mapping module, and a motion retargeting solution module.

[0009] The motion capture data interface is used to receive source motion data from different motion capture devices and send the source motion data to the data mapping module; The data mapping module is used to execute any of the motion-oriented data processing methods described above; The motion redirection solution module is used to receive intermediate representation description information and intermediate representation frame data provided by the data mapping module, and parse the intermediate representation frame data based on the intermediate representation description information to perform motion redirection calculation; wherein, the intermediate representation description information includes marker declaration information, and the intermediate representation frame data includes marker point data corresponding to the marker declaration information.

[0010] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the motion-oriented data processing method described above.

[0011] Fourthly, a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the motion-oriented data processing method as described above.

[0012] Fifthly, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the motion-oriented data processing method as described above.

[0013] The data processing method, system, device, and storage medium for motion redirection provided in this application acquire source motion data and mapping configuration information, and determine intermediate representation description information based on the mapping configuration information. This intermediate representation description information includes label declaration information and data organization information for multiple redirection markers. For a target motion frame in a motion frame, the motion data of the source motion nodes is mapped using the mapping configuration information to obtain the label data of each redirection marker. The label data is organized according to the method indicated by the data organization information to generate intermediate representation frame data corresponding to the target motion frame. The intermediate representation description information and the intermediate representation frame data are respectively provided to the robot's motion redirection solving module, so that the motion redirection solving module can parse the intermediate representation frame data based on the intermediate representation description information. By constructing intermediate representation description information separate from intermediate representation frame data, a data format contract is established between the motion capture device and the robot's redirection solving module. This eliminates the need for the motion capture device and robot to understand each other's private protocols. Both the motion capture device and the robot uniformly follow this universal and independent intermediate representation format. This transforms the original requirement to develop and maintain M×N functionally overlapping but incompatible dedicated data conversion adapters into M data conversion adapters developed for M types of motion capture devices and N data conversion adapters developed for N types of robots. This significantly reduces the integration complexity between various motion capture devices and various heterogeneous robots. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating the motion redirection-oriented data processing method provided in an embodiment of this application.

[0016] Figure 2 This is a schematic diagram of the structure of a motion redirection-oriented data processing system provided in an embodiment of this application.

[0017] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0018] To achieve the common goal of accurately mapping operator motion intentions to robot bodies with different shapes, a dedicated data conversion pipeline can be constructed for specific equipment combinations. Specifically, this solution involves developing independent data conversion adapters for each motion capture device and each robot. These adapters integrate core functions such as skeletal model parsing, coordinate transformation, and retargeting mapping. The adapters directly read the structured data output by the specific motion capture device, parse the joint hierarchy, position, and rotation information, and calculate the corresponding joint control commands frame by frame based on a pre-defined robot kinematic model. This tightly coupled end-to-end design allows for deep optimization for fixed equipment combinations, achieving low latency and high data fidelity in a single, stable hardware environment.

[0019] However, when this solution is applied to system integration scenarios requiring compatibility with multiple heterogeneous motion capture devices and various robot configurations, its inherent point-to-point connection architecture leads to a multiplicative increase in the number of data conversion adapters with the number of device types, resulting in a sharp increase in integration development complexity. In a system containing M types of motion capture devices and N types of robots, it is necessary to develop and maintain M×N functionally overlapping but incompatible dedicated data conversion adapters. For example, in redirection tasks involving two data sources—optical motion capture systems and inertial motion capture systems—and two target platforms, each frame of data must include the unique marker name, type definition, and coordinate system metadata of its source device. This results in a large amount of static information being repeatedly sent during high-frequency transmission, leading to consistently high bandwidth consumption. Furthermore, replacing any device requires redeveloping and verifying the corresponding module, resulting in low code reusability.

[0020] To address the aforementioned technical issues, this application proposes a data processing method for motion retargeting. This method involves acquiring source motion data and mapping configuration information, and determining intermediate representation description information based on the mapping configuration information. This intermediate representation description information includes label declaration information and data organization information for multiple retargeting markers. For a target motion frame within a motion frame, the motion data of the source motion nodes is mapped using the mapping configuration information to obtain the label data for each retargeting marker. The label data is then organized according to the data organization information to generate intermediate representation frame data corresponding to the target motion frame. The intermediate representation description information and the intermediate representation frame data are then provided to the robot's motion retargeting solution module, enabling the motion retargeting solution module to parse the intermediate representation frame data based on the intermediate representation description information. By constructing intermediate representation description information separate from intermediate representation frame data, a data format contract is established between the motion capture device and the robot's redirection solving module. This eliminates the need for the motion capture device and robot to understand each other's private protocols. Both the motion capture device and the robot uniformly follow this universal and independent intermediate representation format. This transforms the original requirement to develop and maintain M×N functionally overlapping but incompatible dedicated data conversion adapters into M data conversion adapters developed for M types of motion capture devices and N data conversion adapters developed for N types of robots. This significantly reduces the integration complexity between various motion capture devices and various heterogeneous robots.

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] Figure 1 This is a flowchart illustrating the motion redirection-oriented data processing method provided in an embodiment of this application, as shown below. Figure 1 As shown in the figure, this application embodiment provides a data processing method for motion redirection, the execution entity of which can be a data mapping module. The method may include the following steps 101 to 104.

[0023] Step 101: Receive source motion data and mapping configuration information. The source motion data includes at least one motion frame. Each motion frame includes node motion data corresponding to multiple source motion nodes. The mapping configuration information is used to characterize the association between multiple redirection markers and one or more of the source motion nodes, as well as the marker generation method corresponding to each redirection marker.

[0024] Specifically, source motion data refers to any raw data sequence that can describe the spatial state of various parts of an object (such as a human body) during motion, and it can come from any type of motion capture device or motion sensor component. For example, source motion data can include at least one of the following: skeletal joint world pose data output by an optical motion capture system, joint rotation and acceleration data output by an inertial motion capture system, and keypoint coordinate data output by a vision-based pose estimation algorithm. Source motion data can include at least one motion frame, and each motion frame records the node motion data corresponding to multiple source motion nodes at a certain moment.

[0025] A motion frame is a time slice of the source motion data, corresponding to a snapshot of the complete motion state of the object at a certain moment. The generation frequency of motion frames depends on the frame rate of the motion capture device, such as 60Hz, 120Hz or higher.

[0026] The node motion data of the source motion node may include position data and quaternion rotation data, or only position data, or only quaternion rotation data.

[0027] Redirection markers are logical data units bound to a specific marker type that transmit spatial constraint information to the robot's motion redirection solution module. Examples include redirection markers such as "left hand," "right hand," and "left foot" used to describe the robot's end effector pose; "left upper arm direction" and "right forearm direction" used to describe limb pointing; "head orientation" redirection markers used to describe head orientation; and "robot center of gravity" redirection markers used to describe the center of gravity position.

[0028] Mapping configuration information is used to characterize the association between multiple redirection markers and one or more source motion nodes, as well as the marker generation method corresponding to each redirection marker. Mapping configuration information may include conversion rules or parameter information from source motion data to intermediate representation frame data. The marker generation method is a specific calculation logic or processing rule defined by the mapping configuration information, used to convert the node motion data of one or more source motion nodes associated with a redirection marker into marker data corresponding to the redirection marker.

[0029] For example, the system receives 120 frames of skeletal data per second from an optical motion capture system. Each frame is a motion frame, and each motion frame contains multiple source motion nodes, such as the hip, left shoulder, left elbow, left wrist, and right ankle. Each source motion node is associated with a set of node motion data, including the 3D position data and quaternion rotation data of the source motion node in the world coordinate system. The mapping configuration information is provided in YAML format and is read when the system starts. The mapping configuration information defines the association between multiple redirection markers and one or more source motion nodes, as well as the marker generation method for each redirection marker.

[0030] Step 102: Based on the mapping configuration information, determine the intermediate representation description information, which includes the tag declaration information and data organization information corresponding to the multiple redirection tag points.

[0031] Specifically, intermediate representation description information is determined based on the received mapping configuration information. This intermediate representation description information may include tag declaration information and data organization information corresponding to multiple redirection tag points.

[0032] The marker declaration information is used to declare the basic attributes of each redirection marker, including at least one of the marker name and the marker type. For example, the marker declaration information may include one or more records, each containing a marker name of type string (e.g., left hand) and a marker type of type enumeration (e.g., pose type marker), or a list of marker declarations in key-value pair form.

[0033] Data organization information is used to indicate how each data component in the marker data corresponding to each type of redirection marker is arranged, grouped, or stored in the intermediate representation frame data.

[0034] Step 103: For the target motion frame in the at least one motion frame, based on the mapping configuration information, perform mapping processing on the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame to obtain the marker point data corresponding to the redirection marker points associated with at least a portion of the source motion nodes in the target motion frame.

[0035] Specifically, for a target motion frame in at least one motion frame, based on the received mapping configuration information, mapping processing is performed on the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame. That is, according to the defined association between redirection markers and source motion nodes and the marker generation method, marker point data corresponding to the redirection markers associated with at least a portion of the source motion nodes is extracted, calculated, or transformed from the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame. The marker point data may include at least one of the position component, orientation component, and rotation component corresponding to the redirection marker.

[0036] The target motion frame refers to the currently selected motion frame to be processed from at least one motion frame. In the embodiments of this application, motion frames are received and processed one by one in chronological order, and each received motion frame can be used as a target motion frame for mapping processing.

[0037] Step 104: Organize the marker point data corresponding to the multiple redirection marker points according to the data organization method indicated by the data organization information, and generate intermediate representation frame data corresponding to the target motion frame.

[0038] Specifically, after obtaining the marker point data corresponding to multiple redirection marker points, the marker point data is organized according to the data organization information in the mapping configuration information, and intermediate representation frame data corresponding to the target motion frame is generated.

[0039] The intermediate frame data is a high-frequency data frame structure that carries the dynamic values ​​of the redirection marker points corresponding to a single motion frame, such as a floating-point array.

[0040] Step 105: Provide the intermediate representation description information and the intermediate representation frame data to the robot's motion relocation solution module, so that the motion relocation solution module can parse the intermediate representation frame data based on the intermediate representation description information.

[0041] Specifically, intermediate representation description information and intermediate representation frame data are provided to the motion redirection solving module. The implementation methods for providing data may include: sending data through a communication interface or bus, transferring data through function calls or application programming interfaces (APIs), or transferring data through storage media.

[0042] A motion retargeting solver module refers to any software or hardware component capable of receiving intermediate representation description information and intermediate representation frame data, parsing the intermediate representation frame data based on the intermediate representation description information, and then performing motion retargeting calculations to generate robot joint control commands. Examples include motion control modules based on inverse kinematics solvers and whole-body motion retargeting solvers based on optimization algorithms.

[0043] Parsing intermediate representation frame data refers to the process by which the motion redirection solving module correctly extracts the data component corresponding to each redirection marker point from the received intermediate representation frame data based on the marker declaration information and data organization information in the intermediate representation description information.

[0044] For example, the marker declaration information in the intermediate representation description information is read to determine the redirection markers contained in the current intermediate representation frame data and the marker type of each redirection marker. Then, based on the declaration order and data organization information in the marker declaration information, the data fields and storage locations corresponding to each data component (such as position, rotation, or orientation components) in the marker data of each redirection marker in the intermediate representation frame data are determined. Finally, the data components of the marker data corresponding to each redirection marker are extracted from the storage locations of the corresponding data fields for use in subsequent motion redirection calculations.

[0045] In this embodiment of the application, the intermediate representation description information is separated from the intermediate representation frame data, that is, the intermediate representation description information and the intermediate representation frame data are independent of each other at least at the level of structure, content and transmission.

[0046] In this context, "structure separation" means that the intermediate representation description information and the intermediate representation frame data exist as independent data objects or data structures, rather than being packaged in the same data structure. For example, the intermediate representation description information is a Profile object, and the intermediate representation frame data is a Frame object.

[0047] Content separation means that the intermediate representation frame data only contains dynamic values ​​(such as position components, rotation components, and orientation components) and does not carry any intermediate representation description information (such as marker point names, associated robot model information, etc.).

[0048] Transmission separation refers to the fact that the intermediate representation description information and the intermediate representation frame data are independent of each other in terms of transmission timing and transmission frequency.

[0049] The motion redirection-oriented data processing method provided in this application establishes a data format contract between the motion capture device and the motion redirection solution module by constructing intermediate representation description information separate from the intermediate representation frame data. This transforms the original requirement of developing and maintaining M×N functionally overlapping but incompatible dedicated data conversion adapters into M data conversion adapters developed for M types of motion capture devices and N data conversion adapters developed for N types of robots, significantly reducing the integration complexity between various motion capture devices and various heterogeneous robots. Simultaneously, the separation of intermediate representation frame data from intermediate representation description information effectively saves transmission bandwidth and supports the robot's motion redirection solution module in parsing the intermediate representation frame data based on the intermediate representation description information. This achieves complete decoupling of the motion capture device, data mapping module, and redirection solution module, improving scalability and flexibility.

[0050] In some embodiments, determining intermediate representation description information based on the mapping configuration information may include: parsing the mapping configuration information to obtain multiple tag generation rules; generating tag declaration information corresponding to the multiple redirection tag points based on the redirection tag points corresponding to each of the tag generation rules; and determining the data organization information based on the tag declaration information corresponding to the multiple redirection tag points.

[0051] Specifically, parsing the mapping configuration information yields multiple tag generation rules, which specify the tag generation method for redirection tags, such as data source and generation logic. One tag generation rule typically determines one redirection tag.

[0052] Based on the redirection markers corresponding to the marker generation rules, multiple marker declaration information is generated for each redirection marker. Data organization information is then determined based on this marker declaration information. The order (or declaration order) of the redirection markers in the marker declaration information determines the storage order of the marker data. Therefore, the marker declaration information implicitly contains the indexing convention for the marker data, and the data organization information can be directly determined by the order of the redirection markers and the marker type in the marker declaration information.

[0053] The motion redirection-oriented data processing method provided in this application parses the tag generation rules from the mapping configuration information and automatically generates tag declaration information and data organization information. This makes the construction process of intermediate representation description information completely driven by the mapping configuration information, eliminating the need to manually encode data contracts for different device or robot interaction scenarios. This simplifies the initialization process and reduces frame parsing misalignment caused by manual declaration errors.

[0054] In some embodiments, generating tag declaration information corresponding to the plurality of redirection tag points based on the redirection tag points corresponding to each tag generation rule may include: determining the tag type corresponding to each redirection tag point from a preset tag type set for each redirection tag point; generating tag declaration information corresponding to each redirection tag point based on the tag point name and tag type corresponding to each redirection tag point; wherein, the preset tag type set includes at least one of pose type, direction type, orientation type, and position type, the pose type is used to characterize the spatial position and rotation state of the redirection tag point, the direction type is used to characterize the spatial pointing of the redirection tag point, the orientation type is used to characterize the rotation state of the redirection tag point, and the position type is used to characterize the spatial position of the redirection tag point.

[0055] Specifically, the preset mark type set is used to uniformly describe all spatial constraint information of the redirection mark points in motion redirection scenarios.

[0056] The marker data corresponding to the pose type repositioning marker may include three-dimensional spatial position components and quaternion rotation components. The pose type is used to fully describe the position and rotation state of the repositioning marker in space. The rotation state refers to the spatial orientation information containing complete pose information, represented by quaternions.

[0057] The marker data corresponding to a redirection marker of a direction type may include a direction component, which describes the spatial orientation of the redirection marker. This spatial orientation can be represented by a direction vector.

[0058] The marker data corresponding to the orientation-type redirection marker may include a rotation component to describe the rotation state of the marker.

[0059] The marker data corresponding to a location-type redirected marker may include a location component, which describes the spatial location of the marker.

[0060] For each of the repositioning markers, the corresponding marker type is determined from a preset marker type set. For example, for a repositioning marker describing a left-hand end effector, its marker type can be determined as a pose type; for a repositioning marker describing forearm orientation, its marker type can be determined as a direction type.

[0061] Based on the marker name and target marker type corresponding to each redirection marker, generate marker declaration information for each redirection marker. That is, combine the marker name and marker type of each redirection marker into a marker declaration information, thereby obtaining marker declaration information for multiple redirection markers.

[0062] The marker name is used to uniquely identify a redirection marker, and redirection markers can be declared and indexed in the intermediate description information.

[0063] Tag declaration information can be recorded in list form and arranged in the order in which the tag generation method appears in the mapping configuration information.

[0064] The motion retargeting data processing method provided in this application clarifies the spatial constraint information to be represented by the retargeting marker points by pre-setting a set of marker types. By generating the marker declaration information corresponding to the retargeting marker points, the identity and marker type of each retargeting marker point can be clearly identified. It is not necessary to be aware of which motion capture device the intermediate representation frame data comes from, nor is it necessary to pay attention to the internal data structure and naming conventions of the motion capture device. This decouples the motion capture device from the robot's inverse motion solution module and reduces the integration complexity between various motion capture devices and various heterogeneous robots.

[0065] In some embodiments, based on the mapping configuration information, mapping processing is performed on the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame to obtain the marker point data corresponding to the redirection marker points associated with at least a portion of the source motion nodes in the target motion frame. This may include: obtaining the redirection marker points associated with one or more source motion nodes in the target motion frame based on the mapping configuration information; and processing the node motion data corresponding to one or more source motion nodes in the target motion frame based on the marker generation method corresponding to the redirection marker points associated with one or more source motion nodes in the target motion frame to obtain the marker point data corresponding to the redirection marker points associated with one or more source motion nodes in the target motion frame.

[0066] Specifically, for a target motion frame within a motion frame, based on mapping configuration information, redirection markers associated with one or more source motion nodes in the target motion frame can be obtained. After determining the redirection markers associated with one or more source motion nodes, the node motion data corresponding to these one or more source motion nodes is read or extracted from the target motion frame. Based on the marker generation method corresponding to the redirection markers, the node motion data corresponding to these one or more source motion nodes is processed to obtain the marker point data corresponding to the redirection markers associated with these one or more source motion nodes.

[0067] For example, for a specific motion frame in the source motion data, i.e. the target motion frame, the redirection markers associated with one or more source motion nodes in the target motion frame can be determined based on the mapping configuration information. For instance, the source motion node "left palm" is associated with the pose-type redirection marker "left hand", the source motion nodes "left elbow" and "left wrist" are associated with the direction-type redirection marker "left forearm direction", the source motion node "head" is associated with the orientation-type redirection marker "head orientation", and the source motion node "hip joint" is associated with the position-type redirection marker "robot center of gravity". The system reads node motion data from one or more source motion nodes in the target motion frame. Based on the marker generation method corresponding to the redirection marker, it processes the read node motion data to obtain the marker data corresponding to each redirection marker. For example, based on the marker generation method corresponding to the pose-type redirection marker "left hand", the system processes the node motion data of the source motion node "left palm" to obtain the marker data corresponding to the redirection marker "left hand". Based on the marker generation method corresponding to the direction-type redirection marker "left lower arm direction", the system processes the node motion data of the source motion nodes "left elbow" and "left wrist" to obtain the marker data corresponding to the redirection marker "left lower arm direction". Based on the marker generation method corresponding to the orientation-type redirection marker "head orientation", the system processes the node motion data of the source motion node "head" to obtain the marker data corresponding to the redirection marker "head orientation". Based on the marker generation method corresponding to the position-type redirection marker "robot center of gravity", the system processes the node motion data of the source motion node "hip joint" to obtain the marker data corresponding to the redirection marker "robot center of gravity".

[0068] The motion redirection-oriented data processing method provided in this application obtains the redirection markers associated with one or more source motion nodes in the target motion frame by defining the association relationships between multiple redirection markers defined in the mapping configuration information and one or more source motion nodes. Based on the marker generation method corresponding to the redirection markers, the node motion data corresponding to one or more source motion nodes is processed into marker point data corresponding to the associated redirection markers. This realizes the conversion from heterogeneous source motion data to marker point data. Moreover, when the source motion data undergoes local changes, the associated redirection markers can be accurately located based on the changed source motion nodes, and their marker point data can be re-determined, thus improving the efficiency of the mapping process. For the same redirection marker, only the mapping configuration information needs to be modified to change one or more source motion nodes associated with it or to change the marker generation method corresponding to the redirection marker, without changing the main frame mapping process, thus improving the flexibility and configurability of the mapping process.

[0069] In some embodiments, based on the marker generation method corresponding to the redirection markers associated with one or more source motion nodes in the target motion frame, the node motion data corresponding to one or more source motion nodes in the target motion frame is processed to obtain marker point data corresponding to the redirection markers associated with one or more source motion nodes in the target motion frame. This may include at least one of the following: when the marker type of the redirection marker is a pose type, generating position and rotation components in the node motion data of the redirection marker based on position and rotation data in the node motion data of the source motion nodes associated with the redirection marker; when the marker type of the redirection marker is an orientation type, generating rotation components in the node motion data of the redirection marker based on rotation data in the node motion data of the source motion nodes associated with the redirection marker; when the marker type of the redirection marker is a position type, generating position components in the node motion data of the redirection marker based on position data in the node motion data of the source motion nodes associated with the redirection marker; when the marker type of the redirection marker is a direction type, generating the corresponding direction component in the marker point data of the redirection marker based on the corresponding position or rotation data in the node motion data of the source motion nodes associated with the redirection marker.

[0070] Specifically, position data refers to the spatial position information of the source moving node, usually represented by a three-dimensional coordinate vector (x, y, z). Rotation data refers to the spatial orientation information of the source moving node, usually represented by a quaternion (qx, qy, qz, qw).

[0071] For pose-type repositioning markers, marker data for the repositioning marker can be generated based on the position and rotation data of the source motion node associated with the repositioning marker. This marker data contains both position and rotation components.

[0072] For example, for the pose-type repositioning marker "left hand", the node motion data of the source motion node associated with the repositioning marker is determined according to the marker generation method corresponding to the repositioning marker. The node motion data includes position data (three-dimensional spatial coordinates) and rotation data (quaternion rotation data). The position data and the rotation data are directly used as the position component and rotation component of the repositioning marker.

[0073] For orientation-type redirection markers, marker data for the redirection marker can be generated based on the rotation data of the source motion node associated with the redirection marker, and the marker data contains only the rotation component.

[0074] For example, for a redirection marker of orientation type "head orientation", the node motion data of the source motion node associated with the redirection marker is determined according to the marker generation method corresponding to the redirection marker. The node motion data includes rotation data (quaternion rotation data), and the rotation data is directly used as the rotation component of the redirection marker.

[0075] For location-type redirection markers, marker data for the redirection marker can be generated based on the location data of the source motion node associated with the redirection marker. This marker data contains only the location component.

[0076] For example, for a redirection marker of the position type "robot center of gravity", the node motion data of the source motion node associated with the redirection marker is determined according to the marker generation method corresponding to the redirection marker. The node motion data includes position data (three-dimensional spatial coordinates), and the position data is directly used as the position component of the redirection marker.

[0077] For a direction-type redirection marker, the marker data for that redirection marker can be generated based on the position or rotation data of the source motion node associated with that redirection marker. This marker data includes a direction component.

[0078] For example, for a redirection marker of the direction type "left forearm direction," the node motion data of the source motion node associated with the redirection marker is determined based on the marker generation method corresponding to the redirection marker. This node motion data can be multiple position data (3D spatial coordinates), and the direction component in the marker data of the redirection marker can be determined based on these multiple position data. Alternatively, the node motion data can be rotation data (quaternion rotation data), and the direction component in the marker data of the redirection marker can be determined based on this rotation data.

[0079] The above four mapping processing methods can be selected to be executed in one or more ways according to the actual application requirements.

[0080] The motion redirection data processing method provided in this application, based on different marker types of redirection markers, processes the source motion node data using a marker generation method matching the marker type, generating standardized, type-matched marker point data. Pose-type redirection markers require both position and rotation components to fully describe their spatial state; orientation-type redirection markers only require rotation components to describe orientation; position-type redirection markers only require position components to describe position; and direction-type redirection markers describe spatial pointing based on direction components. This on-demand processing method reduces unnecessary data computation and transmission, achieving accuracy and efficiency in data processing, and flexibly adapting to various application scenarios.

[0081] In some embodiments, when the marker type of the redirection marker is a direction type, generating the direction component in the marker data of the redirection marker based on the position data or rotation data in the node motion data of the source motion node associated with the redirection marker may include: determining direction data from the first source motion node to the second source motion node based on the position data in the node motion data of the first source motion node and the position data in the node motion data of the second source motion node; wherein the first source motion node and the second source motion node are two source motion nodes on the limb segment of the operation object, and the first source motion node and the second source motion node are used to determine the direction of the limb segment; or, determining the direction data corresponding to the local coordinate axis of the source motion node in the target coordinate system based on the rotation data in the node motion data of the source motion node; and normalizing the direction data to obtain the direction component in the marker data of the redirection marker.

[0082] Specifically, for a direction-type redirection marker, its corresponding marker data contains only a normalized direction component. There are several methods for generating markers for direction-type redirection markers.

[0083] One method for generating redirection markers of a certain direction type involves calculating the positional difference between two source motion nodes, determining the direction data from the first source motion node to the second source motion node, and then normalizing this data to obtain the marker data, i.e., the direction component, for the redirection markers of that direction type. This generation method is suitable for limb segments with a defined skeletal structure, such as the upper arm (from the shoulder joint to the elbow joint), forearm (from the elbow joint to the wrist joint), thigh (from the hip joint to the knee joint), or lower leg (from the knee joint to the ankle joint).

[0084] Normalization refers to the process of dividing the calculated direction vector by its magnitude to make its length 1.

[0085] The first source motion node and the second source motion node are two source motion nodes on the limb segment of the object being manipulated. The first source motion node can be a joint of the limb segment close to the torso, and the second source motion node can be a joint of the limb segment far from the torso. For example, for calculating the direction of the upper arm, the first source motion node is the shoulder joint, and the second source motion node is the elbow joint. By calculating the direction vector from the first source motion node to the second source motion node, the direction data of the limb segment can be obtained.

[0086] For example, the source motion nodes include the left elbow (first source motion node) and the left wrist (second source motion node). Based on the mapping configuration information, the redirection markers associated with these two source motion nodes are determined to include the direction-type redirection marker "left forearm direction". The node motion data of these two source motion nodes, including the left elbow position data and the left wrist position data, are read from the target motion frame. Based on the left elbow and left wrist position data, a direction vector pointing from the left elbow to the left wrist can be calculated. Then, the magnitude of this direction vector is calculated to obtain the normalized direction data, which is the marker data (i.e., direction component) of the direction-type redirection marker.

[0087] Another type of redirection marker generation method uses the orientation data obtained by extracting the direction of the local coordinate axis of the source motion node in the target coordinate system. The target coordinate system can refer to the world coordinate system. This generation method is suitable for scenarios where only joint rotation data is provided, but position data is not provided, and the source motion node is a single source motion node used to extract orientation data, such as joints with a clear local coordinate axis orientation, such as wrist joints, ankle joints, and head joints. The local coordinate axis refers to the coordinate axis in the source motion node's own local coordinate system, usually one of the X-axis, Y-axis, or Z-axis. In this generation method, the local coordinate axis can be determined through mapping configuration information, and the orientation data of the local coordinate axis in the target coordinate system can be calculated using the rotation data in the node motion data of the source motion node.

[0088] For example, the source motion node includes the left shoulder joint. Based on the mapping configuration information, the retargeting marker associated with this source motion node is determined to be the direction-type retargeting marker "left upper arm direction". The node motion data of the left shoulder joint is read from the target motion frame, including the rotation data of the left shoulder joint [0, 0, 0.707, 0.707]. According to the mapping configuration information, the local coordinate axis is determined to be the X-axis, i.e., the unit vector [1, 0, 0]. The local X-axis unit vector of the left shoulder joint is transformed through its rotation data to obtain the direction of this local coordinate axis in the world coordinate system: [0, 0, 0.707, 0.707] × [1, 0, 0] = [0, -1, 0]. Since the extracted direction data itself is already a unit vector, the result after normalization remains unchanged. The extracted direction data is directly used as the marker data (i.e., direction component) of this direction-type retargeting marker.

[0089] The motion redirection-oriented data processing method provided in this application provides multiple ways to generate marker point data for redirection marker points of various directions. It can flexibly select based on the node motion data and mapping configuration information of the source motion node received in the actual application. While ensuring the accurate generation of marker point data for redirection marker points of various directions, it enhances the adaptability of the marker point data generation method.

[0090] In some embodiments, organizing the marker point data corresponding to the plurality of redirection marker points according to the data organization method indicated by the data organization information to generate intermediate representation frame data corresponding to the target motion frame may include: determining, based on the data organization information, the data fields corresponding to each data component in the marker point data corresponding to the plurality of redirection marker points; writing each data component in the marker point data corresponding to each of the plurality of redirection marker points into the corresponding data fields according to the declaration order of the plurality of redirection marker points in the intermediate representation description information; and generating the intermediate representation frame data based on the written data fields.

[0091] Specifically, according to the data organization method indicated by the data organization information, the data fields corresponding to each data component in different types of marker point data are determined, that is, the data fields that each data component should be written into. This data field refers to the structured storage area in the intermediate representation frame data used to store each data component in the marker point data. Data fields can be divided according to the type of marker point data, and each data field consists of a continuous sequence of values.

[0092] Then, for the data fields corresponding to the same type of marker point data, according to the declaration order of multiple redirected marker points in the intermediate representation description information, that is, the arrangement order of multiple redirected marker points recorded in the marker declaration information in the intermediate representation description information, each data component in the marker point data corresponding to each redirected marker point is written into the corresponding data field, thereby generating intermediate representation frame data.

[0093] For example, the intermediate description information sequentially declares the pose-type redirection marker "left hand," the orientation-type redirection marker "left forearm direction," the facing-type redirection marker "head facing," and the position-type redirection marker "robot center of gravity." After mapping processing, the marker data corresponding to these redirection markers is obtained, including the marker data for "left hand" (including position and rotation components), "left forearm direction" (including orientation components), "head facing" (including rotation components), and "robot center of gravity" (including position components). Based on the data organization information, the data fields corresponding to each data component in the marker data corresponding to the redirection markers are determined, including data fields storing position components, rotation components, and orientation components. Based on the order in which these redirection markers are declared in the intermediate representation description information—namely, the order of left hand, left forearm direction, head orientation, and robot center of gravity—each data component of the marker data corresponding to each redirection marker is written into its corresponding data field. Specifically, the position component of each redirection marker is sequentially written into the data field storing position components, the rotation component is sequentially written into the data field storing rotation components, and the orientation component is sequentially written into the data field storing orientation components. These three data fields are then packaged into a complete data frame structure to generate the final intermediate representation frame data.

[0094] The motion redirection-oriented data processing method provided in this application transforms the scattered data components of marker point data with different data structures and semantics into highly structured data fields containing only pure numerical values, according to predetermined data organization information. Since the division of data fields strictly follows the data organization information, and the writing order of data components in each data field strictly follows the declaration order of the redirection marker points in the intermediate representation description information, the motion redirection solving module can accurately parse the data components of the marker data corresponding to each redirection marker point from the frame data according to the declaration order without relying on any additional protocols or conventions, thus achieving efficient and accurate transmission of marker data.

[0095] In some embodiments, writing each data component of the marker data corresponding to each redirection marker point into the corresponding data field may include: when the marker type of the redirection marker point is a pose type, writing the position component of the marker data corresponding to the redirection marker point into a position data field and writing the rotation component of the marker data corresponding to the redirection marker point into a rotation data field; when the marker type of the redirection marker point is an orientation type, writing the rotation component of the marker data corresponding to the redirection marker point into a position data field; when the marker type of the redirection marker point is a direction type, writing the direction component of the marker data corresponding to the redirection marker point into a direction data field; when the marker type of the redirection marker point is a position type, writing the position component of the marker data corresponding to the redirection marker point into a position data field; wherein at least one of the position data field, the rotation data field, and the direction data field is stored using a flattened array.

[0096] Specifically, the data organization information indicates that the position component of the marker point data is written into the position data field, the rotation component into the rotation data field, and the orientation component into the orientation data field. Different data fields correspond to different identifiers, which can be strings or numbers, such as the identifier for the position data field being "position" or "1". During the parsing phase (i.e., when parsing the intermediate representation frame data based on the intermediate representation description information), the data organization information is used to enable the robot's motion relocation solution module to know what data component is stored in each data field of the intermediate representation frame data (e.g., the data field with the identifier 1 stores the position component); and the declaration order of each relocation marker point in the intermediate representation description information is used to indicate the relocation marker point corresponding to each data component in each data field (e.g., the position component in the data field with the identifier 1 corresponds to both the pose type relocation marker point and the position type relocation marker point).

[0097] Multiple redirection markers in the middle represent the redirection markers in the description information, assuming the declaration order is: pose type redirection markers, orientation type redirection markers, facing type redirection markers, and position type redirection markers. All redirection markers are traversed according to the declaration order, and the position components from the marker data of each redirection marker are continuously written into the position data field. That is, the position components from the marker data of pose type redirection markers and the position type redirection markers are sequentially and continuously written into the position data field. Similarly, all redirection markers are traversed according to the declaration order, and the rotation components from the marker data of each redirection marker are continuously written into the rotation data field. That is, the rotation components from the marker data of pose type redirection markers and the facing type redirection markers are sequentially and continuously written into the rotation data field. Finally, all redirection markers are traversed according to the declaration order, and the orientation components from the marker data of each redirection marker are continuously written into the position data field. That is, the orientation components from the marker data of orientation type redirection markers are written into the orientation data field.

[0098] In the data organization method indicated by the data organization information, the data fields are divided according to the type of marker point data, which can be divided into position data fields, rotation data fields and orientation data fields. At least one of these data fields is stored using a flat array.

[0099] In this context, a flattened array refers to a data structure consisting of a continuous sequence of numerical values ​​in a single dimension. In the embodiments of this application, data components of the same category (such as all position components, all rotation components, or all direction components) are stored continuously in a one-dimensional floating-point array according to the declaration order in the label declaration information. This one-dimensional floating-point array is the flattened array.

[0100] For example, based on data organization information, the data fields corresponding to different types of data components are determined, including position data fields, rotation data fields, and orientation data fields. According to the declaration order of the redirection markers in the description information (i.e., the declaration order of left hand, left forearm direction, head orientation, and robot center of gravity), the data components in the marker data corresponding to each redirection marker are written into the corresponding data fields. Specifically, this includes: first, sequentially writing the position components [-0.5, 0.8, 0.15] of the "left hand" redirection marker and [0.1, 0.8, 0.0] of the "robot center of gravity" into the position data field, at which point the content of the position data field is [-0.5, 0.8, 0.15, 0.1, 0.8, 0.0]; then sequentially writing the redirection markers... The rotation components [0.0, 0.0, 0.0, 1.0] for the "left hand" marker and [0.0, 0.707, 0.0, 0.707] for the "head orientation" marker are written into the rotation data field, resulting in the current content of the rotation data field: [0.0, 0.0, 0.0, 1.0, 0.0, 0.707, 0.0, 0.707]. The direction components [-0.696, -0.696, 0.174] for the "left forearm direction" marker are then written into the direction data field, resulting in the current content of the direction data field: [-0.696, -0.696, 0.174]. These three data fields are then packaged into a complete data frame structure to generate the final intermediate representation frame data.

[0101] The motion redirection-oriented data processing method provided in this application uses data field partitioning and flattened array storage to store the three types of data components—position, rotation, and orientation—in separate, continuous arrays. This allows the motion redirection solving module to quickly locate and extract each data component from the marker data of each redirection marker point based on the data organization and marker declaration information in the intermediate representation description information. Furthermore, the use of flattened array storage in this application saves bandwidth, significantly reduces network load and serialization overhead, and improves data processing efficiency.

[0102] In some embodiments, the marker declaration information may include the marker name and marker type corresponding to the plurality of redirection markers; the intermediate representation frame data may include at least one data component among position component, rotation component and orientation component.

[0103] Specifically, the intermediate description information may include tag declaration information corresponding to multiple redirection tag points. This tag declaration information may include the tag point names and tag types corresponding to the multiple redirection tag points. Among them, the tag point name uniquely identifies a redirection tag point.

[0104] The intermediate frame data contains data components but does not include descriptive information related to the redirection markers, such as marker name and marker type.

[0105] The data component refers to the data component that changes over time during the motion process, and may include at least one of the position component, rotation component, and orientation component.

[0106] The motion redirection-oriented data processing method provided in this application decouples intermediate representation description information from intermediate representation frame data in terms of content. Static metadata such as marker names and marker types exist only in the intermediate representation description information, while the intermediate representation frame data contains only pure numerical data components. This makes the intermediate representation frame data lightweight and significantly reduces metadata redundancy in the transmission of intermediate representation frame data. Furthermore, since all the metadata information required for parsing is included in the intermediate representation description information, the motion redirection solving module can independently and accurately complete the parsing of the intermediate representation frame data without relying on any external information.

[0107] In some embodiments, determining intermediate representation description information based on the mapping configuration information may further include: if the mapping configuration information also includes a robot model identifier, writing the robot model identifier into the intermediate representation description information; or, if the mapping configuration information also includes a coordinate system convention after the source motion data is converted, writing the coordinate system convention after the source motion data is converted into the intermediate representation description information; or, if the mapping configuration information also includes a description information version number, writing the description information version number into the intermediate representation description information; wherein, the robot model identifier is used for the motion redirection solving module to determine the kinematic model corresponding to the robot, the coordinate system convention is used for the motion redirection solving module to parse the intermediate representation frame data according to the corresponding coordinate system, and the description information version number is used to perform consistency verification between the intermediate representation frame data and the intermediate representation description information.

[0108] Specifically, the mapping configuration information may also include at least one of the following: the robot model identifier corresponding to the robot, the coordinate system convention after the source motion data is transformed, and the description information version number. At least one of these pieces of information can be added to the intermediate representation description information along with the tag declaration information and data organization information.

[0109] If the mapping configuration information also includes a robot model identifier, this identifier can be written into the intermediate representation description information. The robot model identifier is a unique identifier for the robot model, used by the motion relocation solution module to determine the corresponding kinematic model of the robot. The robot model identifier can be a robot model name (in string form), such as "A3" representing an A3 series humanoid robot, and "Biped_General" representing a general-purpose bipedal robot; the robot model identifier can also be a predefined identifier value, with each identifier value corresponding to a robot model, such as identifier value "1" representing an A3 series humanoid robot, and "2" representing a general-purpose bipedal robot.

[0110] In this embodiment, the intermediate representation description information carrying the robot model identifier is sent to the robot's motion redirection solving module. The motion redirection solving module parses the intermediate representation description information to obtain the robot model identifier, such as "A3". The motion redirection solving module searches for and loads the kinematic model of the A3 series humanoid robot in the internal model library, including parameters such as the number of joints, joint type, link length, and joint limit. These parameters are used to map the marker point data into specific joint angle control commands in the future.

[0111] When changing the robot being used, only the robot model identifier in the mapping configuration information needs to be modified, such as changing it from "A3" to "Biped_General". After receiving the robot model identifier from the intermediate representation description information, the motion relocation solver module searches for and loads the kinematic model named "Biped_General" from its internal model library, thus adapting it to a general-purpose bipedal robot without modifying the data mapping module code. When supporting a new robot model, only the robot model identifier of that robot needs to be added to the mapping configuration information, and a kinematic model of that robot needs to be added to the model library of the motion relocation solver module. This achieves rapid adaptation and flexible expansion between multiple robot models.

[0112] If the mapping configuration information also includes the coordinate system convention after the source motion data is converted, this coordinate system convention can be written into the intermediate representation description information. The coordinate system convention refers to the convention information regarding the coordinate system type, coordinate axis orientation, and positive direction followed by each data component in the intermediate representation frame data. This can be used by the motion relocation solution module to parse the intermediate representation frame data according to the corresponding coordinate system. For example, the coordinate system convention may include at least one of the following: right-handed Z-Up, right-handed Y-Up, left-handed Z-Up, and left-handed Y-Up. The declaration of the coordinate system convention ensures that the source motion data output by different motion capture devices can be uniformly converted to the agreed target coordinate system during the mapping stage. This ensures that the data mapping module and the robot's motion relocation solution module understand the data in the same coordinate system, reducing motion distortion caused by coordinate system mismatch.

[0113] For example, optical motion capture systems default to a right-handed Y-axis-up coordinate system. During the mapping phase, based on the mapping configuration information, the coordinate system convention after the source motion data transformation is extracted, which is a right-handed Z-axis-up coordinate system. The data mapping module converts all source motion data from the Y-axis-up coordinate system to the Z-axis-up coordinate system, obtaining intermediate representation frame data. This coordinate system convention (right-handed Z-axis-up coordinate system) is carried in the intermediate representation description information, which is then sent to the motion redirection solving module. Upon receiving this information, the motion redirection solving module parses all intermediate representation frame data in the right-handed Z-axis-up coordinate system according to the coordinate system convention in the intermediate representation description information.

[0114] For example, inertial motion capture systems typically use a right-handed Z-axis-up coordinate system. During the mapping phase, the coordinate system convention in the mapping configuration information is a right-handed Z-axis-up coordinate system, eliminating the need for coordinate system transformation in the data mapping module. The intermediate representation description information carries this coordinate system convention (right-handed Z-axis-up coordinate system) and sends the intermediate representation description information and intermediate representation frame data to the motion redirection solution module. Upon receiving this information, the motion redirection solution module parses all intermediate representation frame data in the right-handed Z-axis-up coordinate system according to the coordinate system convention in the intermediate representation description information.

[0115] If the mapping configuration information also includes a description information version number, then this description information version number can be written into the intermediate representation description information. The description information version number refers to a version identifier that monotonically increases with the number of times the content of the intermediate representation description information changes, for example, an incrementing version number of an unsigned integer type (uint32).

[0116] In some embodiments, the description information version number can be used to implement consistency verification between intermediate representation frame data and intermediate representation description information.

[0117] Specifically, when an intermediate representation frame is generated, a version number corresponding to the intermediate representation description information is added to the intermediate representation frame data, and the intermediate representation frame data is sent to the motion relocation solving module. After receiving the intermediate representation frame data, the motion relocation solving module reads the version number of the description information within it and compares this version number in the intermediate representation frame data with the version number of the description information in the intermediate representation description information. If the two version numbers match, the consistency check passes, and the motion relocation solving module proceeds normally, parsing the intermediate representation frame data based on the intermediate representation description information and extracting the data components of each relocation marker point for subsequent motion relocation calculations. If the two version numbers do not match, the consistency check fails.

[0118] If the consistency check fails, the motion redirection solution module can discard the current intermediate representation frame data and send a request to the data mapping module to retransmit the intermediate representation description information; alternatively, it can wait for the data mapping module to provide updated intermediate representation description information until the description information version number in the intermediate representation description information matches the description information version number carried in the current intermediate representation frame data before resuming parsing and redirection calculation. By checking the consistency of the description information version number, the consistency between the intermediate representation frame data and the intermediate representation description information is ensured, reducing data parsing errors caused by data mismatches.

[0119] The motion retargeting data processing method provided in this application embodiment may further include at least one of robot model identifier, coordinate system convention, and description information version number in the intermediate representation description information. This information enables the motion retargeting solution module to automatically load the appropriate kinematic model based on the robot model identifier, achieving rapid adaptation and flexible expansion among multiple robot models; it can also parse the intermediate representation frame data in a unified spatial reference system according to the coordinate system convention, reducing motion distortion caused by coordinate system mismatch; and it can also perceive and adapt to changes in the intermediate representation description information through the description information version number, reducing data misalignment parsing caused by configuration changes.

[0120] In some embodiments, providing the intermediate representation description information and the intermediate representation frame data to the robot's motion relocation solution module may further include: providing the intermediate representation description information to the motion relocation solution module when a data transmission channel is established or the mapping configuration information changes; and after the intermediate representation description information is provided, providing the intermediate representation frame data that can be parsed by the intermediate representation description information to the motion relocation solution module in sequence.

[0121] Specifically, the transmission timing or frequency of intermediate representation description information and intermediate representation frame data are separate. The transmission of intermediate representation description information is triggered when the data mapping module and the motion redirection solution module establish a data transmission channel for the first time, or when the mapping configuration information changes.

[0122] A data transmission channel refers to the logical or physical data communication link established between the data mapping module and the motion relocation solution module. A data transmission channel can include at least one of the following: a network connection channel, an inter-process communication pipe, or an API.

[0123] A change in mapping configuration information refers to any event in which any part of the mapping configuration information changes, such as adding, deleting, or modifying marker generation rules, switching robot model identifiers, or adjusting coordinate system conventions. This event triggers the data mapping module to regenerate intermediate representation description information and re-provide it to the motion relocation solution module to ensure that the intermediate representation description information is consistent with the version of the intermediate representation frame data.

[0124] Once the intermediate representation description information is provided, meaning the intermediate representation description information has been completely transmitted to the motion redirection solving module, the data mapping module triggers the transmission of intermediate representation frame data once it generates one or more frames of intermediate representation frame data that can be parsed by the intermediate representation description information. The intermediate representation frame data corresponding to each of the multiple motion frames are then provided to the motion redirection solving module in chronological order.

[0125] In this embodiment, the intermediate representation description information can be used as low-frequency changing static parsing information (such as marker point declaration information, data organization information, robot model identification, coordinate system convention, and description information version number). Since this information usually does not change with the update of motion frames, there is no need to repeatedly carry the same content (i.e., intermediate representation description information) in each intermediate representation frame data.

[0126] For example, when the data mapping module and the motion relocation solving module initially establish a data transmission channel, the data mapping module immediately provides the intermediate representation description information generated based on the mapping configuration information to the motion relocation solving module. Upon receiving the intermediate representation description information, the motion relocation solving module parses it to obtain information such as the tag declaration information and the description information version number. Subsequently, the data mapping module begins receiving source motion data and generates the first frame of intermediate representation data, providing it to the motion relocation solving module. Then, for the second frame, third frame, and so on, the data mapping module sequentially generates and provides them to the motion relocation solving module.

[0127] For example, after data processing and parsing have been in progress for a period of time, suppose the object being operated switches to teleoperation mode via the control interface. The mapping configuration information is thus updated. Upon detecting this change, the data mapping module regenerates intermediate representation description information (version number updated to 2.0) based on the updated information and provides this updated intermediate representation description information to the motion redirection solution module. After the updated intermediate representation description information is provided, multiple subsequently generated intermediate representation frame data that can be parsed using the updated information can be sent and parsed sequentially.

[0128] The motion redirection-oriented data processing method provided in this application limits the transmission of intermediate representation description information to events such as data transmission channel initialization or mapping configuration information update, and transmits intermediate representation frame data at high frequency. This reduces the repeated transmission of intermediate representation description information in continuous motion frames, reduces the data volume and bandwidth usage of a single frame, and improves the efficiency of the motion redirection solution module in receiving and parsing intermediate representation frame data.

[0129] In some embodiments, before mapping the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame, the motion-oriented data processing method may further include: obtaining the measured size information of the operation object under a preset calibration posture; determining runtime normalization parameters based on the target size information corresponding to the robot and the measured size information; and scaling the position data in the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame based on the runtime normalization parameters.

[0130] Specifically, before performing mapping processing on the source motion nodes in the target motion frame, runtime normalization processing can be performed on the size differences between the manipulated object and the robot. This runtime normalization processing can be completed during the calibration phase or before the mapping processing, or it can be re-executed after the manipulated object is switched, the robot is switched, or the calibration information is updated.

[0131] During runtime normalization, the measured dimensions of the object being operated on are first obtained under the preset calibration posture.

[0132] The preset calibration posture is used to ensure the body dimensions of the manipulated object are in a stable state that facilitates measurement and comparison. For example, the preset calibration posture can be a T-Pose, a natural standing posture, or other postures that can stably estimate dimensions such as height, shoulder width, arm length, and leg length. In the preset calibration posture, the joints of the manipulated object are in fixed relative positions, enabling stable measurement of key dimensions such as height, arm length, and leg length, avoiding measurement errors caused by posture changes. The data mapping module can acquire the measured dimensions of the manipulated object in this preset calibration posture through motion capture equipment, a skeleton tracking module, a depth measurement device, or calibration input information. The measured dimensions reflect the actual body dimensions of the current manipulated object, thereby reducing the problem of excessive or insufficient translation amplitude when different manipulated objects share the same mapping configuration.

[0133] Then, based on the target size information and measured size information of the robot, the runtime normalization parameters are determined.

[0134] The target size information for the robot can be used to characterize the robot's equivalent body dimensions during motion retargeting. Target size information may include the robot's equivalent height, or dimensions such as shoulder width, arm length, leg length, or torso length related to specific mapped parts.

[0135] Runtime normalization parameters can be obtained by comparing the target size information with the measured size information of the object being manipulated. When using overall height normalization, the runtime normalization parameters can be determined by the ratio between the robot's equivalent height and the measured height of the object being manipulated.

[0136] Runtime normalization parameters primarily affect the position data of the node motion data corresponding to the source motion node in the target motion frame, adjusting the scale of this position data. Specifically, the translation component of the source motion node in the world coordinate system, human coordinate system, or acquisition coordinate system can be multiplied by the corresponding runtime normalization parameter to obtain scaled position data. If the node motion data also includes rotational components, these components remain unchanged, ensuring that the action posture, joint orientation, and relative rotation relationships are not altered by size normalization.

[0137] For example, if the measured height of the object being manipulated is 1.7m and the equivalent height of the robot is 1.8m, then the runtime normalization parameter is 1.8 / 1.7≈1.0588, indicating that the joint position data of the object being manipulated needs to be scaled by approximately 5.88% to match the size of the robot. For each target motion frame, before performing mapping processing, the position data of the source motion nodes is first scaled. For example, after scaling the original position data of the left wrist [-0.5, 0.8, 0.15], the scaled position data [-0.5294, 0.8470, 0.1588] is obtained.

[0138] By normalizing parameters at runtime, objects of different heights can achieve consistent motion mapping effects.

[0139] For example, for operators of different heights, such as operator A (1.7 meters tall) and operator B (1.9 meters tall), with the robot's equivalent height being 1.8 meters, the runtime normalization parameter for operator A would be 1.8 / 1.7 ≈ 1.0588, meaning that operator A's joint position data needs to be enlarged by approximately 5.88% to match the robot's size. The runtime normalization parameter for operator B would be 1.8 / 1.9 ≈ 0.9474, meaning that operator B's joint position data needs to be reduced by approximately 5.26% to match the robot's size. Through runtime normalization parameters, when operators of different heights operate the same robot, the robot's range of motion can automatically adapt to the operator's body size. The movements of the shorter operator A are amplified, while the movements of the taller operator B are reduced, ultimately resulting in the robot exhibiting a movement effect consistent with the operator's actual movements.

[0140] The motion retargeting data processing method provided in this application can dynamically adjust the scaling ratio of position data according to the actual size information of the object being manipulated, so that the robot's motion amplitude is consistent with the action of the object being manipulated, reducing the impact of individual differences in the size of the object being manipulated on the motion retargeting result. Moreover, the scaling process only affects the position data, while the rotation data remains unchanged, ensuring the accurate transmission of motion posture.

[0141] In some embodiments, scaling the position data in the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame based on the runtime normalization parameters may include: obtaining basic scaling parameters, wherein the basic scaling parameters are initial calibration parameters or identity matrices; determining composite scaling parameters based on the basic scaling parameters and the runtime normalization parameters; scaling the position data in the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame based on the composite scaling parameters, while maintaining the rotation data in the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame.

[0142] Specifically, a base scaling parameter can be introduced during runtime normalization and then combined with runtime normalization parameters. The base scaling parameter can be an initial calibration parameter or an identity matrix; the runtime normalization parameter is used to express the dynamic dimensional ratio between the current object being manipulated and the robot.

[0143] The base scaling parameter can be an initial calibration parameter. For example, it can be obtained by placing the manipulated object and the robot in the same preset calibration pose (such as a T-Pose) and measuring the distance ratio of key joints. If the manipulated object's arm length is 0.6m and the robot's corresponding arm length is 0.55m, then the initial calibration parameter is 0.55 / 0.6≈0.9167. This initial calibration parameter serves as the base scaling parameter. Alternatively, the base scaling parameter can be an identity matrix. The composite scaling parameter obtained based on the base scaling parameter and the runtime normalization parameter is the runtime normalization parameter.

[0144] The basic scaling parameter can be a uniform scalar or multiple scaling factors corresponding to motion nodes in different directions, different body parts or different sources. This application does not limit this.

[0145] Specifically, the base scaling parameter and the runtime normalization parameter are combined to obtain the composite scaling parameter. When both are scalars, the composite scaling parameter can be obtained by multiplying the base scaling parameter and the runtime normalization parameter; when both contain multiple components, they can be combined item by item according to direction components, node groups, or part groups. After obtaining the composite scaling parameter, it can be applied to the position data of at least a portion of the source motion nodes in the target motion frame.

[0146] For example, during the calibration phase, both the object and the robot are in a T-Pose posture. The measured length of the object's left forearm (distance from the elbow to the wrist) is 0.3m, while the robot's corresponding left forearm length is 0.28m. The initial calibration parameter is calculated as 0.28 / 0.30≈0.9333, indicating that the object's forearm length needs to be reduced by approximately 6.67% to match the robot's forearm size. The object's height is 1.7m, and the robot's equivalent height is 1.8m. The calculated runtime normalization parameter is 1.8 / 1.7≈1.0588. Multiplying the basic scaling parameter by the runtime normalization parameter yields a composite scaling parameter of 0.9333×1.0588≈0.9882, indicating that the object's position data needs to be scaled to approximately 98.82% of its original value. During the mapping process, the position data of the source motion nodes is multiplied by 0.9882 for scaling, while the rotation data remains unchanged.

[0147] For example, suppose the robot has a long torso and short legs, while the object being manipulated has the opposite proportions. Therefore, multiple sub-parameters are configured for the base scaling parameters, each applied to different body parts. The base scaling parameter for the upper body is 1.08 to match the longer robot torso, and the base scaling parameter for the lower body is 0.95 to match the shorter robot legs. The object's height is 1.8m, and the robot's equivalent height is 1.7m. The calculated runtime normalized parameter is 1.7 / 1.8 ≈ 0.9444. Multiplying the base scaling parameter by the runtime normalized parameter yields the upper body composite scaling parameter as 1.08 × 0.9444 ≈ 1.020, and the lower body composite scaling parameter as 0.95 × 0.9444 ≈ 0.897. During the mapping process, different composite scaling parameters are applied according to the body parts to which the motion nodes from different sources belong. For motion nodes from the upper body (such as shoulder joint, elbow joint, wrist joint, head, etc.), their position data is multiplied by 1.020, and for motion nodes from the lower body (such as hip joint, knee joint, ankle joint, etc.), their position data is multiplied by 0.897, while the rotation data remains unchanged.

[0148] In this embodiment, the rotation data in the node motion data of at least a portion of the source motion nodes in the target motion frame is not scaled by the composite scaling parameter. Directly applying the composite scaling parameter to the rotation data could potentially disrupt the attitude orientation, joint relative rotation relationships, or subsequent motion solution constraints. Therefore, scaling is performed only on the position data, while preserving the original attitude representation of the corresponding rotation data.

[0149] The motion retargeting data processing method provided in this application combines basic scaling parameters with runtime normalization parameters. It can handle both static scaling requirements that are predetermined and do not change with the operation object, and dynamic scaling requirements that change with the operation object in real time. It can ensure the accurate transmission of motion posture while adapting to operation objects and robots of different sizes.

[0150] In some embodiments, the motion-oriented data processing method may further include: If the offset between the position component in the marker data of the redirected marker and the reference position component under the preset calibration attitude exceeds a preset position offset threshold, obtain the position offset parameter corresponding to the redirected marker and inject the position offset parameter into the marker generation method corresponding to the redirected marker; and / or, if the offset between the rotation component in the marker data of the redirected marker and the reference rotation component under the preset calibration attitude exceeds a preset rotation offset threshold, obtain the rotation offset parameter corresponding to the redirected marker and inject the rotation offset parameter into the marker generation method corresponding to the redirected marker.

[0151] Specifically, during initial calibration, reference position components and / or reference rotation components of multiple repositioning markers are determined based on the source motion data of the manipulated object in a preset calibration posture. For example, if the manipulated object maintains a T-Pose posture, the data mapping module receives 100 frames of node motion data in the T-Pose posture, including the position and rotation components corresponding to the left hand of the repositioning marker. By taking the median or mean of these 100 position components and the median or mean of these 100 rotation components, the reference position and reference rotation components of the repositioning marker in the preset calibration posture can be obtained.

[0152] In some embodiments, after obtaining the marker data of the redirected markers through mapping processing, the offset between the position component in the marker data and the reference position component under the preset calibration attitude is calculated, and the offset between the position component and the reference position component is compared with a preset position offset threshold. If the offset between the position component of the redirected marker and the reference position component does not exceed the preset position offset threshold, the acquisition and injection of the position offset parameter will not be triggered, and the marker generation method corresponding to the redirected marker will remain unchanged.

[0153] If the offset between the position component of the redirected marker and the reference position component exceeds the preset position offset threshold (which may be due to a slight change in posture caused by the fatigue of the object being operated on, or a slow drift of the motion capture device), then the position offset parameter corresponding to the redirected marker is obtained and injected into the marker generation method corresponding to the redirected marker.

[0154] For example, after obtaining the marker data for the "left hand" of the redirected marker through mapping processing, the reference position component of the pre-acquired "left hand" of the redirected marker is read from memory. The deviation between the position component in the marker data of the redirected marker and the reference position component is calculated, and this deviation is converted into Euclidean distance as the offset. This offset is compared with a preset position offset threshold. If the offset exceeds the preset position offset threshold, calibration is triggered: First, the position offset parameter corresponding to the redirected marker is obtained, and then the position offset parameter is injected into the marker generation method corresponding to the redirected marker. After injection, the marker generation method of the redirected marker is updated to: position data in the node motion data of the source motion node associated with the redirected marker × scaling factor (i.e., runtime normalization parameter or composite scaling parameter) + position offset parameter corresponding to the redirected marker.

[0155] In some embodiments, after obtaining the marker data of the redirected markers through mapping processing, the offset between the rotation component in the marker data and the reference rotation component under the preset calibration attitude is calculated, and the offset between the reference rotation component and the reference rotation component is compared with a preset rotation offset threshold. If the offset between the rotation component of the redirected marker and the reference rotation component does not exceed the preset rotation offset threshold, the acquisition and injection of the rotation offset parameter will not be triggered, and the marker generation method corresponding to the redirected marker will remain unchanged.

[0156] If the offset between the rotation component of the redirected marker and the reference rotation component exceeds a preset rotation offset threshold, the rotation offset parameter corresponding to the redirected marker is obtained and injected into the marker generation method corresponding to the redirected marker.

[0157] For example, after obtaining the "head orientation" marker data of the redirected marker point through mapping processing, the reference rotation component of the pre-acquired "head orientation" of the redirected marker point is read from memory. The rotational offset angle between the rotation component in the marker point data of the redirected marker point and the reference rotation component is calculated, which is the offset. This offset is compared with a preset rotational offset threshold. If the offset exceeds the preset rotational offset threshold, calibration is triggered: First, the rotational offset parameter corresponding to the redirected marker point is obtained. Then, the rotational offset parameter is injected into the marker generation method corresponding to the redirected marker point. After injection, the marker generation method of the redirected marker point is updated to: rotational data in the node motion data of the source motion node associated with the redirected marker point + rotational offset parameter corresponding to the redirected marker point.

[0158] For example, after obtaining the marker data for the "left foot" redirection marker through mapping processing, the pre-acquired reference position component and reference rotation component of the "left foot" redirection marker are read from memory. The deviation between the position component and the reference position component in the marker data of the redirection marker, and the rotational offset angle between the rotation component and the reference rotation component are calculated, yielding the position offset and rotational offset, respectively. The position offset is compared with a preset position offset threshold for the rotation component of the redirection marker. If the position offset exceeds the preset position offset threshold, the rotational offset is compared with a preset rotational offset threshold. Calibration is triggered. The calibration process includes: obtaining the left foot position offset parameter, adding the left foot position offset parameter to the position component in the marker data of the redirection marker to correct the position component; and obtaining the left foot local rotational offset parameter, adding the left foot local rotational offset parameter to the rotation component in the marker data of the redirection marker to correct the rotation component. The correction process for the position and rotation components in the marker data of the "right foot" redirection marker is the same. After updating the marker generation method corresponding to the feet, it is usually necessary to update the marker generation method corresponding to the redirection marker point "robot center of gravity" based on the left foot local position offset parameters and the right foot position offset parameters. The average value of the left foot local position offset parameters and the right foot position offset parameters can be used as the calibration parameters of the redirection marker point "robot center of gravity" and injected into its corresponding marker generation method to achieve trunk or limb height compensation, thereby correcting the position of the robot's center of gravity.

[0159] In some embodiments, the position offset parameters and rotation offset parameters can be obtained externally, such as by a separate calibration system, a user, or an upper-level controller.

[0160] In some embodiments, only the offset monitoring and offset calibration of the redirected markers that allow the injection of calibration parameters (such as reference position components or reference rotation components) are required. When determining the reference position components and / or reference rotation components corresponding to multiple redirected markers, only the reference position components and / or reference rotation components corresponding to the redirected markers that allow the injection of calibration parameters are determined. The mapping configuration information can record the injection permission and injection position of calibration parameters for multiple redirected markers respectively. The injection permission can be used to indicate whether the marker generation method corresponding to a certain redirected marker allows receiving calibration parameters. For redirected markers that do not declare that injection is allowed, their original marker generation method can be kept unchanged, thereby reducing the misapplication of calibration parameters to irrelevant markers. For example, in the mapping configuration information, pose-type redirected markers such as "left foot" and "right foot" are configured to allow the injection of calibration parameters, while pose-type redirected markers such as "left hand" and "right hand" are configured to disallow the injection of calibration parameters.

[0161] The motion redirection-oriented data processing method provided in this application automatically triggers calibration parameter injection when a significant offset is detected in the position or rotation component of the redirection marker data relative to the reference value under the preset calibration posture. Subsequent mapping processing is then performed based on the marker generation method of the injected calibration parameters, thereby improving the motion mapping accuracy.

[0162] In some embodiments, while the data transmission channel remains operational, updated calibration parameters can be received; the marker generation method corresponding to the redirection marker point can be updated based on the updated calibration parameters; wherein, the intermediate representation frame data generated after updating the marker generation method is obtained based on the updated calibration parameters.

[0163] Specifically, updated calibration parameters can be received while the data transmission channel is continuously running, and these parameters can be written into the marker generation method corresponding to the redirection marker point. The updated calibration parameters can come from recalibration, manual parameter adjustment, or an external calibration module, and their arrival does not require pausing source data acquisition or recreating the data transmission channel.

[0164] Upon receiving the updated calibration parameters, the system can locate the currently used marker generation method based on the target marker name or marker type carried by the calibration parameters, and update the calibration parameters in that marker generation method. The motion relocation solution module can still parse the intermediate representation frame data according to the loaded intermediate representation description information, without needing to re-adapt the interface due to changes in calibration parameters.

[0165] After updating the calibration parameters, when generating intermediate representation frame data frame by frame, the original marker generation process can continue to read the source motion data, and the updated calibration parameters can be introduced when generating the position or rotation components of the redirected marker points.

[0166] The motion redirection-oriented data processing method provided in this application receives updated calibration parameters while the data transmission channel remains operational, and updates the marker generation method corresponding to the redirection marker point based on the updated calibration parameters. This allows for hot updating of calibration parameters while maintaining continuous operation of the data transmission channel, enabling the motion redirection system to continuously absorb new calibration results during real-time teleoperation or online calibration, and ensuring that the subsequent output intermediate representation frame data reflects the latest compensation state.

[0167] In some embodiments, providing the intermediate representation description information and the intermediate representation frame data to the motion redirection solving module may include: providing the intermediate representation description information to the motion redirection solving module so that the motion redirection solving module establishes a parsing relationship corresponding to the plurality of redirection markers based on the intermediate representation description information; and providing the intermediate representation frame data to the motion redirection solving module so that the motion redirection solving module extracts marker point data corresponding to the plurality of redirection markers from the intermediate representation frame data based on the parsing relationship.

[0168] Specifically, during the connection establishment or initialization phase, intermediate representation description information can be sent to the motion redirection solving module, enabling the module to determine the parsing basis for subsequent frame data based on this information. During the frame-by-frame execution phase, continuously generated intermediate representation frame data is then sent to the motion redirection solving module, allowing it to read the corresponding dynamic data according to the determined parsing basis.

[0169] After reading the intermediate representation description information, the motion redirection solution module can obtain the marker name, marker type, arrangement order, data organization information, and coordinate system conventions of each redirection marker point, and establish the analytical relationship between the redirection marker points and the data components in each data field of the intermediate representation frame data.

[0170] The motion redirection-oriented data processing method provided in this application embodiment eliminates the need for the motion redirection solving module to parse the bone names, node levels, or original data structures of the source motion capture device during frame-by-frame solving. Instead, it parses unified intermediate representation description information and intermediate representation frame data. This reduces the amount of data in the intermediate representation frame data while ensuring that each frame of intermediate representation frame data can still be correctly interpreted, making it easier for the motion redirection solving module to stably obtain input data corresponding to multiple redirection markers.

[0171] Figure 2 This is a schematic diagram of the structure of a motion redirection-oriented data processing system provided in an embodiment of this application, as shown below. Figure 2 As shown in the figure, this application provides a data processing system for motion retargeting, which may include a motion capture data interface 201, a data mapping module 202, and a motion retargeting solution module 203.

[0172] The motion capture data interface 201 can be used to receive source motion data from different motion capture devices and send the source motion data to the data mapping module. The motion redirection solution module 203 can be used to receive intermediate representation description information and intermediate representation frame data provided by the data mapping module, and parse the intermediate representation frame data based on the intermediate representation description information to perform motion redirection calculation; wherein, the intermediate representation description information includes marker declaration information, and the intermediate representation frame data includes marker point data corresponding to the marker declaration information.

[0173] The data mapping module 202 can be used to execute a motion-oriented data processing method, which may include: receiving source motion data and mapping configuration information, wherein the source motion data includes at least one motion frame, each motion frame includes node motion data corresponding to multiple source motion nodes, and the mapping configuration information is used to characterize the association between each of the multiple redirection markers and one or more of the source motion nodes, as well as the marker generation method corresponding to each redirection marker; determining intermediate representation description information based on the mapping configuration information, wherein the intermediate representation description information includes marker declaration information and data organization information corresponding to the multiple redirection markers; and for the at least one motion frame... Based on the mapping configuration information, the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame are mapped to obtain the marker point data corresponding to the redirection marker points associated with at least a portion of the source motion nodes in the target motion frame. The marker point data corresponding to the multiple redirection marker points are organized according to the data organization method indicated by the data organization information to generate intermediate representation frame data corresponding to the target motion frame. The intermediate representation description information and the intermediate representation frame data are respectively provided to the robot's motion redirection solving module, so that the motion redirection solving module parses the intermediate representation frame data based on the intermediate representation description information.

[0174] Specifically, the motion-oriented data processing system provided in this application embodiment can implement all the method steps implemented in the motion-oriented data processing method embodiment and achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0175] It should be noted that the division of units / modules in the above embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0176] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 3 As shown, the electronic device may include: a processor 301, a communication interface 302, a memory 303, and a communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304. The processor 301 can call logical instructions in the memory 303 to execute a motion-oriented data processing method. This method may include: receiving source motion data and mapping configuration information, wherein the source motion data includes at least one motion frame, each motion frame includes node motion data corresponding to multiple source motion nodes, and the mapping configuration information is used to characterize the association between multiple redirection markers and one or more source motion nodes, as well as the marker generation method corresponding to each redirection marker; determining intermediate representation description information based on the mapping configuration information, wherein the intermediate representation description information includes marker declaration information and data organization information corresponding to the multiple redirection markers; and for the at least... In a target motion frame within a motion frame, based on the mapping configuration information, the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame are mapped to obtain marker point data corresponding to the redirection marker points associated with at least a portion of the source motion nodes in the target motion frame; the marker point data corresponding to the plurality of redirection marker points are organized according to the data organization method indicated by the data organization information to generate intermediate representation frame data corresponding to the target motion frame; the intermediate representation description information and the intermediate representation frame data are respectively provided to the robot's motion redirection solving module, so that the motion redirection solving module parses the intermediate representation frame data based on the intermediate representation description information.

[0177] Specifically, the processor 301 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD). The processor may also adopt a multi-core architecture.

[0178] When the logical instructions in memory 303 can be implemented as software functional units and sold or used as independent products, they can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0179] In some embodiments, a computer program product is also provided, the computer program product including a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the motion-oriented data processing method provided in the above-described method embodiments. The method may include: receiving source motion data and mapping configuration information, the source motion data including at least one motion frame, each motion frame including node motion data corresponding to multiple source motion nodes, the mapping configuration information being used to characterize the association relationship between multiple redirection markers and one or more of the source motion nodes and the marker generation method corresponding to each redirection marker; determining intermediate representation description information based on the mapping configuration information, the intermediate representation description information being packaged... The method includes marker declaration information and data organization information corresponding to the plurality of redirection markers; for a target motion frame in the at least one motion frame, based on the mapping configuration information, the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame are mapped to obtain marker point data corresponding to the redirection markers associated with at least a portion of the source motion nodes in the target motion frame; the marker point data corresponding to the plurality of redirection markers are organized according to the data organization method indicated by the data organization information to generate intermediate representation frame data corresponding to the target motion frame; the intermediate representation description information and the intermediate representation frame data are respectively provided to the robot's motion redirection solving module so that the motion redirection solving module parses the intermediate representation frame data based on the intermediate representation description information.

[0180] Specifically, the computer program product provided in this application embodiment can implement all the method steps implemented in the above method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.

[0181] In some embodiments, a computer-readable storage medium is also provided, the computer-readable storage medium storing a computer program, the computer program being configured to cause a computer to execute the motion-oriented data processing method provided in the above method embodiments, the method comprising: receiving source motion data and mapping configuration information, the source motion data including at least one motion frame, each motion frame including node motion data corresponding to multiple source motion nodes, the mapping configuration information being used to characterize the association relationship between each of the multiple redirection markers and one or more of the source motion nodes and the marker generation method corresponding to each of the redirection markers; and determining intermediate representation description information based on the mapping configuration information, the intermediate representation description information including the node motion data corresponding to the multiple redirection markers. The system includes: label declaration information and data organization information; for a target motion frame in at least one motion frame, based on the mapping configuration information, mapping processing is performed on the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame to obtain the label point data corresponding to the redirection label points associated with at least a portion of the source motion nodes in the target motion frame; the label point data corresponding to the plurality of redirection label points is organized according to the data organization method indicated by the data organization information to generate intermediate representation frame data corresponding to the target motion frame; the intermediate representation description information and the intermediate representation frame data are respectively provided to the robot's motion redirection solving module so that the motion redirection solving module parses the intermediate representation frame data based on the intermediate representation description information.

[0182] Specifically, the computer-readable storage medium provided in the embodiments of this application can implement all the method steps implemented in the above method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.

[0183] It should be noted that the computer-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic storage (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical storage (e.g., CD, DVD, BD, HVD), and semiconductor storage (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).

[0184] It should also be noted that the terms "first," "second," etc., in the embodiments of this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that the embodiments of this application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are usually of the same class, and the number of objects is not limited. For example, the first object can be one or more.

[0185] In the embodiments of this application, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0186] In the embodiments of this application, the term "multiple" refers to two or more, and other quantifiers are similar.

[0187] In this application's embodiments, "determine B based on A" means that factor A must be considered when determining B. It is not limited to "B can be determined based solely on A," but should also include: "determine B based on A and C," "determine B based on A, C, and E," "determine C based on A, and further determine B based on C," etc. Additionally, it can include using A as a condition for determining B, for example, "when A meets the first condition, determine B using the first method"; another example, "when A meets the second condition, determine B," etc.; another example, "when A meets the third condition, determine B based on the first parameter," etc. Of course, it can also be a condition where A is a factor in determining B, for example, "when A meets the first condition, determine C using the first method, and further determine B based on C," etc.

[0188] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0189] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0190] These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the processor-readable memory produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0191] These processors can execute instructions that can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0192] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. A data processing method for motion redirection, characterized in that, include: Receive source motion data and mapping configuration information. The source motion data includes at least one motion frame, and each motion frame includes node motion data corresponding to multiple source motion nodes. The mapping configuration information is used to characterize the association relationship between multiple redirection markers and one or more of the source motion nodes, as well as the marker generation method corresponding to each redirection marker. Based on the mapping configuration information, intermediate representation description information is determined, which includes marker declaration information and data organization information corresponding to the plurality of redirection marker points; For a target motion frame in the at least one motion frame, based on the mapping configuration information, the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame are mapped to obtain the marker point data corresponding to the redirection marker points associated with at least a portion of the source motion nodes in the target motion frame; According to the data organization method indicated by the data organization information, the marker point data corresponding to the plurality of redirection marker points are organized to generate intermediate representation frame data corresponding to the target motion frame; The intermediate representation description information and the intermediate representation frame data are respectively provided to the robot's motion relocation solution module, so that the motion relocation solution module can parse the intermediate representation frame data based on the intermediate representation description information.

2. The data processing method for motion redirection according to claim 1, characterized in that, The step of determining intermediate representation description information based on the mapping configuration information includes: The mapping configuration information is parsed to obtain multiple tag generation rules; Based on the redirection marker points corresponding to each of the aforementioned marker generation rules, generate marker declaration information corresponding to the plurality of redirection marker points; The data organization information is determined based on the tag declaration information corresponding to the multiple redirection tag points.

3. The data processing method for motion redirection according to claim 2, characterized in that, The step of generating tag declaration information corresponding to the multiple redirection tag points based on the redirection tag points corresponding to each of the tag generation rules includes: For each of the redirection markers, the marker type corresponding to each redirection marker is determined from a preset marker type set; Based on the marker name and marker type corresponding to each redirection marker, generate marker declaration information corresponding to each redirection marker; The preset marker type set includes at least one of pose type, direction type, orientation type and position type. The pose type is used to characterize the spatial position and rotation state of the redirection marker point, the direction type is used to characterize the spatial orientation of the redirection marker point, the orientation type is used to characterize the rotation state of the redirection marker point, and the position type is used to characterize the spatial position of the redirection marker point.

4. The data processing method for motion redirection according to claim 1, characterized in that, The step of mapping the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame based on the mapping configuration information to obtain the marker point data corresponding to the redirection marker points associated with at least a portion of the source motion nodes in the target motion frame includes: Based on the mapping configuration information, obtain one or more redirection marker points associated with source motion nodes in the target motion frame; Based on the marker generation method corresponding to the redirection marker points associated with one or more source motion nodes in the target motion frame, the node motion data corresponding to one or more source motion nodes in the target motion frame is processed to obtain the marker point data corresponding to the redirection marker points associated with one or more source motion nodes in the target motion frame.

5. The data processing method for motion redirection according to claim 4, characterized in that, The method of generating markers based on the redirect markers associated with one or more source motion nodes in the target motion frame, which processes the node motion data corresponding to one or more source motion nodes in the target motion frame to obtain the marker data corresponding to the redirect markers associated with one or more source motion nodes in the target motion frame, includes at least one of the following: When the marker type of the redirection marker is a pose type, the position component and rotation component in the marker data of the redirection marker are generated based on the position data and rotation data in the node motion data of the source motion node associated with the redirection marker. When the marker type of the redirection marker is an orientation type, the rotation component in the marker data of the redirection marker is generated based on the rotation data in the node motion data of the source motion node associated with the redirection marker. When the marker type of the redirection marker is a location type, the location component in the marker data of the redirection marker is generated based on the location data in the node motion data of the source motion node associated with the redirection marker; When the marker type of the redirection marker is a direction type, the direction component in the marker data of the redirection marker is generated based on the position data or rotation data in the node motion data of the source motion node associated with the redirection marker.

6. The data processing method for motion redirection according to claim 5, characterized in that, When the marker type of the redirection marker is a direction type, the direction component in the marker data of the redirection marker is generated based on the position data or rotation data in the node motion data of the source motion node associated with the redirection marker, including: Based on the position data in the node motion data of the first source motion node and the position data in the node motion data of the second source motion node, direction data from the first source motion node to the second source motion node is determined; wherein, the first source motion node and the second source motion node are two source motion nodes on the limb segment of the operation object, and the first source motion node and the second source motion node are used to determine the direction of the limb segment; or, Based on the rotation data in the node motion data of the source motion node, determine the direction data of the local coordinate axis of the source motion node in the target coordinate system; The direction data is normalized to obtain the direction component in the marker data of the redirection marker.

7. The data processing method for motion redirection according to claim 1, characterized in that, The step of organizing the marker point data corresponding to the plurality of redirection marker points according to the data organization method indicated by the data organization information to generate intermediate representation frame data corresponding to the target motion frame includes: Based on the data organization information, determine the data fields corresponding to each data component in the marker data of the multiple redirection markers; According to the declaration order of the multiple redirection markers in the intermediate representation description information, each data component in the marker data corresponding to each redirection marker is written into the corresponding data field; Based on the data fields after they have been written, the intermediate representation frame data is generated.

8. The data processing method for motion redirection according to claim 7, characterized in that, The step of writing each data component in the marker data corresponding to each of the redirected marker points into the corresponding data field includes: When the marker type of the redirected marker is pose type, the position component in the marker data corresponding to the redirected marker is written into the position data field, and the rotation component in the marker data corresponding to the redirected marker is written into the rotation data field; When the marker type of the redirection marker is orientation type, the rotation component in the marker data corresponding to the redirection marker is written into the position data field; When the marker type of the redirection marker is a direction type, the direction component in the marker data corresponding to the redirection marker is written into the direction data field; When the marker type of the redirected marker is a location type, the location component in the marker data corresponding to the redirected marker is written into the location data field; Wherein, at least one of the position data field, the rotation data field, and the orientation data field is stored using a flattened array.

9. The data processing method for motion redirection according to claim 1, characterized in that, The tag declaration information includes the tag name and tag type corresponding to the plurality of redirection tag points; The intermediate representation frame data includes at least one data component among position, rotation, and orientation components.

10. The data processing method for motion redirection according to claim 1, characterized in that, The step of determining intermediate representation description information based on the mapping configuration information further includes: If the mapping configuration information also includes a robot model identifier, the robot model identifier is written into the intermediate representation description information; or, If the mapping configuration information also includes a coordinate system convention after the source motion data is converted, the coordinate system convention after the source motion data is converted is written into the intermediate representation description information; or, If the mapping configuration information also includes a description information version number, the description information version number is written into the intermediate representation description information; The robot model identifier is used by the motion redirection solution module to determine the kinematic model corresponding to the robot, the coordinate system convention is used by the motion redirection solution module to parse the intermediate representation frame data according to the corresponding coordinate system, and the description information version number is used to verify the consistency between the intermediate representation frame data and the intermediate representation description information.

11. The data processing method for motion redirection according to claim 1, characterized in that, The step of providing the intermediate representation description information and the intermediate representation frame data to the robot's motion redirection solution module further includes: In the event that a data transmission channel is established or the mapping configuration information changes, the intermediate representation description information is provided to the motion redirection solution module; After the intermediate representation description information is provided, the intermediate representation frame data that can be parsed by the intermediate representation description information are sequentially provided to the motion redirection solving module.

12. The data processing method for motion redirection according to claim 1, characterized in that, Before mapping the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame, the method further includes: Obtain the measured dimensions of the object under a preset calibration posture; Based on the target size information of the robot and the measured size information, the runtime normalization parameters are determined; Based on the runtime normalization parameters, the position data in the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame are scaled.

13. The data processing method for motion redirection according to claim 12, characterized in that, The scaling process for position data in the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame, based on the runtime normalization parameters, includes: Obtain the basic scaling parameters, which are initial calibration parameters or an identity matrix; Based on the basic scaling parameters and the runtime normalization parameters, the composite scaling parameters are determined; The position data in the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame are scaled based on the composite scaling parameters, while maintaining the rotation data in the node motion data corresponding to at least a portion of the source motion nodes in the target motion frame.

14. The data processing method for motion redirection according to claim 1, characterized in that, The method further includes: If the offset between the position component in the marker data of the redirected marker and the reference position component under the preset calibration attitude exceeds a preset position offset threshold, the position offset parameter corresponding to the redirected marker is obtained, and the position offset parameter is injected into the marker generation method corresponding to the redirected marker; and / or, If the offset between the rotation component in the marker data of the redirected marker and the reference rotation component under the preset calibration attitude exceeds the preset rotation offset threshold, the rotation offset parameter corresponding to the redirected marker is obtained, and the rotation offset parameter is injected into the marker generation method corresponding to the redirected marker.

15. A data processing system for motion redirection, characterized in that, include: The motion capture data interface is used to receive source motion data from different motion capture devices and send the source motion data to the data mapping module; The data mapping module is used to execute the motion-oriented data processing method as described in any one of claims 1 to 14; The motion redirection solution module is used to receive intermediate representation description information and intermediate representation frame data provided by the data mapping module, and parse the intermediate representation frame data based on the intermediate representation description information to perform motion redirection calculation; wherein, the intermediate representation description information includes marker declaration information, and the intermediate representation frame data includes marker point data corresponding to the marker declaration information.

16. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the motion-oriented data processing method as described in any one of claims 1 to 14.

17. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores a computer program that, when executed by a processor, implements the motion-oriented data processing method as described in any one of claims 1 to 14.

18. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the motion-oriented data processing method as described in any one of claims 1 to 14.

Images