Object motion pose reproduction method based on Marker ball

Abstract

An object motion pose reproduction method based on a Marker ball comprises the following steps that firstly, size measurement is carried out on a target object, a model file is generated through three-dimensional scanning, and denoising and zooming optimization are carried out; then, a reflective Marker ball is pasted on the surface of an object, and three-dimensional coordinate data of the reflective Marker ball are collected through an optical motion capture system; then, a template group is constructed based on the initial Marker ball position of the object, a target group is constructed in combination with motion capture data, and a rigid body transformation matrix containing a rotation matrix and a displacement vector is solved through least square optimization; and finally, acting the transformation matrix on the model file to reproduce the pose, and reversely optimizing the modeling precision or the Marker point position through error analysis. The object motion pose reproduction method is high in precision, strong in anti-interference capability, simple in algorithm and suitable for the fields of industrial automation, movie and television games and the like.

Description

Technical Field

[0001] The present invention relates to motion capture and object posture reproduction technology, and in particular to a method for reproducing the motion posture of an object based on a marker ball. Background Art

[0002] Motion capture technology is widely used in modern film, gaming, animation, and other fields. It captures movements in real time and converts them into digital signals, enabling the recording and reproduction of motion trajectories. Furthermore, in the 3D field, for example, in the generation of interactive motions between digital humans and objects, the development of motion capture technology has led to the emergence of numerous interactive datasets, laying a solid foundation for research on the generation of interactive motions between humans and objects.

[0003] However, reproducing object motion trajectories remains challenging due to the time-consuming and complex nature of capturing and reproducing them. Trajectory reproduction methods based on inertial measurement data utilize inertial measurement units (IMUs) attached to key locations on an object to collect inertial data such as acceleration and angular velocity in real time. Integration algorithms are then used to infer the object's position and attitude changes, thereby reproducing the motion trajectory. However, due to bias drift and noise interference in inertial sensors, cumulative errors in the integration calculations occur over time, significantly reducing the accuracy of trajectory reproduction. For example, when reproducing the trajectory of a person walking or a drone flying for extended periods, the trajectory reproduced based on inertial data will exhibit significant deviations after a few minutes, making it difficult to meet high-precision application requirements. Alternatively, trajectory reproduction techniques based on RGB video utilize computer vision algorithms to extract and track objects in video sequences, thereby achieving trajectory reproduction. However, under complex lighting conditions, such as strong direct sunlight or shadows, object feature extraction can be subject to deviation or loss, leading to trajectory tracking failure. When an object moves rapidly or is interfered with by similar-looking objects, the algorithm struggles to accurately distinguish the target object, similarly resulting in trajectory reproduction errors. Moreover, RGB video data only contains two-dimensional image information of objects and lacks depth data, which often results in large errors when recovering three-dimensional trajectories.

[0004] In summary, the existing object motion posture reproduction technology has certain limitations in data processing and algorithm application, and it is difficult to meet the trajectory reproduction requirements in high-precision and complex scenarios.

[0005] It should be noted that the information disclosed in the above background technology section is only used to understand the background of this application, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0006] The main purpose of the present invention is to overcome the defects in the above-mentioned background technology and provide a method for reproducing the motion posture of an object based on a marker ball.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for reproducing the motion posture of an object based on a marker ball comprises the following steps:

[0009] S1. Object Modeling and Optimization: Measure the dimensions of the target object, generate an object model file through 3D scanning, and perform denoising and scaling optimization on the model.

[0010] S2. Optical Motion Capture: Multiple reflective marker balls are attached to the surface of an object, and the optical motion capture system is used to collect the three-dimensional coordinate data of the marker balls during the object's motion.

[0011] S3. Pose parameter solution: Build a template group based on the marker ball position of the object's initial state, build a target group based on the motion capture data, and solve the rigid body transformation matrix of the object relative to the initial state for each frame through least squares optimization. The transformation matrix includes a rotation matrix and a displacement vector.

[0012] S4. Motion trajectory reproduction and optimization: Apply the transformation matrix to the object model file to reproduce the object's motion posture, and reversely optimize the modeling accuracy or marker point position through error analysis.

[0013] Furthermore, step S1 specifically includes:

[0014] Measure the dimensions of key parts of objects;

[0015] Generate object model files through 3D scanning software;

[0016] Eliminate model noise in mesh processing software;

[0017] Scale the model based on the measured dimensions.

[0018] Furthermore, in step S2:

[0019] The number of the marker balls is 4-6, and the pasting positions are not on the same plane;

[0020] The motion capture environment meets the conditions of dark background and no reflective interference.

[0021] Furthermore, in step S3:

[0022] Convert motion capture raw data into standard matrix format;

[0023] Position the vertex coordinates of the marker ball on the surface of the object model to construct a template group.

[0024] Furthermore, the pose solution in step S3 specifically includes:

[0025] Calculate the centroid coordinates of the template group and the target group and perform de-centroiding;

[0026] Construct the covariance matrix and solve the optimal rotation matrix through singular value decomposition (SVD);

[0027] Calculates the displacement vector based on the center of mass coordinates and the rotation matrix.

[0028] Furthermore, the objective function of the least squares optimization is defined as the sum of squares of the Euclidean distances between the transformed template group and the target group;

[0029] The orthogonality of the rotation matrix is ​​ensured by the SVD decomposition.

[0030] Furthermore, step S3 further includes:

[0031] Compare the transformed template group coordinates with the target group coordinates and calculate the Euclidean distance error point by point;

[0032] When the error exceeds the threshold, the optimization mechanism of step S4 is triggered.

[0033] Furthermore, the optimization mechanism of step S4 includes:

[0034] Adjust the marker point selection position on the object model surface;

[0035] Or improve the accuracy of 3D scanning modeling.

[0036] Furthermore, the rigid body transformation matrix is ​​a 4×4 homogeneous transformation matrix, which is composed of a 3×3 rotation matrix and a 3×1 displacement vector.

[0037] A computer program product includes a computer program, which implements the object motion posture reproduction method when executed by a processor.

[0038] The present invention has the following beneficial effects:

[0039] The present invention provides a method for reproducing the motion posture of an object based on a marker ball, introduces the shape alignment theory in statistics into the motion posture reproduction of an object based on a marker ball, and significantly improves the accuracy and robustness of the object motion posture reproduction by optimizing the data processing process and the posture reproduction algorithm. Compared with the inertial measurement scheme that is prone to cumulative errors and the RGB video scheme that is interfered by light, the method of the present invention effectively resists ambient light interference under a dark background environment by combining a highly reflective marker ball with an optical motion capture system, and achieves trajectory reproduction with millimeter-level accuracy. The present invention innovatively integrates a full-process optimization mechanism: from size calibration and denoising processing in the object modeling stage to standardized conversion of motion capture data, and then utilizes an algorithm based on statistical shape analysis (such as TrajectoryCal algorithm) to solve the least squares optimization problem, and performs high-precision rigid body transformation alignment on the initial marker coordinates (template group) and the motion capture data (target group), significantly reducing the computational complexity of the posture solution. At the same time, the method of the present invention has outstanding engineering practical value. It provides a standardized operation chain (Luma / Blender modeling, optical capture, matrix solving), enabling technicians to deploy without complex algorithm development. Through a closed-loop error feedback mechanism (comparing real data with calculated results), it reversely optimizes modeling accuracy or marker point selection positions, ensuring system scalability. This design is not only suitable for digital interactive scenarios such as film and television animation and game development, but also flexibly adapts to areas with strict real-time and precision requirements, such as industrial robot arm calibration, providing an efficient and reliable motion trajectory reproduction solution for multiple industries.

[0040] Compared with the traditional method, the significant advantages of the present invention are embodied in the following aspects:

[0041] (1) The present invention relies on an optical motion capture system to capture the motion of an object based on a marker ball, with high accuracy. Based on the reflective properties of the optical motion capture system and the marker ball, the rotation matrix and displacement vector are solved in combination with the TrajectoryCal model algorithm, which significantly improves the accuracy of trajectory alignment. For example, in the embodiment, the coordinate reproduction error of the marker ball is controlled at the millimeter level (the average error of the embodiment is less than 20mm, see Table 1), which is more advantageous than traditional inertial measurement (the cumulative error may increase over time) and RGB video (susceptible to light interference) solutions.

[0042] (2) Strong anti-interference ability. The marker ball is made of highly reflective material. Through the optical motion capture system, it can be significantly distinguished from the surface of objects and surrounding interference in a dark background environment, reducing the interference of ambient light such as natural light and artificial light on visual recognition.

[0043] (3) The algorithm of the present invention is simple, has low computational complexity, and is highly scalable. It uses the linear algebra optimization method SVD to solve the transformation parameters, which has low computational time complexity and is suitable for real-time or large-scale data processing. It is also compatible with a variety of object types, such as rigid bodies and regular geometric bodies. By adjusting the number of marker balls and modeling accuracy, it can be flexibly adapted to various scenarios such as industrial robotic arms and film and television motion capture.

[0044] (4) The present invention provides a full-process tool chain and parameter settings (such as the number of marker balls to be pasted and site environment requirements) from object modeling using Luma and Blender, optical motion capture to data processing and pose solution. Technicians can quickly reproduce through standardized operations without the need for complex algorithm development experience. It is especially suitable for deployment by small and medium-sized enterprises or research teams.

[0045] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is an overall flow chart of a preferred embodiment of the present invention;

[0047] Figure 2 This is a flowchart of object scanning, modeling, and processing according to a preferred embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the object marker ball pasting position in a preferred embodiment of the present invention;

[0049] Figure 4 1 is a flow chart of step S3 of a preferred embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of the TRC format of motion capture data according to a preferred embodiment of the present invention;

[0051] Figure 6 This is a schematic diagram of a CSV file of motion capture data according to a preferred embodiment of the present invention;

[0052] Figure 7 It is a schematic diagram of recording the initial state position coordinates of the marker ball in the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope of the present invention and its application.

[0054] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0055] The present invention proposes a method for reproducing the motion posture of an object based on a marker ball. The shape alignment theory in statistics is introduced into the method. By optimizing the data processing flow and the posture reproduction algorithm, the high-precision solution of the object motion posture parameters is achieved, effectively improving the accuracy and robustness of the motion posture reproduction.

[0056] See Figure 1 The embodiment of the present invention provides a method for reproducing the motion posture of an object based on a marker ball, comprising:

[0057] Step S1. Object modeling and optimization: Measure the dimensions of the target object, generate an object model file through 3D scanning, and perform denoising and scaling optimization on the model.

[0058] In some embodiments, step S1 specifically includes: measuring the dimensions of key parts of the object; generating an object model file using 3D scanning software; eliminating model noise using mesh processing software; and scaling the model based on the measured dimensions.

[0059] Step S2. Optical motion capture: attach multiple reflective marker balls to key locations on the object's surface, and use an optical motion capture system to collect the three-dimensional coordinate data of the marker balls during the object's motion.

[0060] In some embodiments, in step S2: the number of the marker balls is 4-6, and the pasting positions are not on the same plane; the motion capture environment meets the conditions of dark background and no reflective interference.

[0061] Step S3. Pose parameter solution: Construct a template group based on the marker ball position of the object's initial state, construct a target group based on the motion capture data, and solve the rigid body transformation matrix of each frame relative to the initial state through least squares optimization. The transformation matrix includes a rotation matrix and a displacement vector.

[0062] In some embodiments, in step S3: the motion capture raw data is converted into a standard matrix format; and the coordinates of the vertices corresponding to the marker balls are positioned on the surface of the object model to construct a template set.

[0063] In some embodiments, the posture solution of step S3 specifically includes: calculating the center of mass coordinates of the template group and the target group and performing de-centering; constructing a covariance matrix and solving the optimal rotation matrix through singular value decomposition (SVD); and calculating the displacement vector based on the center of mass coordinates and the rotation matrix.

[0064] In some embodiments, the objective function of the least squares optimization is defined as the sum of squares of the Euclidean distances between the transformed template group and the target group; and the orthogonality of the rotation matrix is ​​ensured by the SVD decomposition.

[0065] In some embodiments, step S3 further includes: comparing the transformed template group coordinates with the target group coordinates, and calculating the Euclidean distance error point by point; and triggering the optimization mechanism of step S4 when the error exceeds a threshold.

[0066] In some embodiments, the rigid body transformation matrix is ​​a 4×4 homogeneous transformation matrix composed of a 3×3 rotation matrix and a 3×1 displacement vector.

[0067] Step S4. Motion trajectory reproduction and optimization: Apply the transformation matrix to the object model file to reproduce the object's motion posture, and reversely optimize the modeling accuracy or marker point position through error analysis.

[0068] In some embodiments, the reverse optimization mechanism of step S4 includes: adjusting the positions of marker points on the surface of the object model; or improving the accuracy of three-dimensional scanning modeling.

[0069] The following further describes specific embodiments of the present invention, its algorithm examples and case verification.

[0070] A method for reproducing the motion posture of an object based on a marker ball comprises the following steps:

[0071] S1: Select a suitable object, record the object size information, scan and model the object, obtain the corresponding OBJ model file, and perform denoising and optimization on the obtained 3D file;

[0072] S2: Perform optical motion capture of the object based on the marker ball to obtain the object motion capture file, that is, the three-dimensional coordinate data of the marker ball during the object's motion;

[0073] S3: Preprocess the motion capture data obtained in step S2, parse the motion data of the marker ball, and convert it into a directly readable data format. Record the position coordinates of the marker ball's initial state as a template group, and use the TrajectoryCal model to solve the object's transformation matrix relative to the initial state in each frame. The goal is to minimize the sum of squared deviations between the motion capture data and the reproduced marker ball position; the transformation matrix includes the object's rotation (rotation) and displacement (translation) data, which together constitute the object's position and posture;

[0074] S4: Combining the object modeling file obtained in step S1 and the object transformation matrix data obtained in step S3, the object motion trajectory is reproduced, and the pose is optimized and verified.

[0075] Preferably, step S1 specifically includes:

[0076] S11: Select the object to be motion captured, and use a rangefinder or tape measure to measure the size of the key parts of the object to facilitate subsequent scaling of the modeling file;

[0077] S12: Use Luma software to model the object and export the corresponding obj format file;

[0078] S13: Eliminate redundant noise points and parts irrelevant to the object in the obj file in Meshlab;

[0079] S14: Referring to the object size data measured in step S11, use Blender to scale the object model optimized in S13, and finally obtain a 3D modeling file that conforms to the actual size of the object.

[0080] Preferably, step S2 specifically includes:

[0081] S21: Using an optical motion capture system, four to six reflective markers (optical landmarks) are attached to key locations on the surface of an object. The motion capture system's camera captures the markers' positions in space. Avoid reflective surfaces to avoid interfering with data collection. If reflective areas are present on the surface, use appropriate materials to block them.

[0082] S22: Pre-plan the action to be captured, start motion capture, and obtain the motion capture file of the object. The present invention obtains a c3d format file.

[0083] Preferably, step S22 specifically includes:

[0084] S221: Choose a suitable venue. A space with minimal human activity should be selected. The motion capture space should minimize floor reflections and use a dark background. Also, minimize distractions and remove reflective objects.

[0085] S222: Create a rigid body and ensure that the marker balls are not in the same plane so that they can form a three-dimensional shape. When creating, keep the relative positions of the marker balls unchanged. The number of marker balls depends on the size and structural complexity of the object. Generally, 4-6 marker balls are sufficient.

[0086] S223: Motion acquisition: capturing the motion of the object according to the pre-set motion, and obtaining motion capture data files in c3d format. These files store the three-dimensional coordinates (x, y, z) of the marker ball in each frame.

[0087] Preferably, step S3 specifically includes:

[0088] S31: Convert the motion capture data in c3d format to trc format through OpenSim, and then store the 3D position data of each marker ball in trc in csv format;

[0089] S32: Open the object modeling file in Blender, find the vertex (vertex) on the surface of the obj model that is close to the marker ball pasted position, record its position coordinates, and record it as matrix M N×3 , where N is the number of marker balls attached to the surface of the object, and 3 represents the coordinate data of three dimensions;

[0090] S33: Calculates the 6D pose data of an object in motion based on the TrajectoryCal model. 6D pose, or six degrees of freedom, consists of three degrees of freedom (translation) and three degrees of freedom (rotation). Displacement represents the position of an object in three-dimensional space, represented by a three-dimensional vector (X, Y, Z). Rotation represents the orientation of an object in three-dimensional space, represented by a rotation matrix relative to a reference. Together, the two constitute the pose of the object during motion.

[0091] Taking displacement as an example, in the three-dimensional homogeneous coordinate representation, any point P = (x, y, z) is calculated by translating the distance t x , t y , t z Add to the coordinates of P and translate to position P'=(x',y',z'),x'=x+t x ,y'=y+t y ,z'=z+t z , expressed as a homogeneous matrix:

[0092]

[0093] For rotation, taking rotation around the z-axis as an example, the rotation expression is: x'=xcosθ-ysinθ, y'=xsinθ+ycosθ, z'=z, and the rotation matrix form is:

[0094]

[0095] Pose is a relative concept that describes displacement and rotation transformation. The displacement and rotation matrices can be written in the same matrix to form a 4*4 homogeneous transformation matrix T:

[0096]

[0097] To reproduce the pose of an object's motion, the key is to solve the rotation matrix and displacement to align the object's initial pose with the real-time pose captured during motion capture. In this invention, the object can be considered a rigid body, whose motion follows Euclidean transformations, which only involve rotation and translation, without scaling deformation.

[0098] Template group (initial position): where n i =[x ’ i ,y ’ i ,z ’ i ] T Represents the three-dimensional coordinates of the i-th Marker ball in the initial state of the object (modeled obj file), with a total of N Marker points.

[0099] Target group (motion capture data): where m i =[x i ,y i ,z i ] T Indicates the real-time coordinate data of the corresponding marker ball during motion capture.

[0100] The rigid body motion satisfies the Euclidean transformation, that is, the coordinates of the template group are matched with the target group after rotation and translation:

[0101] m i =n i R+t+ε i (i=1,2,...,N)

[0102] where ε i is the noise term, is the orthogonal rotation matrix (R T R=I), is the displacement vector.

[0103] To minimize the matching error, the least squares objective function is defined as:

[0104]

[0105] This function represents the sum of the squares of the Euclidean distances between all marker point coordinates and the template group after transformation. That is, the transformation parameter that can make the two sets of data match best, and it is also the core optimization problem of solving the motion posture of the object.

[0106] To simplify the solution of the objective function, the two sets of data are first de-centred. Calculate the centroid of the template group and the target group:

[0107] Template group centroid: Target group centroid: Subtract the centroids from the original coordinate points to get the de-centroided point set matrix:

[0108]

[0109] At this time, the displacement vector t can be expressed as After substituting into the objective function, the t term can be eliminated, and the objective function is simplified to a problem that only needs to optimize R:

[0110]

[0111] in,

[0112] E(R) can be further converted into matrix form:

[0113]

[0114] Here, ||·|| F is the Frobenius norm, which is defined as the square root of the sum of the squares of the matrix elements, that is, for the matrix Its Frobenius norm is

[0115]

[0116] To solve the optimal R, singular value decomposition (SVD) is introduced. First, the covariance matrix H is calculated:

[0117] H=N′ T M′

[0118] Perform SVD decomposition on H: H = UΣV T , where U and V are orthogonal matrices, ∑ is a diagonal matrix whose diagonal elements are the singular values ​​of H.

[0119] By taking the derivative of the objective function E(R) with respect to R and combining the properties of the orthogonal matrix (R T R = I), and a new objective function is constructed using the Lagrange multiplier method to solve it. When E(R) reaches the minimum value, the optimal rotation matrix R is obtained. Combined with the center of mass coordinates during decentralization, the displacement vector t is calculated:

[0120]

[0121] At this point, we've obtained the transformation parameters R,t that align the initial position N (template group) of the marker sphere on the object obj model surface to the actual motion capture data M (target group). By transforming the initial state with these parameters, we can reproduce the motion pose of the rigid body based on the marker sphere, providing an accurate data foundation for subsequent motion analysis, virtual simulation, and other applications.

[0122] S34: Analyze the error between the real motion capture data and the trajectory data calculated by the model.

[0123] Preferably, step S4 specifically includes:

[0124] S41: Import the object modeling obj file obtained in step S1;

[0125] S42: Based on the transformation matrix calculated by the model in step S3, the motion trajectory of the object can be reproduced.

[0126] S43: Posture optimization and verification. If the reproduced object motion has a large error compared to the real motion, optimization can be performed in terms of object scanning modeling accuracy, marker position accuracy selected from the object model surface, etc.

[0127] Examples

[0128] refer to Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 、 Figure 7 The object motion posture reproduction method based on the marker ball includes the following steps:

[0129] S1: Object size measurement, scanning modeling and optimization.

[0130] First, select a suitable object for motion capture. In order to facilitate the subsequent steps, the object needs to be measured, scanned, modeled, and optimized.

[0131] Specifically, step S1 includes:

[0132] S11: Select Figure 2For the chair shown in (a), the dimensional data of key parts of the object can be recorded using distance meter software or a tape measure. For example, the distance from the chair seat surface to the ground can be measured to be approximately 74 cm, which facilitates the subsequent calculation of the scaling parameters for object modeling.

[0133] S12: Place the object in a clean background and shoot videos from multiple angles around the object. To ensure the quality of the model, try to remain stable during the shooting process and capture as many structural details of the object as possible. After completing the video recording, use Luma software to import the video, generate the corresponding object modeling file, and export it in obj format. Figure 2 (b) shows the initial shape of the chair after modeling.

[0134] S13: The object modeling obtained in step S12 contains noise or redundant parts in the scene. Select the noise points in Meshlab, such as Figure 2 As shown in the red part in (c), this part is eliminated to obtain the denoised object obj model.

[0135] S14: Combine the object size data measured in step S11, use Blender, measure the size of the model in Blender, calculate the scaling parameters, and then scale the model to obtain a 3D modeling file that meets the actual size of the object, such as Figure 2 (d) shown.

[0136] S2: Optical motion capture.

[0137] Using an optical motion capture system, affix an appropriate number of reflective markers (optical markers) to key locations on the surface of an object. The motion capture system's camera collects information about the markers' positions within the space. Choose a suitable location with minimal human activity. Minimize floor reflections and use a dark background. Also, minimize distractions and remove reflective objects from the space.

[0138] In this embodiment, four marker balls are attached to the surface of the chair. Figure 3 As shown, marker balls are attached to the chair seat at four locations: front, back, left, and right. Once the venue is set up and the marker balls are attached, motion capture of the objects can be performed as planned. After motion capture is complete, a c3d file is generated, storing the 3D coordinates of the marker balls for each frame of motion.

[0139] S3: Process the motion capture data and solve the object's rotation and displacement data.

[0140] In the present invention, the object's motion pose includes data on six degrees of freedom: rotation and translation. These data constrain the object's position and orientation during motion. To this end, the present invention calculates the object's motion pose based on the motion data of the marker ball.

[0141] Specifically, if Figure 4 As shown, step S3 includes:

[0142] S31: Since the c3d data format is not convenient for visual viewing, the c3d format motion capture data is converted into trc format in Matlab through OpenSim. In this embodiment, the trc data format is as follows: Figure 5 As shown, in order to facilitate data reading, the three-dimensional position data of each Marker ball in trc is converted into csv format, as shown Figure 6 As shown, the unit of the coordinate data in this embodiment is millimeter (mm).

[0143] S32: Import the object modeling file into Blender, refer to the actual position of the marker ball pasted on the object, find the vertex (vertex) on the surface of the obj file that is close to the marker ball pasted position, record it as the initial position of the marker ball, check and record its three-dimensional coordinates, such as Figure 7 In this embodiment, the marker coordinates are: (-0.061855m, -0.18409m, 0.62278m), (0.091619m, -0.007383m, 0.63352m), (-0.071962m, 0.16252m, 0.63105m), and (-0.25375m, -0.003273m, 0.61932m). The marker positions should be selected as accurately as possible to lay the foundation for pose calculation.

[0144] S33: Calculate the 6D pose data of the object based on the TrajectoryCal model. In this embodiment, the template group (initial position) matrix is ​​as follows (unit: mm):

[0145]

[0146] In this embodiment, one frame of the motion sequence is selected, and the coordinate data of the marker ball in the frame is used as the target group (motion capture data) matrix as follows (unit: mm):

[0147]

[0148] The rigid body motion satisfies the Euclidean transformation, that is, the coordinates of the template group are matched with the target group after rotation and translation:

[0149] m i =n i R+t+ε i (i=1,2,...,N)

[0150] where ε i is the noise term, is the orthogonal rotation matrix (R T R=I), is the displacement vector.

[0151] To minimize the matching error, the least squares objective function is defined as:

[0152]

[0153] This function represents the sum of the squares of the Euclidean distances between all marker points and the template group after transformation. That is, the transformation parameters that can make the two sets of data best match each other, and it is also the core optimization problem of solving the motion posture of the object in the embodiment.

[0154] After solving, the rotation matrix corresponding to the object in the embodiment in this frame is:

[0155]

[0156] The displacement matrix t is:

[0157] t=[-1056.2046 1883.5005 109.6211] T

[0158] The transformation matrix T can be expressed as:

[0159]

[0160] S34: Analyze the error between the real motion capture data and the model calculation results. According to the object pose calculated in step S33, that is, the rotation and displacement matrix, the template group data is rotated and displaced and compared with the target group data. The comparison results of the embodiment of the present invention are shown in Table 1:

[0161] Table 1 Error of the embodiment of the present invention (unit: mm)

[0162]

[0163]

[0164] In this embodiment, the error between the calculation result and the target group data is small and meets the requirements, that is, the calculated rotation and displacement matrices basically conform to the posture of the object during the movement process.

[0165] S4: Reproduce the object motion and analyze the optimization results.

[0166] If the error is large, you can improve the accuracy of the data by improving the accuracy of the object scanning model and the accuracy of the marker points selected from the object model file surface. If the error meets the requirements, you can apply the transformation matrix to the object model file to reproduce the object's motion pose.

[0167] In other embodiments, the present invention may also apply multimodal fusion (fusion of IMU data or point cloud) and deformation reproduction of objects extending from rigid bodies to flexible objects.

[0168] In summary, the present invention proposes a method for reproducing the motion pose of an object based on a marker ball. Its innovative work and important features include:

[0169] (1) A pose reproduction framework combining optical motion capture and rigid body transformation: A marker-based pose reproduction process is proposed. Through the entire process of "object modeling → optical capture based on marker balls → data preprocessing → transformation matrix solution → error analysis", high-precision mapping from physical entities to digital pose trajectories is achieved.

[0170] (2) 6D pose solution algorithm based on TrajectoryCal model: The marker ball position of the object in the initial state and the motion capture data are constructed as a template group and a target group respectively, and the least squares objective function is defined. The rotation matrix R and the displacement matrix t are solved by the idea of ​​SVD decomposition to minimize the matching error, and the statistical shape analysis theory is applied to the alignment of the marker ball coordinate position.

[0171] (3) Multi-stage data optimization mechanism:

[0172] Modeling optimization: Through Meshlab denoising and Blender scaling, the obj file size is ensured to be consistent with the actual object, improving the benchmark accuracy;

[0173] Motion capture data preprocessing: converting C3D format to CSV format to facilitate data parsing and matrix operations, and reduce format compatibility errors;

[0174] Error feedback: By comparing the real motion capture data with the model calculation results, the modeling accuracy and marker ball selection positions are reversely optimized to form a closed-loop optimization link.

[0175] Through the above-mentioned innovative solution, the present invention has achieved the significant advantages of high precision (the marker ball coordinate reproduction error is controlled at the millimeter level, and it is simpler than traditional inertial measurement and RGB video solutions), strong anti-interference ability (it can be significantly distinguished from the surface of the object and surrounding interference in a dark background environment), simple algorithm and low computational complexity (suitable for real-time or large-scale data processing, with strong scalability), and providing a full-process tool chain and parameter settings (technicians can quickly reproduce through standardized operations, suitable for deployment by small and medium-sized enterprises or research teams).

[0176] Application scenarios of the present invention may include:

[0177] Industrial automation: robotic arm trajectory calibration, assembly process simulation;

[0178] Film, television, and games: object motion capture, interaction between virtual characters and objects, and special effects scene reconstruction;

[0179] Sports Science: analysis of athletes' movements and postures, prevention of sports injuries;

[0180] Medical rehabilitation: Patient gait analysis and prosthetic motion trajectory optimization. By attaching marker balls to the patient's joints (such as the hip, knee, and ankle), the joint motion trajectory is reproduced. By comparing the shape differences between the patient's joint posture and that of a healthy person's template, pathological indicators such as knee varus / valgus angle and stride abnormality are quantified to provide data support for rehabilitation program design.

[0181] An embodiment of the present invention further provides a storage medium for storing a computer program, which, when executed, at least performs the object motion posture reproduction method described above.

[0182] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the object motion posture reproduction method as described above when executing the computer program.

[0183] An embodiment of the present invention further provides a processor, which executes a computer program and at least performs the object motion posture reproduction method as described above.

[0184] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disc or a read-only optical disc (CD-ROM); the magnetic surface memory can be a magnetic disk memory or a magnetic tape memory. The storage medium described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0185] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0186] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0187] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0188] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0189] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0190] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0191] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0192] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0193] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. Those skilled in the art will recognize that, without departing from the scope of the present invention, several equivalent substitutions or obvious variations can be made, and the performance or use of the same should be considered to fall within the scope of protection of the present invention.

Claims

1. A method for reproducing the motion posture of an object based on a marker ball, characterized in that: The following steps are involved: S1. Object Modeling and Optimization: Measure the dimensions of the target object, generate an object model file through 3D scanning, and perform denoising and scaling optimization on the model. S2. Optical Motion Capture: Multiple reflective marker balls are attached to the surface of an object, and the optical motion capture system is used to collect the three-dimensional coordinate data of the marker balls during the object's motion. S3. Pose parameter solution: Build a template group based on the marker ball position of the object's initial state, build a target group based on the motion capture data, and solve the rigid body transformation matrix of the object relative to the initial state for each frame through least squares optimization. The transformation matrix includes a rotation matrix and a displacement vector. S4. Motion trajectory reproduction and optimization: Apply the transformation matrix to the object model file to reproduce the object's motion posture, and reversely optimize the modeling accuracy or marker point position through error analysis.

2. The object motion posture reproduction method according to claim 1, characterized in that: Step S1 specifically includes: Measure the dimensions of key parts of objects; Generate object model files through 3D scanning software; Eliminate model noise in mesh processing software; Scale the model based on the measured dimensions.

3. The object motion posture reproduction method according to claim 1 or 2, characterized in that: In step S2: The number of the marker balls is 4-6, and the pasting positions are not on the same plane; The motion capture environment meets the conditions of dark background and no reflective interference.

4. The method for reproducing the motion posture of an object according to any one of claims 1 to 3, wherein: In step S3: Convert motion capture raw data into standard matrix format; Position the vertex coordinates of the marker ball on the surface of the object model to construct a template group.

5. The object motion posture reproduction method according to claim 4, characterized in that: The pose solution in step S3 specifically includes: Calculate the centroid coordinates of the template group and the target group and perform de-centroiding; Construct the covariance matrix and solve the optimal rotation matrix through singular value decomposition (SVD); Calculates the displacement vector based on the center of mass coordinates and the rotation matrix.

6. The method for reproducing the motion posture of an object according to claim 5, wherein: The objective function of the least squares optimization is defined as the sum of squares of the Euclidean distances between the transformed template group and the target group; The orthogonality of the rotation matrix is ​​ensured by the SVD decomposition.

7. The method for reproducing the motion posture of an object according to any one of claims 1 to 6, wherein: Step S3 further includes: Compare the transformed template group coordinates with the target group coordinates and calculate the Euclidean distance error point by point; When the error exceeds the threshold, the optimization mechanism of step S4 is triggered.

8. The method for reproducing the motion posture of an object according to any one of claims 1 to 7, wherein: The optimization mechanism of step S4 includes: Adjust the marker point selection position on the object model surface; Or improve the accuracy of 3D scanning modeling.

9. The method for reproducing the motion posture of an object according to any one of claims 1 to 8, wherein: The rigid body transformation matrix is ​​a 4×4 homogeneous transformation matrix, which is composed of a 3×3 rotation matrix and a 3×1 displacement vector.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for reproducing the motion posture of an object as described in any one of claims 1 to 9 is implemented.

Images