Motion capture-based data processing method, device, equipment and storage medium

Abstract

The present disclosure provides a motion capture-based data processing method and device, equipment and storage medium, relates to the technical field of image processing, especially to the field of computer vision and virtual reality. The specific implementation scheme is: after obtaining the motion parameter observation of the joint of the motion capture object by observing the motion capture object, the motion parameter observation of each joint is processed by using a first jitter elimination algorithm to obtain a first reference quantity, and a second jitter elimination algorithm to obtain a second reference quantity. According to the difference between the first reference quantity and the second reference quantity, a target jitter elimination algorithm matched with the difference is selected to correct the motion parameter observation. Through the difference between the results of the two different jitter elimination algorithms, the motion speed state of the motion capture object is embodied, and then the motion parameter observation is corrected by selecting the matched target jitter elimination algorithm based on the different motion speed states, thereby improving the motion restoration degree of the motion capture object.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, specifically computer vision and virtual reality technology, and can be applied to animation scenarios. In particular, it relates to data processing methods, devices, equipment, and storage media based on motion capture. Background Technology

[0002] Motion capture technology uses optical, inertial, and visual methods to capture the body features, joint positions, and other movement states of humans or animals to obtain motion data. Because the raw motion data obtained from motion capture generally suffers from jitter and uneven trajectories, it must be manually refined by animators or undergo post-processing methods to recreate the motion of the captured subject to a certain extent. Summary of the Invention

[0003] This disclosure provides a data processing method, apparatus, device, and storage medium based on motion capture.

[0004] According to one aspect of this disclosure, a data processing method based on motion capture is provided, comprising:

[0005] The motion capture object is observed to obtain the motion parameters of the joints of the motion capture object;

[0006] The observed motion parameters of the joint are processed using a first jitter elimination algorithm to obtain a first reference value, and processed using a second jitter elimination algorithm to obtain a second reference value; wherein the smoothing intensity of the second jitter elimination algorithm is different from that of the first jitter elimination algorithm;

[0007] Based on the difference between the first reference value and the second reference value, a target jitter elimination algorithm matching the difference is selected to correct the motion parameter observation.

[0008] According to another aspect of this disclosure, a motion capture-based data processing apparatus is provided, comprising:

[0009] The observation module is used to observe the motion capture object in order to obtain the motion parameters of the joints of the motion capture object.

[0010] The processing module is used to process the observed motion parameters of the joint using a first jitter elimination algorithm to obtain a first reference value, and to process them using a second jitter elimination algorithm to obtain a second reference value; wherein the smoothing intensity of the second jitter elimination algorithm is different from that of the first jitter elimination algorithm.

[0011] The correction module is used to select a target jitter elimination algorithm that matches the difference between the first reference value and the second reference value to correct the motion parameter observation.

[0012] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect of this disclosure.

[0013] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the method described in the first aspect of this disclosure.

[0014] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in the first aspect of this disclosure.

[0015] The motion capture-based data processing method, apparatus, device, and storage medium disclosed herein obtain motion parameter observations of the joints of the motion capture object through observation. Then, for each joint's motion parameter observation, a first jitter elimination algorithm is applied to obtain a first reference value, and a second jitter elimination algorithm is applied to obtain a second reference value. Based on the difference between the first and second reference values, a target jitter elimination algorithm matching the difference is selected to correct the motion parameter observations. Since the smoothing intensity of the second jitter elimination algorithm differs from that of the first jitter elimination algorithm, the difference between the results of these two different algorithms reflects the speed of the motion capture object's movement. Therefore, by selecting a matching target jitter elimination algorithm based on different speeds of movement to correct the motion parameter observations, the corrected motion parameter observations can smooth the trajectory while maintaining the original rhythm of the movement, thus improving the motion reproduction accuracy of the motion capture object.

[0016] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0017] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0018] Figure 1This is a flowchart illustrating a motion capture-based data processing method provided in an embodiment of this disclosure.

[0019] Figure 2 This is a schematic diagram of a joint;

[0020] Figure 3 A flowchart illustrating another motion capture-based data processing method provided in this embodiment of the present disclosure;

[0021] Figure 4 A schematic diagram illustrating the principle of the motion capture-based data processing method provided in this embodiment of the disclosure;

[0022] Figure 5 A flowchart illustrating another motion capture-based data processing method provided in this embodiment of the present disclosure;

[0023] Figure 6 A schematic diagram of the structure of a motion capture-based data processing device 600 provided in an embodiment of this disclosure;

[0024] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0025] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0026] In related virtual reality or animation technologies, in order to solve the problems of data jitter and uneven trajectory during motion capture, the industry generally uses filtering methods to reduce jitter in post-processing. This method will lose the accuracy of motion reproduction, increase motion delay, and make the tracking of fast motion worse.

[0027] To ensure both smoothness and a certain degree of tracking accuracy, this embodiment of the invention observes the motion capture object to obtain motion parameter observations of its joints. Then, each joint's motion parameter observation is processed using a first jitter elimination algorithm to obtain a first reference value, and a second jitter elimination algorithm to obtain a second reference value. Since the smoothing intensity of the second jitter elimination algorithm differs from that of the first, the difference between these two algorithm results reflects the speed of the motion capture object's movement. Therefore, based on different speed states, a matching target jitter elimination algorithm is selected to correct the motion parameter observations. This allows the corrected motion parameter observations to smooth the trajectory while maintaining the original rhythm of the movement, improving the motion reproduction accuracy of the motion capture object.

[0028] Figure 1 This is a flowchart illustrating a motion capture-based data processing method provided in an embodiment of this disclosure. The method provided in this embodiment can be executed by a motion capture device, such as... Figure 1 As shown, it includes:

[0029] Step 101: Observe the motion capture object to obtain the motion parameter observations of the joints of the motion capture object.

[0030] The motion capture target can be a human or an animal.

[0031] Motion capture technology includes optical, inertial, and visual methods to acquire motion information of humans or animals. In this embodiment, the acquired motion parameters of joints related to the motion state are observed.

[0032] Motion parameter observations, such as Figure 2 This is a schematic diagram of a joint; the observable motion parameters can be... Figure 2 The movement speed, posture, position coordinates, and angle of the joint shown are not limited in this embodiment. Figure 2 In this context, joints connected by lines belong to the same joint kinematic chain.

[0033] As one possible implementation, the joint positions of the motion capture object are observed to obtain the joint position coordinates of each joint; based on the joint position coordinates of each joint, the joint angles of each joint are determined; and the position coordinates and / or joint angles are used as the motion parameter observations.

[0034] As another possible implementation, the joint angles of the motion capture object are observed to obtain the joint angles of each joint; a kinematic model is used to identify the relative positions of each joint point based on the joint angles of each joint, so that the relative positions and / or joint angles are used as the motion parameter observations.

[0035] Step 102: The observed motion parameters of the joint are processed using a first jitter elimination algorithm to obtain a first reference value, and processed using a second jitter elimination algorithm to obtain a second reference value, wherein the smoothing intensity of the second jitter elimination algorithm is different from that of the first jitter elimination algorithm.

[0036] As one possible implementation, the first and second jitter removal algorithms use different filter orders, meaning their filter formulas have different orders. For example, the first jitter removal algorithm could be based on a first-order filter, while the second jitter removal algorithm could be based on a second-order filter.

[0037] As another possible implementation, the first and second jitter elimination algorithms use the same filtering order, but the parameters of the filtering formula are different.

[0038] It should be noted that both the first and second jitter elimination algorithms are based on historical motion parameters observed before the current motion parameter observations are observed, or motion parameters obtained after historical jitter reduction processing. The algorithms use a filter function to smooth the motion parameters, thereby achieving the jitter reduction effect.

[0039] Step 103: Based on the difference between the first reference value and the second reference value, select a target jitter elimination algorithm that matches the difference to correct the motion parameter observations.

[0040] In this embodiment, after obtaining the motion parameter observations of the joints of the motion capture object through observation, the motion parameter observations of each joint are processed using a first jitter elimination algorithm to obtain a first reference value, and a second jitter elimination algorithm to obtain a second reference value. Based on the difference between the first reference value and the second reference value, a target jitter elimination algorithm matching the difference is selected to correct the motion parameter observations. The difference between the results of the two different jitter elimination algorithms reflects the speed state of the motion capture object. Therefore, based on different speed states of motion, a matching target jitter elimination algorithm is selected to correct the motion parameter observations, thereby improving the motion reproduction accuracy of the motion capture object.

[0041] Figure 3 A flowchart illustrating another motion capture-based data processing method provided in this disclosure embodiment is shown below. Figure 3 As shown, it includes:

[0042] Step 301: Observe the motion capture object to obtain the motion parameter observations of the joints of the motion capture object.

[0043] Optionally, joint angles are observed on the motion capture object to obtain joint angles J as raw motion capture data. Based on a pre-established kinematic model K, the three-dimensional coordinate position P of the joint can be determined based on the joint angle J.

[0044] Among them, the kinematic model K is a model used to determine the three-dimensional coordinate position of the joint based on the joint angle.

[0045] Step 302: The motion parameter observations of the joint are processed using a first jitter elimination algorithm to obtain a first reference value, and processed using a second jitter elimination algorithm to obtain a second reference value.

[0046] The smoothing intensity of the second jitter elimination algorithm differs from that of the first jitter elimination algorithm.

[0047] Optionally, a first-order filter is applied to the joint angle J. This first-order filter can alleviate significant jitter in the data while keeping fast movements unaffected, resulting in joint angle J1. Then, a second-order filter is applied to the joint angle J. This second-order filter can eliminate all jitter, and simultaneously, fast / large-amplitude movements will be significantly reduced or their amplitude will disappear, resulting in joint angle J2.

[0048] Using kinematic model K, the 3D position P1 of joint J1 is calculated. Similarly, using kinematic model K, the 3D position P2 of joint J2 is calculated.

[0049] To eliminate data jitter without affecting the motion effect of fast motion, a method combining first-order and second-order filtering is proposed.

[0050] It should be noted that J, J1, J2, P, P1, and P2 mentioned above can be vectors representing angles or coordinates in three-dimensional space. Taking multiple motion parameters into account improves recognition accuracy.

[0051] Steps 301 and 302 can be specifically referred to as steps 101 and 102 in the foregoing embodiments. They will not be repeated in this embodiment.

[0052] Step 303: When there are multiple joints for which the observed motion parameters are obtained, a first matrix, a second matrix, and a third matrix are generated based on at least two target joints belonging to the same joint kinematic chain among the multiple joints.

[0053] In the first matrix, each row vector corresponds to a target joint and is used to indicate the first reference value of the corresponding target joint in three-dimensional space.

[0054] Each row vector in the second matrix corresponds to a target joint and is used to indicate the second reference value of the corresponding target joint in three-dimensional space.

[0055] Each row vector in the third matrix corresponds to a target joint and is used to indicate the observed motion parameters of the corresponding target joint in three-dimensional space.

[0056] Optionally, a joint kinematic chain can be pre-defined, representing how the movement of a parent node typically influences the movement of its child nodes. For example, all the joints in an arm form a joint kinematic chain. Joint movements within the same kinematic chain will influence each other, while movements in different kinematic chains have little impact on each other. In this embodiment, considering all joints within the same kinematic chain comprehensively to identify the speed of movement helps improve recognition accuracy.

[0057] For example, based on the joint kinematic chain C, the first matrix [J1] and [P1], the second matrix [J2] and [P2], and the third matrix [J] and [P] are generated. Each row of [J1] corresponds to the J1 of each joint, and J1 is a row vector representing the angle. The generation methods of [P1], [J2], [P2], [J], and [P] are similar to [J1], with row vectors corresponding to each joint. The only difference is that the row vectors are P1, J2, P2, J, and P, respectively, which will not be elaborated further here.

[0058] Furthermore, based on the parent-child relationships among the multiple joints, at least two target joints belonging to the same joint kinematic chain are identified. For example... Figure 2 As shown, joints connected by the same line are target joints belonging to the same kinematic chain. During movement, these joints have some interrelationships, meaning that the angles or positions between the joints affect each other. Therefore, identifying whether a target joint is in a fast or slow state based on its kinematic chain will yield more accurate results and improve recognition accuracy.

[0059] Step 304: The distance between the first matrix and the second matrix, and the distance between the third matrix and the second matrix are weighted and summed to obtain the distance value.

[0060] Substitute the first, second, and third matrices into the decision criteria to determine whether the kinetic chain is in a fast or slow state. Specific decision criteria can be:

[0061] H=R1×F1([J1]–[J2])+R2×F2([P1]–[P2])+R3×F1([J]–[J2])+R4×F2([P]–[P2]).

[0062] Where F1([J1]–[J2]) represents the spherical distance between [J1] and [J2], and F2([P1]–[P2]) represents the Euclidean distance between [P1] and [P2]. ​​Similarly, F1([J]–[J2]) represents the spherical distance between [J] and [J2], and F2([P]–[P2]) represents the Euclidean distance between [P] and [P2].

[0063] R1 to R4 are the weights of the corresponding sub-items.

[0064] Substituting the first, second, and third matrices into the decision criteria yields the distance value H. In this embodiment, not only is the distance value H for identifying fast and slow states derived from the differences in results between two different jitter elimination algorithms, but also the original motion parameter observations are introduced to compare the results of the jitter elimination algorithm with stronger smoothing intensity, thereby making the correlation between the distance value H and the fast / slow state stronger.

[0065] Step 305: Based on the distance value, select the matching target jitter elimination algorithm to correct the motion parameter observations.

[0066] Optionally, based on the range of values ​​to which the distance value belongs, a target jitter elimination algorithm corresponding to the range of values ​​is selected to correct the motion parameter observation.

[0067] For example, when H is greater than the threshold h, it is determined to be a fast motion state and a first-order filter is applied; otherwise, it is determined to be a second-order filter.

[0068] It should be noted that those skilled in the art will understand that the range of H values ​​can be further divided into multiple stages, for example, into several more refined value intervals corresponding to low-low speed, medium-low speed, high-low speed, medium speed, high-low speed, medium-high speed, high-high speed, etc. Each value interval is configured with a corresponding target jitter elimination algorithm to correct the motion parameter observations, so as to make the jitter elimination algorithm more accurate.

[0069] In one possible implementation, the motion parameter observations are joint angles. Figure 4 This is a schematic diagram illustrating the principle of the motion capture-based data processing method provided in the embodiments of this disclosure, as shown below. Figure 4 As shown, the observed joint angles are input into the kinematic model to obtain the joint position coordinates. For target joints belonging to the same kinematic chain, first-order and second-order filtering methods are used to filter the position coordinates of the target joints respectively. Based on the difference between the first-order and second-order filtering of each target joint in the same kinematic chain, it is determined whether the motion capture object's motion state is fast or slow.

[0070] In fast-moving situations, a jitter reduction algorithm is needed that minimizes jitter without affecting the rapid movements, thus achieving... Figure 4 As shown, a first-order filtering algorithm is used to filter the position coordinates of the target joint during rapid motion to eliminate the impact of jitter without affecting the rapid movement.

[0071] In slow motion, a jitter reduction algorithm is needed that effectively eliminates jitter but negatively impacts fast movements. Therefore, a solution is required... Figure 4 As shown, in slow motion, a second-order filtering algorithm is used to filter the position coordinates of the target joint to eliminate the effect of jitter without worrying about its impact on fast movements.

[0072] In this embodiment, after obtaining the motion parameter observations of the joints of the motion capture object through observation, the motion parameter observations of each joint are processed using a first jitter elimination algorithm to obtain a first reference value, and a second jitter elimination algorithm to obtain a second reference value. Based on the difference between the first reference value and the second reference value, a target jitter elimination algorithm matching the difference is selected to correct the motion parameter observations. The difference between the results of the two different jitter elimination algorithms reflects the speed state of the motion capture object. Therefore, based on different speed states of motion, a matching target jitter elimination algorithm is selected to correct the motion parameter observations, thereby improving the motion reproduction accuracy of the motion capture object.

[0073] Figure 5 A flowchart illustrating another motion capture-based data processing method provided in this disclosure embodiment is shown below. Figure 5 As shown, it includes:

[0074] Step 501: Observe the motion capture object to obtain the motion parameter observations of the joints of the motion capture object.

[0075] Step 502: The motion parameter observations of the joint are processed using a first jitter elimination algorithm to obtain a first reference value, and processed using a second jitter elimination algorithm to obtain a second reference value.

[0076] The smoothing strength of the second jitter elimination algorithm is higher than that of the first jitter elimination algorithm.

[0077] Steps 501 and 502 can be specifically referred to as steps 101 and 102 in the foregoing embodiments. They will not be repeated in this embodiment.

[0078] Step 503: When there are multiple joints for which the observed motion parameters are obtained, the difference is compared with a threshold for at least two target joints belonging to the same joint kinematic chain among the multiple joints.

[0079] Furthermore, based on the correspondence between joint kinematic chains and thresholds, the threshold corresponding to the joint kinematic chain to which each target joint belongs is determined.

[0080] Furthermore, based on the parent-child relationships among the multiple joints, at least two target joints belonging to the same joint kinematic chain are identified. For example... Figure 2 As shown, joints connected by the same line are target joints belonging to the same kinematic chain. During movement, these joints have some interrelationships, meaning that the angles or positions between the joints affect each other. Therefore, identifying whether a target joint is in a fast or slow state based on its kinematic chain will yield more accurate results and improve recognition accuracy.

[0081] Step 504: If at least a number of target joints in at least two target joints satisfy the difference being greater than the threshold, select the first jitter elimination algorithm as the target jitter elimination algorithm, and use the first reference value of each target joint in the joint kinematic chain to correct the motion parameter observation.

[0082] Step 505: If fewer than the set number of target joints in at least two of the target joints satisfy the condition that the difference is greater than the threshold, select the second jitter elimination algorithm as the target jitter elimination algorithm to correct the motion parameter observation using the second reference value of each target joint in the joint kinematic chain.

[0083] Based on the relationship between the difference and the threshold, the motion parameter observation is corrected by using the second reference value or the first reference value of each target joint in the joint kinematic chain. While improving the motion reproduction of the motion capture object, the amount of computation is also less. Only two jitter elimination algorithms need to be calculated and the differences need to be compared, and the appropriate algorithm can be selected for jitter elimination.

[0084] In this embodiment, after obtaining the motion parameter observations of the joints of the motion capture object through observation, the motion parameter observations of each joint are processed using a first jitter elimination algorithm to obtain a first reference value, and a second jitter elimination algorithm to obtain a second reference value. Based on the difference between the first reference value and the second reference value, a target jitter elimination algorithm matching the difference is selected to correct the motion parameter observations. The difference between the results of the two different jitter elimination algorithms reflects the speed state of the motion capture object. Therefore, based on different speed states of motion, a matching target jitter elimination algorithm is selected to correct the motion parameter observations, thereby improving the motion reproduction accuracy of the motion capture object.

[0085] Figure 6A schematic diagram of the structure of a motion capture-based data processing device 600 provided in an embodiment of this disclosure is shown below. Figure 6 As shown, it includes: observation module 601, processing module 602, and correction module 603.

[0086] The observation module 601 is used to observe the motion capture object in order to obtain the motion parameters of the joints of the motion capture object.

[0087] The processing module 602 is used to process the observed motion parameters of the joint using a first jitter elimination algorithm to obtain a first reference value, and to process them using a second jitter elimination algorithm to obtain a second reference value; wherein the smoothing intensity of the second jitter elimination algorithm is different from that of the first jitter elimination algorithm.

[0088] The correction module 603 is used to correct the motion parameter observation by selecting a target jitter elimination algorithm that matches the difference between the first reference value and the second reference value.

[0089] As one possible implementation, the correction module 603 includes:

[0090] The generation unit is configured to generate a first matrix and a second matrix based on at least two target joints belonging to the same joint kinematic chain when multiple joints have been observed to obtain the kinematic parameter observations. In the first matrix, each row vector corresponds to a target joint and is used to indicate the first reference value of the corresponding target joint in three-dimensional space. In the second matrix, each row vector corresponds to the target joint and is used to indicate the second reference value of the corresponding target joint in three-dimensional space.

[0091] The selection unit is used to select the target jitter elimination algorithm to correct the motion parameter observations based on the distance between the first matrix and the second matrix.

[0092] Optionally, the selection unit is configured to: generate a third matrix based on the at least two target joints, wherein each row vector in the third matrix corresponds to a target joint and is used to indicate the motion parameter observation of the corresponding target joint in the three-dimensional space; perform a weighted summation of the distance between the first matrix and the second matrix and the distance between the third matrix and the second matrix to obtain a distance value; and select a matching target jitter elimination algorithm to correct the motion parameter observation based on the distance value.

[0093] The selection unit is used to: select the target jitter elimination algorithm corresponding to the value range to correct the motion parameter observation based on the value range to which the distance value belongs.

[0094] As another possible implementation, the smoothing strength of the second jitter elimination algorithm is higher than that of the first jitter elimination algorithm. Based on this, the correction module 603 is used to:

[0095] When there are multiple joints from which the observed motion parameters are obtained, the difference is compared with a threshold for at least two target joints belonging to the same joint kinematic chain among the multiple joints.

[0096] If at least a number of target joints in at least two target joints satisfy the difference being greater than the threshold, the first jitter elimination algorithm is selected as the target jitter elimination algorithm, so as to correct the motion parameter observation using the first reference value of each target joint in the joint kinematic chain;

[0097] If fewer than the set number of target joints in at least two of the target joints satisfy the condition that the difference is greater than the threshold, the second jitter elimination algorithm is selected as the target jitter elimination algorithm, so as to correct the motion parameter observation using the second reference value of each target joint in the joint kinematic chain.

[0098] Optionally, the correction module is further configured to: determine the threshold corresponding to the joint kinematic chain to which each target joint belongs based on the correspondence between the joint kinematic chain and the threshold.

[0099] Optionally, the motion capture-based data processing device 600 further includes a determination module for determining at least two target joints belonging to the same joint kinematic chain based on the parent-child relationship between the plurality of joints.

[0100] In some possible embodiments, the observation module 601 is used for:

[0101] The joint angles of the motion capture object are observed to obtain the joint angles of each joint;

[0102] A kinematic model is used to identify the relative positions of the joints based on the joint angles of each joint, so that the relative positions and the joint angles are used as the kinematic parameter observations.

[0103] Alternatively, in some other possible embodiments, the observation module 601 is used for:

[0104] The joint positions of the motion capture object are observed to obtain the joint position coordinates of each joint;

[0105] Based on the joint position coordinates of each joint, the joint angle of each joint is determined;

[0106] The position coordinates and the joint angles are used as the motion parameter observations.

[0107] In this embodiment, after obtaining the motion parameter observations of the joints of the motion capture object through observation, the motion parameter observations of each joint are processed using a first jitter elimination algorithm to obtain a first reference value, and a second jitter elimination algorithm to obtain a second reference value. Based on the difference between the first reference value and the second reference value, a target jitter elimination algorithm matching the difference is selected to correct the motion parameter observations. The difference between the results of the two different jitter elimination algorithms reflects the speed state of the motion capture object. Therefore, based on different speed states of motion, a matching target jitter elimination algorithm is selected to correct the motion parameter observations, thereby improving the motion reproduction accuracy of the motion capture object.

[0108] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0109] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0110] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in ROM (Read-Only Memory) 702 or a computer program loaded from storage unit 708 into RAM (Random Access Memory) 703. RAM 703 can also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. I / O (Input / Output) interface 705 is also connected to bus 704.

[0111] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0112] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above. For example, in some embodiments, the motion capture-based data processing method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform motion capture-based data processing methods by any other suitable means (e.g., by means of firmware).

[0113] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0114] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0115] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (CompactDisc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0116] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0117] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.

[0118] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0119] It's important to note that artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0120] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0121] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A data processing method based on motion capture, comprising: The motion capture object is observed to obtain the motion parameters of the joints of the motion capture object; The observed motion parameters of the joint are processed using a first jitter elimination algorithm to obtain a first reference value, and processed using a second jitter elimination algorithm to obtain a second reference value; wherein, the smoothing intensity of the second jitter elimination algorithm is different from that of the first jitter elimination algorithm, the first jitter elimination algorithm is a jitter elimination algorithm based on first-order filtering, and the second jitter elimination algorithm is a jitter elimination algorithm based on second-order filtering. Based on the difference between the first reference value and the second reference value, a target jitter elimination algorithm that matches the difference is selected to correct the motion parameter observation; The step of selecting a target jitter elimination algorithm that matches the difference between the first reference value and the second reference value to correct the motion parameter observation includes: When multiple joints are observed to have the observed motion parameters, a first matrix, a second matrix, and a third matrix are generated based on at least two target joints belonging to the same joint kinematic chain among the multiple joints. In the first matrix, each row vector corresponds to a target joint and is used to indicate the first reference value of the corresponding target joint in three-dimensional space. In the second matrix, each row vector corresponds to the target joint and is used to indicate the second reference value of the corresponding target joint in three-dimensional space. In the third matrix, each row vector corresponds to a target joint and is used to indicate the observed motion parameters of the corresponding target joint in three-dimensional space. The distance between the first matrix and the second matrix, and the distance between the third matrix and the second matrix are weighted and summed to obtain the distance value; If the distance value is greater than the threshold, the first jitter elimination algorithm is selected; otherwise, the second jitter elimination algorithm is selected.

2. The method according to claim 1, wherein, The step of selecting a matching target jitter elimination algorithm based on the distance value to correct the motion parameter observations includes: Based on the range of the distance value, the target jitter elimination algorithm corresponding to the range of the distance value is selected to correct the motion parameter observation.

3. The method according to claim 1, wherein, The method further includes: Based on the parent-child relationships among the multiple joints, at least two target joints belonging to the same joint kinematic chain are identified.

4. The method according to any one of claims 1-2, wherein, The observation of the motion capture object to obtain the motion parameter observations of the joints of the motion capture object includes: The joint angles of the motion capture object are observed to obtain the joint angles of each joint; A kinematic model is used to identify the relative positions of the joints based on the joint angles of each joint, so that the relative positions and the joint angles are used as the kinematic parameter observations.

5. The method according to any one of claims 1-2, wherein, The observation of the motion capture object to obtain the motion parameter observations of the joints of the motion capture object includes: The joint positions of the motion capture object are observed to obtain the joint position coordinates of each joint; Based on the joint position coordinates of each joint, the joint angle of each joint is determined; The position coordinates and the joint angles are used as the motion parameter observations.

6. A data processing device based on motion capture, comprising: The observation module is used to observe the motion capture object in order to obtain the motion parameters of the joints of the motion capture object. The processing module is used to process the observed motion parameters of the joint using a first jitter elimination algorithm to obtain a first reference value, and to process them using a second jitter elimination algorithm to obtain a second reference value; wherein, the smoothing intensity of the second jitter elimination algorithm is different from that of the first jitter elimination algorithm, the first jitter elimination algorithm is a jitter elimination algorithm based on first-order filtering, and the second jitter elimination algorithm is a jitter elimination algorithm based on second-order filtering. The correction module is used to select a target jitter elimination algorithm that matches the difference between the first reference value and the second reference value to correct the motion parameter observation. The correction module includes: A generation unit is configured to generate a first matrix and a second matrix based on at least two target joints belonging to the same joint kinematic chain among the multiple joints when multiple joints have been observed to obtain the kinematic parameter observations. In the first matrix, each row vector corresponds to a target joint and is used to indicate the first reference value of the corresponding target joint in three-dimensional space. In the second matrix, each row vector corresponds to the target joint and is used to indicate the second reference value of the corresponding target joint in three-dimensional space. A selection unit is used to generate a third matrix based on the at least two target joints, wherein each row vector in the third matrix corresponds to a target joint and is used to indicate the motion parameter observation of the corresponding target joint in the three-dimensional space; the distance between the first matrix and the second matrix and the distance between the third matrix and the second matrix are weighted and summed to obtain a distance value; if the distance value is greater than a threshold, the first jitter elimination algorithm is selected, otherwise the second jitter elimination algorithm is selected.

7. The apparatus according to claim 6, wherein, The selection unit is used for: Based on the range of the distance value, the target jitter elimination algorithm corresponding to the range of the distance value is selected to correct the motion parameter observation.

8. The apparatus according to claim 6, wherein, The device further includes: The determination module is used to determine at least two target joints belonging to the same joint kinematic chain based on the parent-child relationship between the plurality of joints.

9. The apparatus according to any one of claims 6-7, wherein, The observation module is used for: The joint angles of the motion capture object are observed to obtain the joint angles of each joint; A kinematic model is used to identify the relative positions of the joints based on the joint angles of each joint, so that the relative positions and the joint angles are used as the kinematic parameter observations.

10. The apparatus according to any one of claims 6-7, wherein, The observation module is used for: The joint positions of the motion capture object are observed to obtain the joint position coordinates of each joint; Based on the joint position coordinates of each joint, the joint angle of each joint is determined; The position coordinates and the joint angles are used as the motion parameter observations.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Images