A method, device and storage medium for predicting the position of a rigid body marker point

Abstract

The present invention discloses a method for predicting the position of a rigid body marker point, by obtaining a tracking mark data set and a position data set of a marker point on a human limb in each frame, and when there are more than 3 marker points on a human limb and at least 2 marker points can be correctly tracked, if the limb P s The i-frame position data of the 0th marker point on the image is #imgabs0#trackable / visible, the i-frame position data of the 1st marker point is #imgabs1#to be predicted / invisible, and the i-frame position data of the 2nd marker point is #imgabs2#trackable / visible. It is judged whether the marker point meets the preset conditions. If not, the prediction of the i-frame position data of the to-be-predicted / invisible marker point fails; if yes, the i-frame position data of the first to-be-predicted / invisible marker point is predicted according to the 0th and 2nd trackable / visible marker point position data and the scale factor, thereby successfully predicting the lost position data. This method has anti-occlusion performance and can improve the tracking success rate of the marker point.

Description

Technical Field

[0001] The present invention relates to the technical field of motion capture, and in particular to a method, a device and a storage medium for predicting the position of a rigid body marker point. Background Art

[0002] Optical motion capture systems can be divided into two categories: with markers and without markers. Motion capture systems with markers are widely used due to their strong real-time performance, high precision, and strong safety. In current optical motion capture systems, human motion capture is generally performed by tracking and calculating body parts through rigid body markers worn on the human body, thereby inferring information such as human body movements. Therefore, the accuracy of marker tracking and positioning is very critical.

[0003] However, the original markers collected by this type of system are scattered point clouds, and there is no fixed order between the markers. In addition, the missing and noisy markers may appear during the movement of the human body due to occlusion and sliding. Real-time matching of markers during motion capture is a key technology for most optical human motion capture, which directly determines the accuracy of motion capture. However, due to various factors in the motion capture environment, markers are often occluded or easily lost due to intense movement, resulting in marker tracking or matching failures. Therefore, it is necessary to propose a method to avoid occlusion or loss of markers to improve the tracking success rate of markers. Summary of the invention

[0004] In order to solve the above technical problems, the present application proposes a method for predicting the position of rigid body marker points to avoid the problem of failure in tracking the position of marker points due to occlusion or loss of rigid body marker points in the case of multi-rigid body and multi-person motion capture.

[0005] According to the first aspect, an embodiment provides a method for predicting the position of a rigid body marker point, comprising the following steps:

[0006] S1: Get the tracking mark data set of the marked points on the human limbs in each frame and location data collection Among them, i is the frame number, s is the limb part;

[0007] S2: When there are more than 3 markers on a human limb and at least 2 markers can be tracked correctly, if the limb P s The position data of the 0th mark point in the i-th frame Trackable / visible, position data of the first marker in the i-th frame To be predicted / invisible, position data of the second marker point in the i-th frame Trackable / visible, determine whether the marker point meets the first condition, if so, proceed to step S3, if not, the prediction of the i-th frame position data of the marker point to be predicted / invisible fails;

[0008] S3: Calculate the distance between the 0th and 2nd trackable / visible markers at the i-2th frame and the i-1th frame, and determine whether the distance meets the second condition. If so, proceed to step S4. If not, the prediction of the i-th frame position data of the to-be-predicted / invisible marker fails.

[0009] S4: Calculate the scale factor of the trackable / visible marker at the i-th frame and the i-1-th frame, and determine whether the scale factor is within the threshold range. If so, proceed to step S5. If not, the prediction of the i-th frame position data of the to-be-predicted / invisible marker fails.

[0010] S5: Predict the position data of the first to-be-predicted / invisible marker point according to the position data of the 0th and 2nd traceable / visible marker points and the scale factor respectively. If the predicted position data satisfies the third condition, the prediction is successful.

[0011] The first condition includes:

[0012] For the 0th and 2nd trackable / visible markers, the tracking marker data at frame i-2 and Tracking mark data at frame i-1 and Both need to be true, and, for the first unpredicted / invisible marker point, the tracking marker data at the i-1th frame Must be true.

[0013] The second condition includes:

[0014] in, is the distance between the second trackable / visible marker at frame i-2 and frame i-1, is the distance between the 0th trackable / visible marker point at the i-2th frame and the i-1th frame, and d is a set threshold.

[0015] The step of determining whether the scale factor is within a threshold range includes: the calculation formula of the scale factor is:

[0016]

[0017] in, It must be within a set threshold range.

[0018] The method of predicting the position data of the first to-be-predicted / invisible marker point according to the position data of the 0th and the 2nd traceable / visible marker points and the scale factor comprises:

[0019] Combine the rotation transformation matrix R and position data between the vectors composed of the 0th and 2nd trackable / visible markers at the i-2th frame and the i-1th frame and and the scale factor c, the position data of the first to-be-predicted / invisible marker point can be predicted.

[0020] The rotation transformation matrix R is calculated by the following formula:

[0021]

[0022] Among them, the rotation angle Rotation axis Normalization of vectors The unit vector of the rotation axis n=(nx,ny,nz), the vector or vector or

[0023] The predicted position data of the to-be-predicted / invisible marker point in the 1st i-th frame is: in,

[0024] The third condition includes: combining the movement speed of the marker point in the i-1th frame and the i-2th frame to calculate the target value of the position data of the to-be-predicted / invisible marker point in the first i-th frame: but It must be less than a set threshold dt.

[0025] According to the second aspect, an embodiment provides a device for predicting the position of a rigid body marker point, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected via lines; the at least one processor calls the instructions in the memory so that the device for predicting the position of a rigid body marker point executes the method described in the first aspect.

[0026] According to a third aspect, an embodiment provides a computer-readable storage medium, comprising a program, wherein the program can be executed by a processor to implement the method described in the first aspect above.

[0027] The beneficial effects of this application are:

[0028] According to a method for predicting the position of a rigid body marker point in the above embodiment, by obtaining a tracking mark data set and a position data set of the marker points on the human limbs in each frame, and when there are more than 3 marker points on the human limbs and at least 2 marker points can be correctly tracked, if the limb P s The position data of the 0th mark point in the i-th frame Trackable / visible, position data of the first marker in the i-th frame To be predicted / invisible, position data of the second marker point in the i-th frame Trackable / visible, determine whether the marker point meets the preset conditions. If not, the prediction of the i-th frame position data of the predicted / invisible marker point fails; if it meets the conditions, the i-th frame position data of the first predicted / invisible marker point is predicted according to the 0th and 2nd trackable / visible marker point position data and the scale factor, thereby successfully predicting the lost position data. This method has anti-occlusion performance and can improve the tracking success rate of the marker point. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the rigid body structure;

[0030] Figure 2 A flowchart of a method for predicting the position of a rigid body marker point;

[0031] Figure 3 Schematic diagram of the device for predicting the position of rigid body marker points. DETAILED DESCRIPTION

[0032] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, and this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0033] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0034] The serial numbers of the components in this document, such as "second", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0035] The inventive concept of the present application is that in an optical motion capture system, multiple rigid bodies with fixed shapes are required for position tracking, and problems such as similarity between rigid bodies and occlusion of rigid bodies need to be avoided as much as possible. The present application further screens the marker points installed on the rigid body, thereby screening out suitable marker points that can improve the accuracy of the motion capture system. In this way, subsequent matching calculations can be performed based on the screened marker points to identify the corresponding rigid body.

[0036] Embodiment 1:

[0037] Please refer to Figure 1 The present application discloses a structural schematic diagram of a rigid body, which includes: a rigid body base 110, a support rod 120, a marking point 130, etc. There are at least 3 marking points to ensure the accuracy of motion capture calculation, and the number of marking points is the same as the number of support rods. In order to facilitate the motion capture system to distinguish different marking points when calculating the coordinates of the marking points, the distances between any two marking points (such as M1, M2, M3) are as different as possible, that is, similar triangles are avoided as much as possible to ensure that the motion capture system can effectively identify the rigid body.

[0038] Embodiment 2:

[0039] Please refer to Figure 2 , after understanding Figure 1 Based on the rigid body structure shown, this embodiment discloses a method for predicting the position information of a rigid body marker point. The method for which protection is sought includes steps S210-S250, which will be described separately below.

[0040] Step S210, obtaining a tracking mark data set of the marking points on the human limbs in each frame and location data collection Among them, i is the frame number, s is the limb part; Represents the position data of the i-th frame of the marker point on the human limb, represents the tracking mark data of the i-th frame of the marker point on the human limb, and There are only two results: true and false. True means that the marker point tracking is successful, and false means that the marker point tracking fails, that is, the marker point is lost.

[0041] Step S220: When there are more than 3 markers on a human limb and at least 2 markers can be correctly tracked, if the limb P s The position data of the 0th mark point in the i-th frame Trackable / visible, position data of the first marker in the i-th frame To be predicted / invisible, position data of the second marker point in the i-th frame Trackable / visible, determine whether these markers meet the first condition, if so, proceed to step S230, if not, the prediction of the i-th frame position data of the marker to be predicted / invisible fails;

[0042] Specifically, the first condition includes:

[0043] For the 0th and 2nd trackable / visible markers, the tracking marker data at frame i-2 and Tracking mark data at frame i-1 and Both need to be true, that is, the 0th and 2nd trackable / visible markers can be tracked successfully at the i-2th and i-1th frames, and for the 1st unpredicted / invisible marker, the tracking marker data at the i-1th frame Must be true, that is, the first unpredicted / invisible marker that failed to be tracked in the i-th frame can successfully track the corresponding position data in the i-1th frame If these conditions are met, proceed to step S230 to continue prediction.

[0044] Step S230, calculating the distance between the 0th and 2nd trackable / visible markers at the i-2th frame and the i-1th frame, and determining whether the distance meets the second condition, if yes, proceeding to step S240, if no, the prediction of the i-th frame position data of the to-be-predicted / invisible marker fails;

[0045] Specifically, the second condition includes:

[0046] in, is the distance between the second trackable / visible marker at frame i-2 and frame i-1, is the distance between the 0th trackable / visible marker point at the i-2th frame and the i-1th frame, and d is a set threshold, generally the diameter of the marker point.

[0047] For two trackable / visible marker points, the distance between them in the i-2th frame and the i-1th frame cannot be too small at the same time to prevent extreme situations. For example, when the untraceable marker point rotates around two stationary trackable marker points, the probability of failure of the algorithm predicting the marker point position will be very high. If the above-mentioned second condition is not met at the same time, the prediction calculation fails, otherwise enters step S240 to continue the prediction.

[0048] Step S240, calculating the scale factor of the trackable / visible marker at the i-th frame and the i-1-th frame, and determining whether the scale factor is within the threshold range, if so, proceeding to step S5, if not, the prediction of the i-th frame position data of the to-be-predicted / invisible marker fails;

[0049] The calculation formula of the scale factor is:

[0050]

[0051] in, It cannot be too small, for example, it cannot be less than 0.02, that is, it must be within a set threshold range, so that the scale factor c calculated thereby must also be within the preset threshold range, otherwise the calculation is terminated, and if c is within the preset threshold range, step S250 is entered to continue the prediction.

[0052] Step S250, predicting the position data of the first to-be-predicted / invisible marker point according to the position data of the 0th and 2nd traceable / visible marker points and the scale factor, if the predicted position data satisfies the third condition, the prediction is successful.

[0053] Predicting the position data of the first to-be-predicted / invisible marker point according to the position data of the 0th and 2nd trackable / visible marker points and the scale factor specifically includes:

[0054] Combine the rotation transformation matrix R and position data between the vectors composed of the 0th and 2nd trackable / visible markers at the i-2th frame and the i-1th frame and and the scale factor c, the position data of the first to-be-predicted / invisible marker point can be predicted.

[0055] Then the rotation transformation matrix R can be calculated by the following formula:

[0056]

[0057] Among them, the rotation angle Rotation axis Normalization of vectors The unit vector of the rotation axis n=(nx,ny,nz), the vector or vector or That is, different vectors v1 and v2 can be obtained through the 0th and 2nd traceable / visible marker points, respectively, thereby obtaining two rotation transformation matrices R and R′. It should be noted that the length of any vector in the calculation process cannot be too small, otherwise the prediction calculation is considered to have failed.

[0058] Passing the 0th mark The missing position data of the first marker point in the i-th frame is calculated as:

[0059]

[0060] Also pass the second mark point The other value of the missing position data of the first marker in the i-th frame is calculated as:

[0061]

[0062] Then the final predicted position data of the to-be-predicted / invisible marker point in the 1st i-th frame is:

[0063] In one embodiment, to further determine The correctness of the third condition is also set, including: The corresponding target value is The target value of the position data of the to-be-predicted / invisible marker point in the first i-th frame can be calculated by combining the movement speed of the marker point in the i-1th frame and the i-2th frame: but It must be less than a set threshold dt, otherwise the prediction calculation will fail.

[0064] If there is no prediction calculation failure in the above steps S220-S250, the prediction calculation of the marker point to be predicted is successful. In addition to the position data of the 1st i-th frame to be predicted / invisible marker point listed above, which can be predicted using the above method, the position data of other untraceable marker points on the limbs can be predicted by referring to the above method.

[0065] Embodiment three:

[0066] Please refer to Figure 3 ,

[0067] Furthermore, the present application also provides a detailed description of the device for predicting the position of the rigid body marker point in this embodiment from the perspective of hardware processing.

[0068] Figure 3 is a schematic diagram of the structure of a device for predicting the position of a rigid body marker point provided in this embodiment. The device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 510 (for example, one or more processors) and a memory 520, and one or more storage media 530 (for example, one or more mass storage devices) storing application programs 533 or data 532. Among them, the memory 520 and the storage medium 530 can be short-term storage or permanent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the device 500. Furthermore, the processor 510 can be configured to communicate with the storage medium 530 and execute a series of instruction operations in the storage medium 530 on the device 500.

[0069] The device 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and / or one or more operating systems 531, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 3 The structure of the device 500 shown does not constitute a limitation on the device for predicting the position of a rigid body marker point provided in the present application, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0070] The present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the above-mentioned method for predicting the position of a rigid body marker point.

[0071] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0072] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art, according to the concept of the present invention, some simple deductions, modifications or substitutions can be made.

Claims

1. A method for predicting the position of a rigid body marker, characterized in that: The following steps are involved: S1: Get the tracking mark data set of the marked points on the human limbs in each frame and location data collection Among them, i is the frame number, s is the limb part; S2: When there are more than 3 markers on a human limb and at least 2 markers can be tracked correctly, if the limb P s The position data of the 0th mark point in the i-th frame Trackable / visible, position data of the first marker in the i-th frame To be predicted / invisible or the position data of the second marker in the i-th frame If it is trackable / visible, it is determined whether the marker point meets the first condition. If so, it proceeds to step S3. If not, the prediction of the i-th frame position data of the marker point to be predicted / invisible fails. S3: Calculate the distance between the 0th and 2nd trackable / visible markers at the i-2th frame and the i-1th frame, and determine whether the distance meets the second condition. If so, proceed to step S4. If not, the prediction of the i-th frame position data of the to-be-predicted / invisible marker fails. S4: Calculate the scale factor of the trackable / visible marker at the i-th frame and the i-1-th frame, and determine whether the scale factor is within the threshold range. If so, proceed to step S5. If not, the prediction of the i-th frame position data of the to-be-predicted / invisible marker fails. S5: predicting the position data of the first to-be-predicted / invisible marker point according to the position data of the 0th and the 2nd traceable / visible marker points and the scale factor respectively. If the predicted position data satisfies the third condition, the prediction is successful. The first condition includes: For the 0th and 2nd trackable / visible markers, the tracking marker data at frame i-2 and Tracking mark data at frame i-1 and Both need to be true, and, for the first unpredicted / invisible marker point, the tracking marker data at the i-1th frame It must be true, which means the marker tracking is successful. The second condition includes: in, is the distance between the second trackable / visible marker at frame i-2 and frame i-1, is the distance between the 0th trackable / visible marker point at the i-2th frame and the i-1th frame, and d is a set threshold; The step of determining whether the scale factor is within a threshold range includes: the calculation formula of the scale factor is: in, It must be within a set threshold range. Indicates the position data of the second mark point in the i-th frame. Indicates the position data of the 0th mark point in the i-th frame, Indicates the position data of the second marker point in the i-1th frame. Indicates the position data of the 0th mark point in the i-1th frame, || || indicates a mathematical operation symbol; The third condition includes: combining the movement speed of the marker point in the i-1th frame and the i-2th frame to calculate the target value of the position data of the to-be-predicted / invisible marker point in the first i-th frame: but It must be less than a set threshold dt.

2. The method for predicting the position of a rigid body marker point according to claim 1, wherein: The method of predicting the position data of the first to-be-predicted / invisible marker point according to the position data of the 0th and the 2nd traceable / visible marker points and the scale factor comprises: Combine the rotation transformation matrix R and position data between the vectors composed of the 0th and 2nd trackable / visible markers at the i-2th frame and the i-1th frame and and the scale factor c, the position data of the first to-be-predicted / invisible marker point can be predicted.

3. The method for predicting the position of a rigid body marker point as claimed in claim 2, characterized in that: The rotation transformation matrix R is calculated by the following formula: Among them, the rotation angle The unit vector of the rotation axis n=(n x ,n y ,n z ),vector or vector or Different vectors v1 and v2 can be obtained by the 0th and 2nd trackable / visible marker points respectively.

4. The method for predicting the position of a rigid body marker point as claimed in claim 3, characterized in that: The predicted position data of the to-be-predicted / invisible marker point in the 1st i-th frame is: Among them, the position data of the first marker point missing in the i-th frame The missing position data of the first marker in the i-th frame is another value 5. A device for predicting the position of a rigid body marker, characterized in that: include: A memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected via a line; The at least one processor calls the instructions in the memory to enable the device for predicting the position of rigid body marker points to execute the method according to any one of claims 1-4.

6. A computer-readable storage medium, characterized in that: The method comprises a program which can be executed by a processor to implement the method according to any one of claims 1 to 4.

Images