Method for positioning the center of rotation of an articulating joint

Abstract

The invention relates to a method for locating the center of rotation (Pc') of an articulation joint connecting two bones or bone sets of the upper or lower limbs of a user (USR), comprising the following steps: a) performing by the user (USR) a series of repetitive movements of one of the bones or bone sets around the joint and simultaneously acquiring the 3D positions of the bone or bone set during the series, thus obtaining a 3D point cloud (P); b) calculating in a 3D search space comprising the 3D positions of the end of the other bone or bone set a point called center point (Pc) which is the search point of the 3D search space for which the standard deviation is lowest when considering the set of distances between the search point and each point of the 3D point cloud; c) converting the 3D point cloud (P) to a plane; d) projecting the center point onto said plane, thus obtaining the center of rotation (Pc') of the joint. The invention also relates to a method for estimating the dimensions of the upper limbs of a user.

Description

Technical Field

[0001] This invention relates to the field of computer programs and systems, and more particularly to the field of digital human body modeling, product review, ergonomic analysis and verification in immersive environments. Background Technology

[0002] In immersive environments, users interact with the 3D scene through a graphical representation (often called an avatar). Whether in first-person visualization (where the user sees only their avatar's arms, not the entire avatar) or third-person visualization (where the user sees only their avatar), the length of the avatar's arms must be estimated as accurately as possible. In particular, ergonomic validation requires precision down to the millimeter for limb length. For example, in simulating a workstation with an emergency stop button, the user's ability to reach the button within the immersive environment is crucial for validating the workstation.

[0003] Therefore, experiences in a virtual immersive environment are only meaningful and realistic when the mapping from the real world to the virtual world is accurate. This aspect is also relevant in scenarios where avatars are used to evaluate object prototypes modeled using virtual reality devices. One of the fundamental requirements in this scenario is the dimensional consistency between the avatar and its real-world counterpart (a person). This can be achieved by modifying the avatar's dimensions using the dimensions of the person guiding the avatar.

[0004] However, modeling the human skeletal structure is complex. The human skeleton can be considered as a set of bones. Hinge joints can be defined as the relative interactions between some of these bones, including rolling, translational, and gliding movements. The behavior of these anatomical joints ranges from simple fixed-axis rotation to very complex multi-axis coupled motion variations. For example, consider the scapula-humerus rhythm, where the scapula and humerus move at a 1 / 2 ratio: when the arm is abducted 180 degrees, the scapula rotates 60 degrees, and the humerus rotates 120 degrees at the shoulder point. In this application, the behavior of the joints will be considered analogous to rotational axes. In fact, the purpose of this invention is not to determine the user's actual skeleton, but to model the length of the limb and the hinge points. Even in this simplified case, accurately capturing these movements and mapping them to a digital human body using a limited number of trackers is a difficult task.

[0005] On one hand, modeling of human anatomy can be performed by placing magnetic and / or optical markers in appropriate locations on the human body. The article “Automatic Joint Parameter Estimation from Magnetic MotionCapture Data” (James F. O'Brien et al., Proceedings of the Graphics Interface 2000 Conference, May 15-17, 2000) discloses a technique for determining joint parameters at the articulation level using magnetic motion capture data. This technique allows for the determination of limb lengths, joint positions, and sensor locations in a human subject without external measurements. According to Table 1 of the article, the disclosed technique is highly accurate: for the upper arm, the difference between the measured (with a ruler) value and the calculated value averages 2 / 3 millimeter. However, this technique requires wearing a motion capture harness equipped with sensors, which is useless for immersive experiments once modeling is complete. Furthermore, the accuracy of this solution is often linked to the accuracy of marker positioning; therefore, setup cannot be performed independently (it is usually done by experts), which is expensive and time-consuming.

[0006] On the other hand, bone length estimation using markerless tracking systems is less accurate compared to marker-based systems. Markerless systems can be found in consumer-grade virtual reality systems (e.g., for video game applications). In those systems, the user inputs their size, and the length of the upper limb is inferred based on statistical considerations. However, there is no true proportional relationship between limb size and length; a tall person may have short arms, and vice versa. Therefore, the technique is inaccurate.

[0007] Therefore, there is a need for a fast and accurate label-free method for modeling the upper or lower limbs of users in immersive environment applications, particularly for locating the rotation center of the articulated joints connecting the bones of the upper or lower limbs, and also for estimating the dimensions of the upper or lower limbs. Summary of the Invention

[0008] The present invention provides a method for locating the rotation center of a hinge joint connecting two bones or a group of bones in the upper or lower limb of a user, comprising the following steps:

[0009] a) A series of repetitive movements of a bone or set of bones sweeping around a joint are performed by the user, and the 3D position of the bone or set of bones is acquired simultaneously during the series, thereby obtaining a 3D point cloud.

[0010] b) Calculate a point called the center point in a 3D search space that includes the 3D location of the end of another bone or set of bones. The center point is a search point in the 3D search space that has the lowest standard deviation when considering the set of distances between the search point and every point in the 3D point cloud.

[0011] c) Convert 3D point clouds into planes;

[0012] d) Project the center point onto the plane to obtain the rotation center of the joint.

[0013] In a preferred embodiment:

[0014] -The method includes preparatory steps for providing two handheld trackers;

[0015] - In step a), a series of repetitive movements are performed while holding one hand tracker, referred to as the movement tracker, and another hand tracker, referred to as the reference tracker, is held close to the end of another bone or bone set.

[0016] In a preferred embodiment, step b) includes a sub-step of recursively subdividing the 3D search space through the following steps:

[0017] b1) Define the 3D search space as a cube with a predefined edge length, and center the cube on the 3D position of the reference tracker, which is called the reference point;

[0018] b2) Receive input from the user a set of points identifying the cube, which together with the reference points constitute a set of search points;

[0019] b3) For each search point, calculate the distance between the search point and every point in the 3D point cloud, and derive the standard deviation of each search point;

[0020] b4) In the set of search points, identify at least one search point with the lowest standard deviation;

[0021] b5) Furthermore, by halving the edge length of the cube and centering the cube around the identified search point, steps b1), b2), b3), and b4) are repeated until the edge length of the cube is less than or equal to a predefined threshold, and then the center point corresponds to the identified search point.

[0022] In a preferred embodiment, in step b2), the cube is subdivided into eight smaller cubes, and the set of search points includes a reference point or identified search point, the center of a small cube, the center of the square of two adjacent smaller cubes, and the midpoint of the edge of two adjacent smaller cubes.

[0023] In a preferred embodiment, step c) includes the following sub-steps:

[0024] c1) Calculate the covariance matrix of points in the 3D point cloud;

[0025] c2) Calculate the eigenvalues ​​and eigenvectors of the covariance matrix;

[0026] c3) Construct a rectangular bounding box that encloses the 3D point cloud. The orientation of the bounding box corresponds to the feature vector, and the dimension of the bounding box corresponds to the feature value.

[0027] c4) Calculate the geometric center of the bounding box;

[0028] c5) Construct a plane defined by two eigenvectors with the largest eigenvalues, and this plane passes through the geometric center of the bounding box.

[0029] In a preferred embodiment, in step a), the acquired 3D positions are stored in frames, each frame including a predefined number of 3D positions, and steps b), c), and d) are performed for all frames whenever a new frame has been acquired.

[0030] In a preferred embodiment, the method includes the following steps:

[0031] - For each implementation of steps a), b), c), and d), store the 3D position of the reference tracker.

[0032] - Calculate the average 3D position of the reference tracker, and

[0033] - If the difference between the 3D position of the reference tracker and the average value of the 3D positions stored in the reference tracker is greater than a predefined value, then the corresponding frame is discarded.

[0034] In a preferred embodiment, the method includes discarding a frame if it is determined in step c2) that there is no single feature value smaller than the other feature values.

[0035] In a preferred embodiment, the method includes the step of sending an indication of the minimum standard deviation to the user in real time.

[0036] The present invention also relates to a method for estimating the dimensions of a user's upper limbs, comprising the following steps:

[0037] S1: Calculate the length of the hand based on the distance between the position of the wrist rotation center calculated using the method according to any one of the preceding claims and the position of the origin of the motion tracker;

[0038] S2: Calculate the length of the forearm based on the distance between the position of the rotation center of the wrist calculated using the method according to any one of the preceding claims and the position of the rotation center of the elbow calculated using the method according to any one of the preceding claims;

[0039] S3: Calculate the length of the upper arm and forearm based on the distance between the position of the wrist rotation center calculated using the method according to any one of the preceding claims and the position of the shoulder rotation center calculated using the method according to any one of the preceding claims;

[0040] S4: When each of the reference tracker and the motion tracker is held in a different hand by the user and waved in front of the user with an extended arm, calculate the length between the left and right shoulders based on the length of the upper arm and forearm, the length of the hand, and the maximum distance between the reference tracker and the motion tracker.

[0041] The present invention also relates to a computer program product stored on a non-transitory computer-readable data storage medium, the non-transitory computer-readable data storage medium including computer-executable instructions for causing a computer system to perform the method according to any one of the preceding claims.

[0042] The present invention also relates to a non-transitory computer-readable data storage medium comprising computer-executable instructions for causing a computer system to perform the aforementioned methods.

[0043] The present invention also relates to a computer system comprising two handheld trackers, a processor coupled to a non-transitory memory, a screen, and the memory storing computer-executable instructions for causing the computer system to perform the aforementioned methods. Attached Figure Description

[0044] Additional features and advantages of the invention will become apparent from the following description taken in conjunction with the accompanying drawings, which illustrate:

[0045] - Figure 1 It is a top view showing the user performing a series of repetitive movements of his hand.

[0046] - Figure 2 It is a diagram of the acquired 3D point cloud, the calculated center point, and the calculated rotation center;

[0047] - Figure 3 This is another illustration of a 3D point cloud, center point, and center of rotation.

[0048] - Figure 4 It is a diagram of a cube including the search points;

[0049] - Figure 5AIt is a side view of the user performing a series of repetitive movements of sweeping his forearm;

[0050] - Figure 5B It is a side view of the user performing a series of repetitive movements of sweeping his arm;

[0051] - Figure 5C It is a front view of the user performing a series of repetitive sweeping movements of both arms;

[0052] - Figure 6 This is a flowchart of a method for estimating the dimensions of a user's upper limbs;

[0053] - Figure 7 It is a computer environment suitable for executing the method according to the present invention. Detailed Implementation

[0054] Figure 1 A top view shows a user performing a series of repetitive sweeping movements of his hand to determine the center of rotation of his wrist.

[0055] In the first step of the method of the present invention, a) the user USR performs a series of repetitive sweeping movements of a bone or bone set around a joint, and simultaneously acquires the 3D position of the bone or bone set during the series, thereby obtaining a 3D point cloud P. Figure 2 The obtained 3D point cloud is displayed in the image.

[0056] In a preferred embodiment, which is particularly well-suited for locating the center of rotation of the user's wrist, the user USR holds the tracker (MT, RT) in each hand.

[0057] The trackers (MT, RT) can be part of a virtual reality system (e.g., the "HTC Vive"™ virtual reality system). Such a system includes at least one handheld tracker (wireless handheld controller) and a virtual reality headset for displaying a 3D scene to the user in an immersive context. Since the claimed method is not directed at the immersive experiment itself, but rather at the pre-calibration of the user's avatar, the implementation of the method of the present invention does not require the use of a virtual reality headset. Furthermore, the present invention requires the use of two handheld trackers, i.e., one for the upper left / lower limb and one for the upper right / lower limb. Therefore, an expensive motion capture system is not required.

[0058] Trackers can be positioned in the coordinate system of the virtual environment in different ways: through cooperation with a base station, through surface detection, marking, environmental recognition, gyroscope control unit, and computer vision. For the method of this invention, utilizing one of these techniques, the location of the origin of each tracker is known.

[0059] If the 3D position can be determined, the tracker does not necessarily have to be used exclusively for virtual reality. The tracker can be used only for the calibration process.

[0060] In this embodiment, the handheld trackers (MT, RT) are identical. The tracker held by the hand, which makes the bone assembly of the hand sweep around the joints in a repetitive motion, is called the motion tracker MT, while the other tracker is called the reference tracker RT.

[0061] Once all the 3D positions of a bone or bone assembly have been obtained, an estimate of the rotation center position can be calculated. Alternatively, the estimate can be calculated whenever a predefined number of points (hereinafter referred to as frames) have been obtained. Therefore, real-time estimation of the rotation center position can be performed, depending on both the tracker hardware and the virtual reality software. For example, the HTC Vive. TM The refresh rate of handheld trackers in virtual reality systems can reach up to 90Hz. To enhance real-time visual rendering, the refresh rate can be set to a value much lower than 90Hz. In the claimed method, the 3D position of the motion tracker (MT) is stored in frames of a predefined length (e.g., 25 points), and the position of the rotation center is calculated first for the first frame, then for two first frames (e.g., 50 points), then for three first frames (e.g., 75 points), and so on. Therefore, the user perceives near real-time measurements, and the accuracy of the measurements increases over time.

[0062] To locate the center of wrist rotation, the user (USR) holds the motion tracker (MT) and the reference tracker (RT) in their hand. For example... Figure 1 As shown, while keeping the reference tracker RT stable and close to the wrist, the user repeatedly swings the motion tracker MT from left to right and in the opposite direction.

[0063] The reference tracker RT has three functions. First, as described below, the reference tracker RT provides an initial estimate of the position of the wrist's rotation center because the user is prompted to keep the reference tracker RT on the forearm close to the wrist (but not directly on the wrist, otherwise the measurement process would be noisy). Second, by keeping the reference tracker RT on the forearm (where the forearm is more stable), the accuracy of the measurement is increased. Third, by knowing the position of the reference tracker RT during the calibration process, only the relative movement of the moving tracker MT with respect to the reference tracker RT can be considered, compensating for the overall movement of the forearm. This increases the stability of the calibration process.

[0064] While the hand can have a wide angular displacement (approximately 130-140°) around the center of rotation of the wrist, it's best to avoid swinging the hand into extreme positions (e.g., articulated termination). This is important for the wrist because at the end of the movement, the user tends to "break" their wrist, i.e., fundamentally altering the plane of point P. This is also important for shoulder movement because if the shoulder movement is too wide, the scapula also moves, thus the movement tends to be very complex, implying multiple articulations. Users are advised against making too wide angular displacements. For example, an example is shown in a snapshot intended as a user-guided video tutorial. Figure 1 , Figure 5A , Figure 5B and Figure 5C In the diagram, the angular range corresponding to extreme postures is displayed differently from the range corresponding to desired postures (e.g., in different colors).

[0065] The user can be prompted to position their arm using visual cues on the screen, such as... Figure 1 As shown. Even for non-professional users, such movement is easy to achieve.

[0066] In the second step (b) of the claimed method, a point called the center point Pc is calculated in a 3D search space that includes the 3D location of the end of another bone or bone assembly. The center point Pc is the point in the search space that minimizes the standard deviation when taking into account the set of distances between the search point and every point in the 3D point cloud.

[0067] The 3D search space first includes the ends of another bone or bone set (according to...) Figure 1 For example, the 3D position of the forearm end near the wrist. Therefore, by stimulating the user to place the reference tracker RT near the articulation joint, stability can be increased (because there is less limb movement other than swinging bones or bone assemblies), and the calculation of the center point Pc starts from a point close to the actual center of rotation, thereby improving the speed of the method.

[0068] Then, in order to find an approximate center of the 3D point cloud P that looks like a bunch of arc points, the method starts with a reference point Porg and eventually finds a point near it that minimizes the standard deviation when taking into account the set of distances between the search point and each point in the 3D point cloud.

[0069] In a preferred embodiment, the center point Pc is calculated based on an octree search algorithm, wherein the 3D search space is recursively subdivided, which includes the following sub-steps of step b):

[0070] In the first sub-step b1), the 3D search space is defined as a cube CUB with a predefined edge length, and the cube CUB is centered at a reference point Porg corresponding to the 3D position of the reference tracker RT. Figure 4 A cube (CUB) is illustrated schematically. For example, the initial predefined edge length could be a few decimeters to accommodate all types of articulation points in the upper limb. In practice, the maximum distance between the center of rotation of the articulated joint and the motion tracker points to a measurement of the entire arm length, which is approximately fifty centimeters, naturally depending on the user. To measure the location of the center of rotation of the lower limb (ankle, knee, or hip), for the same reason, the initial predefined edge length could be one meter. The initial predefined edge length could, for example, be equal to 2... 9 =512mm, which is a good trade-off considering performance and accuracy.

[0071] Then, in the second sub-step b2), a set of points of the cube CUB is identified. This set of identified points, together with the reference point Porg, constitutes the set of search points Ps. The identified points, for example, form a network of points periodically arranged around the reference point Porg.

[0072] In a preferred embodiment, for the first iteration of step b), the cube CUB is subdivided into eight smaller cubes SCUB (voxels), and the set of search points Ps includes the following points:

[0073] - Reference point Porg, which is located at the center of cube CUB;

[0074] -Eight small cubes at their centers;

[0075] - The centers of the twelve squares of two adjacent smaller cubes SCUB; and

[0076] - The six midpoints of the edges of two adjacent smaller cubes SCUB.

[0077] Another option is to search for points. However, it was found that using these points provided excellent results in terms of computational speed.

[0078] Then, in the third sub-step b3), for each search point Ps of the cube CUB, the distance between the search point Ps and each point in the 3D point cloud P is calculated. It should be understood that when this method is implemented while the 3D points are being acquired, the distance between the search point Ps and the points in the 3D point cloud P is calculated only for the acquired 3D points.

[0079] Suppose that at time t, N points have been acquired, and there are Nps search points. For each search point Ps in the cube CUB, calculate N distances. For the set of these N distances, calculate the standard deviation. For the Nps search points Ps, calculate Nps standard deviations.

[0080] Then, in the fourth sub-step b4), at least one search point Ps with the lowest standard deviation is identified. The search point(s) with the lowest standard deviation is determined by defining an indicator that the distance between the search point and each point in the 3D point cloud P tends to be close to the average of that distance.

[0081] Then, steps b1), b2), b3), and b4) are repeated. Instead of centering the cube CUB around the reference point Porg (corresponding to the position of the reference tracker RT), the cube CUB is centered around the search point Ps (called the identified search point) with the lowest standard deviation. For each new iteration, the edge length of the cube CUB is halved.

[0082] In a preferred embodiment, in substep b4), two search points with the lowest standard deviation are identified, and iterations are performed in parallel for both points. Therefore, the number of identified search points doubles after each iteration. Defining more than 27 search points Ps can improve the accuracy of the method if only one search point is identified after each iteration.

[0083] In any case (whether one search point is identified in each iteration, or more than one search point is identified in each iteration (e.g., two search points are identified)), the iteration continues once the edge length of the cube is less than or equal to a predefined threshold, and a single search point is identified after each iteration. For example, the threshold could be 16 mm.

[0084] Then, if the edge length of the cube is less than or equal to another predefined threshold (e.g., 2 mm), the iteration stops. The center point Pc is located in the cube CUB whose edge length is less than or equal to 2 mm. Therefore, in this case, the resolution is equal to 1 mm (the search point of the last iteration is located at the center of the cube with an edge length of 2 mm), which has a high resolution compared to the aforementioned method.

[0085] The result of step b), and the result of the octree search in the preferred embodiment, is the center point Pc.

[0086] Alternatively, instead of using an octree search algorithm (which has logarithmic complexity) to compute the centroid Pc, a brute-force linear search (which has linear complexity) can be used to compute the centroid Pc.

[0087] Then, in step c), the 3D point cloud P is transformed into a plane. The goal of step c) is to map the 3D point cloud P into two dimensions, while losing too much information.

[0088] In a preferred embodiment, the 3D to 2D conversion is performed using a statistical process described below as “principal component analysis”.

[0089] c1) Calculate the covariance matrix of the points in the 3D point cloud (P);

[0090] c2) Calculate the eigenvalues ​​and eigenvectors of the covariance matrix;

[0091] c3) Construct a rectangular bounding box that encloses the 3D point cloud P. The orientation of the bounding box corresponds to the feature vector, and the dimension of the bounding box corresponds to the feature value.

[0092] c4) Calculate the geometric center of the bounding box;

[0093] c5) Construct a plane defined by the two eigenvectors with the largest eigenvalues, and this plane passes through the geometric center of the bounding box. Assume the user follows the process displayed on the screen and performs the sweeping movement correctly. The 3D point cloud P should be almost on the same plane and appear curved. Therefore, typically, one eigenvalue representing the thickness of the point cloud should be much smaller than the other two.

[0094] Once a series of repetitive movements have been completed, a plane can be constructed. In a preferred embodiment, the plane is constructed after acquiring frames of a predefined length, which contain all the acquired frames. Therefore, step c) is also performed near real-time, and the user can correct their movement if the stability of the plane's construction is insufficient. Specifically, if, during step c3), the three eigenvalues ​​are determined to be very similar, for example based on tolerance margins, a frame can be discarded. Thus, the 3D-to-2D conversion is accomplished with minimal information loss.

[0095] Alternatively, instead of using the "principal component analysis" method, a plane can be constructed first by calculating the rectangular bounding box that encloses the 3D point cloud. Then, a plane parallel to the two largest faces of the bounding box is calculated, and this plane divides the bounding box into two equal volumes.

[0096] In the final step d) of the claimed method, the center point Pc, calculated in step b), is projected onto the plane calculated in step c). The projection of the center point Pc corresponds to the position of the rotation center Pc' of the joint. In a particular embodiment of the invention where the user holds the motion tracker MT, the position of the rotation center Pc' refers to the 3D position relative to the origin of the motion tracker MT.

[0097] Figure 2 and Figure 3 The 3D point cloud P, center point Pc, and rotation center Pc' are shown.

[0098] In a preferred embodiment, steps a), b), c), and d) are iterated whenever a new frame (e.g., including 25 points) has been acquired. In other words, when a new frame has been acquired, the position of the rotation center Pc' is recalculated using all the 3D positions. The 3D positions of the reference tracker RT are stored in real time, and the average position of all stored positions of the reference tracker RT is calculated. If the difference between the 3D position of the reference tracker and the average of the stored 3D positions of the reference tracker is greater than a predefined value, the new incoming frame is discarded. Therefore, if the user has moved too much during a series of movements, the 3D position of the reference tracker RT will be unstable, and discarding the corresponding frame avoids calculating the rotation center Pc' for the user's unstable position.

[0099] It can calculate the distance d between the rotation center Pc' and every point in the 3D point cloud P. k Assume Dc = {d0, ..., dn} n-1} is a set of n distances (n = Card(P)). The average value of Dc is the approximate distance between the rotation center Pc' and the origin of the motion tracker MT. The length of the bone or set of bones can be easily derived from the approximate distance: by requiring the user to press a button on the motion tracker MT with a specific finger while performing a series of movements, the position of the finger's end is known (corresponding to the position of the pressed button), so the distance between the finger's end and the rotation center Pc' can be calculated.

[0100] The standard deviation of Dc can be calculated to estimate the quality of the results. If the standard deviation is too high, the estimate will be unreliable. For example, if the standard deviation is less than 10 mm, the quality of the results is suggested to be acceptable; if the standard deviation is between 10 mm and 15 mm, the quality of the results is questionable; and if the standard deviation exceeds 15 mm, the quality of the results is suggested to be poor. The standard deviation or quality level (acceptable / questionable / poor) of Dc can be sent (displayed) to the user in real time, allowing the user to correct a series of movements to improve the quality level. Therefore, the user can see the quality of the measurement through visual feedback indicating the standard deviation. More generally, the method of claiming protection can be completed by the user himself, without the help of experts or even non-experts.

[0101] Based on empirical testing, the inventors found that for approximately 1000 points in a 3D cloud, good measurement accuracy could be achieved as long as stability standards were met. This was achieved using a tracker with a 90Hz refresh rate (used in the HTC Vive). TMIn the case of a virtual reality system, the acquisition takes approximately 11 seconds. A threshold number of points can be set (e.g., 1000 points), after which the process will stop. If the results are very stable, the process may stop before acquiring the threshold number of points.

[0102] To estimate the dimensions of the user's upper limbs, the length L1 of the hand is first measured using the aforementioned method. It represents... Figure 6 Step S1 is shown in the flowchart. Steps a)-d) are performed as described above, and if the measurement is stable and noise-free, the same process is performed for the forearm to calculate the rotation center of the elbow (step S2): The user USR keeps one controller (reference tracker RT) stable and close to the elbow, and performs multiple flexion / extension movements and opposite swinging of the other controller (motion tracker MT), as... Figure 5A As shown.

[0103] Since the position of the wrist rotation center (relative to the origin of the motion tracker MT) has been previously calculated, the forearm length L2 can be calculated based on the elbow rotation center and the wrist rotation center.

[0104] Perform steps a)-d) above, and if the measurement is stable and noise-free (noisy measurements may occur when the reference tracker RT moves over a wide range or if the motion tracker MT does not move in the plane, contrary to the recommendation), then perform the same process for the forearm to calculate the rotation center of the shoulder (step S3). Perform the same process for the arm to calculate the rotation center of the shoulder. The user USR keeps one controller (reference tracker RT) stable and close to the shoulder, and attempts to keep the arm straight by repeatedly swinging the other controller (motion tracker MT) up and down and in the opposite direction, as follows. Figure 5B As shown.

[0105] Since the position of the wrist rotation center (relative to the origin of the motion tracker MT) has been previously calculated, the length L3 of the arm (forearm and upper arm) can be calculated based on the shoulder rotation center and the wrist rotation center.

[0106] Finally, while performing a series of repetitive up-and-down sweeping extensions of both arms, calculate the length L4 between the left and right shoulders, such as... Figure 5C As shown. User USR holds the reference tracker RT in one hand and the moving tracker RT in the other hand. For this step, both trackers are moved.

[0107] Assume that the length of the hand L1 and the length of the arm L3 are the same; therefore, the length of the hand L1 and the length of the arm L3, which have been calculated for one side, are considered to be the same for the other side.

[0108] During a series of repeated movements, the length Lt between the reference tracker RT and the moving tracker RT is calculated. The trackers move as far as possible in front of the user to obtain the maximum value of length Lt. Then, the length L4 between the left and right shoulders is calculated using the following formula:

[0109] L4 = max(Lt) - 2 × (L3 + L1)

[0110] Alternatively, the length of the hand L1 (L1l = length of the left hand and L1r = length of the right hand) may have been calculated previously for both sides of the user, and the length of the arm L3 (L3l = length of the left arm and L3r = length of the right arm) may have been calculated previously for both sides of the user.

[0111] In this case, the length L4 between the left and right shoulders can be calculated using the following formula:

[0112] L4=max(Lt)–(L1l+L1r+L3l+L3r)

[0113] Considering that each step in steps S1-S4 is performed at a refresh rate of 90Hz for approximately one thousand points, a complete estimate of the dimensions of the upper limb takes less than a minute, which is very fast compared to techniques using magnetic and / or optical markings.

[0114] In an optional preparatory step, prior to step a) in S1, the user may be prompted to input the size of the USR. Thus, a very rough estimate of the length of each segment may be displayed at the beginning of steps S1-S4 before calculating the dimensions of the bone or bone assembly based on the estimated center of rotation.

[0115] The aforementioned method can also be implemented using a tracker that can be worn on the foot to measure the dimensions of the user's lower limbs. The articulated joints can be the ankle, knee, and hip.

[0116] The method of the present invention can be executed by a suitably programmed general-purpose computer or virtual reality system, which may include a computer network, store the suitable program in non-volatile form on a computer-readable medium such as a hard disk, solid-state disk or CD-ROM, and execute the program using its microprocessor(s) and memory(s).

[0117] refer to Figure 7 A computer CPT suitable for performing methods according to exemplary embodiments of the present invention is described. Figure 7In this context, the computer CPT includes a central processing unit (CPU) that executes the method steps described above while running an executable program (i.e., a computer-readable instruction set). The executable program is stored in a memory device such as RAM M1 or ROM M2 or a hard disk drive (HDD) M3, DVD / CD drive M4, or is stored remotely.

[0118] The claimed invention is not limited to the form of a computer-readable medium on which computer-readable instructions and / or data structures containing the processes of the invention are stored. For example, the instructions and files may be stored on a CD, DVD, or in flash memory, RAM, ROM, PROM, EPROM, EEPROM, hard disk, or any other information processing device (e.g., a server or computer) in communication with a computer. The program and files may be stored on the same memory device or on different memory devices.

[0119] Furthermore, computer programs suitable for performing the methods of the present invention can be provided as utility applications, background daemons, or components or combinations thereof of an operating system, thereby executing in conjunction with a central processing unit (CPU) and an operating system (e.g., Microsoft Vista, Microsoft Windows 10, UNIX, Solaris, LINUX, Apple MAC-OS, and other systems known to those skilled in the art).

[0120] The central processing unit (CPU) can be a Xenon processor from Intel (USA) or an Opteron processor from AMD (USA), or it can be other processor types, such as a Freescale ColdFire, IMX, or ARM processor from Freescale Corporation (USA). Alternatively, as those skilled in the art will recognize, the CPU can be a processor such as the Core 2 Duo from Intel Corporation (USA), or it can be implemented on an FPGA, ASIC, PLD, or using discrete logic circuitry. Furthermore, the central processing unit can be implemented as multiple processors that work together to execute computer-readable instructions of the present invention described above.

[0121] Figure 7The virtual reality system also includes a network interface (NI) for connecting to networks (e.g., local area networks (LANs), wide area networks (WANs), the Internet, etc.), such as the Intel Ethernet PRO network interface card from Intel Corporation, USA. The virtual reality system also includes a head-mounted display (HMD) with a head-tracking device (HED). A general-purpose I / O interface (IF) connects to the tracker (RT, MT). The tracker (MT, RT) can be part of the virtual reality system (e.g., "HTC Vive"). TM It is part of a virtual reality system. Each wireless handheld controller in a virtual reality system includes a tracker (reference tracker or motion tracker). Tracking of the handheld controller can be done in different ways: through cooperation with a base station, through surface detection, marking, environmental recognition, gyroscope control unit, and computer vision.

[0122] The monitor, keyboard, and pointing devices, together with the display controller and I / O interfaces, form a graphical user interface (GUI). Users use the GUI to provide input commands, and the computer uses the GUI to display 3D objects.

[0123] The disk controller DKC connects the HDD M3 and DVD / CD M4 to the communication bus CBS, which can be an ISA, EISA, VESA, PCI, or similar device used to interconnect all components of the computer.

[0124] Any method steps described herein should be understood as representing modules, fragments, or portions of code, including one or more executable instructions for implementing specific logical functions or steps in the process, and alternative implementations are included within the scope of exemplary embodiments of the invention.

Claims

1. A method for locating the rotation center (Pc') of a hinge joint of two bones or a set of bones connecting the upper or lower limb of a user (USR), comprising the following steps: a) The user (USR) performs a series of repetitive movements of one bone or set of bones sweeping around a joint, and simultaneously acquires the 3D position of the bone or set of bones during the series of repetitive movements, thereby obtaining a 3D point cloud (P). b) Calculate a point called the center point (Pc) in a 3D search space that includes the 3D location of the end of another bone or bone set, the center point being a search point in the 3D search space that has the lowest standard deviation when taking into account the set of distances between the search point and every point in the 3D point cloud. c) Convert the 3D point cloud (P) into a plane by applying principal component analysis to the 3D point cloud (P) or based on the rectangular bounding box surrounding the 3D point cloud; d) Project the center point onto the plane to obtain the rotation center (Pc') of the joint.

2. The method according to claim 1, wherein: - The method includes a preparatory step of providing two handheld trackers (RT, MT); - In step a), the series of repetitive movements are performed while holding one hand tracker, referred to as the movement tracker (MT), and another hand tracker, referred to as the reference tracker (RT), is held close to the end of the other bone or bone set.

3. The method according to claim 2, wherein, Step b) includes recursively subdividing the 3D search space into sub-steps through the following steps: b1) Define the 3D search space as a cube (CUB) with a predefined edge length, and the cube (CUB) is centered on the 3D position of the reference tracker (RT), which is called the reference point (Porg). b2) Receive input from the user a set of points that identify the cube (CUB), the set of points together with the reference point (Porg) forming a set of search points (Ps); b3) For each search point (Ps), calculate the distance between the search point (Ps) and each point in the 3D point cloud (P), and derive the standard deviation of each search point (Ps); b4) In the set of search points (Ps), identify at least one search point (Ps) with the lowest standard deviation. b5) Furthermore, by halving the edge length of the cube (CUB) and making the cube (CUB) centered on the identified search point, steps b1), b2), b3), and b4) are repeated until the edge length of the cube is lower than or equal to a predefined threshold, and then the center point (Pc) corresponds to the identified search point.

4. The method according to claim 3, wherein, In step b2), the cube (CUB) is subdivided into eight smaller cubes (SCUB), and the set of search points (Ps) includes the reference point (Porg) or the identified search point, the center of the small cube, the center of the square of two adjacent smaller cubes (SCUB), and the midpoint of the edge of two adjacent smaller cubes (SCUB).

5. The method according to any one of the preceding claims, wherein, Step c) includes the following sub-steps: c1) Calculate the covariance matrix of the points in the 3D point cloud (P); c2) Calculate the eigenvalues ​​and eigenvectors of the covariance matrix; c3) Construct a rectangular bounding box that surrounds the 3D point cloud (P), wherein the orientation of the bounding box corresponds to the feature vector and the dimension of the bounding box corresponds to the feature value; c4) Calculate the geometric center of the bounding box; c5) Construct a plane defined by two eigenvectors having the largest eigenvalues, and the plane passes through the geometric center of the bounding box.

6. The method according to any one of claims 2-4, wherein, In step a), the acquired 3D positions are stored in frames, each frame including a predefined number of 3D positions, and steps b), c), and d) are implemented for all frames whenever a new frame has been acquired.

7. The method according to claim 6, comprising the following steps: - For each implementation of steps a), b), c), and d), store the 3D position of the reference tracker. - Calculate the average value of the 3D position of the reference tracker, and - If the difference between the 3D position of the reference tracker and the average value of the 3D positions stored in the reference tracker is greater than a predefined value, the corresponding frame is discarded.

8. The method according to claim 5, comprising: If it is determined in step c2) that there is no single feature value smaller than the other feature values, then the frame is discarded.

9. The method according to any one of claims 1-4, comprising the following steps: The minimum standard deviation indication is sent to the user in real time.

10. The method according to claim 1, wherein, In step a), the acquired 3D positions are stored in frames, each frame including a predefined number of 3D positions, and steps b), c), and d) are implemented for all frames whenever a new frame has been acquired.

11. A method for estimating the dimensions of a user's upper limbs, comprising the following steps: S1: Calculate the length of the hand (L1) based on the distance between the position of the wrist rotation center calculated using the method according to any one of the preceding claims and the position of the origin of the motion tracker (MT). S2: The length of the forearm (L2) is calculated based on the distance between the position of the rotation center of the wrist calculated using the method according to any one of the preceding claims and the position of the rotation center of the elbow calculated using the method according to any one of the preceding claims. S3: Calculate the length (L3) of the upper arm and forearm based on the distance between the position of the rotation center of the wrist calculated using the method according to any one of the preceding claims and the position of the rotation center of the shoulder calculated using the method according to any one of the preceding claims. S4: When each of the reference tracker (RT) and the motion tracker (MT) is held in a different hand by the user (USR) and waved in front of the user with an extended arm, calculate the length between the left and right shoulders (L4) based on the length of the upper arm and forearm (L3), the length of the hand (L1), and the maximum distance (Lt) between the reference tracker (RT) and the motion tracker (MT).

12. A computer program product stored on a non-transitory computer-readable data storage medium (M1, M2, M3, M4), the non-transitory computer-readable data storage medium (M1, M2, M3, M4) comprising computer-executable instructions for causing a computer system to perform the method according to any one of the preceding claims.

13. A non-transitory computer-readable data storage medium (M1, M2, M3, M4) comprising computer-executable instructions for causing a computer system to perform the method according to any one of claims 1-11.

14. A computer system comprising two handheld trackers (RT, MT), a processor (CPU) coupled to non-transitory memory (M1, M2, M3, M4), and a screen (DY), said memory storing computer-executable instructions for causing the computer system to perform the method according to any one of claims 1-11.

Images