How to find the center of rotation of a joint

Abstract

To find a center of rotation of a joint.SOLUTION: A method for finding a center of rotation (Pc') of joint which connects two bones or a set of bones of the upper limb or the lower limb of a user (USR) includes: a) a step which performs a series of repeating operations to sweep bones or one of the set of bones around the joint and, at the same time, acquires a 3D position of the bones or the set of bones during the series of repeating operations, and thus acquires a group (P) of 3D points; b) a step which calculates a point called a center point (Pc) in a 3D search space including the 3D position of an end of other bones or a set of bones, the center point being a search point having the lowest standard deviation considering a set of distance between the search point and the respective points of the group of 3D points; c) a step which converts the group of 3D points into a planar surface; and d) a step which projects the center point onto a planar surface and thus acquires the center of rotation (Pc') of the joint.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to the field of computer programs and systems, and in particular to the field of digital human modeling, product review, ergonomic analysis and validation in immersive environments. [Background technology]

[0002] In an immersive environment, the user interacts with the 3D scene through a graphical representation, commonly called an avatar. Whether in first-person visualization (where the user sees his avatar's arms but not the whole avatar) or third-person visualization (where the user sees his avatar), the estimation of the avatar's arm length must be as accurate as possible. In particular, ergonomic validation requires an accuracy of a few millimeters for limb lengths. For example, in a simulation of a workstation with an emergency stop button, whether the user can reach it in an immersive environment is very important for validating the workstation.

[0003] Therefore, experiences in virtual immersive environments are only meaningful and realistic if the mapping from the real world to the virtual world is accurate. This aspect also concerns scenarios where avatars are used to evaluate prototypes of modeled objects using virtual reality devices. One of the basic requirements in this scenario is the uniformity of dimensions between the avatar and its counterpart in the real world (the human). This can be achieved by modifying the dimensions of the avatar with the dimensions of the human guiding it.

[0004] However, modeling the human skeletal structure is complex. The human skeleton can be viewed as a group of bones. Articulated joints can be defined as the relative interactions between some of these bones, and these interactions include bone rotation, translation, and sliding motion. The motion of these anatomical joints can vary from simple fixed rotational axes to very complex multi-axis coupled motions. For example, consider the scapular-humeral rhythm, where the scapula and humerus move in a 1 / 2 ratio. When the arm abducts 180 degrees, 60 degrees is caused by the rotation of the scapula and 120 degrees is caused by the rotation of the humerus at the shoulder end. In this application, we consider the joints to behave like rotational axes. In fact, our goal is not to determine the user's actual skeleton, but to model the lengths and articulations of the limbs. Despite its simplification, accurately capturing these motions using a limited number of trackers and mapping them to a digital human is a challenging task.

[0005] On the other hand, modeling of the human anatomy can be performed by placing magnetic and / or optical markers at appropriate positions on the human body. Non-Patent Document 1 discloses a method for determining the joint parameters of an articulated hierarchy using magnetic motion capture data. This method allows determining the length of the subject's limbs, the position of the joints, and the placement of sensors without external measurements. According to Table 1 of Non-Patent Document 1, the disclosed method is very accurate. For the upper arm, the difference between the measured value (using a ruler) and the calculated value is equal to 2 / 3 millimeters on average. However, this method requires wearing a motion capture harness equipped with sensors, which becomes useless for an immersive experience once the modeling is completed. Moreover, the accuracy of this kind of solution is usually related to the accuracy of the positioning of the markers, so the setup cannot be done alone (usually by an expert) and is costly and time-consuming.

[0006] On the other hand, bone length estimation using markerless tracking systems is less accurate when compared to marker-based systems. Markerless systems are found, for example, in consumer virtual reality systems for video game applications. In those cases, the user inputs their size and the length of the upper limbs is derived based on statistical considerations. However, there is no real proportional relationship between size and limb length, and tall people can have short arms and vice versa. Thus, this method is not accurate.

[0007] Therefore, there is a need to provide a fast and accurate markerless method for modeling a user's upper or lower limbs for immersive environment applications, particularly for locating the centers of rotation of the joints connecting the bones of the upper or lower limbs, and also for estimating the dimensions of the upper or lower limbs. [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] James F. O'Brien et al., “Automatic Joint Parameter Estimation from Magnetic Motion Capture Data”, Proceedings of the Graphics Interface 2000 Conference, May 15-17, 2000 Summary of the Invention

[0009] The subject of the present invention is a method for finding the centre of rotation of a joint connecting two bones or sets of bones of an upper or lower limb of a user, said method comprising the steps of: a) performing, by a user, a series of repetitive movements of sweeping one of a bone or set of bones around a joint and simultaneously acquiring 3D positions of the bone or set of bones during said series of repetitive movements, thereby acquiring a 3D cloud of points; b) calculating a point, called the center point, in a 3D search space that includes the 3D positions of the extremities of the other bone or set of bones, the center point being the search point with the lowest standard deviation considering a set of distances between the search points of the 3D search space and each point of the 3D point cloud; c) transforming the 3D point cloud into a plane; d) projecting the center point onto said plane, thereby obtaining the center of rotation of the joint; Includes.

[0010] In a preferred embodiment, The method includes the preliminary step of providing two handheld trackers; - In step a), a series of repetitive movements are performed while holding one of the handheld trackers, referred to as the mobile tracker, and the other handheld tracker, referred to as the reference tracker, maintained relative to an end of another bone or set of bones.

[0011] In a preferred embodiment, step b) comprises b1) defining a 3D search space as a cube of predefined edge length, said cube being centered on the 3D position of a reference tracker, called the reference point; b2) receiving input from a user identifying a set of cubic points which together with said reference points constitute a set of search points; b3) for each search point, calculating the distance between the search point and each point of the 3D point cloud and deriving a standard deviation for each search point; b4) identifying from the set of search points at least one search point having the lowest standard deviation; b5) repeating steps b1), b2), b3), and b4) by halving the edge length of the cube and centering the cube on the identified search point until the edge length of the cube is equal to or less than a predefined threshold, where the center point then corresponds to the identified search point; The method includes the sub-step of recursively subdividing the 3D search space by

[0012] In a preferred embodiment, in step b2), the cube is subdivided into eight smaller cubes and the set of search points includes the reference point or identified search point, the center of the smaller cube, the square centers of two adjacent smaller cubes, and the midpoints of the edges of the two adjacent smaller cubes.

[0013] In a preferred embodiment, step c) comprises: c1) calculating a covariance matrix of the points of the 3D point cloud; c2) calculating the eigenvalues ​​and eigenvectors of the covariance matrix; c3) creating a rectangular bounding box around the 3D point cloud, a bounding box orientation corresponding to the eigenvectors, and a bounding box dimension corresponding to the eigenvalues; c4) calculating the geometry center of the bounding box; c5) creating a plane defined by the two eigenvectors with the largest eigenvalues ​​and passing through the geometric center of the bounding box; Includes.

[0014] In a preferred embodiment, in step a), the acquired 3D positions are stored in frames, each frame containing a predefined number of 3D positions, and steps b), c) and d) are performed for every frame whenever a new frame is acquired.

[0015] In a preferred embodiment, the method comprises the steps of: - storing the 3D position of the reference tracker for each execution of steps a), b), c) and d); - calculating an average of the 3D positions of the reference trackers; - discarding a corresponding frame if the 3D position of the reference tracker differs from the average of the stored 3D positions of the reference tracker by more than a predefined value; Includes.

[0016] In a preferred embodiment, the method includes the step of discarding the frame if it is determined in step c2) that no single eigenvalue is smaller than the others.

[0017] In a preferred embodiment, the method includes the step of transmitting an indication of the minimum standard deviation to a user in real time.

[0018] The present invention also relates to a method for estimating a size of an upper limb of a user, the method comprising the steps of: S1: Calculating a hand length based on a distance between a position of a rotation center of a wrist calculated by any of the methods according to the above claims and a position of an origin of a mobile tracker; S2: A step of calculating a length of the forearm based on a distance between a position of a rotation center of the wrist calculated by the method according to any one of the above claims and a position of a rotation center of the elbow calculated by the method according to any one of the above claims; S3: A step of calculating the length of the upper arm and the forearm based on the distance between the position of the rotation center of the wrist calculated by the method according to any one of the claims and the position of the rotation center of the shoulder calculated by the method according to any one of the claims; S4: Calculating the length between the left and right shoulders based on the length of the upper arm and forearm, the length of the hand, and the maximum distance between the reference tracker and the mobile tracker when the reference tracker and the mobile tracker are held in different hands of the user and swung in the frontal plane of the user with the arm extended. Includes.

[0019] The invention also relates to a computer program product stored on a non-transitory computer readable data storage medium comprising computer executable instructions for causing a computer system to perform a method according to any of the above claims.

[0020] The invention also relates to a non-transitory computer readable data storage medium containing computer executable instructions for causing a computer system to perform the aforementioned method.

[0021] The present invention also relates to a computer system comprising two handheld trackers, a processor coupled to a non-transitory memory, a screen, and a memory storing computer-executable instructions for causing the computer system to perform the aforementioned method.

[0022] Additional features and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0023] [Figure 1] FIG. 1 is a top view of a user performing a series of repetitive hand sweeping movements. [Diagram 2] FIG. 1 shows the acquired 3D point cloud, the calculated center points, and the calculated center of rotation. [Diagram 3] FIG. 13 is another diagram showing a 3D point cloud, a center point, and a center of rotation. [Figure 4] FIG. 13 illustrates a cube containing search points. [Figure 5A] FIG. 1 is a side view of a user performing a series of repetitive forearm sweeping movements. [Figure 5B] FIG. 1 is a side view of a user performing a series of repetitive arm sweeping movements. [Figure 5C] FIG. 1 is a front view of a user performing a series of repetitive arm sweeping movements. [Figure 6] 1 is a flowchart of a method for estimating a user's upper extremity size. [Figure 7] FIG. 1 illustrates a computer environment adapted to carry out the method according to the invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] FIG. 1 shows a top view of a user performing a series of repetitive hand sweeping movements to determine the center of rotation of the wrist.

[0025] In a first step of the method of the present invention, a) a user USR performs a series of repetitive movements sweeping one or a set of bones around a joint and simultaneously acquiring 3D positions of the bone or set of bones during said series of repetitive movements, thereby acquiring a 3D point cloud P. The acquired 3D point cloud is shown in FIG.

[0026] In a preferred embodiment, which is particularly suited to finding the center of rotation of a user's wrist, the user USR holds trackers (MT, RT) in both hands.

[0027] The trackers (MT, RT) can be part of a virtual reality system, such as for example the "HTC Vive" (trademark) virtual reality system. Such a system comprises at least one handheld tracker (wireless handheld controller), a virtual reality headset for displaying a 3D scene to the user in an immersive context. The implementation of the invented method does not require the use of a virtual reality headset, since the claimed method is not directed to the immersive experience itself, but to the pre-calibration of the user's avatar. Furthermore, the invention requires the use of two handheld trackers, one for the left upper / lower limb and one for the right upper / lower limb. Thus, no expensive motion capture system is required.

[0028] Through cooperation with the base station, the trackers can be placed in the coordinate system of the virtual environment in different ways: by surface detection, markers, environmental recognition, gyroscope control units, computer vision, etc. In the invented method, the location of the origin of each tracker is known by one of these methods.

[0029] As long as the 3D position can be determined, the tracker does not have to be dedicated to virtual reality - it can be used just for the calibration process.

[0030] In this embodiment, the handheld trackers (MT, RT) are identical: the tracker held by the hand performing a repetitive motion of sweeping a set of hand bones around the wrist is referred to as the mobile tracker MT, and the other tracker is referred to as the reference tracker RT.

[0031] An estimate of the position of the center of rotation can be calculated after all 3D positions of a bone or set of bones have been acquired. Alternatively, the estimate may be calculated whenever a predefined number of points have been acquired, hereinafter the predefined number of points is referred to as a frame. Thus, depending on the tracker hardware, and also depending on the virtual reality software, a real-time estimation of the position of the center of rotation can be performed. For example, the handheld tracker of the "HTC Vive" (trademark) virtual reality system has a refresh rate of up to 90Hz. To enhance real-time visual rendering, the refresh rate can also be set to a value much lower than 90Hz. In the claimed method, the 3D position of the mobile tracker MT is stored for a predefined length of frames (e.g., 25 points), and the position of the center of rotation is calculated first for the first frame, then for two first frames (e.g., 50 points), then for three first frames (e.g., 75 points), and so on. Thus, the user feels a near real-time measurement, and the accuracy of the measurement improves over time.

[0032] To find the center of rotation of the wrist, the user USR holds the mobile tracker MT and the reference tracker RT in his / her hand. As shown in Figure 1, while keeping the reference tracker RT stable and close to the wrist, the user swings the mobile tracker MT from left to right and back multiple times.

[0033] The reference tracker RT has three functions. First, as explained below, since the user is asked to hold the reference tracker RT on the forearm near the wrist (but not at the wrist, otherwise the measurement process would be noisy), it provides a first estimate of the position of the wrist's center of rotation. Second, by holding the reference tracker RT on the forearm, the forearm is more stable and therefore the accuracy of the measurement is improved. Third, by knowing the position of the reference tracker RT during the calibration process, only the relative movement of the mobile tracker MT with respect to the reference tracker RT can be taken into account, and the global movement of the forearm can be compensated for. This will improve the stability of the calibration process.

[0034] The hand can have a wide angular displacement (approximately 130-140°) around the wrist center of rotation, but it is better not to swing the hand until it reaches an extreme posture, such as a stop of the joint. It is important for the wrist because at the end of the movement, the user tends to "break" the wrist, that is, to radically change the plane of point P. It is also important for the shoulder movement because if the shoulder movement is too wide, the scapula also moves, so that movements that show several joints tend to become very complicated. The user is asked not to make too wide angular displacements. For example, in Figures 1, 5A, 5B, and 5C, which show examples of snapshots from a video tutorial intended as a guide for users, the angular ranges corresponding to extreme postures are displayed in a different way (e.g., in different colors) than the ranges corresponding to desired postures.

[0035] The user USR can be asked through visual instructions on the screen to position his / her arms as shown in Figure 1. Such movements are easy to perform even for non-expert users.

[0036] In a second step b) of the claimed method, a point called the centre point Pc is calculated in a 3D search space that includes the 3D positions of the extremities of the other bone or set of bones, the centre point Pc being the point in the search space that minimises the standard deviation given the set of distances between the search point and each point of the 3D point cloud.

[0037] The 3D search space initially includes the 3D location of the extremity of the other bone or set of bones (according to the example of Fig. 1, the extremity of the forearm close to the wrist). Thus, by inviting the user to place the reference tracker RT close to the joint, stability is improved (due to less limb movement except for shaking the bone or set of bones) and the calculation of the center point Pc starts from a point closer to the actual center of rotation, thereby speeding up the method.

[0038] Then, to find the approximate center of the 3D point cloud P, which looks like a bundle of points in an arc shape, the method starts from the reference point Porg and finally finds a point in its neighborhood that minimizes the standard deviation given a set of distances between the search point and each point in the 3D point cloud.

[0039] In a preferred embodiment, the centre point Pc is calculated based on an octree search algorithm, in which the 3D search space is recursively subdivided, and which comprises the following sub-steps of step b):

[0040] In a first sub-step b1), the 3D search space is defined as a cube CUB of predefined edge length, said cube CUB being centered on a reference point Porg corresponding to the 3D position of the reference tracker RT. Figure 4 shows the cube CUB diagrammatically. For example, the first predefined edge length may be equal to a few decimeters to be adapted to all kinds of joints of the upper limbs. In fact, the maximum distance between the rotation center of the joint and the mobile tracker covers the measurement of the length of the entire arm, which is about 50 centimeters long, depending of course on the user. For measuring the position of the rotation center of the lower limbs (ankle, knee or hip), for the same reason, the first predefined edge length may be equal to one meter. The first predefined edge length may be, for example, 2 9 = 512 mm, which is a good trade-off for performance and accuracy reasons.

[0041] Then, in a second sub-step b2), a set of points of the cube CUB is identified. This set of identified points, together with the reference point Porg, constitutes a set of search points Ps. The identified points form a network of points arranged around the reference point Porg, for example in a periodic arrangement.

[0042] In a preferred embodiment, for the first iteration of step b), the cube CUB is subdivided into 8 smaller cubes SCUB (voxels) and the set of search points Ps is the following points: - a reference point Porg located in the center of the cube CUB, - the centers of 8 small cubes, -12 centers of squares of two adjacent smaller cubes SCUB, and - The 6 midpoints of the edges of two adjacent smaller cubes SCUB Includes.

[0043] Other choices of search points can be made, however, it has been found that using these points gives superior results in terms of computation speed.

[0044] Then, in a third sub-step b3), for each search point Ps of the cube CUB, the distance is calculated between the search point Ps and each point of the 3D point cloud P. It is understood that if the method is implemented such that a frame of 3D points is acquired, the distance between the search point Ps and the points of the 3D point cloud P is calculated only for the acquired 3D points.

[0045] Assume that at time t, N points are obtained and there are Nps search points. For each search point Ps in the cube CUB, N distances are calculated. For this set of N distances, the standard deviation is calculated. For the Nps search points Ps, Nps standard deviations are calculated.

[0046] Then, in a fourth sub-step b4), at least one search point Ps with the lowest standard deviation is identified. The search point(s) with the lowest standard deviation, by definition, indicates that the distances between the search point and each point of the 3D point cloud P tend to be close to the average of the distances.

[0047] Then steps b1), b2), b3) and b4) are repeated. Instead of centering the cube CUB on the reference point Porg (corresponding to the position of the reference tracker RT), the cube CUB is centered on the search point Ps (called the identified search point) with the lowest standard deviation. With each new iteration, the edge length of the cube CUB is halved.

[0048] In a preferred embodiment, in sub-step b4), the two search points with the lowest standard deviation are identified and iterations are performed on both points in parallel. Thus, after each iteration, the number of identified search points doubles. If only one search point is identified after each iteration, the accuracy of the method is improved by defining 27 or more search points Ps.

[0049] In any case, whether there is one identified search point in each iteration or multiple identified search points in each iteration (e.g., two identified search points), once the cube edge length is equal to or less than a predefined threshold, the iterations continue and a single search point is identified after each iteration. For example, the threshold may be equal to 16 mm.

[0050] Then, the iteration stops when the edge length of the cube is less than another predefined threshold, for example 2 mm. The center point Pc is in the cube CUB whose edge length is less than 2 mm. Therefore, the resolution in this case is equal to 1 mm (the search point of the last iteration is in the center of the cube with an edge length of 2 mm), which is a higher resolution compared to the previous method.

[0051] The result of step b), and in the preferred embodiment the result of the octree search, is a centre point Pc.

[0052] Alternatively, instead of calculating the center point Pc based on an octree search algorithm (of logarithmic complexity), the center point Pc may be calculated based on a brute-force linear search (of linear complexity).

[0053] Then, in a third step c), the 3D point cloud P is transformed into a plane. The goal of step c) is to map the 3D point cloud P in two dimensions while losing a lot of information.

[0054] In a preferred embodiment, the 3D to 2D conversion may be performed by using a statistical procedure called "principal component analysis", which is described below.

[0055] c1) Calculate the covariance matrix of the points in the 3D point cloud (P).

[0056] c2) Calculate the eigenvalues ​​and eigenvectors of the covariance matrix.

[0057] c3) Create a rectangular bounding box that encloses the 3D point cloud P, the bounding box orientations corresponding to the eigenvectors, and the bounding box dimensions corresponding to the eigenvalues.

[0058] c4) Calculate the geometric center of the bounding box.

[0059] c5) Create a plane defined by the two eigenvectors with the largest eigenvalues ​​and passing through the geometric center of the bounding box. Assume that the user follows the on-screen instructions to properly perform the sweep operation. The 3D point cloud P is almost on the same plane and looks like an arc shape. Therefore, one of the eigenvalues ​​representing the thickness of this point cloud is usually much smaller than the other two eigenvalues.

[0060] Once the series of iterative movements is completed, a plane can be created. In a preferred embodiment, the plane is created after the acquisition of a predefined length of frames together with all already acquired frames. Thus, step c) is also performed in near real time, allowing the user to correct his movements if the stability of the creation of the plane is insufficient. In particular, if during step c3) it is determined that the three eigenvalues ​​are very similar, for example based on a tolerance margin, the frame can be discarded. Thus, the 3D to 2D conversion is performed with minimal loss of information.

[0061] Alternatively, instead of using the "principal component analysis" method, the plane can be created by first computing a rectangular bounding box that encloses the 3D point cloud. Then, a plane is computed that is parallel to both of the largest faces of the bounding box and divides the bounding box into two equal volumes.

[0062] In the final step d) of the claimed method, the center point Pc calculated in step b) is projected onto the plane calculated in step c). The projection of the center point Pc corresponds to the position of the joint's center of rotation Pc'. In a particular embodiment of the invention in which a user holds a mobile tracker MT, the position of the center of rotation Pc' refers to a 3D position relative to the 3D position of the origin of the mobile tracker MT.

[0063] 2 and 3 show a 3D point cloud P, a center point Pc, and a center of rotation Pc'.

[0064] In a preferred embodiment, steps a), b), c) and d) are repeated whenever a new frame (e.g., containing 25 points) is acquired. In other words, when a new frame is acquired, the position of the center of rotation Pc' is recalculated for all 3D positions. The 3D position of the reference tracker RT is saved in real time and the average position of all saved positions of the reference tracker RT is calculated. If the 3D position of the reference tracker differs from the average of the saved 3D positions of the reference tracker by more than a predefined value, the new incoming frame is discarded. Thus, if the user moves too much during a sequence of movements, the 3D position of the reference tracker RT becomes unstable and the corresponding frame is discarded to avoid calculating the center of rotation Pc' for an unstable position of the user.

[0065] Distance d between the rotation center Pc' and each point of the 3D point group P k Dc={d0, d n-1} be a set of n distances, where n=Card(p) The average of Dc is the approximate distance between the center of rotation Pc' and the origin of the mobile tracker MT. The length of a bone or set of bones can be easily derived from the approximate distances. By asking the user to press a button on the mobile tracker MT with a particular finger while performing a series of movements, the position of the finger's extremity is known (which corresponds to the position of the pressed button) and the distance between the finger's extremity and the center of rotation Pc' can be calculated.

[0066] To estimate the quality of the result, the standard deviation of Dc may be calculated. If the standard deviation is too high, the estimation is not reliable. For example, the quality of the result is indicated as acceptable if the standard deviation is less than 10 mm, questionable if it is between 10 mm and 15 mm, and bad if it is more than 15 mm. The standard deviation of Dc, or the quality level (acceptable / questionable / bad) can be transmitted (displayed) to the user in real time, so that the user can correct the current sequence of movements to improve the quality level. Thus, the user can check the quality of the measurement through visual feedback showing the standard deviation. More generally, the claimed method can be performed by the user himself and does not need to be assisted by an expert or even a non-expert.

[0067] The inventors have empirically tested that good accuracy of measurement is achieved for a 3D point cloud of about 1000 points if the stability criteria are met. Using a tracker with a refresh rate of 90Hz (for the virtual reality system "HTC Vive" (trademark)), acquisition takes place in about 11 seconds. A threshold number of points can be set, for example 1000 points, beyond which processing stops. If the results are very stable, processing may stop before acquiring the threshold number of points.

[0068] To estimate the dimensions of the user's upper limb, first the length L1 of the hand is measured as described above. This represents step S1 shown in the flowchart of FIG. 6. If the above steps a)-d) have been performed and the measurements are stable and not noisy, the same process is performed on the forearm to calculate the elbow rotation center (step S2). Here, the user USR keeps one controller (reference tracker RT) stable and close to the elbow, as shown in FIG. 5A, and swings the other controller (mobile tracker MT) by performing multiple flexion / extension movements and vice versa.

[0069] Since the position of the wrist's center of rotation (relative to the origin of the mobile tracker MT) has been previously calculated, the forearm length L2 can be calculated based on the elbow's center of rotation and based on the wrist's center of rotation.

[0070] Once the above steps a)-d) have been performed and the measurements are stable and not noisy (contrary to recommendations, noisy measurements may occur if the reference tracker RT moves over a wide range or if the mobile tracker MT does not move in a plane), the same process is performed on the forearm (step S3) to calculate the center of rotation of the shoulder. The same process is performed on the arm to calculate the center of rotation of the shoulder. The user USR holds one controller (reference tracker RT) stable and close to the shoulder, keeps the arm straight, and swings the other controller (mobile tracker MT) up and down and vice versa multiple times by the arm, as shown in FIG. 5B.

[0071] Since the position of the wrist's center of rotation (relative to the origin of the mobile tracker MT) has been previously calculated, the length L3 of the arm (forearm and upper arm) can be calculated based on the shoulder's center of rotation and based on the wrist's center of rotation.

[0072] Finally, the length between the left and right shoulders L4 is calculated while performing a series of repeated movements of extending both arms and sweeping them up and down, as shown in Fig. 5C. The user USR holds the reference tracker RT in one hand and the mobile tracker RT in the other hand. In this step, both trackers are moved.

[0073] The hand length L1 and arm length L3 are assumed to be the same, therefore the hand length L1 and arm length L3 previously calculated on one side are considered to be identical on the other side.

[0074] During a series of repeated movements, the length Lt between the reference tracker RT and the mobile tracker RT is calculated. The tracker is moved as far as possible in the frontal plane of the user so that the maximum value of the length Lt is obtained. Then, the length L4 between the left and right shoulders is calculated by the following formula:

[0075] L4 = max(Lt) - 2 × (L3 + L1) Alternatively, the hand length L1 may have been previously calculated on both sides of the user (L1l = left hand length, and L1r = right hand length), and the arm length L3 may have been previously calculated on both sides of the user (L3l = left arm length, and L3r = right arm length).

[0076] In this case, the length L4 between the left and right shoulders is calculated using the following formula.

[0077] L4 = max(Lt) - (L1l + L1r + L3l + L3r) Considering that each step S1-S4 is performed at approximately 1000 points and with a refresh rate of 90Hz, a complete estimation of the upper limb dimensions is completed in less than a minute, which is very fast compared to techniques using magnetic and / or optical markers.

[0078] In an optional preliminary step, before step a) of S1, the user USR may be asked to input his / her size. Thus, very rough estimates of the length of each element may be displayed at the start of steps S1-S4, before calculating the dimensions of the bone or set of bones based on the estimated centre of rotation.

[0079] The above method may also be implemented to measure the dimensions of a user's lower limbs, with the joints being the ankle, knee and hip, by means of a tracker that may be supported by the foot.

[0080] The methods of the present invention may be implemented by a suitably programmed general-purpose computer or virtual reality system, optionally including a computer network, storing a suitable program in non-volatile form on a computer readable medium such as a hard disk, solid state disk or CD-ROM, and executing said program using its microprocessor(s) and memory.

[0081] A computer CPT suitable for carrying out the method according to an exemplary embodiment of the invention will now be described with reference to Fig. 7. In Fig. 7, the computer CPT comprises a central processing unit CPU which carries out the steps of the above method whilst executing an executable program, i.e. a set of computer readable instructions, stored in a memory device such as a RAM M1 or a ROM M2, or a hard disk drive (HDD) M3, a DVD / CD drive M4, or stored remotely.

[0082] The claimed invention is not limited by the form of the computer readable medium on which the computer readable instructions and / or data structures of the processes of the invention are stored. For example, the instructions and files may be stored on a CD, DVD, flash memory, RAM, ROM, PROM, EPROM, EEPROM, hard disk, or other information processing device with which the computer communicates, such as a server or computer. The programs and files may be stored on the same memory device or on different memory devices.

[0083] Furthermore, a computer program suitable for carrying out the methods of the present invention may be provided as a utility application, a background daemon, or a component of an operating system, or a combination thereof, running in conjunction with a central processing unit CPU and an operating system, such as Microsoft VISTA, Microsoft Windows 10, UNIX, Solaris, LINUX, Apple MAC-OS, and other systems known to those skilled in the art.

[0084] The central processing unit CPU may be a Xenon processor from Intel Corporation, USA or an Opteron processor from AMD Corporation, USA, or other processor types such as a Freescale ColdFire, IMX, or ARM processor from Freescale Corporation, USA. Alternatively, the CPU may be a processor such as a Core2Duo from Intel Corporation, USA, or may be implemented on an FPGA, ASIC, PLD, or using discrete logic circuitry, as will be appreciated by those skilled in the art. Additionally, the central processing unit may be implemented as multiple processors working in concert to execute the computer readable instructions of the process of the present invention as described above.

[0085] The virtual reality system of FIG. 7 also includes a network interface NI, such as Intel Ethernet PRO network interface card from Intel Corporation, USA, for interfacing with a network, such as a local area network (LAN), a wide area network (WAN), or the Internet. The virtual reality system further includes a head mounted display device HMD having a head tracking device HED. The generic I / O interface IF interfaces with a tracker (RT, MT). The tracker (MT, RT) can be part of a virtual reality system, such as the "HTC Vive" (trademark) virtual reality system. Each wireless handheld controller of the virtual reality system includes a tracker (reference tracker or mobile tracker). Tracking of the handheld controller can be performed in various ways, such as by surface detection, markers, environment recognition, gyroscope control unit, computer vision, through cooperation with a base station.

[0086] The display, keyboard, and pointing device, together with the display controller and the I / O interface, form a graphical user interface, used by the user to provide input commands and by the computer to display 3D objects.

[0087] The disk controller DKC connects the HDD M3 and the DVD / CD M4 to a communication bus CBS, which can be ISA, EISA, VESA, PCI or similar, for interconnecting all the components of the computer.

[0088] Any method steps described herein should be understood as representing a module, segment, or portion of code that contains one or more executable instructions for implementing a particular logical function or step in a process, and alternative implementations are included within the scope of the exemplary embodiments of the present invention.

Claims

1. 1. A computer-implemented method for finding a center of rotation (Pc′) of a joint connecting two bones or sets of bones in an upper or lower limb of a user (USR), comprising: a) acquiring 3D positions of the extremities of one of the two bones or set of bones during a series of repetitive movements by the user (USR) of sweeping one of the two bones or set of bones around the joint, thereby acquiring a 3D point cloud (P); b) calculating a point, called the centre point (Pc), in a 3D search space that includes the 3D positions of the other extremities of one of the two bones or sets of bones, said centre point being the search point with the lowest standard deviation of a set of distances between the search points of the 3D search space and each point of the 3D point cloud; c) transforming the 3D point cloud (P) into a plane by applying principal component analysis to the 3D point cloud (P) or based on a rectangular bounding box surrounding the 3D point cloud; d) projecting said center point onto said plane, thereby obtaining said center of rotation (Pc') of said joint; The method according to claim 1, further comprising:

2. The method of claim 1, characterized in that the series of repetitive actions are performed while holding one of a handheld tracker, referred to as a mobile tracker (MT), and another handheld tracker, referred to as a reference tracker (RT), which are maintained relative to the other end of one of the two bones or sets of bones.

3. Step b) is b1) defining the 3D search space as a cube (CUB) of predefined edge length, the cube (CUB) being centered on the 3D position of the Reference Tracker (RT), called a Reference Point (Porg); b2) receiving an input from the user identifying a set of points of the cube (CUB) which together with the reference point (Porg) constitute a set of search points (Ps); b3) for each search point (Ps), calculating the distance between said search point (Ps) and each point of said 3D point cloud (P) and deriving a standard deviation of said set of distances; b4) identifying from the set of search points (Ps) at least one search point (Ps) for which the standard deviation of the set of distances is lowest; b5) repeating steps b1), b2), b3) and b4) by halving the edge length of the cube (CUB) until the edge length of the cube is equal to or less than a predefined threshold and centering the cube around at least one search point with the identified lowest standard deviation, where the center point (Pc) then corresponds to the identified at least one search point with the lowest standard deviation; 3. The method of claim 2, further comprising the substep of recursively subdividing the 3D search space by:

4. 4. The method according to claim 3, characterized in that in step b2), the cube (CUB) is subdivided into eight smaller cubes (SCUB), and the set of search points (Ps) includes the reference point (Porg) or the identified search point, the centers of the eight smaller cubes, the centers of the squares of two adjacent smaller cubes (SCUB), the midpoints of the edges of two adjacent smaller cubes (SCUB).

5. Step c) is c1) calculating a covariance matrix of the points of said 3D point cloud (P); c2) calculating eigenvalues ​​and eigenvectors of said covariance matrix; c3) creating a rectangular bounding box around said 3D point cloud (P), the orientation of said bounding box corresponding to said eigenvectors, and the dimensions of said bounding box corresponding to said eigenvalues; c4) calculating the geometric center of said bounding box; c5) creating the plane defined by the two eigenvectors with the largest eigenvalues ​​and passing through the geometric center of the bounding box; 5. The method according to claim 1, further comprising:

6. 6. The method according to claim 1, wherein in step a) the acquired 3D positions are stored in frames, each frame containing a predefined number of 3D positions, and steps b), c) and d) are performed for all said frames whenever a new frame is acquired.

7. storing the 3D position of the reference tracker for each execution of steps a), b), c) and d); calculating an average of the 3D positions of the reference trackers; discarding a corresponding frame if the 3D position of the reference tracker differs from the average of the stored 3D positions of the reference tracker by more than a predefined value; The method of claim 6 when dependent on claim 2, comprising:

8. A method according to claim 6 or 7 when dependent on claim 5, characterized in that it comprises the step of discarding the frame if it is determined in step c2) that there is no single eigenvalue smaller than the others.

9. 9. A method according to any preceding claim, comprising the step of transmitting an indication of the minimum standard deviation to the user in real time.

10. 1. A computer-implemented method for estimating a user's upper extremity size, comprising: S1: Calculating the hand length (L1) based on the distance between the position of the rotation center of the wrist calculated by the method according to any one of claims 2 to 9 and the position of the origin of the mobile tracker (MT); S2: Calculating a length (L2) of the forearm based on a distance between the position of the rotation center of the wrist calculated by the method according to any one of claims 2 to 9 and the position of the rotation center of the elbow calculated by the method according to any one of claims 2 to 9; S3: Calculating the length (L3) of the upper arm and the forearm based on the distance between the position of the rotation center of the wrist calculated by the method according to any one of claims 2 to 9 and the position of the rotation center of the shoulder calculated by the method according to any one of claims 2 to 9; S4: Calculating the length between the left and right shoulders based on the length of the upper arm and forearm (L3), the length of the hand (L1), and the maximum distance between the reference tracker (RT) and the mobile tracker (MT) when the reference tracker (RT) and the mobile tracker (MT) are held in different hands of the user (USR) and swung in the frontal plane of the user with the arm extended; The method according to claim 1, further comprising:

11. A computer program stored on a non-transitory computer readable data storage medium (M1, M2, M3, M4) for causing a computer system to carry out the method according to any of claims 1 to 10.

12. A non-transitory computer readable data storage medium (M1, M2, M3, M4) comprising computer executable instructions for causing a computer system to perform the method according to any of claims 1 to 10.

13. A computer system comprising two handheld trackers (RT, MT), a processor (CPU) coupled to a non-transient memory (M1, M2, M3, M4) and a screen (DY), said memory storing computer executable instructions for causing the computer system to perform a method according to any of claims 1 to 10.

Images