Movement comparison method

Abstract

The present invention relates to a method for comparing movement series performed by two people. Sequences of images corresponding to the movement series are temporally aligned. Several types of offsets between the images are calculated and used to establish a movement execution score indicating a degree of similarity between the two series of movements performed. The invention also relates to a device for implementing the method and comprising, in particular, image capture devices for recording the sequences of images and a computation module for carrying out certain steps of the method.

Description

Motion comparison method Technical field

[0001] The present invention relates to a method for comparing movement and a device suitable for implementing this method. State of the art

[0002] Although algorithms for analyzing and comparing movements based on video recordings are now widespread, particularly in the world of video games, most of them only allow for the comparison of a movement performed by two different people based on a very limited number of parameters. Indeed, in many sports disciplines, several other factors must be taken into account in order for the comparison to be relevant to the discipline in question. In the field of dance, for example, factors such as delay or advance in relation to the tempo can be just as important as the position of the limbs.

[0003] In order for the comparison to be used for improvement, for example in the case where one of the two people is a teacher and the other is a student, as many of these factors as possible must be taken into account when comparing a student's movement with the teacher's movement.

[0004] Document US9358456 describes in particular a method for comparing movements between two players, consisting of displaying instructions to the first player so that he creates and performs a dance movement, recording the dance movement of the first player, generating a representation in the form of an icon of the first player performing his dance movement and displaying this icon to the second player so that he also performs this dance move, record the second player's dance move and compare the two dance moves.

[0005] This method is thus limited to comparing the positions of a member of the first player with respect to a member of the second player. Brief summary of the invention

[0006] An object of the present invention is to provide a motion comparison method free from the limitations known in the prior art.

[0007] Another aim of the invention is to propose a method for comparing movement making it possible to take into account several factors relevant to the comparison.

[0008] Another aim of the invention is to propose a method of comparing movement making it possible to detail the comparison on particular areas of the body.

[0009] According to the invention, these aims are achieved in particular by means of a movement comparison method comprising the steps of: a. recovering a model sequence of images, each image representing a model individual performing a model sequence of movements; b. producing, by means of a computer calculation module such as a computer, a mobile phone / smartphone, or any other device equipped with a calculator, a model skeletonization comprising, i. for at least three images of the model sequence of images, at least one model articulation position of at least one articulation of the model individual represented on each of the at least three images, ii. a model index corresponding to the position of the image in the model sequence and iii. a model time associated with the image; c. retrieving a user sequence of images, each image representing a user performing a user sequence of movements; d. producing, by means of a computing device such as a computer, a mobile phone / smartphone, or any other device equipped with a calculator, a user skeletonization comprising, i. for at least three images of the user sequence of images, at least one user joint position of at least one joint of the user represented on each of the at least three images, ii. a user index corresponding to the position of the image in the user sequence, iii. and a user time associated with the image; e.associating with each of the at least three images of the model sequence and the at least one articulation of the model individual represented on the image, at least one image of the user sequence by applying a dynamic temporal deformation algorithm (also called registration) based on the difference between a model articulation position and a user articulation position; f. for each of the at least three images of the model sequence and the at least one articulation of the model individual represented on the image, selecting at least one reference image of the at least one associated image maximizing a difference between the model index and the user index; g. for at least one image of the model sequence and for at least one articulation of the model individual represented on the image, calculating an offset between the image and the corresponding reference image; h.calculating a user movement execution score based on at least one offset, the score corresponding to a similarity. between the motion model suite and the user motion suite.

[0010] The producing step may include producing a skeletonization model file comprising, for each image in the model sequence of images, a model joint position for each joint of the model individual represented in the image, a model index corresponding to the position of the image in the model sequence, and a model time associated with the image.

[0011] The retrieving step may include retrieving a user sequence of images, each image representing a user performing a user sequence of movements.

[0012] The producing step may include producing a user skeletonization file including, for each image in the user sequence of images, a user joint position for each joint of the user depicted in the image, a user index corresponding to the position of the image in the user sequence, and a user time associated with the image.

[0013] The association step may comprise associating to each image of the model sequence and for each joint of the model individual represented in the image, a set of images of the user sequence by applying a dynamic time warping algorithm based on the difference between the model joint position and the user joint position.

[0014] The selection step may comprise, for each image of the model sequence and for each articulation of the model individual represented on the image, the selection of a reference image from the set of images maximizing a difference between the model index and the user index.

[0015] The calculation step may comprise, for each image of the model sequence and for each joint of the model individual represented on the image, the calculation of an offset between the image and the corresponding reference image.

[0016] The step of retrieving the model sequence of images may include: • recording the template sequence of images using a first image capture device; and / or wherein the step of retrieving the user sequence of images comprises • recording the user sequence of images using a second image capture device.

[0017] An offset can be a time shift determined during a step including: • for the at least three images of the model sequence and for the at least one articulation of the model individual represented on the image, calculation of the time shift on the basis of the model time and the user time of the reference image associated with the image of the model sequence, the movement execution score being calculated on the basis of the time shifts.

[0018] An offset may be a spatial offset determined during a step comprising: • For the at least three images of the model sequence of images and for the at least one joint of the model individual represented in the image, calculate the spatial offset between the model position and the user position, the movement execution score being calculated on the basis of the spatial offsets.

[0019] An offset may be a speed offset determined during a step including: • for the at least three images of the model sequence of images and for the at least one articulation of the model individual represented in the image, calculating a speed offset between an articulation speed of the model individual and an articulation speed of the user, the movement execution score being calculated on the basis of the speed offsets.

[0020] An offset may be an acceleration offset determined during a step including: • for the at least three images of the model sequence of images and for the at least one joint of the model individual represented in the image, calculating an acceleration offset between a joint acceleration of the model individual and a joint acceleration of the user, the movement execution score being calculated on the basis of the acceleration offsets.

[0021] Spatial, velocity, and acceleration offsets can be calculated for each frame of the model sequence and for each joint of the model individual.

[0022] Spatial, velocity, and acceleration offsets can be calculated between the raw data of the model and user skeletons, but can also be calculated between mathematical transformations of this data, such as low-pass, high-pass, or band-pass filters, moving averages, or any other filter.

[0023] The execution score calculation step may include sub-steps of: • for at least one articulation of the model individual, calculate an articulation execution score based on the temporal, spatial, velocity and / or acceleration offsets; • calculate the execution score based on the articulation execution score.

[0024] The execution score may be a weighted average of articulation execution scores of a plurality of articulations of the model individual.

[0025] A weighted average weighting can be done based on movement type, movement difficulty and / or user level.

[0026] An offset may be a torsional offset determined during a step comprising: • for each image of the model sequence of images, determine: o a model torsion angle of a portion of the body of the model individual represented in the image; and o a user torsion angle of a portion of the body of the user represented in the image; • calculate a torsion offset based on the model torsion angle and the user torsion angle; the movement execution score is further calculated based on the torsion offsets.

[0027] The model torsion angle can be formed by a straight line passing through points in the skeletonization model file representing hips of the model individual and a reference line corresponding to a horizontal line of a terrestrial reference frame, and in which the user torsion angle is formed by a straight line passing through points in the user skeletonization file representing hips of the user and the reference line.

[0028] The model twist angle may be formed by a straight line passing through points in the skeletonization model file representing hips of the model individual and a straight line passing through points in the skeletonization file representing shoulders of the model individual, and wherein the user twist angle is formed by a straight line passing through points in the user skeletonization file representing hips of the user and a line passing through points in the user skeleton file representing the user's shoulders.

[0029] The model twist angle may be formed by a straight line passing through points in the skeletonization model file representing shoulders of the model individual and a straight line passing through points in the skeletonization model file representing the midpoint of the segment joining ears and a nose of the model individual, and wherein the user twist angle is formed by a straight line passing through points in the user skeletonization file representing shoulders of the user and a straight line passing through points in the user skeletonization file representing the midpoint of the segment joining ears and a nose of the user.

[0030] The model twist angle may be formed by a straight line passing through points in the skeletonization model file representing a thumb and a little finger of the model individual and a straight line passing through points in the skeletonization file representing an elbow and a wrist of the model individual, and wherein the user twist angle is formed by a straight line passing through points in the user skeletonization file representing a thumb and a little finger of the user and a straight line passing through points in the user skeletonization file representing an elbow and a wrist of the user.

[0031] The model twist angle may be formed by a straight line passing through points in the skeletonization model file representing a heel and a big toe of the model individual and a straight line passing through points in the skeletonization file representing hips of the model individual, and wherein the user twist angle is formed by a straight line passing through points in the user skeletonization file representing a heel and a big toe of the user and a straight line passing through points in the user skeletonization file representing hips of the user.

[0032] An offset may be a bending offset determined during a step including: • for each image of the model sequence of images, determine: o a model flexion angle of a portion of the body of the model individual represented in the image; and o a user flexion angle of a portion of the body of the user represented in the image; • calculate a flexion offset based on the model flexion angle and the user flexion angle; the movement execution score is further calculated based on the flexion offsets.

[0033] The model flexion angle can be obtained by calculating the angle formed by a straight line passing through points in the skeletonization model file representing shoulders of the model individual and a straight line passing through points in the skeletonization file representing hips of the model individual, in a plane containing the points corresponding to the hips of the model individual and the midpoint of the segment joining the shoulders of the model individual, and in which the user flexion angle is obtained by calculating the angle formed by a straight line passing through points in the user skeletonization file representing shoulders of the user and a straight line passing through points in the user skeletonization file representing hips of the user, in a plane containing the points corresponding to the hips of the user and the midpoint of the segment joining the shoulders of the user.

[0034] The model flexion angle can be obtained by calculating the angle formed by a straight line passing through points in the skeletonization model file representing shoulders of the model individual and a straight line orthogonal to the plane containing points in the skeletonization model file corresponding to ears to a nose of the model individual, in a plane containing the points corresponding to the shoulders of the model individual and the midpoint of the segment joining points corresponding to the ears of the model individual, and in which the user flexion angle is obtained by calculating the angle formed by a straight line passing through points in the user skeletonization file representing the user's shoulders and a line orthogonal to the plane containing points of the user skeletonization file corresponding to the user's ears and nose, in a plane containing the points corresponding to the user's shoulders and the midpoint of the segment joining points corresponding to the user's ears.

[0035] The model flexion angle can be obtained by calculating the angle formed by a straight line passing through points in the skeletonization model file representing the midpoint of the segment joining shoulders of the model individual and the midpoint of the segment joining hips of the model individual, and a straight line orthogonal to the plane formed by points in the skeletonization model file representing ears and a nose of the model individual, in a plane orthogonal to the straight line passing through points in the skeletonization model file representing the shoulders of the model individual, and in which the user flexion angle is obtained by calculating the angle formed by a straight line passing through points in the user skeletonization file representing the midpoint of the segment joining shoulders of the user and the midpoint of the segment joining hips of the user,and a straight line orthogonal to the plane formed by points in the user skeletonization file representing the user's ears and nose, in a plane orthogonal to the straight line passing through points in the user skeletonization file representing the user's shoulders.,

[0036] Torsion shifts can be calculated between raw model and user skeleton data, but can also be calculated between mathematical transformations of that data, such as low-pass, high-pass, or band-pass filters, moving averages, or any other filter.

[0037] These goals are also achieved by means of a motion comparison device comprising: • a first image capture device configured to capture a model sequence of images of a model individual performing a model sequence of movements; • a second image capture device configured to capture a user sequence of images of a user performing a user sequence of movements; and • a computer calculation module allowing the implementation of the method described above. The first and second devices can be the same device or two separate devices. Brief description of the figures

[0038] Examples of implementation of the invention are indicated in the description illustrated by the appended figures in which: • Figure 1 illustrates an image sequence alignment using a dynamic time warping algorithm. • Figure 2 illustrates the determination of a torso torsion angle. • Figure 3 illustrates the determination of a head twist angle. • Figure 4 illustrates the determination of an arm torsion angle. • Figure 5 illustrates the determination of a leg twist angle. • Figure 6 illustrates the determination of a torso flexion angle. • Figure 7 illustrates the determination of a lateral head flexion angle. Figure 8 illustrates the determination of a forward-backward flexion (nodding) angle of the head. Example(s) of embodiment of the invention

[0039] The method of the present invention aims to compare a movement or a series of movements of a model individual (or leader), typically a teacher or a reference person, to a movement or a series of movements of a user (or follower), typically a student or a person wishing to improve the performance of the movement.

[0040] The process is suitable for many sports or activities based on a movement technique in which the progression of a practitioner is based at least in part on the acquisition of certain movements or movement sequences. For example, learning a type of dance, martial arts, but also certain sports such as golf in which the importance of posture or rhythm is preponderant is essentially based on the reproduction (and therefore the adequacy) of certain movements. There is also a comparable interest in learning and perfecting movements during industrial handling and manipulation activities to increase the safety, well-being and efficiency of employees.

[0041] In the context of the present invention, the characteristics, attributes, quantities, etc., associated with the model individual (respectively with the user) are designated by the expression “model” (respectively “user”), e.g. a “model movement” (respectively a “user movement”), a model image (respectively a “user image”), a model position (respectively a “user position”), etc. Certain occurrences of the terms “model” and “user” may be omitted when the context is sufficiently clear in order to simplify the reading of the present text.

[0042] The first step of the method involves retrieving a model sequence of images, each image representing the model individual performing a model sequence of movements.

[0043] A sequence of images typically refers to a video recording consisting of several images arranged chronologically. The sequence of images can also be the result of an edit, for example, to include a sequence of specific movements.

[0044] According to one embodiment, retrieving a model sequence of images involves uploading or downloading a digital video file. This file may be stored on a dedicated server or on the memory of a device of the model individual such as a smartphone, a tablet, a computer, etc.

[0045] Alternatively or additionally, the recovery of the model sequence of images may also include the actual recording of the sequence.

[0046] According to one embodiment, the model individual records himself using an image capture device (camera, smartphone, tablet, etc.) and transmits the recording to a dedicated server.

[0047] Based on this model sequence of images, a skeletonization file comprising for each image of the sequence, a model joint position for each joint of the model individual represented on the image, a model index corresponding to the position of the image in the model sequence and a model time associated with the image, is produced.

[0048] Many file formats can be used for the skeleton file, such as Comma Separated Value (.csv), Pickle (.pki or .pickle), Parquet (.parquet), Feather (.fea or .feather), Avro (or ORC), or can be stored in a database in another format. The file can be stored in a database.

[0049] The skeletonization file is created by a skeletonization algorithm that allows the position to be recorded in three dimensions of joints of the model individual represented on each image of the sequence. Skeletonization can be carried out from images representing the joints in 2D or 3D.

[0050] In addition to the position of the joints, a model index designating the position of the image within the sequence is associated with each image. As detailed below, the ability to order the images in the sequence is essential to the implementation of the method.

[0051] Finally, a model time associated with the image corresponding to the time interval elapsed since the first image of the model sequence is also included for each image in the skeletonization file.

[0052] Steps for recovering an image sequence and producing a skeletonization file are also carried out for the user.

[0053] More particularly, the method comprises a step including retrieving a user sequence of images, each image representing the user performing a user sequence of movements. The user sequence of movements typically corresponds to a model sequence of movements to which the user wishes to compare his own.

[0054] As in the case of the model individual, retrieving the user sequence of images may include uploading or downloading a digital video file. This file may be stored on a dedicated server or on the memory of a user device such as a smartphone, tablet, computer, etc.

[0055] Alternatively or additionally, retrieving the user sequence of images may also include actually recording the sequence.

[0056] According to one embodiment, the user records using an image capture device (camera, smartphone, tablet, etc.) and transmits the recording to a dedicated server.

[0057] A user skeletonization file of the user sequence is then produced, the file including for each image in the sequence, a user joint position for each joint of the user represented in the image, a user index corresponding to the position of the image in the user sequence and a user time associated with the image, is produced.

[0058] The above considerations regarding the skeletonization model file format and the skeletonization algorithm are equally applicable to the skeletonization user file format and the skeletonization algorithm.

[0059] The two skeletonization files can then be compared by starting with a step comprising the association with each image of the model sequence and for each joint of the model individual represented on the image, a set of images of the user sequence by applying a dynamic time warping algorithm (“Dynamic Time Warping” (DTW) based on the difference between the model joint position and the user joint position.

[0060] Dynamic time warping algorithms aim to enable the comparison of two time series despite possible speed variations between the series.

[0061] According to an embodiment illustrated in Figure 1, this algorithm involves creating a matrix with the positions of a model joint on the abscissa, and the positions of the corresponding user joint on the ordinate. The interior of the matrix is ​​made up of the distances between each point of the sequences of positions with all the points of the other sample. The distance between points is a formula based on the Euclidean distance combined with the distance from the diagonal.

[0062] As a non-limiting illustration, if we pose L, the sequence of positions of the shoulder of the model individual, image after image until a final image "m": and F , the sequence of positions of the user's shoulder, image after image, until the final image "n": P = fl>f 2,..; fj . fn and if we also set d(i,j) as the Euclidean distance (e.g. measured in meters) between the shoulder of the model individual in frame j and that of the user in frame j, we then write the distance to be entered in the matrix between the shoulder of the leader in frame j and that of the follower in frame j:

[0063] The goal is to find a path to go from the top left point of the matrix to the bottom right while minimizing the sum of all the boxes we have passed through.

[0064] By indicating by W the path made up of all the w boxes: W = w lt w2, —,w k , then, all paths respecting: • monotonicity: all the squares on the path always progress either downwards, or to the right, or both • continuity: the squares must touch, either by height, by width, or by the corner. • the border: it starts at the top left and ends at the bottom right. are eligible. The original algorithm consists of calculating all eligible paths and taking only the one that minimizes the sum of the boxes w, this path gives the agreement between the sample points. DTW L, F) = min

[0065] Calculating all the paths can quickly become very time-consuming and computationally intensive. It is therefore advisable to optimize the search for the diagonal to speed up the work. To do this, it is advantageous to use the principle of "divide and conquer": adding the calculation on small structures that make up the large structure often takes less time than a single calculation on a large structure.

[0066] This optimization consists of dividing the weight matrix in two and applying the same algorithm recursively to the two upper left and lower right matrices.

[0067] The first separation divides the matrix into two equal parts in the width direction. Then on the separation column, it finds, around the central point, the minimum. This minimum point must not be moved away from more than the "permissible offset deviation", that is to say, the temporal difference that we estimate to be maximum between the movement of the model individual and that of the user. This minimum point therefore separates the matrix into four.

[0068] The algorithm is then applied to the top left matrix as well as the bottom right one, which will also be divided into four using the same process, and so on until we arrive at a matrix whose width or height is less than or equal to 2. The paths are then found in this matrix. By traversing it in its longest direction, we calculate the weight of each path that respects the classic rules of DTW paths: • monotonicity: all the squares on the path always progress either downwards, or to the right, or both • continuity: the boxes must touch, either by height, by width, or by the corner. • the border: it starts at the top left and ends at the bottom right The path with the minimum weight is retained for this matrix and will be concatenated to the paths of the other matrices.

[0069] Variants and / or optimizations of this dynamic time warping algorithm may be considered by those skilled in the art without departing from the scope of the present invention.

[0070] This step therefore makes it possible to associate with each image of the model sequence, a set (which may contain only one image in certain cases) of images of the user sequence which minimizes the difference (i.e. the error) of position of the user joint compared to the position of the model joint. The association can be carried out using the model and user indices associated with each image of the model and user sequences. Indeed, it is thus possible to easily assign to a model index one or more user indices in order to associate the corresponding images.

[0071] This step is illustrated in the left graph of Figure 1 where: • some template images are associated with only one user image; • some template images are associated with multiple user images; • some user images are associated with multiple template images.

[0072] Referring to the number matrix in Figure 1, the path determined by the algorithm produces a sequence of positions that minimizes the error between the model joint position and the user joint position, this path being given by the pairs of model and user indices (corresponding to the matrix entries indicated in bold): (1,1), (2,2), (3,3), (3,4), (3,5), (4,6), (5,6), (6,6), (7,6), (8,7), (8,8), (9,9), (10,10). This association means that the user was synchronized (eg in rhythm if it is a dancer) with the model individual on the images corresponding to model indices 1, 2, 3 and 6 but desynchronized on the images corresponding to model indices 4, 5, 7-10 (and more precisely late on the images corresponding to model indices 4 and 5, then early on the images corresponding to model indices 7 to 10).

[0073] The method then comprises a step including the association with each image of the model sequence and for each articulation of the model individual, of a single user reference image from among the images of the set determined during the previous step. This association is carried out by maximizing a difference between the model index and the user index. This means that, among all the images of the user sequence of the set associated with an image of the model sequence, a reference image is selected on the basis of a comparison of the user indices of the images of the set with the model index of the image of the model sequence.

[0074] The reference image is selected by keeping only the image of the set whose difference between the user index and the model index is the greatest. In other words, the selected reference image corresponds to the image of the set on which the user is the most ahead or the most behind the model individual, at least with regard to the articulation in question. If for an image of the model sequence, the user started ahead and finished behind, if the delay was greater than the advance, it is the corresponding image that is selected, if it is the opposite, it is the image corresponding to the advance that will be kept. A new coordinated list is therefore created by keeping only the maximization of the differences between the images of the model sequence and the images of the user sequence.

[0075] This association of one and only one user image to each model image then makes it possible to calculate offsets which will serve as a basis for the comparison between the two sequences of movements. Several types offsets can thus be determined in order to compare the two image sequences according to several parameters.

[0076] As detailed below, the offset can be, for each joint, temporal, spatial, velocity, acceleration, torsion, flexion, etc. Joints can also be grouped into limbs or body portions (e.g. arms, legs, torso and head, hips and shoulders, etc.) so that offsets can be calculated per limb or body portion.

[0077] Finally, the method includes calculating a movement execution score taking into account the previously determined offsets. This score corresponds to a similarity between the model movement sequence and the user movement sequence.

[0078] Many types of scores can be calculated based on the data available as well as on the interests or instructions given by the model individual, the user or even an automatic learning program, etc. In addition, since the offsets are determined image by image as well as joint by joint, the execution score can also relate to a particular sub-sequence of images, to a particular joint, to a grouping of joints (e.g. a limb, the whole skeleton, etc.), to an aggregation or an average of several intermediate scores with possibly weightings introduced according to predetermined factors.

[0079] According to one embodiment, one or more steps of the method are adapted to be carried out consecutively via an API or a mobile application. Typically, the application can retrieve the model sequence by proposing to the model individual to upload a video recording to a dedicated server or directly to make a video recording via a camera integrated into a device (smartphone, tablet, computer equipped with a webcam, etc.), then execute a program allowing the creation of the skeletonization file corresponding. The application then stores the video recording and the skeletonization file on the server. A user of the application can then select the recording made available by the application and be offered to record themselves during the execution of their user movement sequence. The application then executes a program for comparing the model and user sequences as described above in order to generate an execution score and display it to the user on their device (smartphone, tablet, computer, etc.) so that the user can see the level of similarity of their movement sequence with that of the model individual. Improvement advice based on the execution score can for example also be offered by the application as well as suggestions for other recordings (e.g. more or less advanced courses, recordings corresponding to the user's interests, etc.).

[0080] As mentioned above, several types of offsets are possible per model image and per joint.

[0081] According to one embodiment, a time shift is determined for each joint between the images of the model sequence and their associated reference images. This time shift is calculated using the model time and the user time associated with each image of the two sequences in their respective skeletonization file. A time shift, for example in milliseconds, can thus be calculated by taking the difference between the two model and user times. The time shift can therefore correspond to the delay or advance of the position of a joint of the user represented on the reference image relative to the joint of the model individual represented on the image.

[0082] In the case where the movements must be performed in rhythm with music for example, knowing the tempo of the music can allow the shift to be expressed as a percentage of bpm (bit per minute).

[0083] An average (to have the overall trend between delay and advance) and an absolute value average (so a delay and an advance do not cancel each other out) per image of the model sequence can be calculated for each articulation. The advantage of both types of averages is that the overall average can indicate a value close to zero: the user is generally on time, while he may be very late at certain times and very early at others. This disparity can thus be indicated by an absolute value average. A large difference between the average and the absolute value average therefore indicates this disparity. Conversely, the user could present a systematic delay, the average and the absolute value average would therefore be relatively close. Per articulation, an average and an absolute value average can also be calculated on all the images of the model sequence.

[0084] The execution score can thus be calculated by averaging in absolute value all the time shifts over all the images of the model sequence and over all the joints, optionally taking into account a possible weighting per image and / or per joint. In cases where the execution score is determined solely on the basis of time shifts, the execution score can for example be expressed in milliseconds (or seconds).

[0085] The execution score can also be given as a percentage. The execution score value, for example in milliseconds, can then be reported, depending on the available data: • to a known value of the tempo of a piece of music to which the movement sequence is to be performed. This value can, for example, be retrieved from a music sharing site, calculated on the fly by an algorithm or indicated by the model individual, • the overall duration of the model image sequence, • to the maximum deviation allowed when determining a path during the dynamic time warping algorithm as explained above.

[0086] Concrete examples of time shifts and execution scores calculated based on these shifts are given below. Per frame of the model sequence and per joint, the time shift can be: • positive for an advance, negative for a delay in milliseconds; • positive for an advance, negative for a delay in percentage; Per image of the model sequence and for the entire skeleton, the time shift can be: • average giving the trend between advance and delay weighted according to the importance of the articulation in milliseconds; • in absolute average so that the delay and advance do not cancel each other out, this average also follows the weighting fixed by the choreography given in milliseconds; • average giving the trend between advance and delay weighted according to the importance of the articulation in percentage; • in absolute average so that the delays and advances do not cancel each other out, this average also follows the weighting fixed by the choreography given as a percentage. For the entire model sequence and per articulation, the time shift can be: • positive for an advance, negative for a delay in milliseconds; • positive for an advance, negative for a delay in percentage; For the entire model sequence and for the entire skeleton, the time shift can be: • average giving the trend between advance and delay weighted according to the importance of the articulation in milliseconds; • on absolute average so that delays and advances do not cancel each other out, this average may possibly follow a weighting fixed by a choreography.

[0087] For each model image - reference image pair, it is therefore possible to calculate a time difference. However, it is also advantageous to calculate a position difference. Indeed, it could be that the user is late, with a gesture consistent with that of the model individual, but it is also possible that the user is on time with a non-compliant gesture.

[0088] Thus, according to one embodiment, a spatial offset is calculated for each image of the model sequence of images and for each joint of the model individual represented in the image. This spatial offset corresponds to the distance between the model position and the user position of a joint. The movement execution score can thus be calculated on the basis of these spatial offsets.

[0089] For each image of the model sequence and for each joint, a spatial offset corresponding to a positioning error is calculated. This is a Euclidean distance in meters as well as a direction vector indicating the direction of the error.

[0090] For each image of the model sequence, an average is calculated over all joints, optionally taking into account weighting by joint, by type of offset, etc. Since the Euclidean distances are unsigned, there is no need to calculate the average in absolute value.

[0091] For the entire model image sequence, an average can be given for each joint. An overall average can also be given based on a weighting by offset type.

[0092] A version of all these distances is provided can also be calculated in percentages. In this case, the distances can be reduced: to one meter; • the distance of the joint from the center of the hip (i.e. from a point in the skeleton file corresponding to a hip); • the maximum distance between, for each joint, the point closest to the hips and the one furthest away. For example, for the hand, the closest point has a distance close to 0, the furthest from the hips has a distance of approximately 1 m20 (when the arm is stretched upwards, the maximum reference distance is then 1 m20). For the head, the minimum distance is around 50 cm (with the back bent to bring the head towards the stomach) to 80 cm, with the head stretched upwards, the maximum distance difference is therefore 0.3 m.

[0093] Concrete examples of spatial shifts and execution scores calculated based on these shifts are given below. Per frame of the model sequence and per joint, the spatial shift can include: • the Euclidean distance in meters; • the distance in percentage; • the direction vector of the error; Per image of the model sequence and for the entire skeleton, the spatial shift can include: • a weighted average over all joints of the distances in meters; • a weighted average over all joints of the distances in percentage; • the sum of all direction vectors and its normalized version; For the entire model image sequence and per joint, the spatial shift can include: an average of the Euclidean distances in meters; an average of the distances in percentage over all images; • a sum of all direction vectors and its normalized version; For the entire model image sequence and for the entire skeleton, the spatial shift can include: • a weighted average over all joints and all images of the model sequence of distances in meters; • a weighted average over all joints and all images of the model sequence of the percentage distances; • a sum of all direction vectors and its normalized version.

[0094] Instead of these different distances, it is also possible to consider a polynomial function of these distances, for example a polynomial of order 2, 3, 4 or higher. In this way, it is possible to vary the importance of the spatial shift in calculating the final execution score.

[0095] According to one embodiment, a velocity offset is calculated for each image of the model sequence of images and for each joint of the model individual represented in the image. The movement execution score can thus be calculated based on these velocity offsets.

[0096] Two versions of speed differences can be calculated: • the Euclidean distance for the same articulation between two images of the model sequence divided by the time between the two corresponding images. • a speed vector, composed of the vector difference between the position of the model individual's joint and that of the user divided by the time difference between successive images. This calculation also makes it possible to have the direction of the speed. Tl

[0097] For the speed based on the Euclidean distance difference, per frame and per joint, a signed difference can be calculated in m / s, this is the speed difference itself, without taking into account its orientation.

[0098] For velocity based on the velocity vector difference, the direction vector can be calculated, as well as the magnitude of the vector difference, in m / s, hence the oriented velocity difference.

[0099] For each frame of the model sequence, a sum of the direction vectors, an average of the magnitude of the weighted speed difference vector, and the average and an absolute value average of the weighted absolute speed difference can be calculated. The sum of the direction vectors will allow us to know if the whole body is moving in the same direction in error, the average of the magnitude allows us to have the speed error including the error of its orientation, the average of the absolute speed indicates whether, overall, the speed is faster or slower, the average in absolute value of the absolute speed allows us to see if the distribution is homogeneous.

[0100] For video and per articulation the average over all frames of the model sequence is proposed for the Euclidean velocity, the magnitude of the velocity difference vector as well as the sum of the unit vector of the velocity error.

[0101] These differences can also be expressed as a percentage, reduced to the maximum speed per articulation of the model sequence.

[0102] Concrete examples of speed offsets and execution scores calculated based on these offsets are given below. Per frame of the model sequence and per joint, the speed offset can include: a Euclidean speed difference in m / s; a Euclidean speed difference in percentage; • a magnitude of the velocity difference vector in m / s; • a magnitude of the velocity difference vector in percentage; • a direction vector of the offset (i.e. of the error); Per frame of the model sequence and for the entire skeleton, the speed shift can include: • a weighted average over all joints of the Euclidean speed differences in m / s; • a weighted average over all joints of the percentage differences in Euclidean speeds; • a weighted average over all joints of the magnitude of the velocity difference vector in m / s; • a weighted average across all joints of the magnitude of the velocity difference vector in percentage; • the sum of all direction vectors and its normalized version; For the entire model and joint image sequence, the velocity offset may include: • an average over all images of the Euclidean speed differences in m / s; • an average over all images of the percentage differences in Euclidean speeds; • an average over all images of the magnitude of the velocity difference vector in m / s; • an average over all images of the magnitude of the velocity difference vector in percentage; • the sum of all direction vectors and its normalized version; For the entire model image sequence and for the entire skeleton, the velocity shift can include: • a weighted average over all joints and all images of Euclidean speeds in m / s; • a weighted average across all joints and all images of the percentage Euclidean velocity differences; • a weighted average over all joints and all images of the magnitude of the velocity difference vector in m / s; • a weighted average over all joints and all images of the magnitude of the velocity difference vector in percentage; • the sum of all direction vectors and its normalized version.

[0103] Instead of the speed difference, it is also possible to consider a polynomial function of this speed difference, for example a polynomial of order 2, 3, 4 or higher. In this way, it is possible to vary the importance of the speed difference in calculating the final execution score.

[0104] According to one embodiment, an acceleration offset is calculated for each image of the model sequence of images and for each joint of the model individual represented in the image. The movement execution score can thus be calculated based on these acceleration offsets.

[0105] Two versions of acceleration differences are calculated: • the Euclidean speed for the same articulation between two images of the model image sequence divided by the time between the two corresponding images. • the acceleration vector, composed of the vector difference between the vector velocity of the model individual's joint and that of the user divided by the time difference between successive images. This calculation also makes it possible to have the direction of the acceleration.

[0106] For Euclidean acceleration based on the Euclidean velocity difference, per frame and per joint, a signed difference can be calculated in m / s 2 , it is the difference in acceleration itself, without taking into account its orientation.

[0107] For velocity based on the acceleration vector difference, the direction vector can be calculated, as well as the magnitude of the vector difference, in m / s 2 so the difference in oriented acceleration.

[0108] Per image, a sum of the direction vectors, an average of the magnitude of the weighted acceleration difference vector as well as the average and an average in absolute value of the difference of the weighted absolute accelerations can be calculated and taken into account to calculate the acceleration offset. The sum of the direction vectors will make it possible to know if the whole body accelerates in an erroneous way in the same direction, the average of the magnitude makes it possible to have the error of acceleration including the error of orientation of this one, the average of the absolute acceleration indicates if, overall, this one is greater or smaller, the average in absolute value of the absolute acceleration makes it possible to see if the distribution is homogeneous.

[0109] For the entire model image sequence and per joint the average over all images is proposed for the Euclidean acceleration, the magnitude of the acceleration difference vector as well as the sum of the unit vector of the acceleration error.

[0110] These differences can be expressed as a percentage, reduced to the maximum acceleration per articulation of the model sequence.

[0111] Concrete examples of acceleration offsets and execution scores calculated based on these offsets are given below. Per frame of the model sequence and per joint, the acceleration offset can include: a difference in Euclidean acceleration in m / s 2 ; a percentage difference in Euclidean acceleration; • a magnitude of the acceleration difference vector in m / s 2 ; • a magnitude of the acceleration difference vector in percentage; • a direction vector of the error; Per frame of the model sequence and for the entire skeleton, the acceleration offset can include: • a weighted average over all joints of the Euclidean accelerations in m / s 2 ; • a weighted average over all joints of the differences in Euclidean accelerations in percentage; • a weighted average across all joints of the magnitude of the acceleration difference vector in m / s 2 : • a weighted average across all joints of the magnitude of the acceleration difference vector in percentage; • a sum of all direction vectors and its normalized version. For the entire model image sequence and per joint, the acceleration offset can include: • an average over all images of the differences in Euclidean accelerations in m / s 2 ; • an average over all images of the Euclidean acceleration differences in percentage; • an average over all images of the magnitude of the acceleration difference vector in m / s 2 ; an average over all frames of the magnitude of the acceleration difference vector in percentage; the sum of all direction vectors and its normalized version. For the entire model image sequence and for the entire skeleton, the acceleration offset can include: • a weighted average across all joints and all images of the acceleration differences; • a weighted average over all joints and all images of the percentage differences in Euclidean accelerations; • a weighted average over all joints and all images of the magnitude of the acceleration difference vector in m / s 2 ; • a weighted average over all joints and all images of the magnitude of the acceleration difference vector in percentage; • the sum of all direction vectors and its normalized version.

[0112] Instead of the difference in accelerations, it is also possible to consider a polynomial function of this difference in accelerations, for example a polynomial of order 2, 3, 4 or higher. In this way, it is possible to vary the importance of the acceleration shift in calculating the final execution score.

[0113] In addition to temporal, spatial, velocity and acceleration shifts, posture shifts (twisting, bending, etc.) can also be calculated and factored into the movement execution score.

[0114] A torsion offset is determined based on angle differences between a position of the model individual and a position of the user. Thus, a torsion offset can be calculated for each image of the model image sequence by determining a model torsion angle of a body portion of the model individual represented in the image and a user torsion angle of a body portion of the user represented in the image and corresponding to the body portion of the model individual. The calculation of the torsion offset is then based on the difference between the model torsion angle and the torsion angle user. The movement execution score can then include a torsion offset.

[0115] According to one embodiment, the torsion angle corresponds to a general orientation of the body. The model (resp. user) torsion angle is an orientation angle of the model individual (resp. user) relative to a fixed horizontal line, for example an axis of a terrestrial reference frame. The model and user torsion angles are compared to give an angular difference in degrees [°] or radians [rad]. They are also proposed as a percentage over 360°: • either 360° • either at the maximum amplitude of the general orientation of the model individual during the model image sequence.

[0116] The angular velocity of orientation (i.e., the rotational speed during a turn) of the model individual and the user is calculated by taking the difference of the torsion angles divided by the time difference between the two corresponding images. The difference in angular velocity is given in degrees per second [7s] or radians per second [rad / s]. They can also be given as a percentage, in which case they are reduced to the maximum torsional velocity of the model individual during the model sequence.

[0117] The orientation angular acceleration of the model individual and the user is calculated by taking the difference of the orientation angular velocities divided by the time difference between the two images. The angular acceleration difference is given in degrees per second squared [7s 2 ] or radians per second squared [rand / s 2]. They can also be given as a percentage, in which case they are related to the maximum lap acceleration of the model individual during the model sequence.

[0118] According to an embodiment illustrated in Figure 2, the torsion angle corresponds to a torsion of the torso of the model individual and the user. The model (resp. user) torsion angle is typically the angle between the projection onto a horizontal plane (i.e. onto the ground) of a straight line d1 passing through points of the model file (resp. user) of skeletonization representing hips of the model individual (resp. of the user) and the projection on a horizontal plane of a straight line d2 passing through points of the model file (resp. user) of skeletonization representing shoulders of the model individual (resp. of the user).

[0119] The model and user twist angles are compared to give an angular difference in degrees [°] or radians [rad]. They can also be calculated as percentages: • either at the maximum angle of torsion of the bust; • either at the maximum angle of torsion of the model individual during the model sequence.

[0120] The angular velocity of the model individual and the user is determined by calculating the difference in the model and user twist angles divided by the time difference between the two corresponding frames. The angular velocity difference is given in degrees per second [7s] or radians per second [rand / s]. They can also be given as a percentage, in which case they are related to the maximum twist velocity of the model individual during the model sequence.

[0121] The torsional angular acceleration of the model individual and the user is calculated by taking the difference of the torsional angular velocities divided by the time difference between the two images. The difference in angular acceleration is given in degrees per second squared [7s 2 ] or radians per second squared [rad / s 2 ]. They can also be given as a percentage, in which case they are related to the maximum torsional acceleration of the model individual during the model sequence.

[0122] According to one embodiment, the torsion angle corresponds to a twist of the head of the model individual and the user. The twist of the head is typically measured relative to the shoulders. As illustrated in Figure 3, the model (resp. user) torsion angle is the angle between: • a straight line d3 passing through the points of the model file (resp. user) of skeletonization representing the middle of the segment joining the two ears and the nose of the model individual (resp. of the user), and • a straight line d4 passing through points in the model file (resp. user) of the skeletonization representing the shoulders of the model individual (resp. user); these two straight lines being projected in a plane orthogonal to a straight line d5 passing through the middle of the segment joining the shoulders and the middle of the segment joining the ears.

[0123] The torsion angles of the model individual and the user are compared to give an angular difference in degrees [°] or radians [rad]. They are also offered as a percentage: • either at the maximum head torsion angle; • either at the maximum angle of head torsion of the model individual during the model sequence.

[0124] The angular velocity of the model individual and the user is calculated by taking the difference of the torsion angles divided by the time difference between the two corresponding frames. The difference in angular velocity is given in degrees per second [7s] or radians per second [rad / s]. They can also be given as a percentage, in which case they are related to the maximum torsion velocity of the model individual during the model sequence.

[0125] The torsional angular acceleration of the model individual and the user is calculated by taking the difference of the torsional angular velocities divided by the time difference between the two corresponding frames. The difference in angular acceleration is given in degrees per second squared [7s 2 ] or radians per second squared [rad / s 2]. They can also be given as a percentage, in which case they are related to the maximum torsional acceleration of the model individual during the model sequence.

[0126] According to one embodiment, the torsion angle corresponds to an arm twist of the model individual and the user. The arm twist is typically measured by an invariant straight line when the arm enters into torsion. As illustrated in Figure 4, the model (resp. user) torsion angle is the angle between: • a straight line d6 passing through points in the model file (resp. user) of skeletonization representing a thumb and a little finger of the model individual (resp. user); and • a straight line d7 passing through points of the model file (resp. user) of skeletonization representing a hip and a wrist of the model individual (resp. user).

[0127] The torsion angles (of the arm) of the model individual and the user are compared to give an angular difference in degrees [°] or radians [rad]. They are also offered as a percentage: • either at the maximum angle of arm torsion • either at the maximum angle of arm torsion of the model individual during the model sequence.

[0128] The angular velocity of the model individual and the user is calculated by taking the difference of the torsion angles divided by the time difference between the two corresponding frames. The difference in angular velocity is given in degrees per second [7s] or radians per second [rad / s]. They can also be given as a percentage, in which case they are related to the maximum torsion velocity of the model individual during the model sequence.

[0129] The torsional angular acceleration of the model individual and the user is calculated by taking the difference of the torsional angular velocities divided by the time difference between the two corresponding frames. The difference in angular acceleration is given in degrees per second squared [7s 2 ] or radians per second squared [rad / s 2 ]. They can also be given as a percentage, in which case they are reduced to the maximum torsional acceleration of the model individual during the model sequence.

[0130] According to one embodiment, the torsion angle corresponds to a leg twist of the model individual and the user. The leg twist is typically measured by an invariant straight line when the leg enters into torsion. As illustrated in Figure 5, the model (resp. user) torsion angle is the angle between: • a straight line d8 passing through points of the model file (resp. user) of skeletonization representing a heel and a big toe of the model individual (resp. user); and • a straight line d9 passing through points in the model file (resp. user) of skeletonization representing the two hips of the model individual (resp. user), these two straight lines being projected into a horizontal plane (i.e. parallel to the ground or perpendicular to the vertical axis in a standard terrestrial reference frame).

[0131] The (leg) torsion angles of the model individual and the user are compared to give an angular difference in degrees [°] or radians [rad]. They are also offered as a percentage: • either at the maximum leg twist angle • either at the maximum angle of leg torsion of the model individual during the model sequence.

[0132] The angular velocity of the model individual and the user is calculated by taking the difference of the torsion angles divided by the time difference between the two corresponding frames. The difference in angular velocity is given in degrees per second [7s] or radians per second [rad / s]. They can also be given as a percentage, in which case they are related to the maximum torsion velocity of the model individual during the model sequence.

[0133] The torsional angular acceleration of the model individual and the user is calculated by taking the difference of the torsional angular velocities divided by the time difference between the two corresponding frames. The difference in angular acceleration is given in degrees per second squared [7s 2 ] or radians per second squared [rad / s 2]. They can also be given as a percentage, in which case they are related to the maximum torsional acceleration of the model individual during the model sequence.

[0134] In addition to torsional offsets, flexion offsets between a body part of the model individual and the corresponding body part of the user can be calculated and taken into account in the movement execution score. Thus, a flexion offset can be calculated for each image of the model image sequence by determining a model flexion angle of a body portion of the model individual represented in the image and a user flexion angle of a body portion of the user represented in the image and corresponding to the body portion of the model individual. The calculation of the flexion offset is then based on the difference between the model flexion angle and the user flexion angle.

[0135] According to an embodiment illustrated in Figure 6, the model (resp. user) flexion angle corresponds to a torso flexion angle. The model (resp. user) flexion angle is the angle between: • a straight line d10 passing through points of the model file (resp. user) of skeletonization representing the shoulders of the model individual (resp. user); and • a straight line d11 passing through points in the model skeleton file (resp. user) representing the hips of the model individual (resp. user), these straight lines being projected into the plane p1 containing the points corresponding to the two hips and to the middle of the segment joining the shoulders of the model individual (resp. user).

[0136] The flexion angles (of the torso) of the model individual and the user are compared to give an angular difference in degrees [°] or radians [rad]. They are also offered as a percentage: • either at the maximum angle of torsion of the bust • either at the maximum angle of torsion of the model individual during the model image sequence.

[0137] The angular velocity of the model individual and the user is calculated by taking the difference of the flexion angles divided by the time difference between the two corresponding images. The difference in angular velocity is given in degrees per second [7s] or radians per second [rad / s]. They can also be given as a percentage, in which case they are reduced to the maximum velocity of the model individual during the sequence of model images.

[0138] The flexion angular acceleration of the model individual and the user is calculated by taking the difference of the flexion angular velocities divided by the time difference between the two corresponding images. The difference in angular acceleration is given in degrees per second squared [7s 2 ] or radians per second squared [rad / s 2 ]. They can also be given as a percentage, in which case they are related to the maximum flexion acceleration of the model individual during the model image sequence.

[0139] Regarding the head, two types of flexion shift can also be calculated. The first corresponds to a lateral flexion of the head, and the second corresponds to head nods, that is, a forward-backward flexion of the head.

[0140] According to an embodiment illustrated in Figure 7, the model (resp. user) flexion angle corresponds to a lateral flexion angle of the head. The model (resp. user) flexion angle is the angle between: • a straight line d12 passing through points of the model file (resp. user) of skeletonization representing the shoulders of the model individual (resp. user); and • a straight line d13 orthogonal to the plane containing the points of the model skeleton file (resp. user) representing the ears and nose of the model individual (resp. user), these straight lines being projected into the plane p2 containing the points corresponding to the two shoulders and the middle of the segment joining the ears of the model individual (resp. user).

[0141] The head flexion angles of the model individual and the user are compared to give an angular difference in degrees [°] or radians [rad]. They are also offered as a percentage: • either at the maximum angle of flexion of the head; • either at the maximum angle of flexion of the model individual during the model image sequence.

[0142] The angular velocity of the model individual and the user is calculated by taking the difference of the head flexion angles divided by the time difference between the two corresponding images. The angular velocity difference is given in degrees per second [7s] or radians per second [rad / s]. They can also be given as a percentage, in which case they are reduced to the maximum velocity of the model individual during the model image sequence.

[0143] The head flexion angular acceleration of the model individual and the user is calculated by taking the difference of the head flexion angular velocities divided by the time difference between the two corresponding frames. The angular acceleration difference is given in degrees per second squared [7s 2 ] or radians per second squared [rad / s 2 ]. They can also be given as a percentage, in which case they are related to the maximum head flexion acceleration of the model individual during the model image sequence.

[0144] According to an embodiment illustrated in Figure 8, the model (resp. user) flexion angle corresponds to a front-back flexion angle of the head, i.e. a head nodding angle. The model (resp. user) flexion angle is then the angle between: • a straight line d14 passing through points in the model file (resp. user) of the skeletonization representing the middle of the segment joining the shoulders of the model individual (resp. user) and the middle of the segment joining the hips of the model individual (resp. user); and • a straight line d15 orthogonal to the plane containing the points of the model file (resp. user) of skeletonization representing the ears and the nose of the model individual (resp. of the user), these straight lines being projected in the plane orthogonal to the straight line passing through the shoulders of the model individual (resp. of the user) and containing the midpoint of the segment joining the shoulders of the model individual (resp. of the user.

[0145] The head flexion angles of the model individual and the user are compared to give an angular difference in degrees [°] or radians [rad]. They are also offered as a percentage: • either at the maximum angle of head flexion (nodding); • either at the maximum angle of flexion (nodding) of the model individual during the model image sequence.

[0146] The angular velocity of the model individual and the user is calculated by taking the difference of the head flexion angles divided by the time difference between the two corresponding images. The angular velocity difference is given in degrees per second [7s] or radians per second [rad / s]. They can also be given as a percentage, in which case they are reduced to the maximum velocity of the model individual during the model image sequence.

[0147] The head flexion angular acceleration of the model individual and the user is calculated by taking the difference of the head flexion angular velocities divided by the time difference between the two corresponding frames. The angular acceleration difference is given in degrees per second squared [7s 2 ] or radians per second squared [rad / s 2 ]. They can also be given as a percentage, in which case they are related to the maximum head flexion acceleration of the model individual during the model image sequence.

[0148] The execution score can take into account one or more offsets of different types, per image, per total model image sequence, per joint, per group of joints or even per skeleton. Thus, the examples cited above can be taken independently or in combination when creating the execution score.

[0149] The calculation of the execution score may thus, for example, include intermediate calculations of articulation execution scores. Each articulation execution score depends on at least one type of offset (temporal, spatial, velocity, acceleration, torsion, flexion, etc.). The articulation execution scores can then be aggregated to form the movement execution score.

[0150] In particular, weightings can be introduced based on specific parameters. These parameters can, for example, be defined in advance by the model individual, by the user, as part of a program for learning a sequence of movements or a discipline, etc. These parameters make it possible, in particular, to accentuate or reduce the importance of certain shifts or certain joints or body portions.

[0151] According to one embodiment, the execution score is weighted across joints. The weights of the weighting depend on parameters such as, for example, the type of movement / sport / arts, the difficulty of the movement and / or the educational choices of the model individual.

[0152] A weighting by movement type can be determined for known genres of movement by a panel of specialists indicating which joints are most important for their disciplines. For example, Indian dance specialists might indicate that finger positions have a high weighting, which would be practically zero for running, or even another dance style like hip-hop. Conversely, these latter types of movements would have a higher weighting on the legs. There would therefore be a weight per joint, applied when the model individual chooses the movement type manually or when software (such as an AI) detects the movement type. When the genre is not known or not specified, the weights are all equal by default.

[0153] Within the same movement type, there may be movements for beginners, advanced, or experts. Or the same movements with beginner, advanced, or expert expectations. In this case, per movement type, the expert panel will designate a weighting distribution according to the articulations. For example, in Latin dance, beginners initially focus on their steps and foot positioning, while advanced dancers add arms, and experts add rib cage movements. The weighting of all these articulations will evolve in the same way.

[0154] When a model wants to offer a movement sequence focusing on a particular point or technique of their discipline, they can assign a particular weighting to the limbs they propose to work on. For example, a traditional Chinese dance teacher may want to make three videos: the first focusing on the feet and legs, the second on the arms, and the third mixing the previous two. In this case, the model will set a higher weighting on the legs and almost zero on the stockings for the first video, an inverse weighting for the second video, and a weighting closer to their type of movement for the third. A model may also choose to make all their videos on their technical specialty, in which case they will choose a basic weighting for all their videos.

[0155] Alternatively or additionally, the weighting can also depend on the type of shift.

[0156] For example, in a tap dance choreography, the weighting of time would be more important; in martial arts disciplines, position would be the most important. For well-known genres of movement, a panel of specialists can indicate which are the most important time components in their discipline. For unknown genres, a generic weighting will be proposed.

[0157] According to one embodiment, a weighting is adapted according to the difficulty of movement. Thus, for a beginner user, the execution score can give more importance to the position offset and minimize the importance of the other offsets. For an advanced user, the speed offsets (the speed distribution) can constitute a significant part of the execution score, whereas for an expert user, the acceleration offsets will be predominant in the calculation of the execution score.

[0158] According to one embodiment, a weighting is adapted according to a type of sequence or the model individual. A sequence of images explaining the principle of a turn in classical dance would have every interest in increasing the importance of orientation shifts, orientation speed, or even acceleration of the turn for experts. Similarly, a model individual presenting only sequences of images on a specific technical point might wish for a particular weighting relevant to the technical point in question to be applied to all of its sequences.

[0159] As mentioned above, the calculation of the execution score can be done from intermediate execution scores (e.g. articulation execution scores). In order for these scores to be aggregated, it is advantageous to express them as a percentage error so that the different errors can be mixed according to their units of measurement.

[0160] The error averages (in time, position, speed, acceleration) weighted on the joints are done according to the method explained above for the limbs and for the skeleton.

[0161] The combinations of errors between • speed and acceleration • time and position • twists and orientations • time, position, velocity, acceleration, torsion and orientation can be performed according to one of the weightings described above. An execution score can thus be given as a percentage, it is equivalent to 100% if there are no errors and equal to where p £ is the weighting associated with the error e £ .

[0162] As concrete examples, the following execution scores can be calculated (per frame of the model sequence or for the entire model sequence, per joint, per limb and for the entire skeleton). By image of the model image sequence: • By joint: Time shift score, position score, velocity score, acceleration score, torsion / flexion score for affected limbs, torsion / flexion speed score for affected limbs, torsion / flexion acceleration score for affected limbs, combined velocity acceleration score, combined time / position / torsion score, overall combined score • Per limb: Time lag score across all weighted limb joints, position score across all weighted limb joints, velocity score across all weighted limb joints, acceleration score across all weighted limb joints, torsion / flexion score for affected limbs across all weighted limb joints, torsion / flexion velocity score for affected limbs across all weighted limb joints, torsion / flexion acceleration score for affected limbs across all weighted limb joints, combined velocity acceleration score across all weighted limb joints, combined time / stance / torsion score across all weighted limb joints, overall combined score across all weighted limb joints • For the skeleton: Time shift score on all weighted joints, position score on all weighted joints, velocity score on all weighted joints, acceleration score on all weighted joints, orientation score, turn velocity score, turn acceleration score, torsion / flexion score for affected limbs on all weighted joints, torsion / flexion velocity score for affected limbs on all weighted joints, torsion / flexion acceleration score for affected limbs on all weighted joints, combined velocity acceleration score on all weighted limbs, combined time / position / torsion score on all weighted joints, overall combined score on all weighted joints. For the entire model sequence: • By joint: Time shift score, position score, velocity score, acceleration score, torsion / flexion score for affected limbs, torsion / flexion speed score for affected limbs, torsion / flexion acceleration score for affected limbs, combined velocity acceleration score, combined time / position / torsion score, overall combined score. • Per limb: Time lag score across all weighted limb joints, position score across all weighted limb joints, velocity score across all weighted limb joints, acceleration score across all weighted limb joints, torsion / flexion score for affected limbs across all weighted limb joints, torsion / flexion velocity score for affected limbs across all weighted limb joints, torsion / flexion acceleration score for affected limbs across all weighted limb joints, combined velocity acceleration score across all weighted limb joints, combined time / stance / torsion score across all weighted limb joints, overall combined score across all weighted limb joints. • For the skeleton: Time shift score on all weighted joints, position score on all weighted joints, velocity score on all weighted joints, acceleration score on all weighted joints, orientation score, turn velocity score, turn acceleration score, torsion / flexion score for affected limbs on all weighted joints, torsion / flexion velocity score for affected limbs on all weighted joints, torsion / flexion acceleration score for affected limbs on all weighted joints, combined velocity acceleration score on all weighted limb joints, combined time / position / torsion score on all weighted joints, overall combined score on all weighted joints.

[0163] The present invention also relates to a movement comparison device comprising a first image capture device configured to capture a model sequence of images of a model individual performing a model sequence of movements, a second image capture device configured to capture a user sequence of images of a user performing a user sequence of movements, and a computer computing module for implementing the method described above.

[0164] As mentioned above, the first and second image capturing devices may include a smartphone, a tablet, a computer with a webcam, a camera, etc.

Claims

Claims 1. A method for comparing movement comprising the steps of: a) retrieving a model sequence of images, each image representing a model individual performing a model sequence of movements; b) producing, by means of a computer calculation module, a model skeletonization comprising, b1) for at least three images of the model sequence of images, at least one model joint position of at least one joint of the model individual represented on each of the at least three images, b2) a model index corresponding to the position of each of the at least three images in the model sequence and b3) a model time associated with each of the at least three images; c) retrieving a user sequence of images, each image representing a user performing a user sequence of movements;d) producing, by means of a computer calculation module, a user skeletonization comprising, d1) for at least three images of the user sequence of images, at least one user joint position of at least one joint of the user represented on each of the at least three images, d2) a user index corresponding to the position of each of the at least three images in the user sequence, d3) and a user time associated with each of the at least three images; e) associating with each of the at least three images of the model sequence and the at least one joint of the model individual represented on the image, at least one image of the user sequence by applying a dynamic time warping algorithm based on the difference between a model joint position and a user joint position; f) for each of the at least three images of the model sequence and the at least one joint of the model individual represented in the image, selecting at least one reference image of the at least one associated image maximizing a difference between the model index and the user index; g) for at least one image of the model sequence and for at least one joint of the model individual represented in the image, calculating an offset between the image and the corresponding reference image, said offset being a time offset, a spatial offset, a speed offset, an acceleration offset or a torsion offset; h) calculating a user movement execution score based on the offset, the score corresponding to a similarity between the model sequence of movements and the user sequence of movements.

2. The method of claim 1, wherein the step of retrieving the template sequence of images comprises: recording the template sequence of images using an image capture device; and / or wherein the step of retrieving the user sequence of images comprises: recording the user sequence of images using an image capture device.

3. Method according to one of claims 1 to 2, an offset being a time offset determined during a step comprising: for the at least three images of the model sequence and for the at least one articulation of the model individual represented on the image, calculation of the time offset on the basis of the model time and the user time of the reference image associated with the image of the model sequence, the movement execution score being calculated on the basis of the time offsets.

4. Method according to one of claims 1 to 3, an offset being a spatial offset determined during a step comprising: for the at least three images of the model sequence of images and for the at least one articulation of the model individual represented on the image, calculating the spatial offset between the model position and the user position, the movement execution score being calculated based on the spatial offsets.

5. Method according to one of claims 1 or 2, an offset being a speed offset determined during a step comprising: for the at least three images of the model sequence of images and for the at least one articulation of the model individual represented on the image, calculating a speed offset between an articulation speed of the model individual and an articulation speed of the user, the movement execution score being further calculated on the basis of the speed offsets.

6. Method according to one of claims 1 or 2, an offset being an acceleration offset determined during a step comprising: for the at least one image of the model sequence of images and for the at least one articulation of the model individual represented on the image, calculating an acceleration offset between an articulation acceleration of the model individual and an articulation acceleration of the user, the movement execution score being further calculated on the basis of the acceleration offsets.

7. Method according to one of claims 3, 4, 5 or 6, wherein the step of calculating the execution score comprises sub-steps of: for at least one articulation of the model individual, calculating an articulation execution score on the basis of the temporal, spatial, velocity and / or acceleration offsets; calculating the execution score on the basis of the articulation execution score.

8. The method of claim 7, wherein the execution score is a weighted average of articulation execution scores of a plurality of articulations of the model individual.

9. Method according to claim 8, wherein a weighting of the weighted average is carried out according to a type of movement, a difficulty of movement and / or a level of the user.

10. Method according to one of claims 1 or 2, an offset being a torsion offset determined during a step comprising: for each image of the model sequence of images, determining: a model torsion angle of a body portion of the model individual represented in the image; and a user torsion angle of a body portion of the user represented in the image; calculating a torsion offset on the basis of the model torsion angle and the user torsion angle; the movement execution score being further calculated on the basis of the torsion offsets.

11. Method according to claim 10, in which the model torsion angle is formed by a straight line passing through points of the skeletonization model file representing hips of the model individual and a reference straight line corresponding to a horizontal straight line of a terrestrial reference frame, and in which the user torsion angle is formed by a straight line passing through points of the user skeletonization file representing hips of the user and the reference straight line.

12. The method of claim 10, wherein the model torsion angle is formed by a straight line (d1) passing through points in the skeletonization model file representing hips of the model individual and a straight line (d2) passing through points in the skeletonization file representing shoulders of the model individual, and wherein the user torsion angle is formed by a straight line passing through points in the user skeletonization file representing hips of the user and a straight line passing through points in the user skeletonization file representing shoulders of the user.

13. The method of claim 10, wherein the model twist angle is formed by a straight line (d4) passing through points in the skeletonization model file representing shoulders of the model individual and a straight line (d3) passing through the points in the skeletonization model file representing the middle of the segment joining ears and a nose of the model individual, and wherein the user twist angle is formed by a straight line passing through points in the user skeletonization file representing shoulders of the user and a straight line passing through the points in the user skeletonization file representing the middle of the segment joining ears and a nose of the user.

14. The method of claim 10, wherein the model torsion angle is formed by a straight line (d6) passing through points in the skeletonization model file representing a thumb and a little finger of the model individual and a straight line (d7) passing through points in the skeletonization file representing a hip and a wrist of the model individual, and wherein the user torsion angle is formed by a straight line passing through points in the user skeletonization file representing a thumb and a little finger of the user and a straight line passing through points in the user skeletonization file representing a hip and a wrist of the user.

15. The method of claim 10, wherein the model torsion angle is formed by a straight line (d8) passing through points in the skeletonization model file representing a heel and a big toe of the model individual and a straight line (d9) passing through points in the skeletonization file representing hips of the model individual, and wherein the user torsion angle is formed by a straight line passing through points in the user skeletonization file representing a heel and a big toe of the user and a straight line passing through points in the user skeletonization file representing hips of the user.

16. Method according to one of claims 1 to 15, an offset being a bending offset determined during a step comprising: for each image of the model sequence of images, determining: a model flexion angle of a body portion of the model individual represented in the image; and a user flexion angle of a body portion of the user represented in the image; calculating a flexion offset based on the model flexion angle and the user flexion angle; the movement execution score being further calculated based on the flexion offsets.

17. The method of claim 16, wherein the model flexion angle is obtained by calculating the angle formed by a straight line (d10) passing through points of the skeletonization model file representing shoulders of the model individual and a straight line (d11) passing through points of the skeletonization file representing hips of the model individual, in a plane (p1) containing the points corresponding to the hips of the model individual and the middle of the segment joining the shoulders of the model individual, and wherein the user flexion angle is obtained by calculating the angle formed by a straight line passing through points of the user skeletonization file representing shoulders of the user and a straight line passing through points of the user skeletonization file representing hips of the user, in a plane containing the points corresponding to the hips of the user and the middle of the segment joining the shoulders of the user.

18. The method of claim 16, wherein the model flexion angle is obtained by calculating the angle formed by a straight line (d12) passing through points in the skeletonization model file representing shoulders of the model individual and a straight line (d13) orthogonal to the plane containing points in the skeletonization model file corresponding to ears to a nose of the model individual, in a plane (p2) containing the points corresponding to the shoulders of the model individual and to the middle of the segment joining points corresponding to the ears of the model individual, and wherein the user flexion angle is obtained by calculating the angle formed by a straight line passing through points in the user skeletonization file representing shoulders of the user and a straight line orthogonal to the plane containing points in the user skeleton file corresponding to the user's ears and nose, in a plane containing points corresponding to the user's shoulders and to the midpoint of the segment joining points corresponding to the user's ears.

19. The method of claim 16, wherein the model flexion angle is obtained by calculating the angle formed by a straight line (d14) passing through points in the skeletonization model file representing the middle of the segment joining shoulders of the model individual and the middle of the segment joining hips of the model individual, and a straight line (d15) orthogonal to the plane formed by points in the skeletonization model file representing ears and a nose of the model individual, in a plane orthogonal to the straight line passing through points in the skeletonization model file representing the shoulders of the model individual, and wherein the user flexion angle is obtained by calculating the angle formed by a straight line passing through points in the user skeletonization file representing the middle of the segment joining shoulders of the user and the middle of the segment joining hips of the user,and a straight line orthogonal to the plane formed by points in the user skeletonization file representing the user's ears and nose, in a plane orthogonal to the straight line passing through points in the user skeletonization file representing the user's shoulders., 20. A movement comparison device comprising: an image capture device configured to capture a model sequence of images of a model individual performing a model sequence of movements; an image capture device configured to capture a user sequence of images of a user performing a user sequence of movements; a computer computing module for implementing the method according to one of claims 1 to 19.

Images